First, understand why you want to do this. Social default (and the safe path) would be to say nothing, so the fact that you're even bringing this up suggests that you want this thing at some level.
Do you want it because you have a lot of built-up emotional drek at this guy and want to give him a piece of your mind? Probably. Unfortunately, this urge is almost entirely unproductive for everything other than just making yourself feel better. Any response that comes from this place will be an attack. That's what this particular desire wants, after all. It will be received as an attack and responded to as if it's an attack, and is highly unlikely to actually cause anything good to happen for anyone. It'll make you feel better, and might make them feel worse, and will alienate someone who, at the end fo the day, will still be a higher-ranking individual at your same company, even leavign aside the issues with potential recommendations in the future.
Do you want it because you seriously think his management methodology is flawed, and you think that your feedback could help him improve it? Possibly. If this really is a significant part of your motivation, then you might still have something to work with. First, you're going to need to process through the drek from the first reason, so that it doesn't get in the way. Then, you're going to have to think about what he does wrong, why it's wrong, what he could be doing instead that would be better (in clear, straightforward, easily implemented terms), and why it would be beneficial to him to do it in that better way. Ways in which it would be beneficial to the company would also be useful, but unless you have reason to think that he has deep wells of company loyalty, then it's not likely to get you anywhere. Try to come up with some benefit that's not just "your employees would be happier and less likely to leave". That one's still too much of an attack.
Once you have that, you can present it as such. "So tell me, why are you leaving?" "I think you would get more productivity out of your workers if you were more open to employee suggestions, and more willing to listen overall." It communicates what you think you need to communicate, and it does so in a way that indicates that you could have made an attack, and perhaps wanted to, but chose not to. Of course, you may wish to modify the phrasing and content to fit your own case, but in essence, the further you can get away from your own hurt, and away from any clawing back of status, and towards providing him with directly useful information he did not previously have, the better your chances are of actually having a positive effect.
As a particular note, talking about how it made you uncomfortable probably isn't helpful. Talking about how he did something wrong absolutely isn't helpful. Every step you take away from the informative and towards the judgemental is actively damaging to your attempt to help. Again, if you actually want to do this, you're going to have to work through your own feelings first, so that they aren't getting in the way.
...or you can decide to do the safer, easier default thing, and not offer any meaningful response. That's also an entirely valid choice.