This is a sad but incredibly common situation.
You have a manager who simply thinks that, whatever hours have been contracted for, are a license to get any extra hours he feels you somehow "owe", because you aren't an hourly wage employee.
A problem you have is that much of the work culture will be on his side, in the US. That doesn't make him right, but it makes it easier for him to feel you do indeed owe whatever hours he thinks 8-5 should really mean.
I've had this problem myself (due to medical sleep cycle stuff I often start late, end very late). I had a manager who wrote a "see me" memo about turning up to work at 10 instead of 9. I was essential to their project and he wasn't so secure, which gave me leverage to be extremely blunt. I emailed him back that I'd gladly watch and comply with the work hours in the morning, and "of course" in return, I expect him to be as respectful of them in the evening. Since I was usually in about an hour late in the morning, and worked 3-4 hours extra evenings, I never heard another word about it, again. Not a single word. Ever. And the performance bonuses and promotions still flowed.
I don't think that exact answer will work for you, but the principle stands. This is a manager who just wants more hours. Or doesn't understand you are efficient, or sees that and just tbinks of it as an entitlement to have more hours anyway, but efficient ones too.
You need to set boundaries. Because my sense is, he will push. So I wouldn't compromise going at 5, and agree to keep the peace, at all. One act of weakness just says that others are possible, to a manager like this. It'll become the norm and he'll be back for the next slice.
That means, not agreeing with his arguments, or conceding points suggesting you could give in.
I'm not an hourly wage person, that's true. I'm an 8 hourly day person. HR and I contracted that you will get my absolute best efforts for 8 hours a day. If you aren't happy that you have my best efforts, please explain, help me improve, or fire me. If you have my best efforts, and you want more hours, then that's a change of contract. Its not something HR asked for, or I offered. We can discuss it, but it needs to be clear, that you're asking me to do extra for free. That doesn't work with any professional, and I don't see why it should be expected of me.
This is pushy - but so is he. Either way he won't like it, but I don't think you have any option he will like other than rolling over and asking however many hours week he would like, and licking his shoes. So I start from an assumption that you will have to draw a line that fundamentally, he won't accept easily or like you for. You'll either succeed, or fail, to set and hold that line. You don't have a way to so it that doesn't involve conflict of some kind. So, unfortunately, I accept that is likely too, and not a deterrent.
If you're good and he knows you are good, then you won't be at risk from being that forcefully assertive. Challenge his assertion that its your job to provide free extra hours, and push back with your own perspective. No other professional would. Nor will you.