I can't go home for the day until my work is done
This is why your manager doesn't value your time. He never experiences the negative consequences of having you stand around doing nothing for an hour, because you always take that hour out of your personal life. It's fairly natural, if a little careless, to value a resource at what it costs you. In this case: nothing.
It may be that reminding him there are competing demands on your time is all it will take him to stop regarding these things as insignificant. Some people just don't like to anticipate problems, they expect you to tell them (gently) when something's inconvenient or that you're busy. Perhaps since you're reluctant to raise the problem he's carelessly assuming there's no problem. Try out, "I already have a full day today so I can't easily add that hours-long meeting". "You've put an hour for this 1-1 meeting in my diary, and I'm going to need to leave when that's up to get to my next thing". "I can do that, but it'll push the report back to Monday and currently we're planning to send it Friday".
If that attempt to solve it piecemeal doesn't work, then you have to raise the general issue more firmly without upsetting him. How to do it depends how he thinks about your role. There are three major lines of approach I can think of:
Correct him. This relies on him seeing you as a wise advisor who can check on his decisions. If he sees you as a lackey who obeys without question, do not do this. But in whatever words your relationship allows, you might say, "look, boss, you always think I need to be in these meetings, but you never call on me and I can read the minutes in 1/10 the time. I don't see why I need to be there, what's going on? Should I be there? Should I even bring my laptop and work through the meetings?". Or, ultimately, if your boss's time is worth more than yours then sometimes you have to wait for him to finish his phone call. The problem here might be that he always thinks the call is just a few minutes, so it's reasonable to ask you to wait. Actually it takes an hour. Maybe he needs you to point out this mistake he keeps making, and expects you to raise problems you see. Or maybe he very much doesn't.
"There is something more valuable I should be doing than those things". Suggest specific things that you would do with the time. When invited to an hours-long meeting, ask in advance what you're there for and say, "that meeting's the same day as X, I won't be able to do X at the same time as being in the meeting, would it be better for me to be doing X?". This is good if he sees you as an independent action-adventurer who can be relied on to find the best thing to be doing. Now, he might say, "oh, just do both", but maybe he expects you (as lead developer) to have an opinion what you should spend your time on and to turn down the things that are less important.
Appeal to work-life balance. Point out to him that in addition to your ordinary 10 hours of work you have to do, he keeps adding 5-hour meetings. Say that while you're not a stickler for hours, these 15-hour days just aren't possible for you, and you need him to help you reduce your workload. Once he realises that it's easier to delegate "doing nothing" to a less senior developer, than it is to delegate your actual work, hopefully he'll see "doing nothing" as the most sensible thing to cut. Or maybe the reverse: he really values you in these meetings so he'll help you find ways to offload what used to be your ordinary work and change your role. This is good if he sees you as a high-value resource that he has to deploy to the right place at the right time. It's bad if he sees you as a 24/7 fanatic who sleeps at your desk and who he plans to grind to dust and then throw away.
Of course he may think of you as a little of all three of these things, so pick a line that feels to you like he'd go for it: if he values your opinion, tell him you're wasting your time there; if he values your prioritisation, tell him you've got something more important to do; if he values your well-being, tell him you don't have time to do it all. If he values none of these, find a better boss ;-) Whatever you choose, do it politely and present it as an unavoidable fact of life that just has to be dealt with.