Other kinds of solutions exist, that might help here and which I did not see yet in the other answers (though 2 is mentioned in a comment):
1. Accepting the task, and indicating he will have it tomorrow morning.
Though perhaps it may be common for you to pick up things right away, it is not mentioned that the task really has to happen now. Something like this could simply be enough:
I won't be able to make it today, should I do it first thing in the morning?
Note that if his common response is to push you to do it today, you may need to play with the wording untill he does not have the urge. An example:
I have to leave for an appointment in a few minutes.
This does not directly give away that you will do it first thing tomorrow, so perhaps it helps because he then 'achieves' something as manager when he at least convinces you to pick it up when you come in.
2. Building some slack into your schedule
Surely it can be morally defended that you should never be required to do overtime, but in practice it can be very nice if you are able to show that you do want to be flexible, and help people out when they really need it.
The solution may be very easy:
Instead of setting your normal workday end at 17:00, set it at 16:30.
It may sound silly at first glance, but this allows you to pick up urgent last minute requests if they would occur. Once the manager sees your regular time to leave is 16:30, he will normally understand that a request that forces you to stay in till 17:00 may be honored, but that a request that forces you to stay in till 17:30 is really a big thing.
Of course this would mean you need to start working a bit earlier (perhaps not if you worked overtime the day before), and sometimes have 'downtime' between end of work and your appointments, but I think this is the best way if you want to introduce flexibility. (Of course moving your appointments backwards has the same effect).
3. Find a way to balance the scales
If I read correctly you working late in your specific situation could be balanced out by you getting a taxi instead of a bus. I would not mention this directly to your boss, but consider discussing with a mentor first whether it would be reasonable to say:
In order to make it to my appointment, picking up that task would
require me to get a cab
Before bringing this up make sure to consider how you feel about who should be paying for the cab in that scenario. In some companies getting money for a cab can be normal, in others it can be unthinkable.