It's just a mannerism like any other. Normally you just get used to things like this. It's harmless enough, it's not a mannerism like hawking and spitting while clearing your throat.
If you must, then just interrupt his apology with... 'You're fine, carry on' or something else short and to the point to forestall him.