You can go about this a few different ways
1.) Ask your new potential employer if there is any possible way you can do it another time, like the day after what is already scheduled or later in the day. Explain to them the situation you are having and that you need to be there for last day formalities. If they are/want to be a successful employer then they would understand and I'm sure they would want the same from one of their employees if they were leaving. If they think that their time is the most valuable and you should drop everything for them then I would remember that and consider that later on when deciding to take a job there. By choosing this option I think the company you are interviewing with would see it as showing your loyalty and that is a valuable asset in employees that is hard to come by.
2.) Ask your current/old employer there is a different way to go about your last days. You could come in on the last day and have lunch and say your goodbyes and such and then leave for your interview. And then come in the next day to do the formalities with HR and your boss. You could probably say goodbyes then instead as well. First explain the situation and I'm sure your current employer could work something out. I don't think you will be "burning any bridges". And if you do "burn bridges" by being professional and explaining the situation and asking in a professional way then you're probably right to leave that company and they will probably only burn more bridges themselves until they are on an island and I don't think that will hurt you. You can explain that to any employer later on if that ever comes up and I'm sure they would understand
Best Option: I think the best option here, and best of both worlds, would be to try and do formalities in the morning and have lunch with your team and then leave after lunch to go to your interview. Or if your interview is in the morning, go to your interview and then come back for lunch and finish formalities in the afternoon. That way there is a compromise. Both sides should understand.