It will vary from person to person.
Some will not require an explanation of your attire at all.
Some will view the explanation as acceptable and think nothing further of it (at least consciously - some will continue thinking of it subconsciously, which can still be to your detriment).
Some will view it as either you not taking the interview particularly seriously, or perhaps as a sign of disrespect.
Some will view the explanation itself negatively (making excuses) (but would've been fine had you said nothing).
Another point to consider - how comfortable would you be sitting there in your training clothes with everyone else wearing suits (for example)? Is there a possibility that you'll get the feeling that you're being judged (for your choice of attire) and that will make you (even more) nervous?
Some options I can think of:
Take a longer lunch to change.
This is a good option if lunch breaks aren't too closely monitored, or occasionally taking a bit longer lunch and working a bit later isn't frowned upon.
Changing should just take a few minutes, assuming you don't have mobility issues.
Take the afternoon / day off.
Giving even a somewhat vague reason (e.g. 'admin') shouldn't be too suspicious, assuming you don't go for interviews too often (if you do, you should probably focus more on roles you're a better fit for anyway).
If you're not comfortable with giving a somewhat vague reason (or lying), you could always plan to do some actual admin task during the afternoon or day as well, as to use that as the reason instead.
Wear half of what you plan on wearing to the interview to work.
Would sport shoes, formal pants and a formal shirt hanging out look like you're overdressing? From there it should be a really quick change. Try to wear as much as possible of the clothes which you're going to wear so changing can be done really quickly.
Start dressing up to work (most days / every day).
As mentioned in the above point, you don't have to wear the full interview attire, nor does it even have to look particularly formal, it just has to be some bits and pieces to make changing less effort.
Incrementally dressing up could be a good idea - wear formal pants with your current attire for a while, then change the shirt, etc.