There are two issues you have to take care of.
Read your contract and the laws of your country to ensure that toilet time is not considered working time. In many places it is. I wouldn't ask in your company. If you can't find it out yourself, ask a workers union, a lawyer or whatever else is available in your country.
Get proof that the office isn't open long enough and communicate it.
You said your colleague with the key lied to the manager.
Tomorrow you will be there at exactly the time your colleague said the office is open. If it isn't open, take a picture. Wait directly there until your colleague arrives and opens the office, if possible also take a picture.
As soon as you are at your desk, you send an email to your team lead and your acting manager (and your personal email in BCC). "Hello Bosses, I was at the office today at [time] where it should open. Unfortunately, it wasn't unlocked by [coworker] until [time]. As you know, the office usually is locked at [time] again, so it isn't possible for me to work seven hours today. Please advise me what to to."
If your colleague asks you to leave, before leaving you send an email to your team lead and the acting manager (and your personal email in BCC). "Hello Bosses, [coworker] is right now asking me to leave because they will close the office! As I informed you in the morning, the office was only opened at [time]. You see, that it is not possible for me to work seven hours under this circumstances. Please tell me how the company will provide the means to work the contracted seven hours to me."
The day after tomorrow, you do the same. And the day after. Until they provide a solution. And if they tell you how to handle the situation, you request to have this in writing. Until you have that in writing, you continue to inform them that you are not able to work seven hours because of circumstances not in your control each day in writing.