Clarify the Norms
Rather than getting defensive, I recommend asking the boss if it's preferable for you to work the same time frames as your colleagues. Point out that you know there's a real motivation from HR to arrive on time, but that you notice that normal behavior in the office is to arrive after working hours.
Ask him whether it's preferable that you:
- arrive on time, and leave on time
- arrive late and leave late
It sounds like the HR warnings are simply not a big deal - the bigger deal, in your supervisor's opinion, is that the team be together in the later hours.
Make your supervisor clarify this - and then figure out if you are willing to change your life accordingly. Working late is great for some people and horrible for others - if this environment doesn't make you happy, find a different workplace.
Give it a Shot
Assuming the boss concurs, that arriving late and getting warnings from HR is less important than working the same hours as the rest of the team - shift your schedule and see how it goes. What's occuring late at night?
What I read in your question is that you assume that your colleagues are leaving after a shift that is equally as long as your own... only later in the day. Is that something you have proof of, or something you've assumed from talking to people? Being around the same time as the team will give you a sense of when they really go home.
It'll also give you a sense of what happens in those later hours. It can be an unspoken thing that certain team communication just happens to occur later in the day. If your boss is there and your team is there, and you're not, they may be frustrated because they can't talk to you and know your part of the work. It would be far more preferable for your boss to be able to clarify this, and adjust the time of this communication, but not every boss is so smart.
Work with your Boss to Define "Enough"
These days, salaried work does not mean 40 hours. It means working until all the work is complete. It's worth it to clarify what "enough" work really is.
For example - in a team working with weekly assignments (like agile software developers) - "complete" means that the whole team has finished the work for the week. If someone finishes their currebt assignment early, they take on work from someone else who is behind. The goal is that team can complete its work, not that a single individual can start and finish what he takes on. It's understood that each week, some work will be harder and some work will be easier, and tasks get juggled, and feedback is given accordingly.
If you're doing what you were told to do, and nothing more, it may not be enough in a team like this - and if the team is reallocating work late at night, you're missing the opportunity to hear about the problems and take on rebalanced work, which means that in the boss' eyes, you are doing less work than other team members, regardless of the rules of coming in on time.
After working late some nights, and getting a sense for whether there are communication patterns you were missing, or added work that you needed to take on - check in with the boss. Talk to him about your observations, and ask him for feedback. If nothing else - it shows you care.
What if I hate this?
It's really up to you. There are offices where punctuality is important, and bosses who judge people by how early they get in in the morning, and not how late they stay. Every office is different and this may not be the right place for you.
But you won't know unless you try.
Playing by the formal rules clearly isn't working - your boss has expressed unhappiness, and you know that you follow a different pattern than your colleagues. You'll have to figure out whether the pattern of the team is something you can deal with in your home life. And if you hate it, you'll need to figure out what your other options are. Generally, they are likely to be:
- work the hours you prefer, and sacrifice opportunities in the current company
- find a company with better hours for your personal needs