I'm not certain from your question whether you said it jokingly -- the fact you expected him to smile, and that others joked about it, suggests maybe you did. As a rule, don't deliver serious information in a joke. In this case, everything you said was true. Sometimes when people are joking, and doing their joke face, what they are saying is not true. Hence the expression, "only joking". Maybe your boss looked perplexed because he isn't entirely sure you really answered his question.
Possibly also your boss looked perplexed because you said something vague ("I wasted two hours"). The detail might not really matter, but as bethlakshmi points out, you've conjured a lot of possibilities. Part of his job is to keep your workday manageable. Perhaps he was briefly indecisive/perplexed as to whether he needed to pry further into what happened that's apparently keeping you in work for two hours later than usual, but decided not to.
Possibly he looked perplexed because he thinks wasting time isn't something you should do in the first place. Of course often one only realises in hindsight that it's a waste, and that's fine, so it shouldn't be inherently perplexing to discover someone has wasted two hours. But if it was "fine" time-wasting then you shouldn't feel the need to make it up, and if it wasn't "fine" then he's wondering why you did it at all. Maybe he doesn't want people hanging around the office late for no good business-driven reason.
I doubt that he's perplexed over your honesty. It's not like you admitted to a crime or anything, so what you're is doing has way more potential to perplex. The fact you're telling the truth about it to your boss should be entirely normal and comprehensible to him.
You should tell the truth, but you should also give him enough information to understand the situation. If he was perplexed, he didn't understand the situation, so you probably should have said more. Then again, it likely didn't matter all that much to him or he'd have asked more questions.