I think it goes without saying that this kind of behaviour is unusual and probably inappropriate in most cultures, but I don't think that needs to preclude giving the person the information that they're seeking. Perhaps even more so in this situation as the individual might not be aware that they're breaching social norms - in all likelihood their blind spots didn't help them during the interview process either.
You'd be well within your rights to turn them away, and in the absence of other responses I'd probably be emphasising that a bit more, but given the responses already cover this I'll focus on another option; I think the likelihood of harm arising from having a conversation with them is overstated in the other responses. Using the time to explain your decision and what they can do to improve their chances in future might be a hugely valuable experience for the candidate.
I think the other answers are needlessly paranoid and too narrowly focused on solely fiduciary obligations, to the point where they seem to amount to countering some perceived entitlement with fear or even more entitlement. There are plenty of completely innocent explanations for this behaviour and while a cursory assessment of their mental state wouldn't be a bad idea I also think the risk of some kind of violent derangement or an impending lawsuit at the slightest item of constructive feedback are just plain silly.
Having said that, this conversation would be a good time to point out how inappropriate it is for them to show up like this, and that they would have improved their chances by simply calling or sending an email for further clarification. This feedback might be more important and useful to the candidate than any constructive commentary on their interview performance.
I initially mentioned a disclaimer that I wasn't from the US, which I took out, but I'm starting to wonder whether the downvotes are related to a jurisdictional/cultural issue. I don't know whether the risk of someone arriving with a weapon or lawsuit are a realistic prospect elsewhere; they definitely are not where I am from.