Give them no information. If they press, refer them to the information in the rejection letter or refer them to contact the recruiter they were working with at your company (assuming it wasn't you). If there are signs that make you uncomfortable, do not hesitate to call your building security or the local police or law enforcement.
Giving ANY information is not going to benefit you personally or your employer. Your employer has already covered all the bases they care about in the rejection letter. That process exists for a purpose, you should respect that. Giving the candidate info likely won't benefit the candidate, either - if they're not fit for the job, giving them info on why won't suddenly get them hired. And if they want to "improve for the future" they certainly can, and should, focus on getting employment support elsewhere (ie a third party recruiter, job training, etc), not from an employer who has rejected them. Your obligation to interact with this individual ended when they got the rejection letter.
Job interviews and hiring processes can be stressful on candidates, and can bring out the worst in people who aren't equipped to handle the stress. You don't know their mental state or how they handle negative feedback, and in a situation where they've just randomly shown up and confronted you, they're already showing signs that they may not be the most stable, by-the-book person. Think of your own personal safety first and do what you can to de-escalate, end the discussion, and get outside help from security or police as needed.