Here's a few of my heuristics:
Expect people to be human
It's pretty reasonable to expect that an interviewee will tell you if they have just had a horrible life event - death in the family, accident on the way to the interview, etc. At this point, I'd suggest changing from expecting for a normal, energetic, enthusiastic demeanor to one that may be somewhat muted, or downright unhappy. The process changes from trying to see if the candidate is engaging to trying to see if they exhibited good judgement in continuing with the interview instead of cancelling - if they can still talk about the nature of the work and answer reasonable interview questions, then you can respect their ability to focus despite personal tragedy. If they can barely follow standard questions, you have to wonder if they have good judgement or whether they should have rescheduled. After all - I wouldn't want to have someone doing any sort of knowledge work for me on a day they were too rattled to make good decisions.
Caveat - different jobs have different expectations. There are certainly some jobs where a nearly maniacal ability to fake emotion may be a required quality. For example - literally last night - an actress I was interviewing told me her worst performing experience was performing while suffering acute food poisoning. If she was a software engineer - I'd think she was nuts - she should go home and rest. But in a theater environment, there may not be an understudy, you can't replace an actress midshow, and emulating emotion in an unnatural context is the name of the game.
How rude is too rude?
I'm a big fan of the unanimous vote on that one - any set of interviewers you come up with will undoubtedly be fewer people than this person will have to interact with. So if you have 5 interviewers and 1 thinks the candidate was rude - it could be (extrapolating wildly) that 20% of the corporate environment he's working in will find him rude. For me - that wouldn't be acceptable unless there was a mitigating circumstance - for example - the guy doesn't respect personal space from an American perspective, but he's flown in for the interview from another country where you know personal space expectations are much smaller in the dominating culture. He'll be working in his native country primarily and only travels for an annual meeting.
Be careful to interview for what you want. In a knowledge working team, I expect people to seem open, honest, and helpful - I don't really care if they react negatively to an uncomfortable or intimidating situation - as the manager it's my goal to reduce these, and my folks have every right to look and feel intimidated when such a thing comes up. Better yet, I hope they'll tell me that something disquieting just happened. I probably wouldn't even bother to consciously test an interviewee for the ability to stay cool while put through a stressful situation of a psychological variety. That's not what I'm hiring them for, and so it's a waste of my time and energy to try to construct such a test.
OTOH - I know that screenings for sales positions and other high-pressure, people facing work deliberately seek to produce a stress reaction typical of what might occur in day to day work to verify that the potential employee will act appropriately and productively on behalf of the company.
There is no candidate that will be perfect for all things - structure both the verbal and body language to get someone who fits the role you want to fill. Avoid constructing situations that don't get you useful information - as not only are they a waste of energy, but they can set up a false expectation on behalf of the candidate - after all - they are evaluating you, too.