Yes.
If you have had experience working with someone and are asked to describe that experience - there is no ethnical question to whether it's OK for you to describe your experience. Good or bad - the candidate's actions and behavior towards others in a workplace is something they should be responsible for whether or not they are working with someone who they've asked to be a reference.
This is done pretty frequently - the disclosed references that a person gives in a job application can be assumed to be favorable. No candidate will ask a person who hates them to give a reference. :) Many managers and companies will look for a personal connection in their own network and follow up out of band, figuring that if a person they trust has had good experience with a candidate, then it's a fairly trustworthy hire.
If a candidate doesn't assume that this process can and will happen, I don't know what to tell them - a person's behavior is what builds a professional reputation and over time, this reputation will affect their ability to be hired.
The Small Print on This Situation
When asked for what I've heard called a "back door reference", there are a couple of things to consider when giving information:
If this is a friend or close coworker - avoid discriminatory or particularly personal details. Even the stuff you may consider to be very good can be discriminatory, for example "I'm so happy that he and his lovely wife are having a baby, they are going to be great parents!" could lead to an employer deciding not to hire a soon-to-be new parent, for fear that family commitments will distract from work. That is discrimination in the US, but it's extremely hard to prove and prosecute, particularly when a back-door reference points it out. Keep it safe - talk about the professional qualities of the person and their actions in work or a work-like setting.
It is OK to be a reference when the connection does not come through a paid position - but it should be doing something that can show some work-like skills. For example - "he's great, we meet up at the park and play frisbee" is not so useful. "We worked in a soup kitchen together, he was always on time, friendly with the folks he worked with and responsible in doing his work" - it really doesn't matter if this a restaraunt job or a software position, those are great qualities in anyone.
Avoid hearsay. If you have witnessed poor behavior, say so. But "there were these rumors about him and a female college getting inappropriate in a bathroom" - you didn't see it, you don't know how the rumor started, stay out of it.
It's best to know and adhere to workplace rules about references - I'll admit, this is one I don't always follow - I have worked in places that instructed employees NEVER to give personal references for any other employee, the work had had hotline that would verify employment history only and that was it. This is a pretty gruesome rule for a new worker, as it makes it hard to collect references. But... it's also a real risk with your employer that if they find out, you could be in trouble - so at least know the rules before you break them.
Be clear about your experience level with the person - "we worked together once", "we were on the same team for 2 years" - are very different.