My personal experience
When I started looking for my first job out of college, a recruiter from a recruiting company contacted me and asked me for my resume, along with my references. Note that he didn't actually have any specific jobs for me in mind yet, he just wanted my information, which I assumed he was going to use to look around for a suitable job match.
I foolishly gave my list of references to him, which was a huge mistake. He ended up contacting one of my references weeks later. I'm not exactly sure why, because this recruiter never got back to me personally about having found a possible job opening. I only found out that the recruiter had contacted my reference because my reference brought it up to me himself.
It was extremely embarrassing. My reference had spent his time giving me a glowing recommendation to this recruiter, who had not contacted me since I gave him my resume and references, and who hadn't contacted me after speaking with my reference either. I have no idea if he was trying to vet me for a specific job, or just vetting me in general. Either way, the point is, my reference's time was being wasted for nothing.
Guard your reference list
I would suggest that you only hand out your list of references to companies who are actually interviewing you directly, instead of passing them off to a 3rd-party. You don't know and have no control over how that 3rd-party is going to use that list.
You can find more answers to that effect at Should references in a resume be upfront, or upon request?.