You shouldn't have to "connect" on a website if there are other means of communication. I've sent message back and forth on LinkedIn with people without having to be connected personally or professionally. If I had to guess, I would guess that recruiters want to connect with you on LinkedIn to see who else they can reach out to and try to place people in positions or sell positions to them.
You may be losing out on potential opportunities, but you also don't have to think about it in this way. By choosing to work with a recruiter, you are placing your job search in their hands. In a way, you are giving up control of your life. That's a choice you can make, and the experience can vary from person to person and recruiter to recruiter. By limiting your network to just people you actually know and trust, that gives you the advantage of having a stronger network of people who could connect you to wonderful opportunities in the future. It's a two-way street, and a good sign of the value you hold for yourself and that you give to your contacts.
One way I would handle these type of things is to keep your messages brief and concise but also maintain a professional tone. Figure out what the recruiters want to hear from you. You are under no obligation to work with them, and I don't think you want them to take over if you are having doubts about how you want to conduct your job search. It can lead to good opportunities so it might not hurt to bite and see what develops. If the recruiter is someone reputable and someone you can trust, then you can decide whether to add that recruiter to your professional network.