The other answers are partially correct - it can take time not only to schedule your on-site interview, but any other candidates. The process requires that HR ensure all of the interviewers have time in their schedule and haven't scheduled vacation.
The missing piece in the other answers is ensuring you've communicated your "advance notice" requirements to the recruiter. I've never experienced a recruiter telling me the day prior to the trip that I would need to fly. I imagine it could happen, but with people leading complex lives, and often needing to schedule a "vacation day", last minute notice is a losing strategy. Typically advance notice times in my experience are in the range of 1-2 weeks. In-town interviews less than 2 hours tend to have less advance notice, and phone screens are usually "pick one of the following times" a few days in advance.
Perhaps the most important piece of the interview process is to keep in mind that your interviewers will have to not only make the time to interview you, but also make up any time on their regular tasks. This is why pressuring a recruiter is a bad idea, and thanking your interviewers for taking the time to interview you is a good idea.