So... you're on the border between "favor" and "manager job duties". Giving you feedback about your work is one of your manager's duties - but it's not his only duty. Giving you a personal or professional reference is a favor. So - first - come in early like he's asking.
Feedback vs. Getting a Reference
They certainly are different. But if you want a GOOD reference, it's good plan to get the feedback first. And meaningful feedback can't be discussed in 5 minutes. The manager may even have wanted time to collect his thoughts privately, as he's clearly pretty busy with other stuff, and may not have had you or your performance top of mind when you asked. Sounds like you have a good first step going - see where you stand. If your boss has a bunch of negative feedback, it's probably not the right time to ask if he'd be a good reference for you. If you want his reference, work on the feedback.
There's usually a few types of references:
- Simple affirmation that you worked at a given company, in a given role, for a given length of time - can provided by HR, or company bureaucracy, and is usually provided AFTER you've passed the hurdles of interviewing & demonstrating skills for the new job. I'd say wait on this one until you are close to getting an offer, and/or look up what the standard process at your company may be.
- Real, content filled, professional reference - often the new position will dictate what they want (a manager, a peer, a more senior co-worker, etc.) - usually such a reference does not HAVE to be your manager. It's usually a good idea to line up a few people, and it's not so common for one of them to be your current boss - for the awkward reasons you mention. It's toughest at the start of a career where your work history is so short you don't have many/any former bosses. At that point it's fair, when interviewing, to say "I believe I can count on my boss for a reference, but I don't want to obtain such a reference until your offer is firm. I'd be glad to provide my current boss' contact info when you give me an offer that is contingent on it." - which gives you plenty of time to iron out any performance issues that come from the meeting you have lined up with your boss.
But... shouldn't this meeting be during work hours?
Both work hours and whether or not this meeting fits within them is highly variable. In many knowledge working places (engineering teams, academia, research, etc) - the time schedule for each individual has a high degree of divergence. Some companies have core hours, others don't. In a lot of high pressure offices, the boss is often working several more hours a day than individual contributors, or may be time-shifting their own schedule to accomodate the needs of the business (for example, working with remote teams in a different time zone). As such - he may be asking you to put some flex into your schedule, as individual contributor schedules are often a LOT more flexible.
Isn't this his job? Yes. In most companies, giving reasonable feedback IS a manager's responsibility. That said, most managers have a responsibility list a mile long, and never enough hours in the day to get it done... and also, a high degree of variability in how MUCH time to spend on any given thing. One company may insist that people development is the most important thing while another may say "screw the people - get good products made at all cost..." - or something even less nice ("help onboard this new, cheaper site so we can save money", or "help plan the layoff... at which point, people development is really low on the list..."). He may be in a situation where it can't be part of the mainstream company hours, simply because the meeting he's been booked for take precedence over what you want.
It could also be that he's got thoughts, but wants enough time and privacy to actually focus and talk to you uninterrupted. Could be bad, could be good. Either way, for big feedback, often a manager will want to be able to close the door and have a private conversation out of earshot of the rest of the team and uninterrupted. If his day is as busy and interrupt driven as it sounds, before normal hours may be the only time this is possible... regardless of whether it's a high priority.
Or he could be lame, terrible at delegating, and not prioritizing the right things. Or a person who just isn't great at this part of his job. Hard to tell.
A way to figure it out is to ask... but in a fairly kind way - "wow, you look super-busy! What is it that takes so much of your time? Any thing I can do to help?" - goes down a lot better than "I really don't want to come in an hour early to get your feedback, can we do this during my regular hours?". Figure out a day that you could get in an hour early and do it, or if it's nigh-impossible to get in an hour early, then offer to stay an hour late, or do it during his lunch time (and offer to get the guy lunch - you don't have to buy it, just offering to procure it will get you good points).