Yeah, this is a really bad idea waiting to happen. In addition to all the stuff that goes with you managing people you had no say in hiring and didn't even know had positions for (figuring out who will do what work, knowing what skillsets the different people have so that you can place them on the right projects, having no idea whether they'll fit into the culture of your existing group), I will bet a dollar that your boss will also expect these people to contribute right away and do so as full-fledged members of your team. Like, if you have 4 devs right now, he'll expect you to produce 50% more output immediately.
Needless to say, this isn't how things work (see: the Mythical Man-Month). And I don't think you're going to be able to explain this to your boss, because if your boss was the kind of person who listened to their subordinates he would have looped you in on the interview process or, you know, the fact that an interview process existed. So... good luck, I guess?
More seriously, this seems like the same advice folks give every time there's a problem, but this is one of these "polish up the resume and look for a better gig" situations. Get out before your reputation is tarnished by this guy.