I've worked in the IT profession for 12 years and participated in maybe ~40 phone interviews as the interviewer and interviewee. Never have I seen or heard of the interviewer's boss silently listening in to the call and not introducing themselves. This happened for the first time I'm aware recently. In this case, the interviewer made an offhanded joke referring to his boss in the room.
I see the red flags in this, as the subordinates may feel pressure to hold back on any perceived negatives. Their boss also called me personally to ask me to apply, as I had a previous working relationship with him elsewhere. I don't want to ignore red flags, but what would be neutral or positive reasons for this?
Edit: I took @mutt's (obvious in hindsight) advice and asked the boss directly. Turns out I made an assumption that the subordinates were in a conference room. They were at their desks which are located next to the boss. While I question not being in a conference room for guaranteed privacy, it wasn't on the boss and more on the subordinates in that case.