First, I find this practice incredibly distasteful and unprofessional.
All job applications should be submitted in confidence. As a hiring manager, you (we) don't know if the candidate is leaving a job quietly, or not. And by contacting "backdoor references" or "non-listed references" you could jeopardize their situation even more. This puts the candidate in an unfair position, especially should they (worst case) get terminated from their current position for looking for work.
Now, if you found the references unacceptable or unsuitable, as a hiring manager it is your prerogative and reasonable to ask for more - or why a person/position was not utilized. However, this too, often puts a candidate in a less than ideal position. As this conversation can lead them to "bad mouth" their current employer inadvertently (or what they say can be perceived as such) - which is also an interview "no-no". If you must ask "why" do so, expecting a response which may be less than apolitical about their current employer, and take into account your question with their response - in terms, of tone, reasoning and adequacy. This way you are not penalizing them and negatively affecting their application for a difficult and (depending on your wording) distasteful question.
As far as your question if this practice is allowed, yes it is. Is it ethical, that I believe is a "no".