Night shifts vary. A relative who worked as a computer operator (mounting tapes, keeping printers happy) says it was a firing on the spot offense to be found asleep and they were supposed to report each other. A friend who drives an ambulance routinely sleeps through night shifts; they wake up if there is a call. Your manual is unlikely to tell you the truth of the culture at your employer.
Regardless of the workplace culture, if the supervisor's nap is causing you an issue - you're feeling unsafe, you don't know what to do, some things are not getting done because there's too much work for one person - raise those issues with the supervisor without mentioning the sleep part. For example, "sorry to disturb you, but [problem.]" or "Last night, while you were unavailable, [problem.] If that happens again tonight, do you want to be informed right away?"
If you are not having problems, and nobody has told you naps are forbidden, then consider the possibility that the culture of this firm allows night shift people to nap when things are slow. Relax a little and enjoy your job. Getting stuff done and being good at what you do are genuinely pleasant experiences. So is the peace and quiet of a still building overnight, or a glimpse of the moon or the stars through an office window, or being part of a team that all support each other in being good rather than sniping about who is better or more deserving than the other.
To discover the culture, you can ask your coworkers. I'm sure you discussed the nap with the supervisor, since you tell us she explained to you her reasoning. Did she ask you to keep it secret? If so, that suggests it is not approved of. If she did not, that might be because it is fine. You could also try asking a peer - not specifically about the supervisor but in general. "Do people nap on night shifts? Is that allowed?" is a relatively neutral question that could get you useful information. If it's really irking you that the supervisor is napping, and you're sure it's not allowed (it truly is allowed in many on-call situations where the workload varies) then when you go to the supervisor, ask, don't tell. "Are we allowed to nap on the night shift?" for example. And if you get a "no", then "what should I do if I see someone napping?" is a natural followon. And then you will know what your manager wants.