You are overthinking
When I worked in an office I had 2 pair of pants, and 4 shirts for Monday - Thursday. And 1 set for Friday.
Every day I wore mostly the same things. Monday was this set, Tuesday was this set, Wednesday was this set, and so on. There's no reason you can ware the same thing every day so long as it's clean and presentable. Other then that it's no ones business.
You mention "being undressed" by someone. Well you were outside of work. Remember to keep your social interactions and work interactions separate. Workplace romances rarely end well. Assuming it was a mutual flirting thing. If it was unwanted, then I suggest the good ole beer over the head (pour not smash).
As for perception, as long as your clean, hygienic, and meeting your companies guidelines on dress code, then no one at work should say anything. It's also possible that the person in question just really was trying to pay a complement. Even though I wore the same cloths every week, I still got compliments. It's a good, informal way to start a conversation.
Don't worry about "angering" some one, or "gaining favor" with someone based on how you dress. You just ware what you want to ware (in side the dress code of course) and ignore anything else.
There is only one exception to the guidelines above. If your "client facing". You should take care to alter your rotation so that a client doesn't see you in the same cloths all the time. For example I wore the same thing every Tuesday. If I had to talk to a client though, I would swap out Tuesday's set for some other days. Because if a client only ever saw me on Tuesday, they might think that I never change cloths, and I didn't want to present a "look" that made me seem "inflexible". So when dealing with clients there may/will be additional considerations, but when dealing with coworkers "screw it" and do what you want (in side the dress code).