Is this worth mentioning to my supervisor?
Nope.
Everything they have done so far has been respectful, and thus management does not need to be involved. If they do anything further that makes you uncomfortable then this question would need to be revisited based on what happened.
Is there anything I should do to protect myself?
There are several things that you and anyone can do to put up barriers that can reduce the risk of interactions like the one you described turning from a proper respectful one into an incident:
Visibility
Maximize public interactions while minimizing private interactions. When others are around it provides accountability and opportunities to get other people's perspective. If they or anyone else says something to you that makes you question if it was inappropriate and someone else heard it you can ask them for a second opinion.
If you see a chance to avoid a situation where it is just you and one other employee do it. This includes working late with just one other person to carpooling to a customer meeting with just one other person. Even if everything is good, it does not stop a toxic employee from gossiping if you and the other person happen to be of compatible ages and opposite genders.
Lastly, any time you have to be in a one-on-one meeting conduct the meeting in a room with a window to the rest of the office area. Having visibility in the office discourages any inappropriate physical interactions, since people at any point can walk on by and see in.
Have Multiple Performance Reviewers
My workplace has a concept of stake holders when it comes to performance reviews. Anyone in a leadership position that has a stake in the results of my work counts as a stake holder and I have the option to ask them to write a performance review for me. If your workplace already has something like this in place use it, if not encourage leadership to look into it.
If you have four people that provided written feedback at the end of the year where three of them give you excellent marks and one gives you strong negative marks, then you can point to the three to show something is wrong. Which in turn leadership can investigate if it is a case of retaliation or something else. Also, in a normal situation instead of having one person singing your praise, you now have four people doing it, which helps when you are trying to get a raise or a promotion.
Have a Mentor
Preferably this mentor is not someone at work, but if the best person happens to be a senior coworker then take it. Having someone that you can ask about day-to-day situations and who can give you sound council and wisdom can be more helpful than a group of people on the internet who do not fully understand nuances of your culture or background.
Also do not limit yourself to one mentor, if there are multiple people in your life that you can count on for good advice take it. You never know when someone will be unavailable or moves away and thus you no longer have easy access to them.
Lastly, if someone starts making you uncomfortable with their romantic advances and you are not sure how to handle it or if it warrants supervisor attention, you have someone that is an independent third party that can give you impartial advice.
Conclusion
Do not go out of your way to avoid one-on-one interactions, nor put up so many barriers that it starts to inhibit your ability to interact with others. However, if there is an opportunity where you have a choice between a more private setting versus a more public setting, choose the public one.
