I think it's deeply sad that even one innocent remark about the looks of some women said in a rather civilized way and in a guys-only setting can also be considered sexual harassment nowadays in some parts of the world. Thank god I live in a country which still is a bit more normal in these matters.
To counter an argument found in another answer, just because you think a women is hot/talented does not mean you think she isn't at good her work (vice versa if you say a woman is good at her work, doesn't automatically mean you think she isn't hot/talented).
So summarized, I don't think you said something wrong, rude, sexist or impolite. However, unfortunately I don't make the rules at your workplace. So it's probably still better to heed your manager's warning.
I also would make a mental note that some guy in your team is definitely not to be trusted if he ratstells on you out for a tiny little thing like this. So definitely don't confide in him when you want complain a bit about your manager, the company or a different colleague.