You're lucky you're only at the introductory stage of the interview when you had this interaction. DON'T WALK, RUN. I can't make it clearer than that.
So, you went to a convention for women and were referred despite being a man. The point is, regardless of your gender, you were impressive enough that the recruiters at the conference thought you would be a good fit for the position, and that's really the only thing that ought to matter, that you're a good developer and impressed their onsite reps at this conference. Your gender should not matter, where and how you were recruited should not matter, your outside work helping whatever organizations should not matter. The only thing that matters is that you have skill.
And yet, based on this interaction, it seems those things do matter. This interviewer would rather not hire you because you're a man and not a woman. This interviewer would rather not hire you because you were a man recruited through a women's forum. While it is illegal (in most locales, unsure of yours) to hire or not based on things like gender, it seems this company engages in such practices, or at least gives the impression that they take those things into account, by making snide remarks about how you're not X or Y. While actually fighting this in a court of law is probably costly and not worth the trouble, at the very least you have an understanding of the culture of this company, although probably not in the way you might think.
You can't make a wide generalization about the company as a whole based on this interaction; the recruiters you spoke to at the conference seemed to be good, nice people, and perhaps this interviewer was simply a "bad apple". So I wouldn't say that the company is full of man-hating sexists, that's not the point. The point is that the interview is a two-way street; you're interviewing them as much as they're interviewing you. And the representative of the company who they sent to be interviewed by you was the type of person who would make snide remarks about your gender. At the absolute least, this shows that the company as a whole has a blatant disregard for these types of remarks; whether they are encouraged or not, or supported or not, or celebrated or not, you can't say, but at the very least they are not abhorred and reprimanded at this company, as they should be. And that's enough for me to say to you:
DON'T WALK, RUN.
As for notifying the recruiter, you can try; it couldn't hurt. However, I wouldn't try too hard; if the company culture is such that their first impression (you know what they say about first impressions) to you is someone of this nature, it's likely the problems are deep-rooted and probably come from management. Your concerns are likely to fall on deaf ears; this is simply the culture at this company. To be honest, if it was me, I wouldn't even do them the favor of explaining to them why they have lost a perfectly capable developer candidate, I'd just let them keep on doing what they're doing and maybe they'll figure it out on their own someday. Right now is not the correct juncture, and you are not the correct person, to be raising heck in this particular situation.