Culture is an incredibly powerful thing.
As a first step I would do a reality check, and make sure that your situation isn't outside what is legal. I don't want you to start some kind of battle with your employer - that pretty much never ends up well - just make sure they're fulfilling their responsibilities as far as this is concerned.
Beyond that, like I said, culture is a very powerful thing, and when there's a great fit between the culture you enjoy working in, and the one in which you actually work, it does amazing things for your job satisfaction.
I've worked in similar places, and have had similar challenges. In any gathering of humans under any organizational structure, you're going to find that how people have a lot to do with the social norms within that group. In many cases, if this becomes a long-term problem for you, you may need to think about changing your role within the organization (to one that is more to your liking), or searching for a company where you can do similarly fulfilling work, but has a culture that is more in line with your personal beliefs about like, work, outside obligations, etc.
Unlike what many believe, not all companies are the same, and cultures within organizations are as diverse as the people who work there. There's way more choice than many people commonly believe in the way work can feel, and how the people around you will naturally approach it when working within various work cultures.