We have mostly open seating too. I share your pain.
You didn't mention visual distraction, but that can be a big contributor too. If you have any leeway at all over this, getting any sort of visual barrier can help immensely. You might not realize how much people walking by in your peripheral vision can distract you until you cut it off.
The biggest problem for most people, in my experience, is the noise. If your objection to wearing headphones is that you're substituting the distraction of music for the distraction of the office, try noise-canceling headphones instead. Some of my coworkers use them and they say it doesn't hinder them the way wearing earplugs would.
If the problem with headphones is physical, not what's coming through them, then see how the people sitting nearby feel about white-noise generation. It doesn't do much for the loud conversations, but it raises the floor some on what's enough noise to be distracting in the first place.
These are only partial mitigations, not real solutions. I suspect the only real solution involves not doing open seating, but that's out of scope.