One thing no one else seems to mention when discussing the issue of noise in an open office environment is:
Management chose to put you into a sub-optimal situation. They are therefore choosing less productivity. Therefore the first thing to do is to stop worrying about the fact that you could do more under better circumstances. When circumstances aren't going to change, sometimes it is best just to accept that there will always be noise and it is always going to be disruptive to some extent.
Management knows you won't be as productive and they choose to save money by putting in open offices. Management knows the sales team is loud and they chose to put them next to the accounting staff or developers who need to concentrate. So if management has decided the productivity trade-off is worth the cost savings, there is nothing you can do to convince them.
So stop worrying about it and, next time you get a new job, weed out the places with open offices from consideration. Once they can't hire new people, these stupid open office plan offices will start to go away. I expect the trend will take 5-10 years to reverse though because the alternatives are expensive so the hit to productivity will have to be really large. And many people won't realize the problem until they work in one and even then, well, working in an open office is better than being unemployed.
Once you stop fighting it, it becomes more possible to ignore the noise. It is your irritation with the noise that is affecting you more than the actual noise much of the time. Once you learn to accept what can't be changed, you are less irritated, and thus less distracted.
For instance, if I happen to notice the air conditioning, I am usually distracted for less than a second and then back to work (unless it sounded like it blew up!). That is because I don't attribute any malice to the air conditioner and it doesn't make me mad that it is running (I like it working actually).
You will notice in these types of questions that people are most upset about what they consider to be inappropriate noise like having a conversation that is not work related (or not related to their work) or someone who they think is deliberating trying to annoy them by, say, typing too loud. I've noticed that the less the person likes the person making the noise, the more distracting it seems to be. That is because it is your response to the noise not the noise itself that is the the real distraction.
Next you need to practice getting back into the groove after being distracted and letting yourself get distracted less and less. The only way to do this is to practice it and gradually, you will find yourself less and less distracted. So the thing is every time you get distracted, you say mentally to yourself, "Oh a noise, back to work" and then turn back to your task. Soon you will get faster at noticing you are distracted and returning to work. The less time you are distracted for, the easier it is to get back into the groove of where you were.
Meditation techniques will help you develop concentration in the face of distractions. When you first start to meditate, you are distracted almost continuously. But as you practice noticing the distraction and deliberately returning to the meditation, you find that you are distracted less and less and that the time to return to task take less and less time. Consider your concentrated work as the meditation and just keep returning to task quietly, without emotional baggage.
You can also consider having a pen and paper next to you and noting down what you are doing at the time of the distraction if it looks to be a major one. A quick reminder will help you get back on track.