Are there any concerns I should have about turning down the promotion?
For instance, what damage, if any, will that do to my career?
It's okay to turn down a promotion, particularly when it takes you away from the type of work you really want to do. Been there, done that.
There will be a few issues, that you'll need to be aware of and to handle.
Expect the company to be concerned, and to ask why you are declining. They feel you are capable of this higher-level job, and feel you will help the company by accepting it. Turning it down may mean that they have to go elsewhere to fill the role. You want to try to reassure the company that you still like the company, like the career path you are on, and like your current job.
You may end up working for the person the company ends up hiring. It might be awkward when this person inevitably learns that you were the first choice. Try to stay positive, and not wonder (or worse, express publicly) how you would handle situations if you had accepted the promotion.
Consider what kind of career path you will have within this company when you turn down the promotion. In some cases, you will be at the top of your path already, with no room to grow without leaving.
Overall, it's okay. You may want to have a long career discussion with your boss when you decline. In my case, that's exactly what I did. It worked out fine for me.