I'm contracting with an awesome company that gives me challenging work and is very supportive. Occasionally I can't figure out why my code isn't working. Classic programming, but I feel bad billing the time spent troubleshooting. (Sometimes I'm stuck for hours.) Do you charge for this kind of "work"?
Are you debugging your code during those hours you are "stuck", or are you walking your dogs? That time is not billable if you are walking your dogs. If, on the other hand, you are debugging the code you have written, that is a core part of the programming process. So is determining what you think you are supposed to do prior to writing the very first line of code. The time spent writing lines upon lines of deliverable code should represent a small fraction of ones time while programming.
Except for toy problems, nobody writes code that is perfect from the very start. Once you get past that initial testing stage, there likely remain ten to fifty bugs for every thousand lines of code you have written. Getting that defect rate below one bug per thousand lines of code is one of the key reasons why concepts such as agile and test-driven development have been developed.
Developing software consists of three steps: Design, coding, fixing problems. No software works after the "coding" step. Since you are paid to develop working software (I assume), the customer needs to pay for all three steps and you are fine to bill for all three steps.
If you think your skills finding and fixing problems are too low, then there are other sites to ask questions about this.
If your agreement with the customer is that you are paid by the hour, then yes. Your customer buys you "as a package" with your things where you are fast (since you already know how not to get stuck) and the thing where you are slow (e.g. get stuck when you don't know the solution already).
See if the quality of the code is right - and not too high. If you get stuck on things which could have been easy, but you took the hard way, consider if that was needed. (e.g. not using an n log n algorithm instead of an n^2 algorithm is not always the end of the world.)
If you or your customer feel very uneasy about this, consider changing to a more well defined way of defining work packages and not being paid by the hour.