Yes. Though its not as simple as the way you put it.
It is your job to figure out how to get the work done. Saying it is not possible looks badly on you and your team but that responsibility falls directly on your shoulders.
The external bosses tell me to do more or that I am putting my job on
This may or may not be true. They may have the power to have you removed or they may not. However just saying its not my fault pushes the blame up the chain to your boss who probably does have the power. It is important that you appear to be attempting to meet their needs. It probably appears to them that you are just ignoring their requests which appears very disrespectful. That is your fault and yes that you can and should be held responsible for.
how can I ensure that my performance review is not tainted by these
warnings over which I have no control?
Your job as a manager is to not just manage your reports but also your customers and your managers. Your customers expect that you will get their requests done. First you should sit down with your team and try to brain storm how you can accomplish what they need done. Being a manager does not mean that you have all the answers, that is why you have a team. Let them help solve the problem.
Figure out a few of the best options, and if they require resources take those to your manager and explain the options and get direction from them. If you can solve the problem with out resources then try and make sure your customers know you are working to solve their issues.
Your biggest problem right now is the optics of the situation. As far as your customers know you tell them you do not have the time and resources to fill their requests, then head off to the golf course(figuratively). You have to change those optics. You need to make sure your customers know you are taking their issues seriously and are working to find a resolution to them, even if that resolution is beyond your ability right now. Once they know you are trying then you have room to breathe and your manager is more likely to try to help you out where he can.