The other answers are all good, and I'm going to repeat a lot of what's in them for emphasis, but I think a couple of other things need to be said.
First, if all else fails and you are faced with an ultimate directive to include the code without attribution, you should get real legal advice from a real lawyer, not just some guesswork from laypeople on SE. Don't assume that just having a paper trail will absolve you from all liability, and don't take SE posts (including this one!) as legal advice.
Second, you should point out that code can be released and used under multiple licenses, and that the rightsholders might be willing to let you include it without displaying notices for a fee, depending on who they are. It might be worth looking into.
Third, if, after all the gentler arguments and paper-trail establishment has been done, and if you're really sure that the notices must be included, your boss still insists that you publish an app containing the copyrighted code without the legally required copyright notices, you can simply refuse to publish the app.
You absolutely should make darn certain that you're right about the legal situation before you push this. It sounds pretty cut-and-dry, but I'm not there and I don't know every tiny detail; it might not be.
You absolutely should attempt to find a conciliatory way to convince your boss either to include the requisite copyright notices or to give you the time to re-implement the necessary functionality in-house. Wildcard's suggestion, in particular, is excellent. Pointing out the legal liability and potentially devastating consequences of including the code without attribution is also a good way to go—you can do this in such a way as to show that you're trying to look out for the good of the company and of your boss, in particular, that your objections come out of supportiveness, not out of argumentativeness.
You absolutely should involve any department at your company that exists to handle matters of this sort—legal or ethical departments, etc. If you have such a resource, then that's their job, and they have authority you don't both to prevent a crime and to protect you. If this department exists and doesn't agree with you, then you should probably listen to what they have to say—they're experts in this sort of thing, and you're not, so they're probably right. Still get your paper trail (see next point), but they're probably right.
You absolutely should also establish a paper trail to protect yourself from retaliation. Get everything you can in hard copy, and keep it off site. Make sure it's very explicit in this documentation that what is being asked of you is illegal, and make your boss state very explicitly that he wants you to do it in spite of the fact that it's illegal if, in fact, that is what he wants.
But, after all this, if you still find yourself in the same situation, then you have the option to refuse. The consequences may be extremely severe; it might destroy your relationship with your boss or get you fired. All the other answers here seem to be focused on preserving your job and limiting your personal liability; these are great things to do, and it's completely understandable that you would choose to do them. You are, according to what you have said, being asked to break the law and to violate what are clearly your ethics, though, and it needs to be said that you have the choice not to.
Please note that I'm not advocating either way; this is your life, and your job, and it has to be up to you.