As an employee, you have a legal contract with your employer. This contract contains a job description in which relevant duties are outlined. Beyond the duties outlined in the contract, there may be duties that an employer reasonable expects you to carry out. If a duty is neither in the contract, nor reasonably understood to be part of one's employment, then you are under no obligation to fulfill that duty. You may choose to, or you may choose not to.
It is important to point this out, because you may feel obligated to comply, even when no such obligation exists. This is because besides a legal contract, you also have an emotional contract with your employer. It is good to remember that the emotional contract is not on paper, and may thus suddenly be worthless. Your employer may variously choose to abide by it, or not. For example, even the most devoted employee may be replaced with someone cheaper, more senior, or otherwise beneficial to the employer.
That said, choosing not to fulfill a duty that you are not legally obligated to fulfill may have repercussions. Given that you're already fired, these repercussions are minimal: pressure to comply (i.e. conflict), and possibly a bad reference. Though the latter may be further minimized by the fact that you're being let go. Perhaps you should not expect a good reference even if you do perform the inventory duty.
Without knowing all the details, it is impossible to say whether the request to do the inventory is reasonably understood to part of your job. And if not, only you can decide whether refusing is worth bearing possible repercussions. If you do want to refuse, the best way to frame the refusal has already been pointed out: argue that it would make more sense for the next person to carry out the task. Any believable reason will suffice.