It's slightly annoying, but then again I'm not losing sleep over it.
What is the best way to deal with this situation?
Since you aren't losing any sleep over it, and it's only slightly annoying, your best way to deal with it is probably just to laugh and ignore it.
But you might want to do a quick self-check here.
Since you admit that you "took this job because it's the only one", you may be giving off some unintended vibes to your coworkers.
Are you unhappy that this was the only job you could get, wishing you were better off, and unintentionally projecting this to your co-workers? I've seen this happen with more than one co-worker. Once it starts, it seems to be difficult to stop.
Sometimes we secretly want others to share in our unhappiness. That's not a good thing to be feeling, and not a good thing professionally. Others may start to view you as a miserable person, and may stop wanting to work with you.
I'm not saying any of this applies to you at all. But it's worth doing a bit of introspection and determining if you need to do something about it or not.
If not, great - just ignore the slight annoyance.