It depends if you care or not. I'm fluent in the language here and hear nasty stuff all the time but couldn't care less usually unless they're talking about my wife or kids or positioning themselves to attempt a mugging. There is an advantage to people not knowing, you can get some interesting insights and information. It's much better to know who dislikes you than not.

Do be aware that people may even know you understand and still say stuff thinking they can get away with it. Which quite possibly could be the case here. It's not uncommon for people to know a bit of the language of the next door country.

I realize that informing them first time would have been best, but it would have been very awkward

That would have been the best time. Or HR or your manager. But you can still do that any time, only now you don't have an immediate grievance, so it may come across as petty.

It depends if you care or not. I'm fluent in the language here and hear nasty stuff all the time but couldn't care less usually unless they're talking about my wife or kids or positioning themselves to attempt a mugging. There is an advantage to people not knowing, you can get some interesting insights and information. It's much better to know who dislikes you than not.

Do be aware that people may even know you understand and still say stuff thinking they can get away with it. Which quite possibly could be the case here. It's not uncommon for people to know a bit of the language of the next door country especially the bad language.

I realize that informing them first time would have been best, but it would have been very awkward

That would have been the best time. Or HR or your manager. But you can still do that any time, only now you don't have an immediate grievance, so it may come across as petty.