Telling your boss would do two things. First, it would say that you think your boss is not able to discover or notice things that you have -- many people would be offended at being patronized with this. Second, it sets up a "hey, let's talk about how maybe this company is going to fire you" career conversation with your boss that inverts the usual relationship and could be profoundly embarrassing even if your assumption (they want to replace your boss) is true, and worse than that if it isn't.
If they do intend to replace your boss, knowing will not enable some dramatic action that somehow prevents it. It would enable a job search, but that search could very well be underway without your knowing about it, and compensation packages when bosses are let go are sized to cover living expenses during a job search.
The upside to telling this person is that you may be seen as caring and supportive, and your boss may remember you fondly when they get a new job somewhere else. The downsides are that you may in fact upset and embarrass them, and they might not be leaving the company. The ad could be a mistake, it could be for a new person to join the team at that level, it could be to replace a different boss you don't know, or many other things. You've jumped to a conclusion. Watch and learn if you were right. That will help you if you find yourself in this situation again.