I used to be in similar boat where I wouldn't even consider jobs that required as much as regular travel - it just wasn't worth the effort and I had plenty of remote offers (although I was a contractor and running small outsourcing business). As a result I was upfront with everyone that I am remote only, and if you want me on site that will cost you through the nose - 2-3x rate was the norm and they can take it, leave it, or reconsider the much cheaper remote option. Most thanked me for the honesty and moved on to someone else because our needs did not match, some converted to embracing the remote working and a few did pay exactly what I was asking for in order to travel.
Any advice on how to approach this, since it kind of decides the flow
of the interview early on?
Exactly, but not sure how to phase it without sounding rude. Yet I
don't wanna exclude the option completely just in case they do go
On one hand you want to set your own terms (that you are remote only) and hold firmly to them, but on the other you also want to keep those people on the hook in a hope that they will pay whatever is the extraordinary amount you are seeking for relocation. This won't work because when you draw a line far away from the other person's expectations then almost no one is willing to cross it, and in that case it is best for everyone to quickly recognize it and move on.
The situation where someone looking for 50k line employee, after interviewing you, will offer you double or triple that for the same position simply doesn't exist, so by filtering those people out you are not missing out.
If you decide to take this approach, you must understand and appreciate that it's a numbers game. If you are that good and worth it then eventually someone will come along and pay the price you are asking, but until then (if that day will ever come, as there are almost always people with lesser demands) you will have to live with a high rejection rate. No matter how you phrase it if you are multiples of the budget they have in mind then you won't be getting it without some extraordinary circumstances.
PS. There is nothing rude in demanding a high amount of money, same as there is nothing rude in offering a low sum. There can be many rude things about salary negotiations, but honestly stating your expectations (for either side) is definitely not going to come through as rude.