If at all possible put a halt to proceedings and take your desk back. It would have been easier for you to say no initially but you can still back out.
If you haven't transferred items between desks yet then it is very straightforward. Simply say that you have changed your mind and that the swap is no longer happening. Something along the lines of:
I'm sorry but it won't be possible to swap. I like my current desk and realised that I need it for easy access to documents/people/resources x, y and z.
Your colleague won't be happy about this but it is tough luck. Remember that they were asking for something ridiculous (they wanted to have a nice desk entirely at your expense). Offer some sympathy and apologies but stop short of actually helping them out. If they talk about your business trips then offer to let them use the desk when you are away.
If you have already swapped desks then the situation is trickier. The best way forwards would be to speak to the office/desk manager (or your line manager if they can dominate the desk manager) and say you have tried out a new seating arrangement but it hasn't worked for you and you want to reverse the process.
All of the above is in lieu of managerial input. If an appropriate manager gets involved then you can make your case but shouldn't go against their decision (notify your line manager if this happens and they might sort it).
You've made a huge mistake. Desk space is the crown jewels of office politics. People can get extremely territorial and, depending on your circumstances and attitude of management, it may be difficult or impossible to gain access to any of the comfortable desks.
There are basically three ways to win at the game of desks:
- Get a manager to allocate you a good desk (often this is favouritism under the guise of reorganisation).
- Steal/scavange/repurpose a desk when someone leaves the organisation
- Cling to what you have. Accept no compromises unless they protect your position.
Remember that everybody else is also playing the game of desks. Most people will have a moderately comfortable arrangement that they have become accustomed to and the thing they fear most is losing it. This motivates people to believe in the sanctity of possession.
"I thought if I wouldn't have gone her way, probably people will think I'm a rude person."
Maybe but probably not. Turn the question around: if you ask for you desk back and she tells you "no" will people think that she is rude?
They might do but ultimately it doesn't matter: she has the desk and the new truth will quickly become an accepted norm. They will only remember it negatively if they think of her in general as being a rude person.