Get some more people involved in the regular breaks.
I know that sounds like the exact opposite of what you want, but when there are 4 or 5 people who regularly get coffee (or whatever) together, then it'll be a lot easier for you to say "Sorry, I can't make it today" without your coworker pressuring you to come - she'll have other people to spend time with, so everyone wins.