- Chill work environment, small company
- Only ones in the office
Yes, you'll be more than fine wearing jeans and t-shirt. Very formally dressed up you'll possibly even look a little bit out of place - as you'll see, there is a different atmosphere on a quiet weekend meeting like this, compared to the usual weekdays.
Casual appears a lot more down-to-earth on the weekend, compared to formal, which in your situation, I'd perceive as stiff, if not insecure. Everyone likes it if the tension if off a little bit over the weekend, including (most likely) your manager. It will contribute to a more relaxed atmosphere, which can be helpful in building a positive work relationship.
On a more controversial note, I think you should avoid asking your manager about this. There is no reason to ask. Asking here can be seen as a lack of confidence/independence that you can't even decide for yourself if casual clothing on the weekend is OK. Even if my boss came formally dressed up (maybe, in turn I would find that a little bit weird - "just for me?"), I'm sure my choice would still be seen as reasonable/normal/down-to-earth.