There are a lot of very similar questions on Workplace SE, but I'll answer simply because your situation is just a little bit complicated by the fact that you don't start for 2 weeks.
In any case, I have the sneaking suspicion that you actually know what the "right thing" to do is, but are hoping someone will point out some other solution out of this moral dilemma.
So here's the #1 thing to always remember:
Always keep your own priorities and goals in mind.
This new job is not the perfect fit for you. There is no question that you must accept the second job offer. You are definitely screwing your boss over by leaving, but always keep in mind that any business will promote their interests first and foremost - even if it detrimental to you!
However, leaving your job is not the real ethical pickle here - it's whether you should keep quiet for the next 2 weeks or not. So let's put our cards on the table:
The High Road
Since you know you're not going to stick around, you should quit right away.
You know you're hurting their business by taking the money with no intention of sticking around, so be polite, and to the point. Simply tell your boss the truth:
I'm very sorry to put you in this situation, and I really appreciate the opportunity you gave me here, but I have just received a job offer from a company that is much closer to home, and is simply a better long term opportunity for me. I sincerely apologize for this, but I am going to accept their offer.
Announcing that you're going to be gone in 2 weeks is almost certainly going to get you fired on the spot. If your finances are strained then the thing to do would be to wait until after your first paycheck before you bounce.
Note: This is by far the lowest thing to do, but hey, you've got to eat.
If you leave before you have the money you may never get it (This will depend on the country you're in, etc, but I can think of a number of reasons right away, such as leaving without giving notice, for example). The only thing worse than being jobless for 2 weeks is working but not getting paid for those 2 weeks.
At any rate, whenever you decide to quit I would recommend leaving immediately. Two week's notice won't really be necessary since you've barely just started there.
Good luck, and remember: always look out for number one - yourself.