Ok so I have left my email logged in at work. I already understand that whatever I do they can see. That's not a problem. What has happened twice now was management going through the accounts. Looking for things. Anything personal they find they don't like they bring it to my attention. Not just emails that were opened and visable on the work computer. But things buried deep in my account they never would of seen. Isn't this illegal? The computer is theirs. But the account is mine. I feel this is the same as a bag of purse. I can leave it on company property but they cant go through it just because it's there. They would need a reason. A probable cause. Or even a warrant if things got really hairy. Am I wrong?
Ethically, you are right. Your personal business is your personal business, as long as it doesn't interfere with your ability to do your job. In no case is it OK for management to use your personal, unrelated to the job information to bully you.
In terms of law, it depends on the jurisdiction and you'd better consult a lawyer.
Legal or not, you should reconsider if you want to work for a company that needs to follow such practices. If you need the job but don't want them to access your personal accounts, better don't use them from work computers and on the company network.
You could check your email from your smartphone, using your own data plan. In any case, you should set up 2-factor authentication for your account and always make sure you've logged out of it when you don't need access.
Data loss prevention is something which most of the companies struggle with. Hence, the fact that your email is left open on their computer could be probable cause enough to check for data loss and look into it. (That may or may not have been their real motive but they can surely argue that.)
Just extending your purse analogy, if they felt that they are losing office stationary, then they can argue that they looked into a bag which was left there overnight. (Legal or not is a different issue and like others mentioned check with a lawyer)
You did not specify what other kind of personal stuff they go through other than email so difficult to comment on that.
Even if you are allowed to check personal emails from office machines, use your personal smart phone for it. I do not think you need to set any extra security or log off after you are done. (At least not for preventing your employer to read your emails!). Your phone is password protected and it will always be with you !