I worked in a company and found some nice LinkedIn contacts. But there is a contact who started to bully me after a while. I think he can't stand me at all. Should I remove him to my LinkedIn contacts? Can this affect my possibilities to get a new job now as I'm unemployed?
Probably 90% of people won't even notice if you remove someone as a Linked-In contact. I surely wouldn't and I keep a pretty active profile.
Unless the company and this person are using Linked-In much, much, much more than average the person probably won't notice.
If they do use it that actively, you can block them completely.
What makes you think that keeping him as a contact is going to improve your chances with any prospective employer? A contact is valuable to you if you think that he could be helpful to you down the line. Do you think he will be helpful to you down the line? If he was bullying you, what are the chances he'll help you get a job? When was the last time that someone who bullied you helped you get a job? If you don't think he's going to be helpful, you know what you want to do to him as a contact.
Should I remove him to my LinkedIn contacts?
Yes, of course. Don't keep contacts that you do not have a good relationship with. LinkedIn is a professional social network. Think of it this way... if you wouldn't want this person serving as a reference for you or vice versa, you don't have the intention of this ever being the case, and you don't intend to work with or do business with this person in the future, there is no reason to maintain the contact on LinkedIn.
Can this affect my possibilities to get a new job now as I'm unemployed?
No, why would your future employer care who you are connected with on LinkedIn? Unless that specific person is someone you want to work for (and they shouldn't be) then there isn't an issue.