Respect is probably what this boils down to. It's hard to 'half respect' someone. You either lean one way or another.
This is a difficult situation. The eagerness to socialize outside of work may be perceived as attempting to compensate for under-performance. Team members can see through this, or perceive things this way even if this is not your intention. I recall one guy on our team long ago whose performance lagged for various reasons. The rest of us flat out did not like him for this, and the little things he did outside of work that didn't quite mesh with the typical expected behavior provided more fodder for (implicitly validated) the dislike.
This was an unfortunate situation because I believe he recognized that he didn't fit in and suffered as a result. Any time someone is perceived as an outsider, various cues communicate this to them, and they become aware of this and deep inside, suffer from this.
I do regret to some extent having been part of the group that positioned itself in contrast to him, even though I do not recall doing anything overtly to make him aware of this shared attitude against him. But another part of me still feels that the collective attitude toward him was at least partially justified. He was under-performing and the result was a somewhat natural edging-out of him by the rest of the team, kind of like a pack of healthy animals edging a sick one to the periphery so he becomes more isolated and susceptible to become a predator's lunch.
Had he been an effective performer, it is more likely that his social quirks would have been glazed over, rather than pointed out, and he would have been able to command greater respect and have a more central role on the team.
The situation was not resolved until he left. Leaving (escaping) was probably the only solution in that case, that could make him feel better. I remember on his last day, as he passed through our row of cubicles on his way out the building, nobody even got up to shake his hand. Everyone just grunted a 'Bye' and that was it. Sad. I don't know how he fared at whatever organization he joined after ours, but I wish him well wherever he ended up.
For me, the moral of that story is that it comes down to performance. Some organizations and teams are more forgiving of low performance, some less. You need to gauge whether the climate on your team affords you an opportunity (buys you time) to improve before the perception of your under-performance becomes more solidified.
Where does good performance come from? I can name three sources: enjoyment of what you do, experience (knowledge and training), and a sense of responsibility for the results.
If you feel you are lacking in some of these areas, think of what you could do over the next 6-12 months to address these gaps.
If you like what you do, there is a powerful incentive to improve knowledge and skills, and thus build experience and expertise. If you do not, there is an incentive to reflect on your priorities and perhaps consider changing jobs or occupations. (Note: "kind of" liking what you do could work as well, but one needs to be more intentional and position oneself strategically so that more of the tasks s/he enjoys, has some proclivity for, and is self-motivated to improve in, flow their way.)
A sense of responsibility for the results is more tricky. It is tied to a more general personality trait of responsibility for the results of one's actions. It is linked to training (for instance, ex-athletes often have this in spades), upbringing (e.g. strict/demanding parents that instilled discipline in one from early childhood), and personal beliefs (e.g. in one's duty to do one's best, help others, sacrifice in the name of greater good, etc.). Development of this trait is a long-term process.
I hope some of my anecdotal personal experience and reflection in this answer helps you think more broadly about the causes and possible pathways to solving the issue you are facing. Good luck!
