Its in the clients best interest to deal with you direct, as then they get to avoid paying the recruiting agent their fee, which can be substantial (say 10% of your first annual salary, on top of your annual salary), which is why the company is trying to engage you directly.
If you agreed with the recruiting agent that they would represent you, potentially that is an enforceable contract which you could be violating if you bypass the agent in any way - it gets especially more dubious if the company tries to bypass the agent as well...
Typically, a company will not have a contract with a recruiting agent - its done on "first presentation" basis, and its a purely honorary agreement rather than anything else. The recruiter presents you to the company, but often they remove any identifiable details from the CV they present so the company has to go through the agent. Often however its possible to identify the candidate through all the other information on the CV, especially if you are good at trawling LinkedIn.
Here in the UK, if a recruiting agent presents a candidate with no solicitation from the company, there is nothing the recruiting agent can do about that - its not breach of contract, as no contract exists. The worst the recruiting agent can do to the company is bad mouth them.
But if you explicitly agreed to be represented, as I said before there is a potential breach of contract between you and the agent, so there is that to consider.
So the question is, do you value the position enough to deal with the company directly and potentially be sued for breach of contract? Do you consider the chances of being sued high enough to force the company to deal with the agent and potentially drop you as a candidate?
Only you can answer those questions :)