This time they would like me to be onsite for 20 hours per week. Is this an unorthodox request?
It isn't. Clients often request or require that contractors work on site at the location of their choosing. I recently worked on site for a client for 13 months. This is pretty standard stuff.
Nevertheless, I understand your other concerns. Have you expressed these to your agency? Are you an employee of the agency? If so, they can require that you honor the clients wishes regarding your work location. The alternative is that you quit your job at the agency. If you are not an employee of the agency and are an independent contractor then the agency can't require you to work at any specific location, but they can choose to no longer use you as a contractor as a result of your refusal to honor the clients wishes.