One thing I would add here (from a US perspective)...
Another answer stated:
In small companies, people usually wear multiple hats to keep the company running so it's unfortunate that your friend is abusing you like this.
I would state it differently - in a small company like this, it's almost inevitable that the lines between different job roles will blur a bit, and everyone will end up helping with tasks that are nominally 'outside scope.' This has been the case when I've worked in start-up sized companies.
A good employer will state this up front, and there should be language in your contract to that effect as well. This would give you an opportunity to more clearly define what you are and are not willing to do, and to understand what they might expect you to do. It does not sound like you had this conversation when you were hired, which doesn't reflect well on the employer. Also, outdoor manual labor is rather different from 'light office work,' although it's not entirely surprising given that you're in a landscaping company.
To answer your actual question:
Is there anything I can do to stop this madness?
You can talk to your employer, since you stated they're a friend, and express your concerns. If that doesn't help you probably want to start looking for a new job. There might be legal solutions in the middle but that depends a lot on location/jurisdiction and likely isn't a route you want to pursue in light of the friendship.
I'm mostly adding this information so that you can have a better idea of what to expect if you start looking for new jobs. Small companies can be great opportunities and great communities, but they don't tend to have the rigidly defined personnel roles of larger organizations. This is something to consider when considering what future jobs to seek, and something to consider when talking with prospective employers.