I am adding a new answer because I don't believe any of the other answers have fully addressed a key component of the your particularly situation: that you may be suffering from *Imposter Syndrome*. 

This is evidenced by the statement:

> I have the role of the tech lead / architect of the team, though I have less experience both in general and in this project than one of the freelancers

If true, the Imposter Syndrome is likely the reason why you had an emotional reaction to your coworker's demands to get involved with the perf testing, and likely holds the clue for how you can easily deal with this kind of behavior.

The key to this is the recognition that the sufferer of imposter syndrome almost always oversimplifies a complex situation to their own detriment. 

In your case it may be an assumption at some level that you are unequipped to take technical leadership of the team because you are less experienced or skilled than other team members.

If this is the case then you should recognise that to provide good leadership, soft skills like communication, political awareness, empathy, and organisational capability far outweigh hard skills, and that you would not be in the position you are in if you did not posess these to some degree. 

It is therefore perfectly normal and reasonable (although it may not feel like it) for you to be 
1. responsible for the team's technical direction, and 
2. taking such decisions that you believe are correct for the team