I would want to better understand the target audience of the voice assistants, what the other test sets are, and how representative the test sets are of the target audience. There are several factors that affect voice assistants and responding to the human voice - age, gender, natively spoken language, and accent (influenced by national origin), among possible others. The testing sets used would need to be appropriate for the target audience of the assistants to ensure that the voice of the user can be recognized and responded to.
I can see a risk in that the high-level managers aren't representative of the target audience of the voice assistants. However, using their voices and queries for design, development, training, and testing of the assistant can lead to better results for them at the expense of other users. If these are the same high-level managers who are responsible for making go/no-go decisions, they could misinterpret the results of highly focused testing or even demonstration to them as readiness for a broad audience. In essence, this focused test set can put up some smoke-and-mirrors and ultimately harm product quality and organizational reputation in the future.
I would also want to better understand how this "VIP Test Set" is used and maintained. Do the high-level managers know their queries are being used this way? What kinds of queries are they making? Did they consent?
It's not clear to me what these voice assistants are designed to help with, but they could be making personal queries. Those queries may not be things that they would want their employees to have access to. Often, in systems that contain personal information, access to real data is highly restricted and is only supposed to be used by people who have a true need.
I don't think that there's enough to say that the behavior is unethical, but there are definitely potential issues around the managers' privacy and confidentiality, potential harm, and bias and discrimination inherent in the developed product.