Answer honestly, if its a startup with only 3 people tell them that.
If you have 30 employees tell them that too.
Don't use any tricks or white lies. If you hire the candidate they will find out very soon how large your company is anyway.
Yes, its disappointing if someone turns down your offer but it will be much worse if you hire someone and they immediately quit, or are disgruntled you lied and work poorly or make trouble.

This is one of those cases its best just to plainly and honestly state the truth.