I'm leaving my current position at a small startup to work elsewhere. While here I've been shocked by the incompetency of my boss, the CTO. The company is very small, < 10 people and everyone works in the same office. The CEO is a nice guy, but doesn't know much about tech, so doesn't realise the CTO's inadequacies. Now that I'm leaving I want to tell the CEO that I think the CTO is incompetent and damaging the company, and that someone else in the department would be better at his job, but I really don't know if there's a political way of doing that?
You need to be very careful flinging around words like "incompetent". It's a very strong accusation. Even if you believe that someone else would be better at the job, that does not make the person "incompetent", and describing him like that is going to put people on the defensive. If the CTO was truly incompetent then other people would have noticed. If you call the CTO "incompetent" and it reaches then ears of the CTO they will be angry, and their view of you will be changed forever. If you tell the CEO they have appointed an "incompetent" person you are also attacking their judgement.
If you decide to pass on your views, then I strongly recommend toning down the accusation. Focus on their actions, not them, and describe them in some other way than "incompetent". If you talk to the CEO, talk about specific decisions that the CTO made that you believe were bad for the company. make very, very sure you can back up your arguments, and be prepared for the possibility that the CTO knew of factors that you didn't in making that decision.
Even if you have now left that company, that doesn't mean you will never encounter the people who worked there again. Even if you don't work with them again, it's more than possible that someone who wants to hire you in the future will know them and ask their opinion of you. Accusations like you are describing won't make them give a positive report.
Be careful about telling people what they don't want to hear. Martyrdom doesn't help anybody, least of all you as the martyr.
The CTO may not be good for anything but if the CTO is good for a reference, he's good for something - which is about time as far as you are concerned. However, if I were to think poorly of the CTO. I wouldn't bother with getting a reference from him as a matter of personal preference.
If the CEO is soliciting your opinion of the CTO and the CEO is explicitly looking for your unvarnished opinion of the CTO, don't state that the CTO is incompetent - That's a judgement call that you may not be entitled to make. Simply state key instances where the CTO made decisions that were suboptimal, and let the CEO draw his own conclusion. Of course, this means that you'll have to get your references from people other than the CTO.
C-suite folks in a startup are often there for VERY different reasons than the C-suite folks in a mature, publicly-traded company.
Have you considered that perhaps the CEO and the CTO were both "thought leaders" in the original concept discussions, and are partners at a much deeper level than is apparent to you? The CTO may be CTO only because he owns 25% of the company, is aware that he's not ideal in that role, and will transition out of it as soon as the climate is right.
It's possible that the CTO is "the VC's oversight guy" who's there at the behest of the board of directors. In fact, he could even be the childhood best buddy of the CEO who has no discernable value to the organization whatsoever, but is there for other reasons.
It's also possible that the CTO is entirely acutely aware of everything you'd call "incompetent," but has constraints that you're not aware of. Not to be discounted as well is the hypothesis that he's a competent guy in his core, but doesn't even realize that he's leaving money on the table, due to his inexperience in that role. i.e. he doesn't know that his knowledge base is compromised.
If it were me in your shoes, I would sit down with both the CEO and the CTO together, and do a bit of a casual exit interview/knowledge transfer session, where you outline a few organizational/structural/technical suggestions that "you feel would be valuable but hadn't found a good opening to introduce previously." More on the lines of "here are some possible 'wins' for the organization given its current technical debt and position" rather than "this is where your CTO bozo is screwing up."
Then shake hands all around, wish everyone well, and go forth!
I suggest you have a better understanding of the role the CTO plays within the company along with why this person was hired in the first place. If you have any technical recommendations, you should have already presented them to the company (and consider yourself to blame as well if you haven't.), but leave your evaluation of the CTO out of it.
A good CTO has to have a balance of: technical, business and persuasive/communication skills. Having someone excel technically is good, but not at the expense of the other skills. Maybe this person lacks technical expertise, but what good is having a better understanding if you can't get those in charge to follow your suggestions? Why focus on the latest and greatest technology when the company has bigger problems?
Companies need technical people in charge to improve the business and not spend all their money on the latest and greatest toys. If your criteria for this position isn't in line with the company, you're going to be viewed as someone who doesn't know what they're talking about and not the CTO.
I'm a programmer and try very hard to make sure I don't misjudge others just because they're not very technical. Can't say I'm always successful.
I think you should tell the CEO. It will make you feel better, and since you are already going to a new job, your need for a reference from this one is unlikely to be important in the future: all you are likely to need from them is confirmation of the dates you worked, and that's assuming the startup still exists next time you switch jobs.
However, do not tell your CEO your conclusions. It's easy for the CEO to dismiss them and believe you're not qualified to draw them. Instead, share some observations. Not of results (we lost the XYZ contract) but of behaviours (A never reviewed the XYZ contract, and when B said we hadn't taken disaster recovery into account, A just said don't worry about it) then let your CEO draw the conclusions.
It may help to have a small written list of these behaviours to guide you in the meeting, which is likely to be intense at times, but that might result in your handing it over, which would be a bad thing. Instead, practice the details of the 2 or 3 incidents you want to describe until you are confident with them and won't get flustered. If you need notes, they should be super terse (XYZ DR) and not embarrass you should someone physically hand them to the CTO later, or call the CTO into the meeting and hand the notes over on the spot.
But let me end with a question. What do you think the CEO should do? Getting rid of the CTO may not be an option. You were technical enough to know the CTO was wrong, but were not able to do anything about it. Should the CEO bring in someone who can report around the CTO and highlight these issues? Should the CEO learn a little more about some aspect of the company? Should a particular person be promoted to a level where they can override the CTO's bad choices? If there's nothing the CEO can do about this "incompetent" CTO, requesting a meeting to report all this will just upset the CEO for no benefit. You're leaving, so your reasons for wanting the meeting are to "get things off your chest" and possibly to "make life better for the friends I'm leaving behind" - but if there isn't a way to improve their life as a result of your report, why report it?