That's a bit of a loaded question, of course. But plenty of companies have a policy of not providing references at all beyond confirming employment. So, you really can't expect to predict with any real accuracy what your employers will say about you. You don't really know whether your "negative" employer would give you a negative reference or not, and the interviewer knows this. So, he's probably looking not just for honesty, but insight and a positive outlook.
It seems entirely honest to me to say that you can't speak for any of your employers (because you can't, really), but that you had both positive and negative experiences with all of your employers. Then perhaps say that you're quite happy to talk about experiences at your old jobs, and ask what he would like to know. Then, if you are specifically asked to talk about negative stuff, try to explain it objectively, as miscommunication, divergent points of view, etc. Then highlight what you learned from the stuff. That will convey a positive outlook, someone who is open-minded, coachable, and learns from his mistakes. Take responsibility for your own side of it, don't criticize the other. For example, if you are asked what you didn't like about working at wherever, say something like "I didn't like the way I handled X. I found myself between rock Y and hard place Z, and I did Q about it. Knowing what I know now, I would have done R." This avoids criticizing your employer for putting you between the rock and the hard place, and turns it into a learning experience for you. This is the way that a person with a positive outlook addresses problems that come up, rather than focusing on the mistakes that others made to put you in a difficult position. After all, even if you were completely mismanaged, and your manager was really weak, there are always ways that you could have improved your handling of the situation. This is equally true when you were well managed. If you have a strong focus on how you can improve yourself, you should come out fine answering this or any other tough question you get in an interview.