Here is an aspect to consider - 

Only one person showed, he was late, he was unprepared, the other person didn't show, and had to be reminded, a week later, that you were supposed to talk.

Now, that might not be indicative of a company in trouble, but, clearly, we can safely say their recruiting and talent acquisition process is dysfunctional.

What is the purpose of the recruiting process?  To identify and bring on board the best talent possible.

If the process for identifying the right people for the company is a complete crap-show of a mess, what are the odds that they are employing the best people and the best fits for the company?  This company is going to be filled with employees gathered through this mess of a process.  Chances are they will have missed or turned-off some of the best candidates out there.  Chances are they will have hired people who are either not good workers or possibly not even technically competent.  You can't take a process that clearly doesn't work and expect it to get results that a working, competent process would.  This is like Demming's red-ball/white-ball experiment.  The people working at the company are likely to be a random reflection of the applicant pool, with some very, very minimal screening out of the most obvious non-fits.

Since this is the process for selecting the employee pool with which you will be working with and interacting, it's difficult to imagine that it not effecting the company, overall, unless this position and these specific people assigned are uniquely new or inexperienced at the task.

How do you deal with this?  Hard to say. If they are interested, find out what the next steps are (good luck with that).  If they kind of stammer, don't know, or seem to be making it up on the spot, then maybe you should just walk away.  If it's just the same two people saying they'll talk some more or make a decision, request some kind of brief interaction with someone higher up the food chain.  A director, VP, or executive level person. A lot of companies do this anyway, so that would be ideal.

When you get in that room, after the usual back and forth, that person will ask you your impressions and if you have questions.  Be very frank and raise the issue exactly as you did here - you like the concept, location, the job seems appealing, but then detail what happened in the interview process and tell him you are unsure if this was just a bad instance, or a reflection on the organization.  His answer will either reassure you or leave you still doubting.  Maybe he'll take offense and decline to hire, but I don't think you'll be worse off than just hoping for the best.