Is drinking on an interview like this a bad idea?
No. If your interviewer ordered an alcoholic drink that's a fairly clear signal that it's acceptable at this company to enjoy an alcoholic beverage during lunch. In this case he even told you as much.
While (light!) alcohol consumption is perfectly acceptable in all the cultures and companies I've experienced, this isn't universally true. Because of that, during an interview you probably never want to take the initiative to order an alcoholic drink because you normally don't know enough about the company's culture. Alcohol is also typically more expensive but as long as you're not ordering champagne that shouldn't be an issue.
But if your business contact ordered an alcoholic drink himself, and especially if they come right out and say that it's fine to order beer or wine, then you should take them at their word. This goes for all meetings with clients or interviewers and it's typically the person paying (interviewer) or with the highest status (client) that would decide on this.
Or could it be the whole lunch and offer of a drink was to show me what would be expected of me if I was hired and to see if I would fit in?
It could be, but it's bloody unlikely. I have heard of stranger hiring rituals and other inane tests so you can never rule this out, but I can say that no well-run company will do or permit such a practice.
If there is indeed a culture of drinking that is likely to play a role in your job, good interviewers or hiring managers will tell you that that's the case and ask for your thoughts on that. They won't craft secret tests and silently judge you on your reaction.
It's always fine to decline an offer of alcohol, in any context. Most interviewers are used to candidates passing on alcohol, typically because they don't want to dull their concentration. Only the most boorish and uncultured folk will look down on you for choosing not to drink alcohol.
There are some obvious exceptions to this rule but you'll know if you're in such a situation. There is still a strong after-work drinking culture in Japan for instance. I've known a brewing conglomerates that had a "beer lady" instead of a "coffee lady" doing rounds in the office every afternoon. The company kept that practice in place until well into the 21st century but I wouldn't call them a good role model.
Alison Green, whom I frequently quote in my answers here actually disagrees with me on this to a certain extent in her article about accepting a drink during an interview. The main reason she gives is that "this is not the time to lower your inhibitions or mellow out with a drink. You want to be at your absolute best, and you don’t want to impact your judgment at all." I do agree with her on that and certainly if you know that you're likely to be affected negatively in any way by even minor alcohol consumption then you should avoid drinking alcohol in situations where you need to be at your best.
But you're asking about how it will affect you as a candidate. Alison also weighs in on that and quotes a 2012 study from the Journal of Consumer Psychology called "The imbibing idiot bias: Consuming alcohol can be hazardous to your (perceived) intelligence". Given that the article felt the need to create the colourful term "imbibing idiot bias" which has less than a 1000 search engine hits, that the Journal seems largely unknown, that the study is, as far as I can tell, not properly peer-reviewed, and the language they use in their abstract, I'm quite doubtful about the value of their conclusions.