You could take them out or bring food in, whichever works best for your setup.
But whatever you choose to do, let them know ahead of time, and ask if they have any food preferences or allergies. That way you won't provide a pepperoni pizza to someone who only eats Kosher, or a sub sandwich to someone who is gluten intolerant.
That also lets them know that there will be food, and whether they will be going out to eat or eating in. Letting them know can ease some of their stress. And how they handle themselves while eating, hopefully while more relaxed, can help you assess them better as well.
The company should pay for the interviewee's meal, of course. And if you go out to a place and the interviewee orders the most expensive item and treats the wait staff poorly, you have learned something valuable, much more valuable than the cost of one meal.