This is your first contact. You should definitely conform to their standards at this point in time. Some cultures would consider bringing a present at the first visit extremely strange indeed. Don't start off by making a strange impression.
It may be appropriate to bring some food (bagels, cake, ...) for a little breakfast with your colleagues, e.g., on your first Friday in the office. Ask your colleagues what is appropriate - especially since you come from a different culture, people will happily help you navigate local customs.
For your particular question, it might be helpful if you could indicate the office's location in the world.
Later on, by all means follow your own cultural customs (and explain them, so people understand what you are doing). For now, tread softly.