First of all, it depends a bit on how the hint was given: If it was worded like "Business casual is sufficient", it might be just a nice hint by the interviewer telling you that you don't have to be too concerned about dressing not formal enough. In this case it may be fine to come in full business attire but then you shouldn't be surprised if your counterparts look somewhat underdressed compared to you.
If it was more like "Please dress in business casual", this might express that they expect to see you in a certain type of attire. In this case, if you appear blatantly overdressed, then yes, it could be seen as a negative point, since you were obviously not able to dress appropriately and/or maybe do not pay attention to orders given to you.
However, all these types of attire have some bandwidth of acceptable clothing, so you have some margin to play with. That means, don't take the lower end of business casual, but go as formal as you can without overdressing.