I do software development but my software development is from the Devops side of the house. As such, I find your interviewer's comment vastly amusing as I am required to know enough, starting from my systems engineering background, about the Agile process as implemented by the software engineering teams to support it effectively. Add to that that, as a systems engineer, I am a bash/python scripting demon and I am not that far from saying that your interviewer does not know what he is talking about.

Having said that, you will have to make a tremendous effort to develop that software engineering capability. Specialization is not necessarily destiny. Specialization is like a door. The door can open into a dead end closet, or it could open a whole new set of doors, depending which door you choose to open.