How would I go about trying to find out about the company and what it does, and how the industry works?
On the job - you can do this by looking just a few steps beyond the information you're being told when you are asked to do work. Start asking the people you work with "why" - why do we need the software to do this? Why is the data structured this way, not that. Then you are giving something (better ideas) while you learn more.
Other options on the job are to network with people who have responsibility for making your software fit the business needs. That's usually:
- Your manager
- The Product Manager
- Program Managers (sometimes)
- Other managers
- Sales people
- Tech support
Never turn down the nugget where someone in engineering actually has a broader thought, as people with this understanding who can relate it to engineering ideas are just incredibly useful to know.
At the company - look for broader company communications:
- Read the newsletter, ask questions if there's a way to do so.
- Go to briefings that are public, but not necessarily in your domain
- Take the training for the sales people - they often have slick training, but it tells you what the company does and why from an insider perspective. Also - how sales are incentivized tells you a lot about what the company thinks of as it's profit sources.
- Read articles about the business from the outside.
- Poke around the internal intranet site.
Lunch with people you get along with from diverse points of the organization can be incredibly useful, but lunch with random people who don't know you is likely to be much less so. To get a really useful understanding, you have to have someone who can explain in a way that makes sense to you - this is much likely to happen once you know someone. Also realize that in a public lunch setting, some folks may NOT want to talk about work ... it's their lunch break!
My favorite - is to make friends, not look for a network. I got some of the coolest and most interesting insights into the businesses I worked in by becoming pals with people in other areas of the company - anything from lawyers, to cleaning people, to security (both infosec and physical), to sales, to just about anything. Everyone has a story, and a mission in their work, and I really like learning it. It starts as a random conversation about absolutely nothing. It goes into a joke, or something that connects us and next thing you know, we find common ground. Eventually learn why they work here, what they do, and why it matters... people love talking about that stuff - once they know you.
Am I doing something wrong, or is the prospect of "start with XXX job and work your way up" just a myth?
I believe you are mixing concepts here. Very few people go from database engineering to business-related management. The first few promotions are usually within your subject domain - so the "business" at hand that you need to know to work your way up is the business of how to do your job efficiently and the business of how your work ties in with the work of other people. That can be remarkably similar from engineering department to engineering department. I've worked in software engineering in the defense industry, federal economics and private service based companies - and it's far more similar from job to job than it is different, even though the way the company generated revenue, viewed it's investments and structured it's strategy was wildly different from place to place.
When you are nearing the top of the technical food chain then you DO need to know how to get your engineering organization how to move in line with the business itself. By this time you will also have mentors and sponsors who are helping you get what you need to move forward.