4

I worked for a couple of years on a research project where I took the initiative to learn about python and SQL. I installed MySQL on my personal Mac OS laptop, I designed a database on Workbench, and I started populating it and querying it using Python.

The database only had 3 tables, and there were only about a thousand rows, and a handful of columns on each table. I used it to compare my model's output with experimental observations taken from a biology lab, and also to store my model's outputs.

I can perform all the standard queries a SQL fundamentals course (say on PluralSight) would teach, and I know the basics of the MySQLdb python package to connect to a database. I am also a Math PhD student, which helps with my understanding of relational calculus - though I suspect employers don't care about this much.

My question is: Does this relate at all to what a data scientist or developer does when they query, maintain, design or update databases at a company big enough to have physical on-site servers with databases filled with millions of entries?

I want to enter the Data Science field, and I am concerned about the SQL/database requirements. Do employers just want you to be able to run some queries, or do you have to really know how a server is installed, what software comes with it, how to connect to it, etc? If they do, does knowing python and some packages help, or are there other tools I need to learn?

P.S. I've tried to find the right place to ask this question, if you think this is not it, I would appreciate an alternative forum!

Improve this question
2
  • Hi Mike, I'd try Stack Overflow. Someone flagged this "off topic" - and I agree. But it is a good question and I am certain by looking at stack overflow you can perhaps find what you ask for.
    – Stian
    Commented Jan 29, 2019 at 14:38
  • This would get closed on Stack Overflow @StianYttervik. It's a much better fit here.
    – Martin Tournoij
    Commented Jan 30, 2019 at 22:19

3 Answers 3

Reset to default
3

As someone in the data/coding industry, I say yes... but it depends on two important factors:

1. What type of job you're looking for

It's not easy to become a Data Scientist just by knowing Python and SQL, but those are definitely two of the most important skills you'll need. However, most data scientists start off as an analyst (business intelligence analyst, data analyst, etc) OR as a developer (software dev, hard-core python development, etc).

If you're looking for a job "querying SQL databases" (as you referred to it), that sounds more like a Database administrator (DBA) or a web developer who needs to pull data into a website. Those jobs probably won't care that you know Python. The DBA job will require a nice amount of IT background, and the web developer job obviously needs HTML/CSS & a whole lot more probably. If either of those two jobs are what you're looking for, then my answer to you is no, you don't really have what you need just with SQL and Python.

However, Data scientists, data analysts, & data engineers spend a lot more time manipulating and transporting the data. So yes, data manipulation is usually SQL, and Python is a bit skill to have too, so you're certainly on the right track if that's the job of job you're hoping for.

The other important part is:

2. If you're willing to take an entry level position.

Like I mentioned, most data scientists start off in a lower-level data position. If you have SQL and Python skills, and can somehow prove that you actually have those skills (such as with projects that you can show or talk about), then it's likely you can find a company who will take the risk of hiring you. A math background definitely helps in that case - I'd recommend making sure to state that clearly in your resume.

In my experience, I've found the business intelligence (BI) has the lowest entry barrier for people new to the data industry. It's mainly because BI professionals don't have to know SQL - I know many excellent BI folks who focus on the functional side (building charts, helping people define KPIs, etc - the business side of BI but they wouldn't build anything in SQL unless their life depended on it. Happens most often in the consulting world.) The other reason for the lower barrier in BI is that many of the tools are meant for people who aren't super technical, so they can use drag-and-drop interfaces or "Excel-like" tools, rather than dealing with heavy data modeling or pure data querying functionality.

But the point is that you have to start somewhere. Ideally go straight for the gold and see if you can land a job doing really cool data stuff, but my personal opinion is: It's more likely that you'll be able to start off as a technical data analyst working in a business intelligence tool. (Which is definitely a good job, don't get me wrong! It's just fairly different than data science.)

Bottom Line

Yes, you're in a good place to move forward with your career goals. There is A LOT more to learn to become a data scientist, but most of it has to be learned on the job by dealing with very large data and just by getting real-world experience. That's just how it goes.

The only other thing I'd suggest learning is a BI tool like Domo (great for getting more experience with MySQL) or Treasure Data (great for getting more experience with Python). Tableau, PowerBI, and others are good too if you want to enter the BI field, but those tools more focused on the visual side & have less SQL/manipulation functionality.

You're definitely in a good spot to get started, though!

Improve this answer
4
  • Thank you very much for your detailed answer. My fear about data analysis is that many employers want you to merely know excel and have some presentation skills. That probably means they don't want or need much processing on their data, just somebody with a keen eye enough to see and represent some basic relationships between columns. That's why I'm trying to learn enough about developing to get a more hands-on role that will bring me closer to data scientists and engineers. Is it unreasonable to fear that Data Analyst roles can be traps with little room for mentorship and growth?
    – Mike
    Commented Jan 25, 2019 at 12:55
  • Yes, you are right to be concerned about that. However, it's the easiest place to get in the door. Try to find a company that promotes internal growth AND has a team of data scientists or very technical data engineers.
    – giraffe306
    Commented Jan 25, 2019 at 16:59
  • Also, to avoid an "excel" job, it might be worth gaining familiarity with data science or R, such as linear regression models or clustering. Even being able to say "Hey I've done XY course on it" will show that you want more than just Excel/basic data stuff.
    – giraffe306
    Commented Jan 25, 2019 at 17:23
  • This strikes at one of the reasons for my question: How much of a (back-end/front-end) developer do I have to be to go after entry-level/junior Data Science positions? I have worked through the Python Data Science Handbook, I've implemented a paper on image pattern recognition using clustering with Python's scikit-learn library, and I have a fair bit of Large Matrix systems background (I've lectured a college Numerical Analysis course for about a year). Although my background is fairly academic, I feel like it's the data/developer part of me that is lacking, not the scientist/statistician.
    – Mike
    Commented Jan 25, 2019 at 19:40
6

Does this relate at all to what a data scientist or developer does...

YES...but...

The huge difference is scale. MySQL is MySQL, weather it's on your laptop or a multi-node cluster with fully sharded tables. The language is the same and works, pretty much the same.

In practical terms, you probably haven't been exposed to the specific query techniques required for large datasets. This is what will differentiate you between someone who can write a query and someone who can write a good query. The difference can be many, many, many orders of magnitude in performance.

Here's the problem I often face. Using data is a developer's task. It's entirely software based with hardware deployed to facilitate the software. Queries are programs and programs require programmers.

But, you do need to know how the database interacts with physical storage, though you don't really need to know how that physical storage is setup. This is where many shops get into trouble. The assume the "DBA" should do both.

My expectation is that a data programmer can consult and guide the DBA's to deploy hardware and server resources to support the expected app requirements. I would expect them to know how to setup MySQL (or SQL Server, or Oracle, or whatever) but not actually do it in most cases.

Improve this answer
4

A little.

You have some experience -- at least you know what a RDBMS is, and know basic SQL. That's more knowledge than many developers ever get, and it may even be enough to do some junior developer tasks.

More senior work, specially when working with large systems, requires more knowledge/experience. It's not just about writing simple queries, but it's also creating the right table(s) for the application, setting up replication structures, optimizing queries, minimizing deadlocks, etc.

Now, that doesn't mean you need to know all of this to land a job. People learn a lot in their jobs as well. In the 1990s, I took a job and had absolutely no database knowledge. I didn't know more than "there is such a thing as a database". Two years later, they fired their DBA for incompetence, and asked me to take over "because I seem to have more database knowledge than anyone else in the company".

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .