3

I'm a recent college grad who never had any job experience other than my job right out of college at a startup in data analysis. I don't know how to manage a database or operate one which is necessary for doing my job. The people hired me didn't know this and thought everything could be done in excel. They didn't know that I'd end up having problems since none of them is data analysts. I've been managing all the data in disorganized folders and excel spreadsheets which are beginning to be overwhelmed and crash due to the incoming hordes of data. As a result of not knowing SQL and the presence of intellectual property laws which prevents good database learning resources online, I can't teach myself SQL.

I've tried to sort through the data manually and because of the tediousness of the task which is comparable of counting the number of people in a large crowd of people or recalling what day of the week 1965 January 5th was without a calendar to look up. I'm the only guy in the company that can do data analysis, and there are barely any people in the company. It wouldn't be so bad if at least one person could help me. I personally blame the company for not hiring someone else who can do data analysis. Leaving a big burden on me.

Eventually, they're going to have me give them a report of all the data the company has obtained and owing to my lack of database management knowledge, I can only give them current reports.

The problem was exacerbated by the fact that coming out of college I didn't know how to plot their graphs, and plotting graphs in excel causes excel to crash, and redoing all the graphs is too tedious.

I work for family, so I'm wondering how to quit this job if I have familial conflict of interests.

I worked this job because I thought it would boost my résumé and owing to bad performance in college I thought I needed a job.

Now I got an offer to go to graduate school elsewhere, so I no longer need this job to make sure my future career prospects don't go to waste and this job isn't related to my graduate school program and isn't going to help me or hurt me.

So if I quit this job I'll be doing nothing all day because there isn't anything to do in the mean time.

If I quit, the company will need to hire someone else to do this job clean all the data, leak more information, have people sign NDAs/CDAs, and since I've done a half-baked job I've done, they might have to start some of the analysis again, and spend money because of that. Which I'll just let them have all the money in my bank account because I've spent 0% of my salary as living at home. I never really liked this job. I don't have to do it anymore since I got into graduate school. I don't mind refunding them my salary. I haven't spent the money on my earnings.

I'm not quitting because I'm lazy, I'm quitting because very soon they're going to ask me to do something I can't do because of the lack of SQL knowledge. I'm not quitting because of grad-school acceptance itis kind of like senioritis. I'm legally not qualified and I have 1000 folders of excel data that are crashing on me because of the sheer size of big data and can't manage data because I don't know apache-spark/Hadoop/XML>

My family shouldn't have hired me in the first place. They were better of just having some professional with a PhD do it.

I can generate reports of the data as it comes. But going back and fixing data reports and keeping track of data and indexing it is hard to do in excel especially when the rows are so many Excel crashes. I'm not completely incompetent, just half-baked.

This company is so small no one can train me to do it. Everyone else is completely incapable of processing data, and I'm sort of capable but not good enough.

This job sucks so much I've been using a growth mindset to see how long I can tolerate cleaning data using inept methods. But I've finally realized my tolerance limit.

I don't need the money from the job and don't plan on doing a job ever like this in the future.

This job isn't impossible to do. Just really tedious, painful, without any motivation to do. Clicking the same buttons all day. Constant software freezing.

  • 174
    "As a result of not knowing SQL and the pressence of intellectual property laws which prevents good data base learning resources online, I can't teach myself SQL" ... Sorry, but this is tripe. It sounds like all you need it a reasonably basic grasp of MySQL or similar, and there's heaps of free online resources for that. Oh, and good on you for saving your salary, but don't even think about forfeiting it. That money is yours, fair and square. – berry120 Oct 2 at 20:58
  • 25
    Why is the TLDR section longer than the first section? – thursdaysgeek Oct 2 at 21:35
  • 14
    There are plenty of online tutorials that will guide you through the process of setting up a SQL database from scratch. I would recommend you look into MYSQL as its free. You mention excel being difficult to use with over 1000 people, however excel should be able to comfortably store up to several GB of data (I've seen 70GB excel files that had to be processed). If your excel file is smaller than 8GB, you might want to look into upgrading your computer because its likely your RAM that is making it so hard to use. – Shadowzee Oct 3 at 2:06
  • 16
    @Germania sorry, but it sounds like it wasn't the job that required you to do manual labour, but your lack of knowledge. There are lots of ways to convert Excel data to SQL tables, Pandas dataframes, you name it. And, once again sorry to disappoint, but most jobs do involve dealing with the real world, in shape of poorly structured input data, incomplete and/or conflicting user requirements, buggy tools and lack of resources. – IMil Oct 3 at 7:00
  • 11
    @Germania IMHO this was a perfect opportunity to learn a skill that you might probably need later on (at least if you plan on working with data analysis). Almost every employers out there WILL give you task you don't have the skills for, with the expectation that you learn to skills. – user3399 Oct 3 at 7:11

11 Answers 11

94

I am answering this as someone who employs family (my own children, and yes of course I have also employed people who are not my children), and sometimes asks my staff to do things I cannot do myself.

When you are asked to do something you can't do, don't quit, instead say

I can't do that.

Go on to say

I don't know how.

or

I am doing all of this by hand in Excel and that would take 3 months in Excel.

Be honest. This is one of the few employers you can be honest with. They love you for who you are and they want you to work there. So tell them.

I think there is a better way with SQL and stuff and I have looked for free courses but I can't find any and I think if I spent 3 days on a real course that costs [whatever] I could figure out a faster way to do this. But I haven't had that training so I don't know how to do it and the way I'm doing it now just can't handle larger data sets and so I can't, I just can't do what you're asking and I know it's important but it's not possible with the skills I have.

And then maybe they will say "ok, we had better hire someone who knows how to do that. You are leaving soon anyway for grad school." Or maybe they will say "ok, let's just stick with the small datasets for the next few months." Or maybe they will get you trained. Anything might happen. Most of the things that are likely to happen are better than you quitting your job when they don't want you to quit. Right?

Most people who are flailing in a new job and want to quit don't dare trust their boss with the struggles they are facing. I often urge such people to try it - if you're at the point of quitting there is nothing to lose. But in this case, when it's family, there really is nothing to lose. If you tell them you can't handle it and you all agree it's best you walk away, you get to walk away with no big conflict and fight. Or perhaps they find a way for you to be able to handle it. Those are both ok outcomes, right?

  • The problem is all the data is in separate files and analyzing the data makes the data more messy because in analyzing the data I need to create more data and folders. Nothing is all in one place. – user106240 Oct 2 at 22:13
  • 40
    But you can learn how to do that. Admit to them that, right now, you don't know how. But at least you know what you need to learn. Let them decide what to do about your not knowing. – Kate Gregory Oct 2 at 22:23
  • 14
    @Germania Dude, this is the best answer. Just be honest, tell them all the problems, give them some options. Just be honest. – Ernest Friedman-Hill Oct 2 at 23:16
  • Slightly OT, but if the data comes from lots of files in different formats, you should look into Microsoft SSIS. It's specifically designed for cases like that, and the programming environment is entirely visual, so it's a lot easier to learn than typical programming languages. – BlueRaja - Danny Pflughoeft Oct 3 at 17:57
31

As a result of not knowing SQL and the pressence of intellectual property laws which prevents good data base learning resources online, I can't teach myself SQL.

To learn SQL, you have following options:

  1. Go to library and rent any book on SQL written in this century (as the SQL standard very rarelly changes) and catchup on recent changes if needed in MySQL/Postgresql documentation.
  2. Buy a book, even most professional books on SQL do not cost more then 20$
  3. Use one of 236 million results returned when you search for "Free SQL Course".
  4. Get completely free open source MySQL or Postgresql database (I highly recommend later), and read the comprehensive guide & documentation attached to both.
  • Even MS offers a free, trimmed down version of SQL Server, as does Oracle for their server (at least they used to... and I think they still do). If you want to maximize your learning, I would stick with standard ANSI/ISO SQL. If you learn standard SQL, that can pretty much be used on any DBMS (Oracle, SQLServer, mySQL, etc...) – gmiley Oct 3 at 16:59
  • @MarcinRaczkowski "Buy a book (you're working after all), even most professional books on SQL do not cost more then 20$" University textbooks usually cost a lot more than $20. – nick012000 Oct 3 at 17:02
  • 1
    @nick012000 Professional books and university textbooks are not the same thing. Spend literally 2 seconds on Amazon and search for SQL and see there are hundreds of books in the $20 range. Even if you wanted a university textbook for some reason you can find hundreds of those used for less than $10 as long as you don't care that they are out of date, which doesn't really matter much to SQL anyway. You can buy a 15 year old text on SQL for $2 and get everything you need to know to use it today. – DetectivePikachu Oct 3 at 17:15
  • 4
    I personally wouldn't recommend MySQL or PostgreSQL for learning SQL, I would use SQLite. Why? No setup AT ALL needed, no servers, no connection issues, no firewall issues... And the syntax is pretty compatible with MySQL. And, again, it is to learn. To apply at work, one could use SQLite, but it's better to use your suggestions. – Ismael Miguel Oct 3 at 17:15
  • Using XAMPP, or similar all-in-one setup tool, gets MySQL, phpMyAdmin, and other tech installed pretty easily and without much pre-knowledge. I've used it multiple times to easily and quickly set up DB backed websites more complicated than the OP wants. At that point, MySQL is pretty easy to use and the SQL learned there is about 95% compatible with MS SQL and even Oracle. I doesn't even need to be on a server, as a desktop/laptop are more than enough for a single user. A desktop set up as a "server" is also more than enough for XAMPP, and I learned MySQL +10 years ago exclusively online. – computercarguy Oct 3 at 18:06
20

What do I do if I realize I'm incapable of doing my job?

You have two choices:

  • Learn to do your job.
  • Find a new company to work for.

There are plenty of free resources available to learn almost anything, and if you are really interested in learning what your job is requiring you can either pay for resources. An even better option is to ask your company to provide training. This does not mean that employees from your company are the ones to train you ( even though some companies have the resources to do this ). The training can be outsourced to companies dedicated to this. There are many companies that will do this at no cost to you. You must, of course, be genuinely interested in learning otherwise you are wasting everyone's time ( including yourself ).

If you do decide to quit, it needs to be your decision alone. Do not worry about what your family or friends think, they are not the ones who have to do the work. Also, do not worry about the company having to clean up after you. It is their responsibility to be prepared for any employee's departure and if you leave for whatever reason it is not your fault or problem if they have not properly prepared. Start applying to other companies and don't hand in your resignation until after you have signed a written offer with a new company

  • This company is so small, no one knows how to train anyone or wants to train anyone and no one knows how to do data analysis. – user106240 Oct 2 at 20:41
  • 14
    @Germania ... The answer was probably aware you would say this: This does not mean that employees from your company are the ones to train you (even though some companies have the resources to do this). – KingDuken Oct 2 at 22:09
  • 4
    @Germania If the company is too small to know how to do data analysis or to properly pay for it, but they also NEED data analysis in order to function, then this is an unsolvable problem. As others have said, just be as flatly honest with them as you possibly can about your abilities and what you can produce, and let them decide what happens next. – Graham Oct 3 at 13:28
  • 2
    Anybody who hasn't already taught himself SQL as he goes along, shouldn't be in a job that requires any SQL. Sure, it's very hard to master (or so it appears to me, not having mastered it), but it's very easy to achieve an adequate level of mediocrity if you're interested in it at all. Nothing wrong with that, but it's time for a new job. – Ed Plunkett Oct 3 at 14:25
16

No need for learning sql and setting up server environments and all the rest.

Lots of companies get by using MS Access, if you have excel you should already have this because it's part of the Office bundle.

It's not a tool I like, but it's there and pretty easy to use. Tutorials on almost every specific problem you're likely to face should already exist.

  • 39
    Then your problem is not that you cannot do the work, it's that you don't want to. – Kilisi Oct 2 at 21:12
  • 31
    Thats just a cop out.... no offence – Kilisi Oct 2 at 21:15
  • 31
    rprogramming.net/connect-to-ms-access-in-r Literally the first link when you google "using Access in R". Your problem is that you're lazy and unwilling to do even basic research. Which makes me doubt every other problem you claim you have is as bad as you claim. – Gabe Sechan Oct 3 at 6:30
  • 7
    @Germania You keep talking about Hadoop. I can absolutely 100% guarantee that you don't need anything remotely close to the processing power of a hadoop cluster. What you want to do can likely be achieved with 4 lines of ANSI SQL that can easily run in any free RDBMS on the market. If you "hate" writing a simple SQL query to the point you flat out refuse to do it then I have to question how you can possibly expect to have any kind of future as data analyst. – Demonblack Oct 3 at 15:34
  • 5
    Also, it's not possible to integrate dplyr and MS Access" is patently untrue. Not only does dplyr have ODBC connectivity capability, but you could just as easily have a middle-man handle conversion if they weren't directly compatible. Again, this sounds like an inability to approach a problem with constructive solutions. – gravity Oct 3 at 15:55
15

As a result of not knowing SQL and the pressence of intellectual property laws which prevents good data base learning resources online, I can't teach myself SQL.

This is nothing more than an excuse, and a really bad one at that. There's a deluge of free resources online to learn SQL. A simple google search for "Sql tutorial" shows 240 MILLION results. Thousands upon thousands of tutorial sites, youtube videos, blogs, official documentation and quickstart guides from vendors, and free e-books and pdfs. All free. All available for anyone in the world. Did you even bother to search to materials on SQL at all? Or did you just quickly devise this excuse about IP laws and other nonsense just to save yourself from having to make any actual effort whatsoever?

  • 1
    Finally someone said it right. Just the straight up truth. – reg Oct 3 at 15:43
4

I thought i'd give an answer to demonstrate that doing all this in SQL may not be as tricky as you thought:

If you have the data in Excel already, then i'd suggest importing it locally into a SQL database. Looks into the program DataGrip. DataGrip has an import wizard where you can get a CSV, and it will create a SQL table in a database when you import the csv. This is actually easy to do

There's a free version of DataGrip, but i'd say get the company to pay for it

Once you've done this, you can run queries against the tables on your own computer

People make this out to be difficult since they usually have some kind of ETL process to automate the getting-the-data-into-the-database bit, but it can be done on just your computer without too much effort

In the below screenshot, i've made a database, added a table, then I have the option to import data into the table from a file (your csv files)

enter image description here

Here's an example of running a SQL statement:

enter image description here

SELECT * means output all columns of the csv i've uploaded (the table's columns) from the general_payments table

LIMIT means I want to only show x number of rows

  • 11
    Not only it is sorta easy, it is sorta close to spoken English. Your query can be read as "Select anything from the general_payments table but limit it to only 20 lines". If the boss asks for "the list of people who's age is between 20 and 35", you can simply write SELECT * FROM people WHERE age BETWEEN 20 and 35. If the O.P. wanted to learn, the O.P. would have learned by now. I get this feeling that O.P. is just making up excuses. – Ismael Miguel Oct 3 at 11:13
  • 8
    Well the OP doesn't want to do the job, doesn't want to search how to do the job (and shows both in their comments as well as post), and here we are spoonfeeding how to do it. – user3445853 Oct 3 at 13:14
  • Since I've never heard of DataGrip before, I can't UV, but this seems like a very reasonable and easy process to get things started. – computercarguy Oct 3 at 18:17
4

You can absolutely do this job. If SQL is what you need to learn, then learn it. You're working for family, so presumably they will understand if you need to take a month to learn SQL. Of course there are resources available online to learn SQL. That's how I learned.

But you could also use Python for a lot of this stuff. Python has data visualization and manipulation libraries and is way easier to learn than SQL is. Start by Googling pandas, scipy, and matplotlib (or bokeh or seaborn).

It sounds like you just have a defeatist attitude and don't like working. So admit that to yourself, own your feelings and decisions, and quit. But don't come here and say you can't teach yourself SQL because that's just laughably false.

Just research some other ways of doing what you want to do. You're already on StackExchange. Most of the answers are somewhere on this website.

You're in a position most people would be thrilled to be in. You're the expert and you have little chance of being actually held responsible for any mistakes you make. I mean, most people wouldn't keep around an employee with this attitude, but you work for family, so there you are. They don't know the first thing about data analysis, so they won't be able to reprimand you if you make some mistakes at the beginning while you're trying to get a grip on new technology. So you have basically all the time in the world to figure this out, because they have nowhere to go but up, and you're family.

Here's a tip: almost every time someone uses the word "can't," they mean "won't."

3

"As a result of not knowing SQL and the presence of intellectual property laws which prevents good data base learning resources online, I can't teach myself SQL."

If an employee came to me with that phrase, then family or not, they'd be out on their ear. Give me a problem, by all means - but also bring with it some suggestions on how to fix it, and don't make up laws off the top of your head.

There are two issues in that sentence. You don't know SQL, and you don't learn well under your own steam. So take the solution to your boss - you need an in-person course on SQL so that you can do your job more efficiently. Do that. And maybe, if you can't learn on your own, grad school isn't going to be your best way out of this...

  • 1
    THere's nothing wrong with someone, especially a junior, calling out a problem he has no idea how to fix. But this particular one is just complete BS, as has been pointed out many times in these answers. His problem is that he doesn't want to learn how to do anything. – Gabe Sechan Oct 3 at 18:22
2

Try to learn the tools you need to succeed at your job. You mentioned SQL: I remember that I found the sqlzoo online training a good (and free) starting point for learning SQL. Alternatively you could buy a book about relevant topics. Anyway, the strategy should be to use your time there to increase your knowledge: Try to automate tasks, if you have coding questions you can ask them on stackoverflow.

You mentioned that you want to offer the company to refund your salary. I wouldn't recommend this, even if it is a family business. Rather explain to your boss proactively that you still need some skills in order to complete the tasks. Make it clear how you can achieve this and that it will cost some time and training. I.e. come up with solutions to the problems. That can be good for you and the company.

If learning the necessary skills in time is out of scope, you should openly communicate your concerns to your boss. Present the facts, then they can decide. As long as you are honest and do not pretend something that is not true, there is no need to resign. Rather consider it as an opportunity to grow and instead of doing everything manually learn how to use appropriate tools.

Needless to say: If you are not happy at the job and if you don't need the money you can always quit. However, sometimes in the long run it is rewarding to go through tough times without quitting the job, too.

Also congratualtions that you got accepted at graduate school! As I understand your description, you have to terminate your job at the start of graduate school anyway? So there is only a limited amount of time left while you work at that company anyway, so you could use that for your motivation and try to learn as much as possible within that limited time period.

  • it's not just the software. SQL is a software. I need a server physically and figure out how to run SQL on it. Then I need to figure out how to protect the server because data is confidential. Might as well get a cyber security degreee. Stuff has to go online for ease of use... that makes it harder. does SQL run offline? Yes but sooner or later the the pendulum will swing in the other direction and I'll need a cyber sec. major's knowledge on how to encrypt – user106240 Oct 2 at 21:25
  • 9
    @Germania This may be a common misunderstanding. A server is just a computer. If your excel files are all on your computer, congrats, thats your server and your computer. As far as installing SQL goes, it's pretty much as simple as "download __.exe, doubleclick, hit 'OK' a few times" – Mars Oct 3 at 5:06
  • 7
    @Germania Yes, SQL can be local. And if your data is not currently encrypted, then running it through SQL locally doesn't need to be encrypted at this point either. In the future, you do need to encrypt things, but that's usually as simple as just calling encrypt(myData). You do NOT need a cybersec degree here – Mars Oct 3 at 5:10
  • 3
    @Germania You don't need a "server physically". You can use XAMPP (security and integrity will suffer), SQLite (no server at all), use a virtual machine with Debian on it and use it as the server or even an old junky PC with 512MB RAM connected somewhere with power and a network cable. My setup at work is just some random pc with 4GB of RAM, running Debian, with 3 VMs. One of which is my testing server, with 1GB of RAM. Even a raspberry pi zero is enough for this task. It's a physical pc that costs dirt cheap. – Ismael Miguel Oct 3 at 9:09
  • 1
    @Germania you should familiarize yourself with the concept of the correct side of the airtight hatchway (see Douglas Adams and/or Raymond Chen). If you are working with the data in form of "disorganized folders and excel spreadsheets", it's already as unsafe as it can be. Migrating it to a SQL DB on the same machine will not make it any less secure. It even might be a first step to some actual security. – IMil Oct 3 at 14:49
2

A number of the comments and answers here point out that you can learn SQL using various "free" sites or references. But I would suggest that just learning SQL will not be enough, and lead to further frustration. SQL is a tool, not a goal. You can learn to use a hammer, but that does not qualify you to build a house.

It also sounds like your frustration and fear of failure in front of your family is making you feel like its impossible. It's not impossible, but you need to calm down, take a deep breath, and come up with a plan to solve to problem.

I would suggest the following:

  • Have a discussion with manager/company officer/Dad or whoever can make the decision and resources available for you to fix this. Explain what you do know, and what you don't know, and why that will produce less than desirable results.
  • Develop a plan to accelerate the process to success. This will include:
    • Learning SQL and databases. Nothing fancy, but something like on Pluralsight or comparable, to give the knowledge of the basics.
    • Hire a consultant versed in your tools, your dplyr, R, whatever you want to use. And the consultant should be experienced in whatever SQL platform you decide. Have the consultant design and build the database, and automate the data collection. Give the job of laying the foundation to an expert that can help.
    • When the consultant is done, you will work to develop the processes to keep the data analysis going after you leave for grad school.

This plan of action will ease your frustration by getting someone to help you, help the company get the best results, and help your experience (and resume) by having a successful implementation. It's not hard, but sometimes it's hard to step back out of the fire to realize an alternate path to success.

Good luck!

  • 1
    The thing is, without failing and doing it wrong, you won't know if you're doing it right. There's a ton of other solutions. So what if the O.P. used a varchar(45) for an tinyint? Who cares? The O.P. is learning! The basic knowledge is all that's needed. I agree with your points. But one thing I feel (and a few others seem to agree) is that the O.P. is dodging the "trying" by throwing obstacles. Getting a contractor is a good idea. Getting the O.P. to learn it with no excuses would be the ideal. – Ismael Miguel Oct 3 at 17:10
  • @IsmaelMiguel, the contractor could have teaching the OP as part of the contract. Due to the OP's unwillingness to learn on their own, I wouldn't put in any performance goals in the contract, just that the contractor need to try to teach the OP the basics. Once things are setup and ready for the contractor to leave, the OP should have enough knowledge to go from there. If not, it's time for them to leave. – computercarguy Oct 3 at 18:24
  • I like this answer because it tackles the problem from a 3rd perspective. The OP has a "bottom of the cliff" perspective", while most answers say to "go around to the other side of the mountain for a less steep incline", and this says "find the road to the top and hitch a ride." All three are valid views, but the OP's is the only one that doesn't currently go anywhere, since the OP doesn't have the "climbing" skills or desire to deal with their situation. – computercarguy Oct 3 at 18:27
  • @IsmaelMiguel, Perhaps I am hearing a different tone from the OP than most here (maybe because I’ve been there myself). I hear someone buried and frustrated and not knowing where to go. Belittling the OP will not help. He is worried about his work, his job, and how the family will view him. It’s just not “learn it”, it’s also how to get the best solution for all facets of the issues. – mharr Oct 3 at 18:33
  • 1
    @mharr Personally, the tone I'm reading is "I don't want to do this, but people say to do this. So, I will make up excuses that justify that I simply don't want to do it". – Ismael Miguel Oct 3 at 20:00
2

You can always tell the truth.

You don't like your job. You don't want to learn it. Learning a toolset is not going to change that.

You can quit and be honest about your reasons. If they understand, good. If they don't, still good. You can't force yourself to do something you don't believe in, something you don't take pleasure doing.

That's a lot better than make nonsense excuses or keep lying to yourself.