I recently got hired as contract data analyst for a company and have been given their datasets.
The data that they provided me with is either not very well measured (via a poorly designed survey) or too few for formal statistical methods.
I'm not sure how to tell my boss that the data is insufficient and still keep my job. Any suggestions?
Clarification: The relationship I have with my boss is good. I just simply don't have enough data to provide an accurate analysis.
Part of your responsibility as a data analyst is to ensure that you have data of sufficient quality to work on.
You can't make a silk purse out of a sow's ear
You've identified that you don't have data that's of adequate quality to work on, so you need to get back to your manager with that.
I'm sorry, but the quality of this data isn't good enough for me to work on with any degree of efficiency.
Give a few examples of why there's not enough data for you to perform the lookups needed and suggest the root cause(s), and state what the minimum requirements that you need in order to meet your objectives.
It's possible that there's been a communication failure and that you're only meant to deal with this sub-set of data in a specific fashion. This seems unlikely, but it might be the case. You'll soon get clarification when you report.
It's not likely that you'll get in trouble for reporting back to your manager with whatever's blocking your work as long as you clearly describe the issues and do what you can to identify the cause of the blockage.
I'm not sure how to tell my boss that the data is insufficient and still keep my job. Any suggestions?
Short Answer: Try your model with that data first, and if it fails after trying several times, then report the need of more resources or data. You will still have to design the model, so you can start with that and add more data as needed. Notify your boss about your worries before proceeding.
Longer Answer: The best way would be to properly design and create your model. Then proceed to evaluate and train it with your current data.
After this, you will be able to obtain some sort of correlation or effectiveness of your approach, which will probably be low due to insufficient data.
After this happens, you can approach your boss and say "Hey boss, I already built and tested the model. However, seems we got a really low performance, and I suspect it is due to insufficient training data, as I have already tried [x,y,z]."
In other words, try your model first, and if it fails due to the few data report such findings, rather than saying it won't work without trying. This way you are actually showing that you tried and this backs up your claim.
As a data analyst myself this is often the case when building models from few data. No matter how much effort and improvements you put to the model or algorithm, more data is sometimes the only way to improve your performance (specially on Deep Learning models).
One way to empirically demonstrate whether the data is sufficient for a particular analysis is to conduct a power analysis to calculate the minimum sample size necessary to discern an effect of a given magnitude.
Review the results of the power analysis with your manager, and ask for her/his advice on how to proceed. Asking rather than telling puts you in a position of collaborating to solve a shared problem instead of a power struggle (pun intended). As one of my former professors, Leland Wilkinson, wrote in Statistical Methods in Psychology Journals, "Even when not asking for money, think about power. Statistical power does not corrupt."
Asking rather than telling puts you in a position of collaborating to solve a shared problem instead of a power struggle (pun intended) I see you are accomplished in statistics and psychology. Here, have a upvote :)
– Tobia Tesan
Nov 24 '17 at 18:01
First you tell your boss about the problem. If he says to go ahead, you generate all your graphs and tables including confidence intervals. That way, your results are correct and show how useful (or not) they are.
A second step, of course, would be to refine the acquisition process in order to produce better results next time. For getting your foot in that door, an accurate show of what the current data set manages or fails to deliver will likely be essential.
Your job is to process the data. Part of that is the confidence evaluation. If you put tangible and interpretable results on that rather than a handwavy "not good enough", the recipients of your analysis will think that its their own verdict rather than yours desiring better data.
And if they don't, you are not to blame.
Sometimes you have to walk away from a contract. But that isn't the most likely outcome here.
As a contractor and subject matter expert, it is your job to advise your customer what the best use of your time will be, in your professional judgement.
This may mean re-writing the scope of your contract part way through. You will have to negotiate the new scope with your customer. While disappointing and time-consuming, this is much more satisfactory than coming to the end of the contract and delivering a report that concludes that none of the hypotheses were proven.
Here is one example of a change to your scope of work: You might want to use your expertise to spend some time designing a better survey or experiment. Then pause your contract while someone else gathers new data. Finally, your customer hires you to come back later to analyse the new data (perhaps paying you a retainer or a premium to guarantee a particular timeslot or lead time.)
You can't guarantee that any dataset will ever be good enough for you to generate a satisfactory model, (and you can't guarantee that you will always use the most productive analyses), but you can take actions that make that outcome more likely.
Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).
Would you like to answer one of these unanswered questions instead?
