I was fortunate to get early opportunities of working on numerous data science projects. I enjoyed this part the most. Even more, when I realized that my efforts will add value to some company.
However, the miserable part was, less than 30% of the data science projects actually got implemented to their potential. I got shattered realizing that my efforts got wasted. But, I wasn’t the only one. Almost, every other analyst had the same feeling of disappointment.
Even today, the real challenge which data science industry faces is the lack of coordination between business folks and analytics folks. To my surprise, I’ve even noticed that these people prefer to sit away from each other in same offices.
Only if both these skills set were common in professionals, we would have seen a much higher project implementations. Over last four years, I’ve spent lots of time thinking of best practices to make a project successful.
I found that, If you have right people sitting in your office, have clearly defined business problems and a culture which motivates out-of-box thinking, you have a break-through in the pipeline.
Therefore, to become successful at your data science / analytics jobs, I’d recommend you to follow the tips mentioned below. These are tried and tested tips. To reap the maximum benefits, I’d suggest you follow them with discipline. I’ve benefited from them. Its your turn now!
I know you are an analyst and all you care about is numbers. But, what differentiates an awesome business analyst from average data analyst? It’s their potential to understand business. You should try to understand business even before you take up your first project. Here are a few things you should definitely explore:
a. Customer level information: Total number of active customers, month on month customer attrition, segments defined by business on portfolio.
b. Business Strategies: How do we acquire new customers, what are the channels. How do we retain valuable customers.
c. Product Information: How does your customer interact with your products? How do you earn money through your product? Is your product a direct revenue maker or is just an engagement tool?
If you can answer all these questions, you are in a good shape to start your first project.
I have observed that analysts aim for objectives which are not even the main concern of the problem. For instance, lets imagine we found that more a customer calls at customer care, higher will be his propensity to abandon the services.
Now, if we begin solving for method to minimize the calls at customer care, we probably won’t reduce the attrition rate. Rather, I already see higher dissatisfaction in your customers if you do not have a human justifying your faults. This is probably an easy kill, and you will refuse to get into such easy traps. But, real life problems are much difficult to find. I will say, it is much easier to solve a defined problem than finding what is the right problem to solve.
This probably is the easiest puzzle to solve for an analyst yet a simple trap to fall in. Let me explain it using few examples.
Suppose, you are trying to build a targeting model for a marketing campaign. Which of the metric will you choose to gauge your model:
I will always choose KS in this case, given that Lift will only give you estimate on a particular decile. Hence, it probably won’t help us to find the total target population and the break point. AUC-ROC will be an estimate for the overall population, which is not our intent in this case. Log-Likelihood is probably the biggest misfit in this case, as all matters to us is the rank order and not the actual probability.
I have seen this as the biggest problem in many functions/industries. Today business leaders seek innovation in every thing they do.
To truly innovate, you need to follow a systematic way to diverge and converge. The extent to which you need to diverge will come along as you get more experienced in this approach. What we mean is to think of all possible ways to crack the problem being unbiased on feasibility, time for development and traditional approaches. Once, you are convinced that you have covered the universe, you now apply all your constraints to narrow down the approaches.
Analytics is being used in every possible industry. But, why do we not go beyond our traditional approaches and look for solution in other industries?
For instance, a recommended video solution implemented in E-Commerce industry can be very well used in a blogging portal like Analytics Vidhya. The only way to do this is to interact with people working in other industry and learn about their efforts using analytics.
Right from the first day of your analysis, you should interact with business partners. One thing which I have seen going wrong in general is that analyst and business partner get in touch on the solution non-frequently. Business partners want to stay away from the technical details and so does the analyst from business. This does no good to the project. It is very essential to maintain constant interaction to understand implementation of the model parallel to building of the model.
I know you are a statistician and love to confuse business people using complex formulations. Bringing such complexity in discussion with business people might help you get out of the immediate conversation but decreases the chances of successful implementation.
Here is what you need to do: Once you have the output variables, try to find out a simple lever which can make it easier for the business to understand. Let me give you an example of this approach. We were trying to find out the right agents who will be top performers once they onboard. We came up with the stratfied population and their expected performance. However, we had to choose a lever which can change the population mix. What we did was very simple: we implemented differential fee strategy to change the mix of applicant and hence the mix of our population.
The target variable is never the end product of your analysis. It is always a business deck! So you need to put a lot of effort while bringing out your idea in a crisp and effective way. Try learning terminologies with which your audience can connect and think of what will business partner look out if you were in their shoes.
I recently started learning chinese for one of my projects. The entire project was extremely easy but I found that even with a robust model, I was doing a bad job at selling it to business. The reason being my gap of understanding their internal discussions. It is very essential to speak a language of your audience. I have seen very simple models being appreciated and smartest models being rejected. The only difference being the ability of the analyst to speak business while presenting their models.
Coming to the last but not the least, what happens once every one is convinced with the effectiveness of your model. Your job is still not done. Set up monthly follow ups with business to understand how project was implemented, is it being used in the right send.
Apart from these 10 basic practices, here are a few on the technical front I would recommend:
One thing which you will realize with time is that analytics industry is extremely dynamic. However, if you are a person who likes to be in their comfort zone, you will soon find your skill sets redundant. One thing which I found extremely useful is participating in data science competitions and competing with your peers/learn from your peers. Kaggle and Analytics Vidhya are good points to take up a few challenges.
I believe this is another way to keep yourself up to date. I recommend reading these lists to start:
Must Read Books for Beginners on Machine Learning and Artificial Intelligence
Must Read Books for Data Scientists
Must Read Books for Data Scientists
Get out of your comfort zone of programming on SAS, R or Python. Try learning upcoming technologies to handle big data. SPARK and JAVA will be my recommendation to start with.
I have witnessed the difference of marrying business with analytics. The list is no way exhaustive but will give you a kickstart.
Did you find this article helpful? Please share your opinions / thoughts in the comments section below.
Tavish, These are truly valuable tips one faces these challenges in real world scenarios. I can`t agree more with what you recommended here. Thnak you for guiding budding Data Analysts.
Great post! I will surely profit from these tips, except the second tip which was a little criptic and not so clear
Extraordinary post! I will definitely benefit from these tips, with the exception of the second tip which was a little criptic and not all that unmistakable