During a panel discussion in Gartner Business Intelligence and Analytics Summit early this year in Barcelona, vendors estimated:
70% of Analytics projects fail to meet expectations
While the exact number is difficult to find and would vary significantly from organization to organization, poor implementation of analytics projects is definitely a huge challenge for Analytics leaders across the globe.
For an industry based on scientific and logical thinking, these kind of failure rates are appalling. More surprisingly, analytics community has not learnt from its mistakes and continues to see poor implementation project after project.
This post aims to bring out some of the common mistakes people make while solving problems through analytics and suggest a few ways to minimize these mistakes.
[stextbox id=”section”]Common reasons for failure of Analytics projects due to analysts[/stextbox]
Most of the analyst related failures arise because analysts have a tendency to assume similarity across projects. A predictive model which works wonders for one company can lead to disasters in other (if copied blindly). Similarly, each Organization has its own way of storing and dealing with data. Same status code could have different definition. Each and every project should be thought through and planned independently. One solution fit all approach doesn’t work in analytics. Following are the top 3 errors which I think cause most of the failures:
- Not planning for and spending time to understand business process: An analyst needs to spend a lot of time to make sure he understands the business processes and gaps before he can start working on an analytics solution. More often than not, process improvements might give you a better solution than applying analytics to a process you are not clear about. Ironically, people do not budget the required time to be spent on understanding business processes in their project planning. Have provided some good practices towards the second half of the article.
- Unclear and non-specific requirement capturing: This reason stands out for the amount of failures it causes. If you are preparing the requirements document and are not clear about any aspect of the problem, ask out! If you do not clarify things now, you will come across surprises which will de-rail your project from timely completion. In one of the project for implementing a BI tool across an MNC (buying more than hundreds of licenses), the analyst spent less than a day, in total across all the departments and created the BRD (Business Requirement Document) on basis of which entire project pricing was arrived at. End result of the project – Shelved after putting in more than 9 months of effort of 5 member team.
- Half baked thoughts on implementation: You may create the best of predictive models, but without a clear implementation in mind, these models are useless. A common mistake while building predictive models is to include variables which can not be controlled. For example, you find out that people who are Self Employed are more profitable and incentivize your Sales teams to bring more of these customers. What are the chances that people who were mentioning occupation as Professional or Salaried would start calling themselves Self employed? How do you avoid this to fool your model?
[stextbox id=”section”]Common reasons for failure of Analytics projects due to stakeholders[/stextbox]
- Limited buy in from customers or stakeholders across levels: You need to ensure a complete buy in from a customer in order to make sure that your project stands a chance to achieve success. If the stakeholders are not convinced about the idea, it will reflect across as poor business understanding of analysts, analysts not being aware of strategic changes impacting their project etc. which will ultimately lead to project failure.
- Not providing required time and inputs to the analyst: A lot of people would want analysts to work on their business problems, but they find it difficult to give them the required time due to their regular commitments. This becomes specially critical at key stages of project like scoping & creation of BRD (see next point for further details), hypothesis generation for predictive model, user testing for an application etc.
- Hasty review of BRD (business requirement document) / scope document: You would be surprised to see the amount of failures which happen because of a poor BRD review. While creating a BRD might be the responsibility of an analyst, there would always be gaps which only a business person will find out. Unless a BRD is reviewed line by line, you can be rest assured that it will come back to hit you and cause a lot of re-work or will set you up for failure.
[stextbox id=”section”]Guidelines to ensure successful implementation of analytics projects[/stextbox]
These guidelines should minimize chances of failure for analytics project, but they can be applied in any project management role.
- Plan, plan and plan: The more you plan upfront, higher are the chances of your success. Keep enough time to understand business process, capture business requirements, brainstorm various hypothesis and finally making sure that the project is implemented exactly in the fashion you had wanted.
- Requirement gathering / Business Understanding stage:
- Spend time with all stakeholders and understand their perspective. At the outset, meet and understand the requirements of the stakeholders. Often, different stakeholders might require different objectives to be fulfilled. Understanding perspective of various stakeholders helps you outline any differences at the start.
- Spend at least a day experiencing the process. If it relates to a call center, go for call listening. Look at how the process works, is the calling based on dialler or manual? If the problem is pertaining to Sales, go for Sales calls.
- Work on requirement gathering iteratively, don’t wait for that perfect document to be created. If you have a working version, discuss that with stakeholders and take their feedback.
- Put in regular catch ups with stakeholders, ideally weekly
- Attend relevant business meetings – May be 1 or 2 weekly business meetings. They provide you rich understanding of challenges which the group faces. Also, it comes in handy when you are thinking of implementation
- Review of BRD: As a thumb rule, a business owner should spend at least 30% of the time the analyst has spent creating this scope document.
- Analytical modeling stage:
- For predictive modeling, make sure that you have captured all business hypothesis. You should spend that extra hour with the stakeholders to make sure nothing is left in their mind. Another practice which I find useful at this stage is unrestrained thinking. Don’t restrain yourself by data which is available, look at what is available and what is not only after you have exhausted all possible hypothesis.
- Think through about implementation at this stage: You need to be clear of project implementation at this stage. How would the data flow? What would the business owner need to change? If you make this change, would it influence customer behaviour? Can it fool your model?
- Implementation stage:
- Similar to start of the project, go back to business process and check whether project has gone live as you wanted. Look at the changes with minute details
- Monitor the project intensely: For initial period, track things closely till the time you get comfort that things are going fine.
- Update your stakeholders about your findings in weekly reviews.
- If implementation falls apart, accept it and quickly move on to see how can it be corrected.
Hopefully, following these guidelines will help you deliver your analytics projects as per expectation. I have found them very useful while working on various analytics projects. If you have any practices, which you follow to ensure successful implementation of projects, do let me know.
Happy implementation!
If you like what you just read & want to continue your analytics learning, subscribe to our emails or like our facebook page
Kunal Jain is the Founder and CEO of Analytics Vidhya, one of the world's leading communities of Al professionals. With over 17 years of experience in the field, Kunal has been instrumental in shaping the global Al landscape. His expertise spans diverse markets, from developed economies like the UK to emerging ones like India, where he has successfully led and delivered complex data-driven solutions. As a recognized thought leader, Kunal has empowered countless individuals to realize their Al ambitions through his visionary approach to Al education and community building. Before founding Analytics Vidhya, Kunal earned both his undergraduate and postgraduate degrees from IIT Bombay and held key roles at Capital One and Aviva Life Insurance across multiple geographies. His passion lies at the intersection of analytics, Al, and fostering a thriving community of data science professionals.
