MyStory: I became a Data Scientist after working for 10 years in IT Industry

Distant Memories (Prologue)

Getting into Analytics or Data science stream was never my dream. I got into this out of an accident. Prior to getting into data sciences, I was a mainframe programmer all through. The only aim that I had for a very long time was to get into a good MBA programme.

Business analytics as a new career Stream

It was 2013. I had 9+ years into software services industry by then. My career was almost stabilized and I could not see much growth out of it. Hence I was planning to turn up to my dreams of management education. But I was quite apprehensive as my long experience would bar me from getting into a decent MBA. It was when I came to know about the new upcoming stream called Business Analytics.

My first step towards getting into analytics was scouting for an opportunity internally. It didn’t worked out for me for two reasons. First, there were not much analytics initiatives happening within the companies at that time. Secondly, as the advanced analytics stream evolved as an extension to the business intelligence division in most of the companies (especially the Information technology services), they could not afford to take a non BI person like me and to train up for their analytics works. Also, there were not much courses online at that time. Or perhaps, I was not aware.

A Rude(Self) awakening

That was the time I decided to reskill myself to suit to the analytics industry. I had applied for couple of long term analytics programmes available in India and I managed to convert one of them into admission. But I never know that the class of business analytics will be math (advanced math) and technology heavy, the areas which I was bit apprehensive, until I went to the first day of my class. Apprehensive was due to the fact that I was working on a very old technology and doesn’t require you update constantly.

The extreme main skill that any data scientist should have – more than the art of storytelling – is the art of questioning and analyzing the information in hand. The very first mistake that I did before getting into business analytics was failing to understand what “Business analytics” professional do in their work – atleast its literal meaning. My cluttered mind strongly correlated “Business analytics” with the term “Business analysis”.  At the end of my first day class, one thing I really understood was – The decision to join the business analytics program was taken based on the gut feeling, rather than an informed decision.

Beginning of a thousand-mile

I had two stages of transition from being a mainframe programmer to getting into data sciences – First one was undergoing a one year programme and the other being the challenges that I face during the day to day work.

Advantages of structured transition

The one year analytics programme was extremely challenging – I would call it as a toughest challenge (yet most rewarding at a later point of time), I had till now in life. Multifaceted responsibility is the first challenge that I had faced as soon as I enrolled to the programme. Being a dad to a 5 year old, a production support lead at the office, coping up with the course was extremely difficult for me. With the kind of rigorousness that came along with the course, be it in terms of week end online classes (apart from the regular classes that we attend at the campus), tests, home works and assignments I was completely bombarded every day.

The one year spent at the programme was a roller coaster ride for all my student mates without any exception. The quality of problems that we were tested as part of assignments and their deadlines driven us to spend sleepless nights.  The effort to put completing every mini project, the early morning hour classes, the late night discussions  along with my family and office responsibilities – all together  would have made my health worse, if had I not been supported by the awesome peer group.  I still remember the days attending an hour of office call and taking up the exam in tandem. But I didn’t realize God was preparing me for the challenges that would follow up later in my career in analytics.

The First Date

Job interviews are always like first date. The outcomes are seldom predictable.

The next challenge that I faced was when I started looking for an opportunity to work in data science.  Hiring managers are typically concerned about taking in someone who comes with considerable experience in non-data science stream.  Luckily, the capstone project that I did as part of the course work with one of the prominent retail brands came in handy for me. The interviewer typically liked the process that was followed in building up the solution during this project. Thanks to awesome professors who mentored continuously.

These days, I also see many people who apply and get selected in interviews just by participating actively in data science competitions. Infact, as far as I have seen, data science competitors outshined the one with real time work experience in the interviews.

The Swell is really big

If the challenges pertaining during the academic stage of my transition was pertaining to the mathematical part, at work I was facing the challenges from the process and domain front. Here with I am briefing some of the lessons that I learned as part of my journey:

Be good in domain – One of the foremost challenge that any data scientist would face is the short-come in domain knowledge.  It is imperative for any data scientist to be good in the domain without which an analytics engagement can never be successful. Without the domain knowledge, a data scientist would never be able to answer any of the business question. He/She cannot build a good predictive model without the right set of variables. The insights that he generates will go in vain without the business knowledge.

Setting the expectation right with the customer – Most important and vital to satisfy every stakeholder. Customers may not know about analytics. All they know is that if the data is given, data scientists do some magic and generate insights which can be used to improve the business.

 A data scientist should exhibit patience and teach them what can be possible with data science and what not. I remember the days I spent with the customer explaining why a statistical model is much better than predicting manually, what is mean by a model by itself, how to interpret confusion matrix etc.

Knowledge of SQL and databases – I had an initial assumption that it is the duty of the data engineer to extract the relevant data required for building the use cases. It is false. Companies may not be interested in investing one more person for pulling the data. A data scientist is expected to know how to extract the data that he requires, and transform the data to the required format.

Patience for analyzing the data– Data analysis has to be done iteratively from various perspectives.  We never know what kind of pattern exist inside the data. However, by following structured data analysis methods and little bit of patience, the pattern that the data follows can be identified.

Presenting the solution– In my view, this is more important than building a model. All the hard work put in by a data scientist can be showcased only in this area. Also it is important to know the context of to whom the presentation is made. We may not talk about that adjusted R square or an ROC curve to end user.

Be comfortable when you fail – Most of the work that I did during the initial days failed to larger extent. Every time I presented the solution to the customer, there was a high probability that it gets rejected. I looked stupid every time I made silly and big mistakes. But I eventually learned that all these are part and parcel of the career in data science.  Just try to have a mentor if you are new to the analytics projects or start small if you don’t have a mentor.

The Meta Critics:

Here are some the questions that you should ask yourself before getting into a career transition:

Why do you want to make a career shift? – I meet many peoples who come to me and say that they wanted to shift to analytics just because they have problem with existing job/manager/company etc. If this is the case with you, try to see if something can be made interesting in your current work. Changing the work stream may not solve your problem.

Shifting career with high work-ex:  If your overall experience is more than 7 years, please think thrice before changing stream.  Please remember that with more than 7 years of experience, you may not be allowed to experiment in your work rather you will be expected to deliver.

Why do you want to get into data science?  If the answer is that data sciences present lucrative opportunity, then you might want to think again!! You will be getting into a field where you can’t sail smooth atleast for few years down the line.  You will be expected to keep yourself updated very frequently and it is very difficult for many people whom I had seen, including myself.

What do you know about analytics to make a career in it? –  Test this by doing some self-learning. Try some online courses in coursera or edx or udacity to check whether you will be comfortable with data science. Typically, you should not take data science as your career if you are averse to mathematics or if you show dislike to databases and querying tools like SQL. If you are new to all these, check these at khanacademy.com (for math courses) and w3schools.

Dare to Dream

Life holds special magic for all those who are dare to dream beyond their abilities. Some tips for the people who would want to hold their dream and looking to get into data sciences.

1. Do minimum level of self-analysis before getting into data sciences. Test your fitment for data sciences in this link.
2. Try to speak to some of them who already transitioned to this field. Their experience can give you lot of insights to your new career. If you don’t know anyone in person, post your questions in forums like discuss. Someone will be there to help you out.
3. Read and subscribe to articles in Analytics Vidhya, data science central before even committing to the pricy courses available. Follow forums like discuss, quora data science Also if possible try to take couple of 101 challenges from datahack (and later kaggle). You will get a first-hand experience of how to solve a data science problem in these portals. I learned more by solving the challenges than working in real time environment.
4. Never believe in claims which promises to make you a data scientist in less than 3/6/12 months. Every career has its own path of learning curve and data science is no exception to it.

Foot notes

1. This article was aimed mostly for the people who want to enter into data sciences / analytics with considerable work experience. The challenges mentioned here may not necessarily apply for a fresher or with less than 5 years’ of work experience.
2. Data science / business analytics are used interchangeably.

karthe