Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

All about Data Science Professionals

Harshit Ahluwalia
Harshit Ahluwalia Last Updated : 12 Jan, 2022
5 min read

Objective

  • Describe the different entities that form the best data ecosystem
  • Describe and differentiate between the different types of data professionals, their roles and responsibilities in the data ecosystem

 

Introduction

In todays’ world, regardless of your professional background, you must have definitely come across the terms like “Data Science”, “Data Engineering”, “Business Analyst”, and more which are driving decision making in today’s professional world. But just like me, I am sure, you must have been bamboozled by what these terms mean and whether there really any difference between them? As it turns out, all of the jobs in the data field have a well-defined objective and in this article, I will tell you the difference between these job roles and the skills required for these roles.

 

Table of Contents

  1. Who are Data engineers and their responsibilities?
  2. What are the skills required to become a Data Engineer?
  3. Who are Data Analyst and their responsibilities?
  4. What are the skills required to become a Data Analyst?
  5. Who are Data Scientists and their responsibilities?
  6. What are the skills required to become a Data Scientist?
  7. Who are Business Analyst and their responsibilities?
  8. What are the skills required to become a Business Analyst?

 

Data Engineering | Data Science Professionals

Who are Data engineers and what are their responsibilities?

These are the people who develop and maintain the data architecture, data infrastructure and make data available for business operations and analysis. This data infrastructure includes databases, big data repositories and data pipelines for transforming and moving data between the data systems. Their goal is to provide quality data available for fact-finding and data-driven decision making. Data Analysts and Data scientists make use of data that data engineers provide. They work with other data professionals to ensure that the data matches their needs. When we look deep into data engineering, it’s all about selecting the right:

  • Databases
  • Storage system
  • Cloud infrastructure or cloud platform

 

When we put together all these things, the data flow inside the organization is seamless. Data Engineers works within the data ecosystem. The main responsibilities of a data engineer are:

  • Collecting source data: This includes extracting, integrating, organising data and data acquisition from multiple sources.
  • Processing data: This includes cleaning, transforming and preparing data to make it valuable. It also includes maintaining distributed systems for processing data.
  • Storing data: Storing data for reliability and easy availability of data. Data Engineer has to make sure that they should use proper and best data stores for the storage of procured data. They also have to ensure data privacy, security, compliance, monitoring, backup and recovery of the data.
  • Making data available to users securely: Data Engineer has to make sure the APIs, services and programs for retrieving data for end-users.

Skills required to become a Data Engineer

  • Good knowledge of programming languages
  • In-depth understanding of relational databases and non-relational databases
  • comfortable with command line
  • Understands the working of the Hadoop ecosystem especially Map-Reduce and YARN
  • Understands the concept of data warehousing
  • Practical knowledge of working with Apache Spark
  • Data ingestion with Apache Kafka
  • Scheduling jobs with Apache Airflow
  • Fundamentals of cloud computing

Who are Data Analysts and what are their responsibilities?

A Data Analyst is a data professional who transforms numbers into plain language, so organisations can make good decisions. These are the people who answer questions such as “Are the user’s search experience is good or bad on our website?” or “Is there a correlation between sales”. The role of a data analyst is to inspect and clean data for delivering insights. Their main task is to identify correlations, find patterns and apply statistical methods to analyse and mine data. Their day-to-day activities also include visualizing data to interpret and present the findings.

Data Analyst | Data Science Professionals

Skills required to become Data Analyst

  • Good knowledge of spreadsheets
  • Proficient in writing queries and using statistical tools to create charts and dashboards
  • Should have a good understanding of programming language
  • Should be able to convert business problems into a data problem
  • Strong analytical skills and storytelling
  • Good command in Data Visualization

Who are Data Scientists and what are their responsibilities?

A Data Scientist analyses data for actionable insights and create predictive models using Machine learning and Deep Learning techniques. A data scientist’s role combines computer science, statistics, and mathematics. They analyze, process, and model data then interpret the results to create actionable plans for companies and other organizations. These are the people who answer questions such as “How many new social media followers am I likely to get in the next quarter?”

 

Skills required to become a Data Scientist

  • Knowledge of a programming language
  • Good hands-on experience of data visualization tools
  • Good understanding of statistics, linear algebra and probability
  • In-depth understanding of machine learning
  • Basic knowledge of recommendation engines
  • Practical knowledge of working with Time Series data
  • Understanding of Deep Learning, Computer Vision
  • Good hands-on knowledge of Natural Language processing
  • Basic of Model Deployment
  • Strong communication skills and presentation skills

Who are Business Analysts and what are their responsibilities?

Let me explain this term by taking an example. Let us suppose that there is a business whose sales are declining and now to find the answer on why sales are declining and what can be done, two parties are involved in this process. The first person is a data analyst, the role of a data analyst is to work with software developers within the same organization and to connect to different databases where they can find data related to sales etc. The role of a data analyst is they connect with the databases, they will build power BI or tableau dashboards or they will do data analytics in Python or excel and their job is to generate insights. So basically, data analysts produce insights and the business analyst role is to consume these insights and to make business decisions. Business analysts are not technical as data analysts but they have soft skills – they are critical thinkers, they are good at communication, they understand the business very well. So, if the data analyst is saying that in the southwest region, the company’s sales are declining and the customer churn out is higher maybe the business analyst will based on his knowledge say that in the southwest region the income level people are less so let’s run a special discount so that the sales will increase. Business Analyst is more focused on soft skills, critical thinking, problem-solving and domain knowledge whereas data analyst is more focused on technical skills.

Business Analysts | Data Science Professionals

Skills needed to become a Business Analyst

  • Good analytical skills
  • Good logical skills and leadership skills
  • Proficient in Microsoft Excel
  • Good negotiation skills
  • Knowledge of Python and R
  • Good hands-on experience of Tableau, Power BI or other dashboarding tools
  • Strong presentation and communication skills

End Notes

In this article, we’ve learnt the different entities that play a crucial role in the data ecosystem. In a nutshell, Data Engineer converts raw data into usable data. Data Analysts use this data to generate insights. Data Scientists use Data Analytics and Data Engineering to predict the future using data from the past. Business Analysts and Business Intelligence Analysts use these insights and predictions to drive business decisions that benefit and grow their businesses.

 

Hope you enjoyed reading. If you would like to read more on Data Science then, head on to our blog.

Harshit Ahluwalia
Harshit Ahluwalia

Growth Hacker | Generative AI | LLMs | RAGs | FineTuning | 62K+ Followers https://www.linkedin.com/in/harshit-ahluwalia/ https://www.linkedin.com/in/harshit-ahluwalia/ https://www.linkedin.com/in/harshit-ahluwalia/

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details