Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

Benefits Of Data Science In AWS

S
Sharvari Last Updated : 23 Apr, 2021
5 min read
This article was published as a part of the Data Science Blogathon.

 

Data Science In AWS
Source: Sigmund on Unsplash

In this article, we will discuss the advantages of building knowledge in data science that comes within the cloud. We begin by discussing the advantages of cloud computing. 

Next, we tend to describe a typical machine learning workflow and also the common challenges to maneuver our models and applications from the prototyping section to production. We touch on the overall advantages of developing data science applications on Amazon Web Services (AWS) and introduce the relevant AWS services for each step of the model development workflow. We will also look additionally share the field of study best practices, specifically around operational excellence, security, dependableness, performance, and price improvement.

 

Benefits of Cloud Computing

Cloud computing permits the on-demand delivery of IT resources via the web with pay-as-you-go pricing. Thus rather than shopping, owning, and maintaining our data centers and servers, we will acquire technology like compute power, storage, databases, And other alternative services on an as-needed basis. Just like a power company sending electricity instantly when we flip a light switch in our home, the cloud provisions IT resources permits with the click of a button or invocation of an API.

“There is no compression algorithm for experience” is a famous quote CEO of AWS. The quote expresses the company’s long-standing expertise in building reliable, secure, and performant services since 2006.

AWS has been regularly increasing its service portfolio to support nearly any cloud work, as well as several services and options within the space of computer science and machine learning. Several of those AI and machine learning services stem from Amazon’s pioneering add recommender systems, computer vision, speech/text, and neural networks over the past twenty years. A paper from 2003 titled “Amazon.com Recommendations: Item-to-Item Collaborative Filtering” recently won the IEEE award as a paper that withstood the “test of your time.” Let’s review the advantages of cloud computing within the context of data science with AWS.

 

Agility

 

Data Science In AWS agility
 Source: Skye Studios on Unsplash

Cloud computing lets us spin up resources as we want them. This allows us to experiment quickly and whenever we want. Perhaps we would like to check a new library to run data-quality checks on our dataset or speed up model training by leveraging the latest generation of GPU compute resources. We would spin up tens, hundreds, or maybe thousands of servers in minutes to perform those tasks. If an experiment fails, we will continually de-provision those resources with no risk.

 

Cost Savings

Cost Savings
Source: Micheile Henderson on Micheile Henderson

Cloud computing permits us to trade capital expenses for variable expenses. We tend to solely pay money for what we use with no upfront for direct investments in hardware which will become obsolete in a very few months. If we tend to spin up compute resources to perform our data-quality checks, date transformations, or model training, we only solely pay money for the time those compute resources are in use.

We will deliver the goods any value savings by investing Amazon EC2 Spot Instances for our model training. Spot Instances allow us to profit from unused EC2 capability within the AWS cloud and are available with up to a 90% discount compared to on-demand instances. Reserved Instances and Savings Plans permit us to avoid wasting money by prepaying for a given quantity of your time.

 

Elasticity

 

Elasticity Data Science in AWS
Source: aimconsulting

Cloud computing permits us to scale our resources up or down to match our application needs. Let’s say we’ve deployed our data science application to production and our model is serving real-time predictions. we will currently automatically rescale the model hosting resources just in case we tend to observe a peak in model requests. Similarly, we will certainly scale down the resources once the quantity of model requests drops. there’s no need for overprovision resources to handle peak loads.

 

Innovate quicker

 

Innovate quicker
Source: Source on Source

Cloud computing permits us to innovate quicker as we will specialize in developing applications that differentiate our business, instead of defrayal time on the uniform work of managing infrastructure. The cloud helps us to experiment with new algorithms, frameworks, and hardware.

 

Deploy Globally in Minutes

 

Deploy Globally in Minutes
Source: iamwire

Cloud computing lets us deploy our data science applications globally within few minutes. In our world economy, it’s vital to be on the point of our customers. AWS has the idea of a Region, that may be a physical location around the world where AWS clusters data centers. Every group of data centers is called an Availability Zone (AZ). Every AWS Region consists of multiple, isolated, and physically separate AZs inside a geographical area. The quantity of AWS Regions and AZs is endlessly growing.

We can leverage the global footprint of AWS Regions and AZs to deploy our data science applications on the point of our customers, improve application performance with fast response times, and comply with the data-privacy restrictions of every Region.]

 

Smooth Transition from epitome to Production

Smooth Transition from epitome to Production
Source 

One of the advantages of developing data science applications that comes within the cloud is that the sleek transition from epitome to production. We can switch from running model code to data-quality checks or distributed model training across petabytes of data in less time. And once we are done, we will deploy our trained models to serve a period or batch predictions for users across the world.

Prototyping usually happens in single-machine development environments victimization Jupyter Notebook, Numpy, and pandas. This approach works fine for not large datasets. While on work with large datasets, we’ll quickly exceed the only machine’s hardware and RAM resources.

Also, we tend to might want to use GPU or multiple machines to accelerate our model training. This is often sometimes impracticable with one machine. The next challenge arises once we wish to deploy our model (or application) to production. we also need to keep in mind that this application will handle thousands of users on a world scale.

Production preparation usually needs a robust collaboration between numerous groups as well as data science, data engineering, application development, and DevOps. And once our application is successfully deployed, we want to endlessly monitor and react to model performance and data-quality problems that will arise when the model is pushed to production.

Thank You For Reading!

About Me:

Technical Writer | AI Enthusiast😎| | Avid Reader | Data Science ❤️ | Open Source Contributor 🌍

Connect with me on:

Twitter: https://twitter.com/aree_yarr_sharu

LinkedIn: https://t.co/g0A8rcvcYo?amp=1

The media shown in this article on Data Science in AWS are not owned by Analytics Vidhya and is used at the Author’s discretion. 

S
Sharvari

I am Sharvari Raut. I love to write. I am a final year student in Computer Science and Engineering from NCER Pune. I have worked as a freelance technical writer for few startups and companies. Having 2 yrs of experience in Technical Writing I have written over 100+ technical articles which are published till now. Writing for Analytics Vidhya is one of my favourite things to do.

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details