Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

Getting started with Cloud Computing using R Programming

A
Analytics Vidhya Last Updated : 19 Jul, 2020
8 min read

Introduction

Almost any domain / business today is being transformed through SMAC. SMAC is a collective term referring to changes happening in Social, Mobile, Analytics and Cloud. The impact of this change has been across the spectrum – Organizations, people and Products. In today’s article, we will enable you to take your analytics capabilities to next level by using Cloud computing.

We have explained the concept of cloud computing using R programming and RStudio using a step-wise methodology. Furthermore, you will also learn about the benefits of using R over cloud as compared to the traditional desktop or Local client / Server architecture.

Getting started with cloud computing using R Programming

Cloud – an enabling platform for data science:

Cloud computing has witnessed an unparalleled growth and penetration in last few years. It has enabled organizations to scale quickly and easily. Using cloud services, companies today collect, store and analyze huge amount of data, which was almost non-thinkable before. However, with services from the likes of Amazon, Google and Microsoft, cloud services are now accessible to any analyst.

Gone are the days, when you purchase a server for a particular capacity and then need to purchase a new one, when you grow out of the previous capacity. For example, most of the analysis I normally do is on a few GBs of data – sufficient to run on my laptop directly. However, recently Microsoft released ~400 GB of data about Malware and viruses on Kaggle. If, I would have thought of solving this problem on my laptop, I would have run out of my internet plan in just downloading the dataset. Analyzing it is a separate challenge in itself.

Even if I would have downloaded the dataset, the only way to do meaning computation through non-cloud way was by buying new machine – not a very practical solution. This is where cloud computing comes in picture!

 

Must Read: Step wise guide to learn R Programming

Why do you need the ‘cloud’?

As discussed in the case study above, cloud is cheaper for handling big data than storage on local desktops, laptops or servers. Wait. Big Data?  Yes! Big Data is an umbrella term that basically denotes data whose Volume and Variety and Velocity is larger than conventional data sources and which requires distributed computing like Hadoop and non-RDBMS storage like NoSQL databases.

 

Must Read: A beginner’s guide to use big data using MongoDB

What is Cloud Computing?

According to the NIST definition of cloud computing,

Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model is composed of five essential characteristics, three service models, and four deployment models.

Cloud Computing consist of 3 components:

  • Infrastructure as a Service(IaaS)
  • Platform as a Service(PaaS)
  • Software as a Service(SaaS)

IaaS– To deploy their applications, cloud users install operating-system images and their application software on the cloud infrastructure. In this model, the cloud user patches and maintains the operating systems and the application software.

PaaS– Cloud providers deliver a computing platform, typically including operating system, programming language execution environment, database, and web server. Application developers can develop and run their software solutions on a cloud platform without the cost and complexity of buying and managing the underlying hardware and software layers

SaaS – In software as a service (SaaS), users are provided access to application software and databases. Cloud providers manage the infrastructure and platforms that run the applications. SaaS is sometimes referred to as “on-demand software”.

pic 1

What are the cost benefit trade-offs of using cloud computing with R versus other applications?

Python is free just like R, but the main reason R scores is that the statistical library of R packages is far more extensive. SAS remains the leading language for corporate analytics on the desktop but it remains expensive for small enterprises and has a significant disadvantage in capital expenditure commitment because of annual license structure instead of one time licence fee.

 

Must Read: A Quick Guide on SAS vs R vs Python

 

What are the advantages of using R on the cloud versus on the desktop?

  1. Since we know R is constrained to handle data only as big as the RAM size, the cloud offers us a quick solution to handle Big Data Science using R. This can be done by simply ramping up the RAM on the virtual machine instance.  You can see the various kinds of RAM options available on the cloud which are simply not affordable  on the local machine.
  2. For big Datasets , it is better to use it on the cloud than to download the dataset, process it and then score it. For example, if you have a competition that uses 30 GB of data, it is best you use it on the cloud. The cloud is thus a great way of learning about Big Data without getting hassled by internet speed.
  3. Cloud has much better bandwidth speed. Therefore, installing software and transferring data is much faster on the cloud.
  4. You can use additional services like AzureML with R on the cloud rather than build your own machine learning service from scratch. You can this tutorial for more information.
  5. Cloud is much more scalable for changes in volume or velocity of data.

Take the Test: Should I become a Data Scientist?

 

How to use R Programming on the cloud?

You can create a instance (a virtual machine that you access remotely) on Amazon Cloud, or on Microsoft Azure or on Google Cloud. You can then simply install R the same way as you use it on your local desktop. You connect to your remote machine through SSH or Remote Desktop.

Here is a step by step process for creating a cloud instance on Amazon Web Services.

Note: Amazon has a free tier that enables you to try out the Amazon cloud for free for 1 year. However this is only for micro instances which have very small RAM and very small disk space. For higher RAM and higher storage you need to pay more. To look at the various instances and their per hour pricing you can see visit here. Basically fees is charged in compute units but this website makes it easy to figure out costs.

cloud computing using R, how to do

First you need to create your Amazon Id. Once you are done, follow the steps below to create a cloud instance on amazon web services:

    1. Login to Amazon Web Services (AWS) Console
    2. Click on Run Instance
    3. Choose operating System for your virtual machine that you will access remotely. Here I have choosen Amazon Linux.
    4. Choose Instance Type (size of RAM  and memory needed). See here to compare prices.
    5. Create a  security key. This is needed for a secure cracker-proof login to the remote machine. Note you can use remote desktop for Windows operating systems but you will need to use SSH for Linux Instances.
    6. Click on Launch  Instance
    7. Connect to the instance using your security key following the instructions given.
    8. Now work on your remote machine just as you would work on a local machine.
    9. Here I am trying to install R                                                                  
    10. Once your work is done- remember to please close the instance lest you incur high monthly bill.

You can choose on demand instances, or even have reserved instances (booking a virtual machine for a fixed period of time and thus at a considerable discount).

 

Take the Test: Should I become a Data Scientist?

 

How to use R on the cloud using RStudio?

RStudio Server edition runs on only Linux. Therefore, we choose  Linux instance on the cloud and then configure R Studio Server. We can then connect to the remote RStudio Server through the browser and use it just the same way.

Here is a step by step way to run RStudio on the cloud.

  • Note- We installed R already using sudo yum install R
  • Download the RStudio server to your virtual machine and then install RStudio Server
$ wget http://download2.rstudio.org/rstudio-server-rhel5-0.99.442-i686.rpm
$ sudo yum install --nogpgcheck rstudio-server-rhel5-0.99.442-i686.rpm
  • You verify the installation
$ sudo rstudio-server verify-installation
  • You open the 8787 port using the security group in AWS Console(left margin Security Groups) by creating a custom TCP rule (click Edit on the tab below)

cloud computing using R, how to do

  • You create a new user with a new password using the SSH terminal for your virtual machine cloud instance
  1. sudo useradd newuser1
  2. sudo passwd newuser1
  • You find the public IP Address of the cloud instance from the Instances tab in the left margin

cloud computing using R, how to do

  • You open your browser to IPAddress: 8787 and then login using the user id and password created above

cloud computing using R, how to do

  • Now you are ready to use R using the cloud through a browser

cloud computing using R, how to do

 

Using R through the Bioconductor cloud?

Bioconductor cloud is an amazing way of kickstarting R on the cloud. You can see the instructions here.

What are the various other cloud computing options?

You can use cloud options from Google and Windows Azure as well. However most of the space is dominated by Amazon Web Services.

Any examples of Using R with Platforms and Other software as a service?

Yes we can use Azure Machine Learning with R on the cloud and also use Google Big Query with R.

Any examples of Big Data using R on the cloud?

Yes there are many examples. Resource 1 and Resource 2.

End Notes

By now, you would have got an overview of how to implement cloud computing using R and R Studio. I really enjoyed writing and curating the useful resources in this article. This article also covers questions which are usually asked by people while learning cloud computing in R. Hence, I have tried to cover all of them in this article. As per my personal experience, I’ve found demonstrating cloud in R is relatively easier as compared to other softwares.

I hope this article helped you to become familiar with cloud computing. We would love to hear about it from you. Did you find it useful? Feel free to post your thoughts through comments below.

If you like what you just read & want to continue your analytics earning, subscribe to our emails, follow us on twitter or like our facebook page.

A
Analytics Vidhya

Analytics Vidhya Content team

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Sanjay
Sanjay

Thanks Ajay for the article. It's precise and helpful.

123

Igor
Igor

Thanks a lot for your post, exactly what I needed for solving the problem - "Error: can not allocate vector (of file) of size ..."!

123

John Mark Agosta
John Mark Agosta

Granted each existing cloud service can bring up VMs on which you can install R, but, for R as a service, in Azure Machine Learning, you can upload your R code directly into a module as part of the dataflow in a project created in Azure Studio. No need to provision and customize VMs. You can have multiple R routines, interwoven with other cloud functions for storing, cleaning, learning and testing with your data in one project, then access your model via an API that is created for you. As disclaimer, I do work for Azure ML. This article would benefit from a bit more breadth about the tools in the cloud that are available for R.

123

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details