Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

What is NLP? Applications of NLP
Understanding Text Pre-processing Tokenization in NLP Byte Pair Encoding Tokenizer Free Language Modeling with Pixels Stopword Removal Stemming vs Lemmatization Text Mining
Spacy Tutorials Gensim Tutorials
What are Regular Expressions? Regular Expressions
String Similarity
Spelling Correction
Introduction to Topic Modeling Latent Dirichlet Allocation (LDA) Implement Topic Modeling
Introduction to Feature Engineering for Text Data Implement Text Feature Engineering Techniques Introduction to One Hot Encoding Implement One Hot Encoding Limitations of One Hot Encoding Count Vectorizer and TF-IDF Solving Text classification using TF-IDF
Information Retrieval System Explained in Simple terms! How does Google Rank Search Results? Knowledge Graph
Understanding Word2Vec Understanding Skip Gram and Continous Bag Of Words Word2Vec Implementation in Gensim Visualizing Word2Vec
Word Senses and Word Sense Ambiguity
Why Are We Interested in Syntatic Strucure? What is a Dependency Grammar? Neural Dependency Parsing
Introduction to Language Models N-Gram Language Models Neural Language models
Why Sequence models? Usecases of Sequence models Introduction to RNN Implement RNN
Shortcomings of RNN What is Long Short Term Memory (LSTM) Implementing LSTM Build Your Own Fake News Classification Model What is Gated Recurrent Unit (GRU)? Implementing GRU
Introduction to Machine Translation Multilingualism in NLP Drawbacks of Seq2Seq model Mathematical Calculation of Attention
Understand Positional Encoding Introducing Transformers Model Key Query Value Attention in Tranformer Encoder
Pretrained Language Models in NLP Generative Pre-training (GPT) for Natural Language Understanding(NLU) Finetuning GPT-2 Understanding BERT Finetune Masked language Modeling in BERT Implement Text Classification using BERT Finetuning BERT for NER Extensions of BERT: Roberta, Spanbert, ALBER MobileBERT GPT-3 Prompt Engineering in GPT-3 Bigbird T5 and large language models
Implement Question Answering on SQUAD
Text Summarization
Named Entity Recognition (NER) in Python with Spacy
Coreference Resolution
Visualizing Sounds Using Librosa Machine Learning Library! Audio Processing Audio Analysis Audio Classification using Deep Learning
Automatic Speech Recognition Implement Automatic Speech Recognition Can Voice Conversion Improve ASR in Low-Resource Settings?
Audio Separation
Building Chatbots Building Chatbots using Rasa
Automate NLP Tasks using EvalML Library

Web Scraping Using RPA Tool UiPath!

Gyan
Gyan Last Updated : 04 Feb, 2021
5 min read

This article was published as a part of the Data Science Blogathon.

The World is rapidly moving towards AI, So it’s better to go with the flow. This line represents the adaptation of technology in the real world to get the results better & faster.

INTRODUCTION

Web Scraping, Web Data Extraction & Web Harvesting is the collection of data from the web. These days everything & everyone needs data to run. Data is the most precious gem to run any organization & the most challenging part is to collect or gather quality data. Finding the data is good; extracting it even better; doing it using automation is perfect.

What is UiPath?

UiPath is an RPA Tool. But wait – what is RPA?

What is RPA?

Quoting from UIPath’s site:

Robotic Process Automation is the technology that allows anyone today to configure computer software. Or in other words, it is a “robot” to emulate and integrate the actions of a human interacting within digital systems to execute a business process. RPA robots utilize the user interface to capture data and manipulate applications just like humans do. They interpret, trigger responses, and communicate with other systems to perform a vast variety of repetitive tasks.

Only substantially better: an RPA software robot never sleeps and makes zero mistakes.

Experiential-Session

Performed on Versions

UiPath – 20.4.3

Let’s perform web scraping using UiPath. Just check the website for the data you want to scrape and check the list of parent and child HTML tags for better understanding.

 

Steps to follow to do Web Scrape

  • Select the Website and the Data
  • Create a Project in your desired directory
  • Create a Flowchart file for Web scraping flow design
  • Design the Flow
  • Run the Automation flow
  • Open the Excel file & Cross-check the Scraped Data

 

Step 1- Select the Website and the Data

I selected this website “https://www.bullion-rates.com/gold/INR/2007-1-history.htm” and want to scrape data of gold rates along with dates.

Web Scraping UiPath data

Source

 

Step 2- Create a Project in your desired directory

Provide the Name, path & short description of your project.

Web Scraping UiPath create project

Source

 

Step 3- Create a Flowchart file

Now create a flowchart file to design your Web scraping flow.

Web Scraping UiPath create flowchart

Source

 

 

Step 3- Design the Flow

a) Choose the open browser from the activity pane

Web Scraping UiPath design the flow

Source

b) Set the Property of Open Browse

i) Choose browser type as Chrome

ii) Set the URL- Insert your URL within quotes here Data To Scrape

iii) Choose New Session as True

Choose New Session as True

Source

iv) Add a delay activity with a duration of 6 sec in the format of 00:00:06, so that the page will load properly, there is another option as well but for now, I’m using the delay option.

 

Add a delay activity

Source

v) Choose the Data Scraping option

a) Select Element option is coming

b) Select the Next option

Next option

Source

                                                                   

c) Now the Element selector highlighter will come, so select the element. Once the elements are selected, you can see the preview data. If the data are coming as expected select the finish button else re-select the data.

Element selector highlighter

Source

d) Now a pop-up box appears which asks for multiple pages scraping, so if you want to do multiple pages scraping then select Yes & choose the element which will redirect you to the next page. In today’s case, I want to scrape one page only so I’m using the No option.

multiple pages

Source

e) Data Scraping activity will appear in the flow design. Select the Extract Structured Data ‘TABLE dtDGrid’ activity & you can notice two things in properties

i) Max number of results default is 100, you can change it as per the records on the page.
ii) In the output section, you can see the Data Table variable is Extract Data Table.
Data Table variable is Extract Data Table.

Source

f) Now, we have to write the scraped data in excel format. So we use the Write Range activity.

i) 1st field is for the path of the excel sheet, provide it as per your excel sheet location.
ii) 2nd field is for sheet name & Cell name, provide the sheet name in quotes & remove the cell name. So that it will create the sheet & write the whole data.
iii) The Last field is for a variable name, in my case variable name is ExtractDataTable.

Source

 

Step 4-Source Run the Automation flow

Click on the run option or press ctrl +f6 to run the automation flow.

Step 5- Open the Excel file & Cross-check the Scraped Data

Source

 

Conclusion

I tried to explain Web Scraping using RPA Tool UiPath in a very simple way, Hope this will help you.

Find full code on- GitHub

If you have any questions about the code or web scraping in general, reach out to me on

Connect to Gyan on Linkedin

We will meet again with something new.

Till then,

Happy Coding..!

 

The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion.

Gyan
Gyan

Data Manipulator | Data Modeller | Data Scientist | Tech Writer | Lifelong learner | Analysising the world through Technology and Data | I don’t write to impress. I write to inform, entertain, inspire. | It's time to Review, Consume and Create. Cut the mustard for your success. |
Let’s connect! | https://www.linkedin.com/in/gyan-vardhan-data-scientist/ | Thanks for your time!

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details