Build Your First Text Classification model using PyTorch

Aravind Pai Last Updated : 06 Nov, 2024

8 min read

Overview

Learn how to perform text classification using PyTorch
Grasp the importance of Pack Padding feature
Understand the key points involved while solving text classification

Introduction

I always turn to State of the Art architectures to make my first submission in data science hackathons. Implementing the State of the Art architectures has become quite easy thanks to deep learning frameworks such as PyTorch, Keras, and TensorFlow. These frameworks provide an easy way to implement complex model architectures and algorithms with least knowledge of concepts and coding skills. In short, it’s a goldmine for the data science community!

In this article, we will use PyTorch, which is well known for its fast computational power. So in this article, we will walk through the key points for solving a text classification problem. And then we will implement our first text classifier in PyTorch!

Note: I highly recommend to go through the below article before moving forward with this article.

A Beginner-Friendly Guide to PyTorch and How it Works from Scratch

1.Why PyTorch for Text Classification?

Dealing with Out of Vocabulary words
Handling Variable Length sequences
Wrappers and Pre-trained models

2.Understanding the Problem Statement
3.Implementation – Text Classification in PyTorch

Why PyTorch for Text Classification?

Before we dive deeper into the technical concepts, let us quickly familiarize ourselves with the framework that we are going to use – PyTorch. The basic unit of PyTorch is Tensor, similar to the “numpy” array in python. There are a number of benefits for using PyTorch but the two most important are:

Dynamic networks – Change in the architecture during the run time
Distributed training across GPUs

I am sure you are wondering – why should we use PyTorch for working with text data? Let us discuss some incredible features of PyTorch that makes it different from other frameworks, especially while working with text data.

1. Dealing with Out of Vocabulary words

A text classification model is trained on fixed vocabulary size. But during inference, we might come across some words which are not present in the vocabulary. These words are known as Out of Vocabulary words. Skipping Out of Vocabulary words can be a critical issue as this results in the loss of information.

In order to handle the Out Of Vocabulary words, PyTorch supports a cool feature that replaces the rare words in our training data with Unknown token. This, in turn, helps us in tackling the problem of Out of Vocabulary words.

Apart from handling Out Of Vocabulary words, PyTorch also has a feature that can handle sequences of variable length!

2. Handling Variable Length sequences

Have you heard of how Recurrent Neural Network is capable of handling variable-length sequences? Ever wondered how to implement it? PyTorch comes with a useful feature ‘Packed Padding sequence‘ that implements Dynamic Recurrent Neural Network.

Padding is a process of adding an extra token called padding token at the beginning or end of the sentence. As the number of the words in each sentence varies, we convert the variable length input sentences into sentences with the same length by adding padding tokens.

Padding is required since most of the frameworks support static networks, i.e. the architecture remains the same throughout the model training. Although padding solves the issue of variable length sequences, there is another problem with this idea – the architectures now process these padding token like any other information/data. Let me explain this through a simple diagram-

As you can see in the diagram (below), the last element, which is a padding token is also used while generating the output. This is taken care of by the Packed Padding sequence in PyTorch.

Packed padding ignores the input timesteps with padding token. These values are never shown to the Recurrent Neural Network which helps us in building a dynamic Recurrent Neural Network.

3. Wrappers and Pretrained models

The state of the art architectures are being launched for PyTorch framework. Hugging Face released Transformers which provides more than 32 state of the art architectures for the Natural Language Understanding Generation!

Not only this, PyTorch also provides pretrained models for several tasks like Text to Speech, Object Detection and so on, which can be executed within few lines of code.

Incredible, isn’t it? These are some really useful features of PyTorch among many others. Let us now use PyTorch for a text classification problem.

Understanding the problem statement

As a part of this article, we are going to work on a really interesting problem.

Quora wants to keep track of insincere questions on their platform so as to make users feel safe while sharing their knowledge. An insincere question in this context is defined as a question intended to make a statement rather than looking for helpful answers. To break this down further, here are some characteristics that can signify that a particular question is insincere:

Has a non-neutral tone
Is disparaging or inflammatory
Isn’t grounded in reality
Uses sexual content (incest, bestiality, pedophilia) for shock value, and not to seek genuine answers

The training data includes the question that was asked, and a flag denoting whether it was identified as insincere (target = 1). The ground-truth labels contain some amount of noise, i.e. they are not guaranteed to be perfect. Our task will be to identify if a given question is ‘insincere’. You can download the dataset for this from here.

It is time to code our own text classification model using PyTorch.

Implementation – Text Classification in PyTorch

Let us first import all the necessary libraries required to build a model. Here is a brief overview of the packages/libraries we are going to use-

Torch package is used to define tensors and mathematical operations on it
TorchText is a Natural Language Processing (NLP) library in PyTorch. This library contains the scripts for preprocessing text and source of few popular NLP datasets.

Python Code:

#deal with tensors
import torch   

#handling text data
from torchtext import data

In order to make the results reproducible, I have specified the seed value. Since Deep Learning model might produce different results each when it is executed due to the randomness in it, it is important to specify the seed value.

	#Reproducing same results
	SEED = 2019

	#Torch
	torch.manual_seed(SEED)

	#Cuda algorithms
	torch.backends.cudnn.deterministic = True

view raw seed.py hosted with ❤ by GitHub

Pre-processsing Data:

Now, let us see how to preprocess the text using field objects. There are 2 different types of field objects – Field and LabelField. Let us quickly understand the difference between the two-

Field: Field object from data module is used to specify preprocessing steps for each column in the dataset.
LabelField: LabelField object is a special case of Field object which is used only for the classification tasks. Its only use is to set the unk_token and sequential to None by default.

Before we use Field, let us look at the different parameters of Field and what are they used for.

Parameters of Field:

Tokenize: specifies the way of tokenizing the sentence i.e. converting sentence to words. I am using spacy tokenizer since it uses novel tokenization algorithm
Lower: converts text to lowercase
batch_first: The first dimension of input and output is always batch size

TEXT = data.Field(tokenize='spacy',batch_first=True,include_lengths=True)
LABEL = data.LabelField(dtype = torch.float,batch_first=True)

Next we are going to create a list of tuples where first value in every tuple contains a column name and second value is a field object defined above. Furthermore we will arrange each tuple in the order of the columns of csv, and also specify as (None,None) to ignore a column from a csv file.

Let us read only required columns – question and label

fields = [(None, None), ('text',TEXT),('label', LABEL)]

In the following code block I have loaded the custom dataset by defining the field objects.

	#loading custom dataset
	training_data=data.TabularDataset(path = 'quora.csv',format = 'csv',fields = fields,skip_header = True)

	#print preprocessed text
	print(vars(training_data.examples[0]))

view raw load.py hosted with ❤ by GitHub

Let us now split the dataset into training and validation data

	import random
	train_data, valid_data = training_data.split(split_ratio=0.7, random_state = random.seed(SEED))

view raw train_test.py hosted with ❤ by GitHub

Preparing input and output sequences:

The next step is to build the vocabulary for the text and convert them into integer sequences. Vocabulary contains the unique words in the entire text. Each unique word is assigned an index. Below are the parameters listed for the same

Parameters:

1. min_freq: Ignores the words in vocabulary which has frequency less than specified one and map it to unknown token.

2. Two special tokens known as unknown and padding will be added to the vocabulary

Unknown token is used to handle Out Of Vocabulary words
Padding token is used to make input sequences of same length

Let us build vocabulary and initialize the words with the pretrained embeddings. Ignore the vectors parameter if you wish to randomly initialize embeddings.

	#initialize glove embeddings
	TEXT.build_vocab(train_data,min_freq=3,vectors = "glove.6B.100d")
	LABEL.build_vocab(train_data)

	#No. of unique tokens in text
	print("Size of TEXT vocabulary:",len(TEXT.vocab))

	#No. of unique tokens in label
	print("Size of LABEL vocabulary:",len(LABEL.vocab))

	#Commonly used words
	print(TEXT.vocab.freqs.most_common(10))

	#Word dictionary
	print(TEXT.vocab.stoi)

view raw gistfile1.py hosted with ❤ by GitHub

Now we will prepare batches for training the model. BucketIterator forms the batches in such a way that a minimum amount of padding is required.

	#check whether cuda is available
	device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

	#set batch size
	BATCH_SIZE = 64

	#Load an iterator
	train_iterator, valid_iterator = data.BucketIterator.splits(
	(train_data, valid_data),
	batch_size = BATCH_SIZE,
	sort_key = lambda x: len(x.text),
	sort_within_batch=True,
	device = device)

view raw iterator.py hosted with ❤ by GitHub

Model Architecture

It is now time to define the architecture to solve the binary classification problem. The nn module from torch is a base model for all the models. This means that every model must be a subclass of the nn module.

I have defined 2 functions here: init as well as forward. Let me explain the use case of both of these functions-

1. Init: Whenever an instance of a class is created, init function is automatically invoked. Hence, it is called as a constructor. The arguments passed to the class are initialized by the constructor.We will define all the layers that we will be using in the model

2. Forward: Forward function defines the forward pass of the inputs.

Finally, let’s understand in detail about the different layers used for building the architecture and their parameters-

Embedding layer: Embeddings are extremely important for any NLP related task since it represents a word in a numerical format. Embedding layer creates a look up table where each row represents an embedding of a word. The embedding layer converts the integer sequence into a dense vector representation. Here are the two most important parameters of the embedding layer –

num_embeddings: No. of unique words in dictionary
embedding_dim: No. of dimensions for representing a word

LSTM: LSTM is a variant of RNN that is capable of capturing long term dependencies. Following the some important parameters of LSTM that you should be familiar with. Given below are the parameters of this layer:

input_size : Dimension of input
hidden_size : Number of hidden nodes
num_layers : Number of layers to be stacked
batch_first : If True, then the input and output tensors are provided as (batch, seq, feature)
dropout: If non-zero, introduces a Dropout layer on the outputs of each LSTM layer except the last layer, with dropout probability equal to dropout. Default: 0
bidirection: If True, introduces a Bi directional LSTM

Linear Layer: Linear layer refers to dense layer. The two important parameters here are described below:

in_features : No. of input features
out_features: No. of hidden nodes

Pack Padding: As already discussed, pack padding is used to define the dynamic recurrent neural network. Without pack padding, the padding inputs are also processed by the rnn and returns the hidden state of the padded element. This an awesome wrapper that does not show the inputs that are padded. It simply ignores the values and returns the hidden state of the non padded element.

Now that we have a good understanding of all the blocks of the architecture, let us go to the code!

I will start with defining all the layers of the architecture:

	import torch.nn as nn

	class classifier(nn.Module):

	#define all the layers used in model
	def __init__(self, vocab_size, embedding_dim, hidden_dim, output_dim, n_layers,
	bidirectional, dropout):

	#Constructor
	super().__init__()

	#embedding layer
	self.embedding = nn.Embedding(vocab_size, embedding_dim)

	#lstm layer
	self.lstm = nn.LSTM(embedding_dim,
	hidden_dim,
	num_layers=n_layers,
	bidirectional=bidirectional,
	dropout=dropout,
	batch_first=True)

	#dense layer
	self.fc = nn.Linear(hidden_dim * 2, output_dim)

	#activation function
	self.act = nn.Sigmoid()

	def forward(self, text, text_lengths):

	#text = [batch size,sent_length]
	embedded = self.embedding(text)
	#embedded = [batch size, sent_len, emb dim]

	#packed sequence
	packed_embedded = nn.utils.rnn.pack_padded_sequence(embedded, text_lengths,batch_first=True)

	packed_output, (hidden, cell) = self.lstm(packed_embedded)
	#hidden = [batch size, num layers * num directions,hid dim]
	#cell = [batch size, num layers * num directions,hid dim]

	#concat the final forward and backward hidden state
	hidden = torch.cat((hidden[-2,:,:], hidden[-1,:,:]), dim = 1)

	#hidden = [batch size, hid dim * num directions]
	dense_outputs=self.fc(hidden)

	#Final activation function
	outputs=self.act(dense_outputs)

	return outputs

view raw architecture.py hosted with ❤ by GitHub

The next step would be to define the hyperparameters and instantiate the model. Here is the code block for the same:

	#define hyperparameters
	size_of_vocab = len(TEXT.vocab)
	embedding_dim = 100
	num_hidden_nodes = 32
	num_output_nodes = 1
	num_layers = 2
	bidirection = True
	dropout = 0.2

	#instantiate the model
	model = classifier(size_of_vocab, embedding_dim, num_hidden_nodes,num_output_nodes, num_layers,
	bidirectional = True, dropout = dropout)

view raw instantiate.py hosted with ❤ by GitHub

Let us look at the model summary and initialize the embedding layer with the pretrained embeddings

	#architecture
	print(model)

	#No. of trianable parameters
	def count_parameters(model):
	return sum(p.numel() for p in model.parameters() if p.requires_grad)

	print(f'The model has {count_parameters(model):,} trainable parameters')

	#Initialize the pretrained embedding
	pretrained_embeddings = TEXT.vocab.vectors
	model.embedding.weight.data.copy_(pretrained_embeddings)

	print(pretrained_embeddings.shape)

view raw arch.py hosted with ❤ by GitHub

Here I have defined the optimizer, loss and metric for the model:

	import torch.optim as optim

	#define optimizer and loss
	optimizer = optim.Adam(model.parameters())
	criterion = nn.BCELoss()

	#define metric
	def binary_accuracy(preds, y):
	#round predictions to the closest integer
	rounded_preds = torch.round(preds)

	correct = (rounded_preds == y).float()
	acc = correct.sum() / len(correct)
	return acc

	#push to cuda if available
	model = model.to(device)
	criterion = criterion.to(device)

view raw optimizers.py hosted with ❤ by GitHub

There are 2 phases while building the model:

Training phase: model.train() sets the model on the training phase and activates the dropout layers.
Inference phase: model.eval() sets the model on the evaluation phase and deactivates the dropout layers.

Here is the code block to define a function for training the model

	def train(model, iterator, optimizer, criterion):

	#initialize every epoch
	epoch_loss = 0
	epoch_acc = 0

	#set the model in training phase
	model.train()

	for batch in iterator:

	#resets the gradients after every batch
	optimizer.zero_grad()

	#retrieve text and no. of words
	text, text_lengths = batch.text

	#convert to 1D tensor
	predictions = model(text, text_lengths).squeeze()

	#compute the loss
	loss = criterion(predictions, batch.label)

	#compute the binary accuracy
	acc = binary_accuracy(predictions, batch.label)

	#backpropage the loss and compute the gradients
	loss.backward()

	#update the weights
	optimizer.step()

	#loss and accuracy
	epoch_loss += loss.item()
	epoch_acc += acc.item()

	return epoch_loss / len(iterator), epoch_acc / len(iterator)

view raw train.py hosted with ❤ by GitHub

So we have a function to train the model, but we will also need a function to evaluate the mode. Let’s do that

	def evaluate(model, iterator, criterion):

	#initialize every epoch
	epoch_loss = 0
	epoch_acc = 0

	#deactivating dropout layers
	model.eval()

	#deactivates autograd
	with torch.no_grad():

	for batch in iterator:

	#retrieve text and no. of words
	text, text_lengths = batch.text

	#convert to 1d tensor
	predictions = model(text, text_lengths).squeeze()

	#compute loss and accuracy
	loss = criterion(predictions, batch.label)
	acc = binary_accuracy(predictions, batch.label)

	#keep track of loss and accuracy
	epoch_loss += loss.item()
	epoch_acc += acc.item()

	return epoch_loss / len(iterator), epoch_acc / len(iterator)

view raw valid.py hosted with ❤ by GitHub

Finally we will train the model for a certain number of epochs and save the best model every epoch.

	N_EPOCHS = 5
	best_valid_loss = float('inf')

	for epoch in range(N_EPOCHS):

	#train the model
	train_loss, train_acc = train(model, train_iterator, optimizer, criterion)

	#evaluate the model
	valid_loss, valid_acc = evaluate(model, valid_iterator, criterion)

	#save the best model
	if valid_loss < best_valid_loss:
	best_valid_loss = valid_loss
	torch.save(model.state_dict(), 'saved_weights.pt')

	print(f'\tTrain Loss: {train_loss:.3f} \| Train Acc: {train_acc*100:.2f}%')
	print(f'\t Val. Loss: {valid_loss:.3f} \| Val. Acc: {valid_acc*100:.2f}%')

view raw run.py hosted with ❤ by GitHub

Let us load the best model and define the inference function that accepts the user defined input and make predictions

	#load weights
	path='/content/saved_weights.pt'
	model.load_state_dict(torch.load(path));
	model.eval();

	#inference
	import spacy
	nlp = spacy.load('en')

	def predict(model, sentence):
	tokenized = [tok.text for tok in nlp.tokenizer(sentence)] #tokenize the sentence
	indexed = [TEXT.vocab.stoi[t] for t in tokenized] #convert to integer sequence
	length = [len(indexed)] #compute no. of words
	tensor = torch.LongTensor(indexed).to(device) #convert to tensor
	tensor = tensor.unsqueeze(1).T #reshape in form of batch,no. of words
	length_tensor = torch.LongTensor(length) #convert to tensor
	prediction = model(tensor, length_tensor) #prediction
	return prediction.item()

view raw inference.py hosted with ❤ by GitHub

Amazing! Let us use this model to make predictions for few questions:

	#make predictions
	predict(model, "Are there any sports that you don't like?")

	#insincere question
	predict(model, "Why Indian girls go crazy about marrying Shri. Rahul Gandhi ji?")

view raw test.py hosted with ❤ by GitHub

End Notes

We have seen how to build our own text classification model in PyTorch and learnt the importance of pack padding. You can play around with the hyper-parameters of the Long Short Term Model such as number of hidden nodes, number of hidden layers and so on to improve the performance even further.

If you have any queries/feedback, leave in the comments section. I will get back to you.

Aravind Pai

Aravind Pai is passionate about building data-driven products for the sports domain. He strongly believes that Sports Analytics is a Game Changer.

Free Courses

4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.

4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.

4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.

4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.

4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

MUID

Used by Microsoft Clarity, to store and track visits across websites.

Expiry: 1 Year

Type: HTTP

_clck

Used by Microsoft Clarity, Persists the Clarity User ID and preferences, unique to that site, on the browser. This ensures that behavior in subsequent visits to the same site will be attributed to the same user ID.

Expiry: 1 Year

Type: HTTP

_clsk

Used by Microsoft Clarity, Connects multiple page views by a user into a single Clarity session recording.

Expiry: 1 Day

Type: HTTP

SRM_I

Collects user data is specifically adapted to the user or device. The user can also be followed outside of the loaded website, creating a picture of the visitor's behavior.

Expiry: 2 Years

Type: HTTP

SM

Use to measure the use of the website for internal analytics

Expiry: 1 Years

Type: HTTP

CLID

The cookie is set by embedded Microsoft Clarity scripts. The purpose of this cookie is for heatmap and session recording.

Expiry: 1 Year

Type: HTTP

SRM_B

Collected user data is specifically adapted to the user or device. The user can also be followed outside of the loaded website, creating a picture of the visitor's behavior.

Expiry: 2 Months

Type: HTTP

_gid

This cookie is installed by Google Analytics. The cookie is used to store information of how visitors use a website and helps in creating an analytics report of how the website is doing. The data collected includes the number of visitors, the source where they have come from, and the pages visited in an anonymous form.

Expiry: 399 Days

Type: HTTP

_ga_#

Used by Google Analytics, to store and count pageviews.

Expiry: 399 Days

Type: HTTP

_gat_#

Used by Google Analytics to collect data on the number of times a user has visited the website as well as dates for the first and most recent visit.

Expiry: 1 Day

Type: HTTP

collect

Used to send data to Google Analytics about the visitor's device and behavior. Tracks the visitor across devices and marketing channels.

Expiry: Session

Type: PIXEL

AEC

cookies ensure that requests within a browsing session are made by the user, and not by other sites.

Expiry: 6 Months

Type: HTTP

G_ENABLED_IDPS

use the cookie when customers want to make a referral from their gmail contacts; it helps auth the gmail account.

Expiry: 2 Years

Type: HTTP

test_cookie

This cookie is set by DoubleClick (which is owned by Google) to determine if the website visitor's browser supports cookies.

Expiry: 1 Year

Type: HTTP

_we_us

this is used to send push notification using webengage.

Expiry: 1 Year

Type: HTTP

WebKlipperAuth

used by webenage to track auth of webenagage.

Expiry: Session

Type: HTTP

ln_or

Linkedin sets this cookie to registers statistical data on users' behavior on the website for internal analytics.

Expiry: 1 Day

Type: HTTP

JSESSIONID

Use to maintain an anonymous user session by the server.

Expiry: 1 Year

Type: HTTP

li_rm

Used as part of the LinkedIn Remember Me feature and is set when a user clicks Remember Me on the device to make it easier for him or her to sign in to that device.

Expiry: 1 Year

Type: HTTP

AnalyticsSyncHistory

Used to store information about the time a sync with the lms_analytics cookie took place for users in the Designated Countries.

Expiry: 6 Months

Type: HTTP

lms_analytics

Used to store information about the time a sync with the AnalyticsSyncHistory cookie took place for users in the Designated Countries.

Expiry: 6 Months

Type: HTTP

liap

Cookie used for Sign-in with Linkedin and/or to allow for the Linkedin follow feature.

Expiry: 6 Months

Type: HTTP

visit

allow for the Linkedin follow feature.

Expiry: 1 Year

Type: HTTP

li_at

often used to identify you, including your name, interests, and previous activity.

Expiry: 2 Months

Type: HTTP

s_plt

Tracks the time that the previous page took to load

Expiry: Session

Type: HTTP

lang

Used to remember a user's language setting to ensure LinkedIn.com displays in the language selected by the user in their settings

Expiry: Session

Type: HTTP

s_tp

Tracks percent of page viewed

Expiry: Session

Type: HTTP

AMCV_14215E3D5995C57C0A495C55%40AdobeOrg

Indicates the start of a session for Adobe Experience Cloud

Expiry: Session

Type: HTTP

s_pltp

Provides page name value (URL) for use by Adobe Analytics

Expiry: Session

Type: HTTP

s_tslv

Used to retain and fetch time since last visit in Adobe Analytics

Expiry: 6 Months

Type: HTTP

li_theme

Remembers a user's display preference/theme setting

Expiry: 6 Months

Type: HTTP

li_theme_set

Remembers which users have updated their display / theme preferences

Expiry: 6 Months

Type: HTTP

Chirag

Getting the following error - NameError: name 'TEXT' is not defined while running - fields = [(None, None), ('text',TEXT),('label', LABEL)] Could you please tell me how to initialise TEXT and LABEL? Great tutorial btw.

Ajay

Thank Aravind for sharing wonderful topic. i m getting error @ fields = [(None, None), ('text',TEXT),('label', LABEL)] NameError: name 'TEXT' / 'LABEL' is not defined

Show 1 reply

Hi, Thanks for pointing it out. I just missed a snippet of code. I have updated it now.

ramesh

you are post good article keep posting

Reading list

Introduction to Deep Learning

Feed Forward Networks

Gradient Descent

Loss Function

Activation Functions

Introduction to Neural networks

Forward and Backward Propagation

Optimizers

Learning Rate Schedulers

NN on Structured Data

Improving the Deep Learning Model

Deep Learning Model Optimization

Unsupervised Deep Learning

AutoDL

Model Deployment

Introduction to PyTorch

Build Your First Text Classification model using PyTorch

Overview

Introduction

Table of Contents

Why PyTorch for Text Classification?

1. Dealing with Out of Vocabulary words

2. Handling Variable Length sequences

3. Wrappers and Pretrained models

Understanding the problem statement

Implementation – Text Classification in PyTorch

End Notes

Login to continue reading and enjoy expert-curated content.

Free Courses

Generative AI - A Way of Life

Getting Started with Large Language Models

Building LLM Applications using Prompt Engineering

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Microsoft Excel: Formulas & Functions

Recommended Articles

Responses From Readers

Write for us

Analytics Vidhya (4)

brahmaid

csrftoken

Identityid

sessionid

Google (1)

g_state

Microsoft (7)

MUID

_clck

_clsk

SRM_I

SM

CLID

SRM_B

Google (7)

_gid

_ga_#

_gat_#

collect

AEC

G_ENABLED_IDPS

test_cookie

Webengage (2)

_we_us

WebKlipperAuth

LinkedIn (16)

ln_or

JSESSIONID

li_rm

AnalyticsSyncHistory

lms_analytics

liap

visit

li_at

s_plt

lang

s_tp

AMCV_14215E3D5995C57C0A495C55%40AdobeOrg

s_pltp

s_tslv

li_theme