NLP Applications in Support Call Centers

This article was published as a part of the Data Science Blogathon.

Introduction

This article is in continuation of my previous article on using Machine learning in Support environments. I shared my views on, how using simple python code we can enrich our call centers/support division activities in our own organization or customer organization. In that article, I shared an insight into what and how we can make a difference to the current environment using ML in giving better service time in resolution of customer calls and avoiding repetition of the same tickets being raised by multiple users of the same customer with slight variation thus adding the ticket backlog count.

Python dictionaries were used to create very useful historical support resolution logs, which would be very handy for L1/L2 ( Level1, Level2) guys who are constantly engaged in firefighting mode and ensuring the SLA is not breached and escalations don’t flow to the next level. You may refer to my previous blog here.

It is generally seen that a considerable amount of time is spent by support staff on resolving the ticket that has occurred in the past, due to different engineers who work in call centers 24×7 and they may not know of resolution already exist or will take some time to extract the same from tools like Jira/ Seibel, etc. Also if we can automate the ticket prioritization it will bring down the current downtime of creating the priority for the ticket and improve the overall process.

In this blog, I will share some more ideas of how we can use NLP in our organization’s Support activities which will help the support staff and also improve the overall process flow to the extent that there will be marked changes visible than how it was happening previously. This blog is divided into 3 Sections/ Topics  as below :

Section1:

Nature/ Criticality of Tickets:  In any supported kind of environment, it is very important to know the kind of tickets we are getting from the production environment and the reason for the same. Here I have shared few simple but very effective analytical reports that can be taken from simple commands in python.

Section 2:

The priority of the support tickets. Usually, the Priorities are set as P1, P2, P3, P4 calls where P1, P2 needs to be addressed immediately as it impacts the business operation. P3, P4 are software bugs that need to be fixed within a specified time, though they are not affecting the operation, so the support staff have some time, say 2-3 days for a typical P3 call, and 1 week for a P4 call. In this section, I have shared how we can use ML to automate the priorities of any support tickets from the past historical corpus and also bring down the time 15-30 min (if any tools like Jira, etc are used ) to 2-3 mins (by using ML to automate it before an engineer can start the ticket analysis.

 Section 3:

Identifying Similar Tickets from the corpus: Are the same tickets being raised from multiple customers with slight variations?  This is very much possible as multiple customers are using our products. In this section, I will tell you how we can check the most similar tickets from the historical corpus, which will help in providing quick resolution time if resolution already existed from the older ticket and there won’t be a need to duplicate the analysis. Even if the tickets were not the same but very similar, an engineer would on the fly get the idea of how to go about the analysis and save some time there also. So, let us go to the topics now.

Topic 1: Few useful but simple analytic/statistical reports

The above chart can be generated with simple python commands but very useful. It depicts the current support backlogs (from a set of all P3, P4’s only) by priority. Next

This one tells Who all are assigned the issues and count of open issues under each Engineer. Likewise some more charts can be taken out (Issue_Resolution_category: {Software bugs: (99),User Error :20, Performance :15,Not Reproducible : 4…….} , Software Issues : {Java Servlets :  (55),Environmental (11), Log growth : (3)…..}  etc.

Topic 2: Automatic Ticket Prioritization

Snapshot of the dataset

data.columns

Output-

Index (['Summary', 'Issue key', 'Issue id', 'Issue Type', 'Status',
       'Project key', 'Project name', 'Project type', 'Project lead',
       'Project description',’Description’,’Priority’,
       ...
       'Comment1', 'Comment2', 'Comment3'], dtype='object', length=341)

The columns I will be using for this article are ‘Summary’, ‘Description’, and ‘Priority’ columns alone for now. Let’s first check the distribution of the Summary and Description.

From the above, it can be seen most of the Summary have a length of 4-8 words and Description length is between 50-65 words.

I have a ticket corpus of past historical data. Using this, we will solve the multiclass classification problem of predicting the priority, given the ticket. In this article, the focus of this particular blog is more on the application than the actual concepts itself. However, I will be highlighting all the key areas and touching the concepts around it.  I will be demonstrating TfidfVectorizer but for the deployable version, we need to make the model more robust and try different pre-trained models like W2Vec or Glove also

Data.shape

Output

(1000, 341)

df = data[['Summary','Description','Priority']]

Output

Sample rows from the data frame

Creation of X, y for the Model:

Once we are done with all the pre-processing steps (that is, impute missing values if any and clean the data using NLTK or Spacy, we will have 2 additional columns created as Summary clean and description clean. Now, create one newer column in the Dataframe as ‘Tickets’ will be an amalgamation of Summary and description as below

 df['Ticket'] = df['clean_Summary'] + ' ' + df ['clean Description']

Label Encode the target (Priority column values in this case)

le = LabelEncoder()
df['Priority'] = le.fit_transform(df['Priority'].values)

Below is the code snippet for the ML model using TfidfVectorizer

….

from sklearn.preprocessing import LabelEncoder
from sklearn.feature_extraction.text import CountVectorizer,TfidfVectorizer
tf = TfidfVectorizer(min_df=5,max_df=0.9)  # remove terms that are less frequent or too frequent
tf.fit(df['Ticket'].values)
td_vec = tf.transform(df['Ticket'].values)

….

target = np.array(df.Priority)
from sklearn.model_selection import train_test_split
X_train,X_val,y_train,y_val = train_test_split(td_vec,target,test_size=0.25,random_state=42,shuffle=True)
((750, 1064), (250, 1064))

…..

Finally the model

from sklearn import naive_bayes
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

 # Naive bayes model
model = naive_bayes.MultinomialNB()
model.fit(X_train,y_train)
nbp = model.predict(X_val)
accuracy_score(y_val,nbp)

0.556

model = LogisticRegression()
model.fit(X_train,y_train)
nbp = model.predict(X_val)
accuracy_score(y_val,nbp)

0.58

Now, how to improve the accuracy score further?  There are quite a few techniques we can try. I will highlight the first one in this article.

1. Text data Augmentation
2. Create a New Weight feature. (High Priority issues will have more weightage than Medium Priority issue etc)
3. Upsampling to balance the target class

Text Data Augmentation

Here we simply randomly shuffle the tokens in each sentence to create a new one. Do remember the context may change, so we need to find some optimal value for newer records being created. Since in my case the average ticket description length is between 50-65 tokens, I have created 5 new issues out of each original issue corpus. (1000 -> 5000). The code snippet below-

# simple text augmentation
from nltk import word_tokenize
nltk.download('punkt')
import random

def augment(sentence,tvalue,n):
    new_sentences = []
    target = []
    words = word_tokenize(sentence)
    for i in range(n):
        random.shuffle(words)
        new_sentences.append(' '.join(words))
        target.append(tvalue)

    new_sentences = list(set(new_sentences))
    return new_sentences,target

After the text data augmentation model accuracy improved as below

# Naive Bayes model
0.81

# Logistic Regression model
0.93
pred = model.predict(X_val)
pred

array([0, 3, 0, ..., 3, 0, 0])
le.inverse_transform(pred)
array(['Critical', 'Medium', 'Critical', ..., 'Medium', 'Critical',
       'Critical'], dtype=object)

Lastly, we can try a few more models like Random forest, XGBoost, LightGBM, etc, and compare the results with other models.

Now to the last section of this blog

Topic 3: Identify most identical issues from the Ticket corpus

This helps in knowing in advance before starting the root cause analysis of any ticket if there were any similar tickets reported and if yes what was the resolution provided for that ticket. Consider the below two ticket descriptions.

'Last Challan number fed in the system is not being validated',
'Last Scroll number fed in the system is not being validated'

Both the above look quite similar and the most probable resolution for both very similar.

Let us see how we can use NLP to detect these types of similarities.

I will be demonstrating with gensim Doc2Vec which is basically an extension of W2Vec but here instead of vectorizing the tokens, we can do paragraph embedding. But I will use this to do sentence comparison. First, we need to create a past ticket corpus (refer to my previous blog post on ‘Modernizing Support logs using simple Python Commands’). Code snippet below

Ticket  = pd.read_csv(filepath',encoding='ISO-8859-1')
islst = ticketlist['Issue'].to_list()     # List of ticket old corpus
def _MSS(most_similar):              # function to output the most similar ticket from the corpus
    for label, index in [('MOST', 0)]:
                   print(u'%s %s: %s\n' % (label, most_similar[index][1], data[int(most_similar[index][0])]))
tagged data = [TaggedDocument(words=word_tokenize(_d.lower()), tags=[str(i)]) for i, _d in enumerate(islst)]

max_epochs = 2000    # Hyperparameters. We can experiment with these until we get optimal output
vec_size = 100            # Embedding size
alpha = 0.025           # lr – Initial learning rate
model = Doc2Vec(size=vec_size,  alpha=alpha,  min_alpha=0.00025,  min_count=1,  dm =1)  # distributed memory’ (PV-DM) instead of distributed bag of words (PV-DBOW)
model.build_vocab(tagged_data)
for epoch in range(max_epochs):  # train the model
    print('iteration {0}'.format(epoch))
    model.train(tagged_data,
                total_examples=model.corpus_count,
                epochs=model.iter)
    # decrease the learning rate
    model.alpha -= 0.00001
    # fix the learning rate, no decay
    model.min_alpha = model.alpha
model. Save("d2v.model")
model= Doc2Vec.load("d2v.model")
test_data = word_tokenize('Last Challan number in the system is not correct'.lower())   # Test the model with new ticket no  
v1 = model.infer_vector(test_data)
_MSS(similar_doc)   # finally # to print most similar sentences from the corpus along with probabilistic score
MOST 0.32075077295303345: Last Scroll number fed in the system is not being validated

Few more validation
test_data = word_tokenize('sales tax field to capture PAN/GIR should be 16 digits'.lower())
MOST 0.6367178559303284: In the sales tax scheme of Tax collection the field meant for capturing the details of PAN/GIR no /Reg. No? others are allowed to enter 14 digits only. Whereas space needs to be increased to 16 digits.

test_data = word_tokenize('In case of Subscription received column in Transaction Maintenance instead of SB A/c No. , it should be Operative A/C No.'.lower())

MOST 0.7423350214958191: In DA Master Maintenance, the system allows us to define DA  %age less than 100%.  If the DA %age is given more than 100 then the message comes:  the DA percent should be less than 100.  The provision for defining  DA %age as per applicable rates, which may be more than 100, be provided.

After identifying the most similar ticket from the history, support staff may get the resolution provided for that ticket from the resolution log dictionary with ease.

That’s it for now.

Thanks

PATTABHIRAMAN