Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

What is NLP?Applications of NLP
Understanding Text Pre-processingTokenization in NLPByte Pair EncodingTokenizer Free Language Modeling with PixelsStopword RemovalStemming vs LemmatizationText Mining
Spacy TutorialsGensim Tutorials
What are Regular Expressions?Regular Expressions
String Similarity
Spelling Correction
Introduction to Topic ModelingLatent Dirichlet Allocation (LDA)Implement Topic Modeling
Introduction to Feature Engineering for Text DataImplement Text Feature Engineering TechniquesIntroduction to One Hot EncodingImplement One Hot EncodingLimitations of One Hot EncodingCount Vectorizer and TF-IDFSolving Text classification using TF-IDF
Information Retrieval System Explained in Simple terms!How does Google Rank Search Results?Knowledge Graph
Understanding Word2VecUnderstanding Skip Gram and Continous Bag Of WordsWord2Vec Implementation in GensimVisualizing Word2Vec
Word Senses and Word Sense Ambiguity
Why Are We Interested in Syntatic Strucure?What is a Dependency Grammar?Neural Dependency Parsing
Introduction to Language ModelsN-Gram Language ModelsNeural Language models
Why Sequence models?Usecases of Sequence modelsIntroduction to RNNImplement RNN
Shortcomings of RNNWhat is Long Short Term Memory (LSTM)Implementing LSTMBuild Your Own Fake News Classification ModelWhat is Gated Recurrent Unit (GRU)?Implementing GRU
Introduction to Machine TranslationMultilingualism in NLPDrawbacks of Seq2Seq modelMathematical Calculation of Attention
Understand Positional EncodingIntroducing Transformers ModelKey Query Value Attention in Tranformer Encoder
Pretrained Language Models in NLPGenerative Pre-training (GPT) for Natural Language Understanding(NLU)Finetuning GPT-2Understanding BERTFinetune Masked language Modeling in BERTImplement Text Classification using BERTFinetuning BERT for NERExtensions of BERT: Roberta, Spanbert, ALBERMobileBERTGPT-3Prompt Engineering in GPT-3BigbirdT5 and large language models
Implement Question Answering on SQUAD
Text Summarization
Named Entity Recognition (NER) in Python with Spacy
Visualizing Sounds Using Librosa Machine Learning Library!Audio ProcessingAudio AnalysisAudio Classification using Deep Learning
Automatic Speech RecognitionImplement Automatic Speech RecognitionCan Voice Conversion Improve ASR in Low-Resource Settings?
Audio Separation
Building ChatbotsBuilding Chatbots using Rasa
Automate NLP Tasks using EvalML Library

New Anaphora and Co-reference Resolution Technique for Biographies

P
Pearl Last Updated : 27 Jul, 2021
5 min read

This article was published as a part of the Data Science Blogathon

Introduction

Biographies of many famous personalities are very insightful and inspiring. Although, one may not want to read the whole document. In order to just get the important points from the biography, one can generate a summary of the biography. The summary is generated by giving weights to all the words. Sometimes, anaphoras can be predicted by the machine as a separate word which in return produces a less accurate result. For example, Jai is playing tennis. He plays very well. Here, Jai is taken as one word and he as another in the machine by semantic analysis when both he and Jai are referring to the same person.

Another scenario could be that one is reading a biography and they leave it in between. When they revisit the document, they do not remember who he refers to. Thus, they have no choice but to read the above lines again to understand the references. Anaphora resolution saves time for individuals and is also used in pre-processing these biographies.

Anaphora and Co-reference Resolution 2

source: https://meta-guide.com/wp-content/uploads/2020/06/storygenerator19.jpg

Concept

The reason for limiting the document input to biographies is that biographies focus on named entities, i.e real-world objects. Biographies consist of the name of the person whose biography it is, where he/she was born or where he/she moved to and where did he/she study or work. These refer to named entities name, location, and organization respectively.

If we talk about essays or text documents in general, there may or may not exist named entities in them. For example, there could be a story about a boy where the boy’s name is never disclosed. Anaphora resolution, in this case, requires the building of a corpus to identify the type of noun and then referring the pronoun attached to it. Using named entity recognition requires no pre-built corpus as whenever a named entity is identified, it is stored in the local storage and along with its subcategory, and when the pronoun of that category is read, it is automatically assigned to the named entity recognized.

For example, ‘Jai is playing tennis. He plays very well. Here Jai is identified as a named entity, it is stored in local storage as ‘Jai, Person’ where the person is the type of entity. When the next line is read, he is identified as a pronoun that is assigned to a person. After checking the local storage, he will be assigned to Jai.

Anaphora and Co-reference Resolution concept

source: https://www.adl.org/sites/default/files/styles/open_graph_image_1200_x_628_/public/2021-01/pronouns-in-english-language-istock-modified.jpg?itok=_3GxxCPw

There are many types of named entities but the ones used in biographies are person names, organization names, and location names. All these three can be identified using Stanford NER which can be downloaded from the link given below:

https://nlp.stanford.edu/software/CRF-NER.html

Code

The python code for anaphora resolution and co-reference resolution of biographies is mentioned below. It can be used directly after changing the path of the Stanford NER file downloaded if required.

from nltk.tokenize import word_tokenize, sent_tokenize
from nltk.tag import StanfordNERTagger
st = StanfordNERTagger('/Users/Downloads/stanford-ner-2020-11-17/classifiers/english.all.3class.distsim.crf.ser.gz',
'/Users/Downloads/stanford-ner-2020-11-17/stanford-ner.jar',
encoding='utf-8')
text_string='''
Sir Jagadish Chandra Bose was a scientist of many talents. He was born on 30 November, 1858 in Bikrampur, West Bengal. He was a polymath, physicist, biologist, botanist and archaeologist. Bose pioneered the study of radio and microwave optics. He joined the Presidency College of the University of Calcutta as a professor of physics. There, despite racial discrimination and a lack of funding and equipment, Bose carried on his scientific research. He made important contributions to the study of plants. He has laid the foundation of experimental science in the Indian sub-continent. He was the first person to use semiconductor junctions to detect radio signals. What’s more, he is also probably the father of open technology, as he made his inventions and work freely available for others to further develop and his reluctance for patenting his work is legendary.
Another of his well known inventions is the crescograph. He measured plant response to various stimuli and hypothesized that plants can feel pain, understand affection etc. At the time when Bose was a student at Cambridge, Prafulla Chandra Roy was a student at Edinburgh. They met in London and became intimate friends.
While most of us are aware of his scientific prowess, we might not be aware of his talent as an early writer of science fiction! He is in fact considered the father of Bengali science fiction.
'''
tokenized_text = sent_tokenize(text_string)
tokenized_tags_list=[]
new_text_string=''
anaphora_p_sing=['he','she','his','her','He','She','His','Her']
anaphora_p_plural=['they','their','They','Their']
anaphora_l=['this place','that place','the place','there,','there.','This place','That place','The place','There']
person_list_last=[]
person_list_sentence=[]
org_list_last=[]
org_list_sentence=[]
loc_list_last=[]
loc_list_sentence=[]
coreference_list=[]
coreference_sol=[]
flag=0
flag_org=0
flag_loc=0
for i in range(0,len(tokenized_text)):
print(tokenized_text[i])
tokenized_text1 = word_tokenize(tokenized_text[i])
classified_text = st.tag(tokenized_text1)
for j in range(0,len(tokenized_text1)):
a=tokenized_text1[j]
b=classified_text[j][1]
if b=='PERSON':
if flag==0:
if classified_text[j][0] in coreference_list:
for k in range(0,len(coreference_list)):
if coreference_list[k]==classified_text[j][0]:
if coreference_sol[k] in person_list_sentence:
pass
else:
person_list_sentence.append(coreference_sol[k])
break
else:
try:
if classified_text[j+1][1]=='PERSON':
flag=1
full_name=classified_text[j][0]+' '
else:
if classified_text[j][0] in person_list_sentence:
pass
else:
person_list_sentence.append(classified_text[j][0])
except:
if classified_text[j][0] in person_list_sentence:
pass
else:
person_list_sentence.append(classified_text[j][0])
else:
full_name+=classified_text[j][0]+' '
try:
if classified_text[j+1][1]=='PERSON':
flag=1
else:
flag=0
full_name=full_name[0:len(full_name)-1]
if full_name in person_list_sentence:
pass
else:
person_list_sentence.append(full_name)
if full_name not in coreference_sol:
individual=word_tokenize(full_name)
for k in individual:
coreference_list.append(k)
coreference_sol.append(full_name)
except:
flag=0
full_name=full_name[0:len(full_name)-1]
person_list_sentence.append(full_name)
individual=word_tokenize(full_name)
for k in individual:
coreference_list.append(k)
coreference_sol.append(full_name)
elif b=='ORGANIZATION':
if flag_org==0:
if classified_text[j][0] in coreference_list:
for k in range(0,len(coreference_list)):
if coreference_list[k]==classified_text[j][0]:
if coreference_sol[k] in org_list_sentence:
pass
else:
org_list_sentence.append(coreference_sol[k])
break
try:
if classified_text[j+1][1]=='ORGANIZATION':
flag_org=1
full_name=classified_text[j][0]+' '
else:
if classified_text[j][0] in org_list_sentence:
pass
else:
org_list_sentence.append(classified_text[j][0])
except:
if classified_text[j][0] in org_list_sentence:
pass
else:
org_list_sentence.append(classified_text[j][0])
else:
full_name+=classified_text[j][0]+' '
try:
if classified_text[j+1][1]=='ORGANIZATION':
flag_org=1
else:
flag_org=0
full_name=full_name[0:len(full_name)-1]
if full_name in org_list_sentence:
pass
else:
org_list_sentence.append(full_name)
if full_name not in coreference_sol:
individual=word_tokenize(full_name)
for k in individual:
coreference_list.append(k)
coreference_sol.append(full_name)
except:
flag_org=0
full_name=full_name[0:len(full_name)-1]
org_list_sentence.append(full_name)
individual=word_tokenize(full_name)
for k in individual:
coreference_list.append(k)
coreference_sol.append(full_name)
elif b=='LOCATION':
if flag_loc==0:
if classified_text[j][0] in coreference_list:
for k in range(0,len(coreference_list)):
if coreference_list[k]==classified_text[j][0]:
if coreference_sol[k] in loc_list_sentence:
pass
else:
loc_list_sentence.append(coreference_sol[k])
break
try:
if classified_text[j+1][1]=='LOCATION':
flag_loc=1
full_name=classified_text[j][0]+' '
else:
if classified_text[j][0] in loc_list_sentence:
pass
else:
loc_list_sentence.append(classified_text[j][0])
except:
if classified_text[j][0] in org_list_sentence:
pass
else:
loc_list_sentence.append(classified_text[j][0])
else:
full_name+=classified_text[j][0]+' '
try:
if classified_text[j+1][1]=='LOCATION':
flag_loc=1
else:
flag_loc=0
full_name=full_name[0:len(full_name)-1]
if full_name in loc_list_sentence:
pass
else:
loc_list_sentence.append(full_name)
if full_name not in coreference_sol:
individual=word_tokenize(full_name)
for k in individual:
coreference_list.append(k)
coreference_sol.append(full_name)
except:
flag_loc=0
full_name=full_name[0:len(full_name)-1]
loc_list_sentence.append(full_name)
individual=word_tokenize(full_name)
for k in individual:
coreference_list.append(k)
coreference_sol.append(full_name)
else:
pass
a=[]
for l in anaphora_p_sing:
if l in tokenized_text1:
if len(person_list_sentence)!=0:
if person_list_sentence[len(person_list_sentence)-1] not in a:
a.append(person_list_sentence[len(person_list_sentence)-1])
else:
try:
a.append(person_list_last[0])
if person_list_last[0] not in person_list_sentence:
person_list_sentence.append(person_list_last[0])
except:
pass
for l in anaphora_p_plural:
if l in tokenized_text1:
if len(person_list_sentence)>1:
for m in person_list_sentence:
if m not in a:
a.append(m)
elif len(person_list_sentence)==1:
if person_list_sentence[0] not in a:
a.append(person_list_sentence[0])
try:
for n in person_list_last:
if n not in a:
a.append(n)
person_list_sentence.append(n)
except:
pass
else:
try:
for n in person_list_last:
if n not in a:
a.append(n)
person_list_sentence.append(n)
except:
pass
for l in anaphora_l:
if l in tokenized_text1 and ',' in tokenized_text1:
if len(loc_list_sentence)!=0:
if loc_list_sentence[len(loc_list_sentence)-1] not in a:
a.append(loc_list_sentence[len(loc_list_sentence)-1])
else:
try:
a.append(loc_list_last[0])
if loc_list_last[0] not in loc_list_sentence:
loc_list_sentence.append(loc_list_last[0])
except:
pass
if len(org_list_sentence)!=0:
if org_list_sentence[len(org_list_sentence)-1] not in a:
a.append(org_list_sentence[len(org_list_sentence)-1])
else:
try:
a.append(org_list_last[0])
if org_list_last[0] not in org_list_sentence:
org_list_sentence.append(org_list_last[0])
except:
pass
person_list_last=person_list_sentence
person_list_sentence=[]
org_list_last=org_list_sentence
org_list_sentence=[]
loc_list_last=person_list_sentence
loc_list_sentence=[]
print('Anaphora resolved:')
print(a)
print('\n')
view raw anaphora_resolution.py hosted with ❤ by GitHub

Description

The code above creates two lists for each entity (name, organization, location). The lists refer to the named entities mentioned in the previous and current sentences respectively. These lists are maintained because sometimes the anaphora refers to the named entity mentioned in the previous line. For example, Jai is playing and he loved to play. Here, he refers to Jai which is mentioned in the current sentence. But the same sentence can be written as: Jai is playing. He loved to play. Here also, he refers to Jai but Jai is mentioned in the previous sentence.

Two other lists are maintained for a co-reference list and co-reference solutions. Co-reference resolution can be explained with the help of an example. Mary Lee is referred to as Mary sometimes and Lee sometimes. Identifying that Mary and Lee refer to the same person is co-reference resolution. Thus, if the sentence is: Lee loves to read. She is a bookworm. Here, if we run this sentence through Stanford NER tagger, it will store Lee as a person and when the machine reads the pronoun she, it resolves it to Lee.

If the co-reference resolution is included in the code, the machine will resolve she to Mary Lee, which is much better for understanding. Thus the lists stored for this purpose as per our code are [‘Mary’, ‘Lee’] and [‘Mary Lee’, ‘Mary Lee’] which implies that Mary is resolved to Mary Lee and Lee is also resolved to Mary Lee so that the machine is able to interpret that we are talking about the same person.

A hardcoded list for many possible pronouns is created in the code. If an element of the list occurs in the sentence, it is resolved. A special case used in the code is the word ‘There’. There is only resolved if it has a comma after in the sentence. This is to avoid errors as there can have two meanings:

1.  There is no doubt that he is a great man.

2. He moved to Delhi. There, he learned music.

The ‘there’ in the second case needs to be resolved it refers to a location. The text string taken from the biography is also hardcoded. It can be changed according to requirements.

 

Output

The code output is shared below in the form of screenshots. It can be seen that ‘he’, ‘his’ has been resolved to Jagadish Chandra Bose in all sentences. In many places, the name Jagadish Chandra Bose was referred to as Bose in the text string, it has been resolved to its full name while generating the desired output.  ‘There has been resolved to an organization. ‘They’ has been resolved to Jagadish Chandra Bose and his friend. An empty list is returned where nothing had to be resolved.

output 1
output 2

Conclusion

The algorithm used in the code above gives better accuracy with respect to many algorithms used for anaphora resolution. For example, Jai is eating. Mary Joins him. They enjoy dinner. Here, using the Lapin and Lease algorithm, They will be resolved as null when it refers to Jai and Mary. This algorithm has been designed in a way such that it is best suited for biographies as mostly named entities are used and facts are provided.

Author name: Pearl Miglani

Contact info: miglanipearl@gmail.com

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.

P
Pearl

Login to continue reading and enjoy expert-curated content.

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details