Create a Multilingual Q&A Chatbot Easily with BLOOM’s Free Tools

Introduction

In the realm of natural language processing (NLP) technology, we encounter different models serving distinct purposes. There are both free and paid models available. In the paid section, the OpenAI Library provides a range of models, all of which are grounded in the transformer architecture. Over time, numerous models have been developed based on the transformer, an advanced architecture that has been meticulously trained on extensive datasets to comprehend and generate human-like text. BLOOM, a cutting-edge model built on this transformer architecture, demonstrates exceptional proficiency in tasks such as token classification and question-answering. I created a Q&A model using Gemini Pro and Langchain, underscoring its versatility for developers, researchers, and businesses seeking to leverage the capabilities of state-of-the-art NLP.

BLOOM offers a free model that stands out for its enhanced power and accuracy, owing to its training on large datasets. This makes it an ideal choice for applications like conversational chatbots, where grasping the subtleties of language is paramount. BLOOM not only showcases the latest AI advancements but also empowers users to construct intelligent, context-aware applications that elevate user interactions and redefine the landscape of conversational AI.

Learning Objectives

• Learn about BLOOM and its architecture.
• Understand the token classification task in BLOOM.
• Follow a step-by-step guide on implementing a conversational chatbot using BLOOM.
• Explore the advantages and limitations. 
• Deploy the chatbot application using the Streamlit framework.

This article was published as a part of the Data Science Blogathon.

What is BLOOM?

BLOOM is a Large Language Model (LLM), and a remarkable aspect of its development is the involvement of 1000 researchers spanning over 70 countries, with 250+ institutions participating. This collaborative effort, unprecedented in the field of generative AI, focuses on creating a freely accessible LLM. Unlike other LLMs, the researchers aim to make the model freely available for use by small companies and academic students. Trained on the Jean Zay supercomputer in Paris, BLOOM boasts 176 billion parameters and the ability to generate text in 46 natural languages and 13 programming languages. This release represents a significant stride in making advanced language models accessible to a broader audience, encouraging collaboration, and providing researchers with the opportunity to explore the inner workings of the model.

Key Points:

1. BLOOM is the first multilingual LLM trained transparently, challenging the exclusivity of access to such models.
2. Developed through the largest collaboration in AI research, involving over 1000 researchers from 70+ countries and 250+ institutions.
3. With 176 billion parameters, BLOOM can generate text in 46 natural languages and 13 programming languages.
4. Training on the Jean Zay supercomputer in Paris took 117 days, supported by a €3M compute grant from French research agencies.
5. Researchers can freely download, run, and study BLOOM, promoting openness and responsible AI practices.

Architecture Of BLOOM

BLOOM, a large language model (LLM), is based on the Transformer architecture. In the context of the Transformer, this architecture elevates generative AI, or NLP, to the next level. This architecture follows the decoder-only concept. Unlike the old encoder-decoder architecture, the Transformer employs only the decoder on both sides. ‘Decoder-only’ implies a focus on predicting the next token in a sequence, akin to models like GPT. The model has been trained with over 100 billion parameters, establishing it as a state-of-the-art language model (LLM).

Key Components:

• BLOOM is built on the Transformer architecture, a foundation widely adopted for large language models (LLMs).
• In contrast to the original Transformer’s encoder-decoder design, BLOOM utilizes a causal decoder-only model, demonstrating its effectiveness in transfer learning.
• BLOOM introduces modifications such as ALiBi Positional Embeddings and Embedding LayerNorm, contributing to smoother training, improved downstream performance, and enhanced stability.

Differences Between BLOOM and Other Large Language Models (LLMs)

Steps to Build a Multi-Lang Conversational Chatbot with BLOOM

Prerequisites

• Python 🐍: Blog
• Visual Studio Code (VSCode) 💻: Download
• Streamlit 🚀: It’s a framework for building interactive UIs for testing code.
• PyTorch: Download

Step 1. Environment Preparation

• Python Installation: Ensure Python is installed on your machine. You can download and install Python from the official
• Virtual Environment Creation: Create a virtual environment using venv:

conda create -p ./venv python=3.8 -y

Virtual Environment Activation: Activate the virtual environment:

conda activate ./env

Step 2. Installing Requirements Packages

• First, install the required packages create the requirements.txt file and paste the packages we want.

## Creating requirements.txt
touch requirements.txt

## inside the requirements.text file 
""""
streamlit
notebook
transformers

"""
## Install the packages
pip install -r requirements.txt

• For this library, we require PyTorch as it is a dependency. I will provide the command for PyTorch installation, but I recommend checking the official page for your specific requirements.

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

Step 3. Import Libraries

• Import the necessary libraries in a Jupyter Notebook..

## Importing the libraries
from transformers import AutoTokenizer, AutoModelForCausalLM

Step 4. Initialize Model

• Initialize the BLOOM model and tokenizer.

# Replace with the desired BLOOM model
model_name = "bigscience/bloom-560m"  
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

Step 5. Define Chatbot Functions

• Create functions for initializing the model and generating responses.

def initialize_model(model_name):
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = AutoModelForCausalLM.from_pretrained(model_name)
    return tokenizer, model

def generate_response(model, tokenizer, user_input, max_length=50):
    input_ids = tokenizer.encode(user_input, return_tensors="pt")
    output = model.generate(input_ids, max_length=max_length, num_beams=5, no_repeat_ngram_size=2)
    response = tokenizer.decode(output[0], skip_special_tokens=True)
    return response

Step 6. Create Chatbot Interaction Loop

• Implement an interactive loop where the chatbot takes user input and generates responses.

def chatbot_interaction(model, tokenizer):
    print("Chatbot: Hello! I'm a BLOOM-based chatbot. Type 'exit' to end the conversation.")
    while True:
        user_input = input("You: ")
        if user_input.lower() == 'exit':
            print("Chatbot: Goodbye!")
            break

        response = generate_response(model, tokenizer, user_input)
        print("Chatbot:", response)

Step 7. Run Chatbot Interaction

• Run the chatbot interaction function.

chatbot_interaction(model, tokenizer)

In the output, check the upper red box. In this box, we want to write our query for interacting with the chatbot. The bottom red box shows that the chatbot model has been successfully initialized.

In the red box, the output is printed. We can also print the complete output according to the token creation. For the above, we can mention (max_length=50).

Building the Streamlit App

To create a simple Streamlit application for your BLOOM-based chatbot, follow these steps:

Step 1. Create the main.py file

• For the creating main.py file just open the terminal and paste the following command.

touch main.py

Step 2. Create the main.py file

• Building the interactive Streamlit application: Add the following code into the main.py and witness the magic. You can choose based on your requirements, such as FastAPI or Flask.

# main.py

import streamlit as st
from transformers import AutoTokenizer, AutoModelForCausalLM

# Function to initialize BLOOM model
def initialize_model(model_name):
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = AutoModelForCausalLM.from_pretrained(model_name)
    return tokenizer, model

# Function to generate response
def generate_response(model, tokenizer, user_input, max_length=50):
    input_ids = tokenizer.encode(user_input, return_tensors="pt")
    output = model.generate(input_ids, max_length=max_length, num_beams=5, no_repeat_ngram_size=2)
    response = tokenizer.decode(output[0], skip_special_tokens=True)
    return response

def main():
    st.title("BLOOM Chatbot")

    # Choose BLOOM model
    model_name = st.selectbox("Select BLOOM Model", ["bigscience/bloom-560m", "other/bloom-model"])
    tokenizer, model = initialize_model(model_name)

    st.sidebar.markdown("## Conversation")

    # Interactive chat
    user_input = st.text_input("You:")
    if user_input:
        response = generate_response(model, tokenizer, user_input)
        st.text_area("Chatbot:", value=response, height=100)

if __name__ == "__main__":
    main()

Step 3. Run the Streamlit App

• Open a terminal and navigate to the directory containing the script. Run the following command.

streamlit run streamlit_chatbot.py

• Copy the address into a browser.

Here, we observe the UI of our application. We inquire about machine learning, and the model predicts the correct answer. Note that the output is approximately 50 words, and you can adjust it based on your requirements.

• Check the following output for the Hindi version.

Note: Follow on GitHub for the complete code.

Advantages of BLOOM Conversational Chatbot

• BLOOM excels in 46 natural languages and 13 programming languages, offering a broad spectrum of applications.
• With transparency in its training, BLOOM allows researchers to explore its inner workings, fostering a deeper understanding of large language models.
• Q&A chatbots quickly address common user queries, leading to faster problem resolution and improved user experience.
• Being a free and open-source model, BLOOM is an accurate and powerful choice for applications like conversational chatbots.
• The implementation of chatbots reduces the need for human intervention in routine tasks, resulting in significant cost savings.
• Developed collaboratively by 1000+ researchers from 70+ countries, BLOOM aims to democratize access to advanced language models, benefitting smaller companies and academic researchers.
• BLOOM is not static; it’s the starting point for a family of models that will continuously improve, accommodating more languages and complex architectures.

Challenges of BLOOM Conversational Chatbot

• Sometimes, AI can accidentally be unfair or give wrong information. It’s important to make sure BLOOM behaves sometimes.
• Using and training big language models like BLOOM requires lots of computer power. Not everyone has access to these resources.
• BLOOM might not understand or respond well in all languages or situations. It needs to get better at handling different ways people express themselves.
• Make sure the BLOOM does not have a memory concept like OpenAI models.
• Language is always changing. BLOOM might struggle to understand the latest slang or trends, and it needs updates to stay smart.

Conclusion

BLOOM represents a groundbreaking contribution to generative AI and natural language processing, boasting an impressive 176 billion parameters. This extensive parameter count underscores the model’s ability to process and generate vast amounts of data. Another notable aspect is BLOOM’s provision as a free and open-source model, emphasizing accessibility and collaboration within the AI community.

As a large language model (LLM), BLOOM excels in text-to-text generation, taking on tasks such as causal language modeling, text classification, token classification, and question answering. Our practical section delves into hands-on experiences, guiding users through the process of building with BLOOM, including parameter adjustments like stop word criteria.

Key Takeaways

• BLOOM stands as a groundbreaking contribution to generative AI and natural language processing, featuring an impressive 176 billion parameters.
• From text-to-text generation to tasks like Causal Language Modeling, Text Classification, Token Classification, and Question Answering, BLOOM showcases capabilities for different languages.
• BLOOM’s free tools make it easy for you to build a versatile and multilingual Q&A chatbot effortlessly.

Frequently Asked Questions

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

suyodhanj6

As a Data Scientist, I leverage my expertise in statistical analysis, machine learning, and data visualization to derive insights and make informed decisions. I have experience working with various programming languages, databases, and machine learning frameworks, enabling me to tackle complex data problems and deliver actionable results. I am a collaborative problem-solver who can work with stakeholders to deliver scalable and secure data solutions.