Alibaba’s Neural Network Model Beat the Highest Human Score in Stanford’s Reading Test

Machines getting the better of humans is no longer a surprise. It started with IBM’s Deep Blue program beating Garry Kasparov in a game of chess more than 20 years ago and with the increasing breakthroughs in the world of machine and deep learning, machines continue to become powerful tools.

                                                                       Source: SFGate

Yesterday, Alibaba developed a model that beat out any human competition in Stanford’s reading comprehension competition. The dataset consists of more than 100,000 questions sourced from more than 500 Wikipedia articles. The purpose of the quiz is to see how long it takes the machine learning models to process all the information, train themselves and then provide precise or accurate answers.

Alibaba used a deep learning framework to build a neural network model. It’s based on the “Hierarchical Attention Network”, which according to the company, works by identifying first paragraphs, then sentences and finally words. The underlying technology has been used previously by Alibaba, in it’s AI-powered chatbot – Dian Xiaomi.

Alibaba achieved a score of 82.44, which beat out the human high of 82.304. Microsoft’s AI achieved a score on 82.650. The website lists that Microsoft submitted their model a couple of days before Alibaba but the team evaluating the models, Squad (Stanford Question Answering Dataset), officially released the results of Alibaba’s model first, and Microsoft’s a day later, thus giving Alibaba the unique distinction.

The competition leaderboard published by Squad

Companies like Google, Tencent, IBM and Samsung (among many others) have also participated in the competition but Alibaba became the first to beat the human best score.

Alibaba have mentioned that they will be sharing the model-building framework with the public in the coming weeks.

Our take on this

This just goes to show that machines are now able to answer complex objective questions with remarkable precision. Remember going to museums or historical monuments with a guide? That will be a thing of the past.

Customer service is expected to be fully automated in the next few years and Alibaba hope to lead the drive using their Natural Language Processing lab. The human input required for these tasks will be minimal.

Pranav Dar

Senior Editor at Analytics Vidhya.Data visualization practitioner who loves reading and delving deeper into the data science and machine learning arts. Always looking for new ways to improve processes using ML and AI.