Why is Microsoft's New Machine Reading Comprehension Dataset a Big Deal?

However, despite the latest advancement of artificial intelligence in the domains of natural language understanding and speech recognition, conversational systems backed by machine learning algorithms do not yet rival their human counterparts in either cognitive capabilities (IQ) or emotional intelligence (EQ). One challenge is the difficulty for machines to infer the true meaning from a span of words or sound bites. Consider the following tweet-sized passage: "Caffeine is found in almost every over-the-counter fat-burning supplement commercially available today. You could burn more fat.” As human readers, we immediately sense the “positive vibe” in the text, which describes one of the benefits of caffeine. Until recently, however, most machine reading comprehension algorithms would pick up a predominantly negative sentiment, because the text sample, on the surface level, includes negative words such as “fat” and “burn.” Another challenge stems from the fact that many knowledge-based questions are ambiguous in nature and require multi-perspective answers. Is caffeine good or bad? How about cholesterol? Are gun control laws effective? As humans, we tend to answer “umm… it depends.” In the domain of machine reading comprehension, however, synthesizing answers from multiple perspectives has proven to be an ever-daunting task. First comes the “pollution” by low-quality content—especially against the backdrop of today’s clickbait practices in digital content creation. Then, we run into the risk of a surface-level interpretation instead of a deep semantic inference. Last, but not least, we need a reliable ranking algorithm to prioritize, merge, and synthesize the answers from multiple perspectives. Conversational AI & Open Datasets Despite the challenges, AI researchers across the globe have been closing the gap in both IQ and EQ between human and machine-learned conversational agents, largely thanks to the advancement of multi-layer artificial neural networks in the AI domain. For example, the Facebook AI Research (FAIR) group has trained a conversational bot to successfully negotiate deals with humans. Microsoft’s AI and Search division has been tackling automated question answering by leveraging deep neural networks (DNNs), and recently exceeded human performance on a key dimension of the Stanford Question Answering Test (an industry standard). Below is a multi-perspective synthesis if you ask, “is cholesterol good?” on the Bing search engine today.