InfoQ Homepage Natural Language Processing Content on InfoQ
-
OpenAI Announces GPT-3 AI Language Model with 175 Billion Parameters
A team of researchers from OpenAI recently published a paper describing GPT-3, a deep-learning model for natural-language with 175 billion parameters, 100x more than the previous version, GPT-2. The model is pre-trained on nearly half a trillion words and achieves state-of-the-art performance on several NLP benchmarks without fine-tuning.
-
Google Open-Sources AI for Using Tabular Data to Answer Natural Language Questions
Google open-sourced Table Parser (TAPAS), a deep-learning system that can answer natural-language questions from tabular data. TAPAS was trained on 6.2 million tables extracted from Wikipedia and matches or exceeds state-of-the-art performance on several benchmarks.
-
Google AI Launches NLU-Powered Tool to Help Explore COVID-19 Literature
Google AI launched COVID-19 Research Explorer, which provides a semantic search interface on top of the COVID-19 Open Research Dataset to help scientists and researchers efficiently analyze all of the dataset’s journal articles and preprints.
-
Blender, Facebook State-of-the-Art Human-Like Chatbot, Now Open Source
Blender is an open-domain chatbot developed at Facebook AI Research (FAIR), Facebook’s AI and machine learning division. According to FAIR, it is the first chatbot that has learned to blend several conversation skills, including the ability to show empathy and discuss nearly any topic, beating Google's chatbot in tests with human evaluators.
-
OpenAI Approximates Scaling Laws for Neural Language Models
Artificial intelligence company OpenAI studies empirical scaling laws for language models using cross entropy loss to determine the optimal allocation of a fixed compute budget.
-
Microsoft and Google Release New Benchmarks for Cross-Language AI Tasks
Research teams at Microsoft Research and Google AI have announced new benchmarks for cross-language natural-language understanding (NLU) tasks for AI systems, including named-entity recognition and question answering. Google's XTREME covers 40 languages and includes nine tasks, while Microsoft's XGLUE covers 27 languages and eleven tasks.
-
ExBERT, a Tool for Exploring Learned Representations in NLP Models
MIT-IBM AI Labs and Harvard NLP Group have released a live demo of their interactive visualization tool for exploring learned representations in Transformers models called exBERT, along with a pre-publication and the source-code.
-
Stanford NLP Group Releases Stanza: a Python NLP Toolkit
The Stanford NLP Group recently released Stanza, a new python natural language processing toolkit. Stanza features both a language-agnostic fully neural pipeline for text analysis (supporting 66 human languages), and a Python interface to the Java Stanford CoreNLP software.
-
Google Open-Sources Reformer Efficient Deep-Learning Model
Researchers from Google AI recently open-sourced the Reformer, a more efficient version of the Transformer deep-learning model. Using a hashing trick for attention calculation and reversible residual layers, the Reformer can handle text sequences up to 1 million words while consuming only 16GB of memory on a single GPU accelerator.
-
Microsoft Open-Sources ONNX Acceleration for BERT AI Model
Microsoft's Azure Machine Learning team recently open-sourced their contribution to the ONNX Runtime library for improving the performance of the natural language processing (NLP) model BERT. With the optimizations, the model's inference latency on the SQUAD benchmark sped up 17x.
-
Google Open-Sources ALBERT Natural Language Model
Google AI has open-source A Lite Bert (ALBERT), a deep-learning natural language processing (NLP) model, which uses 89% fewer parameters than the state-of-the-art BERT model, with little loss of accuracy. The model can also be scaled-up to achieve new state-of-the-art performance on NLP benchmarks.
-
Uber Open-Sources Plug-and-Play Language Model for Controlling AI-Generated Text
Uber AI open-sourced the plug-and-play language model (PPLM) which can control the topic and sentiment of AI-generated text. The model's output is evaluated by human judges as achieving 36% better topic accuracy compared to the baseline GPT-2 model.
-
Google Cloud Team Releases AutoML Natural Language
The Google Cloud team recently announced the generally available (GA) release of AutoML Natural Language framework. AutoML Natural Language supports features for data processing and common machine learning tasks like classification, sentiment analysis, and entity extraction.
-
Microsoft Releases DialogGPT AI Conversation Model
Microsoft Research's Natural Language Processing Group released dialogue generative pre-trained transformer (DialoGPT), a pre-trained deep-learning natural language processing (NLP) model for automatic conversation response generation. The model was trained on over 147M dialogues and achieves state-of-the-art results on several benchmarks.
-
Google Applies NLP Algorithm BERT to Search
BERT, Google's latest NLP algorithm, will power Google search and make it better at understanding user queries in a way more similar to how humans would understand them, writes Pandu Nayak, Google fellow and vice president for Search, with one in 10 queries providing a different set of results.