InfoQ Homepage Deep Learning Content on InfoQ
-
University Researchers Publish Results of NLP Community Metasurvey
Researchers from New York University, University of Washington, and Johns Hopkins University have published the results of the NLP Community Metasurvey, which compiles the opinions of 480 active NLP researchers about several issues in the natural language processing AI field. The survey also includes meta-questions about the perceived opinions of other researchers.
-
OpenAI Releases 1.6 Billion Parameter Multilingual Speech Recognition AI Whisper
OpenAI recently released Whisper, a 1.6 billion parameter AI model that can transcribe and translate speech audio from 97 different languages. Whisper was trained on 680,000 hours of audio data collected from the web and shows robust zero-shot performance on a wide range of automated speech recognition (ASR) tasks.
-
Microsoft Trains Two Billion Parameter Vision-Language AI Model BEiT-3
Researchers from Microsoft's Natural Language Computing (NLC) group announced the latest version of Bidirectional Encoder representation from Image Transformers: BEiT-3, a 1.9B parameter vision-language AI model. BEiT-3 models images as another language and achieves state-of-the-art performance on a wide range of downstream tasks.
-
Google Open-Sources Natural Language Robot Control Method SayCan
Researchers from Google's Robotics team have open-sourced SayCan, a robot control method that uses a large language model (LLM) to plan a sequence of robotic actions to achieve a user-specified goal. In experiments, SayCan generated the correct action sequence 84% of the time.
-
Amazon SageMaker Provides New Built-in TensorFlow Image Classification Algorithms
Amazon is announcing a new built-in TensorFlow algorithm for image classification in Amazon Sagemaker. The supervised learning algorithm supports transfer learning for many pre-trained models available in TensorFlow Hub.
-
MIT Researchers Develop AI Model to Solve University-Level Mathematics Problems
Researchers at MIT have developed an AI model that can solve problems used in university-level mathematics courses. The system uses the OpenAI Codex engine to generate programs that output the problem solution, including graphs and plots, achieving an accuracy of 81% on the MATH benchmark dataset as well as on real problems from MIT courses.
-
Near-Optimal Scaling of Large Deep Network Training on Public Cloud
A recently published study, MiCS, provides experimental evidence that the infrastructure used to carry out model training should be taken into account, especially for large deep neural networks trained on the public cloud. The article shows distributing the model weights unevenly between GPUs decreases inter-node communication overhead on AWS V100 and A100 instances.
-
Stability AI Open-Sources Image Generation Model Stable Diffusion
Stability AI released the pre-trained model weights for Stable Diffusion, a text-to-image AI model, to the general public. Given a text prompt, Stable Diffusion can generate photorealistic 512x512 pixel images depicting the scene described in the prompt.
-
AWS Deep Graph Knowledge Embedding for Bond Trading Predictions
AWS developed the Deep Graph Knowledge Embedding Library (DGL-KE), a knowledge graph embedding library built on the Deep Graph Library (DGL). DGL is a scalable, high performance Python library for deep learning in graphs. This library is used by the advanced machine learning systems developed with Trumid to build a credit trading platform.
-
Meta Open-Sources 175B Parameter Chatbot BlenderBot 3
Meta AI Research open-sourced BlenderBot 3, a 175B parameter chatbot that can learn from live interactions with users "in the wild." In evaluations by human judges, BlenderBot 3 achieves a 31% rating increase compared to the previous BlenderBot version.
-
Berkeley Researchers Announce Robot Training Algorithm DayDreamer
Researchers from University of California, Berkeley, recently announced DayDreamer, a reinforcement-learning (RL) AI algorithm that uses a world model, which allows it to learn more quickly without the need for interacting with a simulator. Using DayDreamer, the team was able to train several physical robots to perform complex tasks within only a few hours.
-
Amazon's AlexaTM 20B Model Outperforms GPT-3 on NLP Benchmarks
Researchers at Amazon Alexa AI have announced Alexa Teacher Models (AlexaTM 20B), a 20-billion-parameter sequence-to-sequence (seq2seq) language model that exhibits state-of-the-art performance on 1-shot and few-shot NLP tasks. AlexaTM 20B outperforms GPT-3 on SuperGLUE and SQuADv2 benchmarks while having fewer than 1/8 the number of parameters.
-
Meta Develops Dataset Pruning Technique for Scaling AI Training
Researchers from Meta AI and Stanford University have developed a metric for pruning AI datasets which improves training scalability from a power-law to exponential-decay. The metric uses self-supervised learning and performs comparably to existing metrics which require more compute power.
-
Machine Learning Systems Vulnerable to Specific Attacks
The growing number of organizations creating and deploying machine learning solutions raises concerns as to their intrinsic security, argues the NCC Group in a recent whitepaper (Practical Attacks on Machine Learning Systems).
-
Meta's Genomics AI ESMFold Predicts Protein Structure 6x Faster Than AlphaFold2
Meta AI Research recently announced ESMFold, an AI model for predicting protein structure from a sequence of genes. ESMFold is built on a 15B parameter Transform model and achieves accuracy comparable to other state-of-the-art models with an order-of-magnitude inference time speedup.