InfoQ Homepage Neural Networks Content on InfoQ
-
Microsoft Unveils VALL-E, a Game-Changing TTS Language Model
Microsoft has introduced VALL-E, a novel language model method for text-to-speech synthesis (TTS) that employs audio codec codes as intermediate representations and can replicate anyone's voice after listening to just three seconds of audio recording.
-
Deep Learning Pioneer Geoffrey Hinton Publishes New Deep Learning Algorithm
Geoffrey Hinton, professor at the University of Toronto and engineering fellow at Google Brain, recently published a paper on the Forward-Forward algorithm (FF), a technique for training neural networks that uses two forward passes of data through the network, instead of backpropagation, to update the model weights.
-
Researchers Publish Survey of Algorithmically-Efficient Deep Learning
Researchers from Lawrence Livermore National Laboratory and MosaicML have published a survey of over 200 papers on algorithmically-efficient deep learning. The survey includes a taxonomy of methods to speed up training as well as a practitioner's guide for mitigating training bottlenecks.
-
Meta's CICERO AI Wins Online Diplomacy Tournament
Meta AI Research recently open-sourced CICERO, an AI that can beat most humans at the strategy game Diplomacy, a game that requires coordinating plans with other players. CICERO combines chatbot-like dialogue capabilities with a strategic reasoning, and recently placed first in an online Diplomacy tournament against human players.
-
PyTorch Becomes Linux Foundation Top-Level Project
PyTorch, the popular deep-learning framework developed by Meta AI Research, has now become an independent top-level project of the Linux Foundation. The project will be managed by the newly-chartered PyTorch Foundation, with support from several large companies including Meta, AWS, NVIDIA, AMD, Google, and Microsoft.
-
University Researchers Publish Results of NLP Community Metasurvey
Researchers from New York University, University of Washington, and Johns Hopkins University have published the results of the NLP Community Metasurvey, which compiles the opinions of 480 active NLP researchers about several issues in the natural language processing AI field. The survey also includes meta-questions about the perceived opinions of other researchers.
-
Transformers Can Mock Part of Human Brain
In recent years, neuroscientists have tried many types of neural networks to model the firing of neurons in the human brain. In a recent project, two researchers Whittington and Behrens found that the hippocampus, a structure of the brain critical to memory, works as a particular kind of artificial neural network called transformer.
-
OpenAI Releases 1.6 Billion Parameter Multilingual Speech Recognition AI Whisper
OpenAI recently released Whisper, a 1.6 billion parameter AI model that can transcribe and translate speech audio from 97 different languages. Whisper was trained on 680,000 hours of audio data collected from the web and shows robust zero-shot performance on a wide range of automated speech recognition (ASR) tasks.
-
MIT Researchers Develop AI Model to Solve University-Level Mathematics Problems
Researchers at MIT have developed an AI model that can solve problems used in university-level mathematics courses. The system uses the OpenAI Codex engine to generate programs that output the problem solution, including graphs and plots, achieving an accuracy of 81% on the MATH benchmark dataset as well as on real problems from MIT courses.
-
Stability AI Open-Sources Image Generation Model Stable Diffusion
Stability AI released the pre-trained model weights for Stable Diffusion, a text-to-image AI model, to the general public. Given a text prompt, Stable Diffusion can generate photorealistic 512x512 pixel images depicting the scene described in the prompt.
-
Berkeley Researchers Announce Robot Training Algorithm DayDreamer
Researchers from University of California, Berkeley, recently announced DayDreamer, a reinforcement-learning (RL) AI algorithm that uses a world model, which allows it to learn more quickly without the need for interacting with a simulator. Using DayDreamer, the team was able to train several physical robots to perform complex tasks within only a few hours.
-
Amazon's AlexaTM 20B Model Outperforms GPT-3 on NLP Benchmarks
Researchers at Amazon Alexa AI have announced Alexa Teacher Models (AlexaTM 20B), a 20-billion-parameter sequence-to-sequence (seq2seq) language model that exhibits state-of-the-art performance on 1-shot and few-shot NLP tasks. AlexaTM 20B outperforms GPT-3 on SuperGLUE and SQuADv2 benchmarks while having fewer than 1/8 the number of parameters.
-
Meta Develops Dataset Pruning Technique for Scaling AI Training
Researchers from Meta AI and Stanford University have developed a metric for pruning AI datasets which improves training scalability from a power-law to exponential-decay. The metric uses self-supervised learning and performs comparably to existing metrics which require more compute power.
-
Meta's Genomics AI ESMFold Predicts Protein Structure 6x Faster Than AlphaFold2
Meta AI Research recently announced ESMFold, an AI model for predicting protein structure from a sequence of genes. ESMFold is built on a 15B parameter Transform model and achieves accuracy comparable to other state-of-the-art models with an order-of-magnitude inference time speedup.
-
PrefixRL: Nvidia's Deep-Reinforcement-Learning Approach to Design Better Circuits
Nvidia has developed PrefixRL, an approach based on reinforcement learning (RL) to designing parallel-prefix circuits that are smaller and faster than those designed by state-of-the-art electronic-design-automation (EDA) tools.