InfoQ Homepage Deep Learning Content on InfoQ
-
Georgia Tech Researchers Create Wireless Brain-Machine Interface
Researchers from Georgia Tech University's Center for Human-Centric Interfaces and Engineering have created soft scalp electronics (SSE), a wearable wireless electro-encephalography (EEG) device for reading human brain signals. By processing the EEG data using a neural network, the system allows users wearing the device to control a video game simply by imagining activity.
-
Facebook Open-Sources Computer Vision Model Multiscale Vision Transformers
Facebook AI Research (FAIR) recently open-sourced Multiscale Vision Transformers (MViT), a deep-learning model for computer vision based on the Transformer architecture. MViT contains several internal resolution-reduction stages and outperforms other Transformer vision models while requiring less compute power, achieving new state-of-the-art accuracy on several benchmarks.
-
PyTorch 1.9 Release Includes Mobile, Scientific Computing, and Distributed Training Updates
PyTorch, Facebook's open-source deep-learning framework, announced the release of version 1.9 which includes improvements for scientific computing, mobile support, and distributed training. Overall, the new release contains more than 3,400 commits since the 1.8 release.
-
OpenAI Announces 12 Billion Parameter Code-Generation AI Codex
OpenAI recently announced Codex, an AI model that generates program code from natural language descriptions. Codex is based on the GPT-3 language model and can solve over 70% of the problems in OpenAI's publicly available HumanEval test dataset, compared to 0% for GPT-3.
-
DeepMind Open Sources Data Agnostic Deep Learning Model Perceiver IO
DeepMind has open-sourced Perceiver IO, a general-purpose deep-learning model architecture that can handle many different types of inputs and outputs. Perceiver IO can serve as a "drop-in" replacement for Transformers that performs as well or better than baseline models, but without domain-specific assumptions.
-
MIT Demonstrates Energy-Efficient Optical Accelerator for Deep-Learning Inference
Researchers at MIT's Quantum Photonics Laboratory have developed the Digital Optical Neural Network (DONN), a prototype deep-learning inference accelerator that uses light to transmit activation and weight data. At the cost of a few percentage points of accuracy, the system can achieve an transmission energy advantage of up to 1000x over traditional electronic devices.
-
Baidu's ERNIE 3.0 AI Model Exceeds Human Performance on Language Understanding Benchmark
A research team from Baidu published a paper on the 3.0 version of Enhanced Language RepresentatioN with Informative Entities (ERNIE), a natural language processing (NLP) deep-learning model. The model contains 10B parameters and achieved a new state-of-the-art score on the SuperGLUE benchmark, outperforming the human baseline score.
-
Google Announces 800M Parameter Vision-Language AI Model ALIGN
Google Research announced the development of A Large-scale ImaGe and Noisy-Text Embedding (ALIGN), an 800M-parameter pre-trained deep-learning model trained on a noisy dataset of 1.8B image-text pairs. The model can be used on several downstream tasks and achieves state-of-the-art accuracy on several image-text retrieval benchmarks.
-
EleutherAI Open-Sources Six Billion Parameter GPT-3 Clone GPT-J
A team of researchers from EleutherAI have open-sourced GPT-J, a six-billion parameter natural language processing (NLP) AI model based on GPT-3. The model was trained on an 800GB open-source text dataset and has performance comparable to a GPT-3 model of similar size.
-
Google Integrates TensorFlow Lite with Android, Adds Automatic Acceleration
Google has announced a new mobile ML stack, dubbed Android ML Platform and built around TensorFlow Lite, which aims to solve a number of problems that developers find when using on-device machine learning.
-
Google Open-Sources Token-Free Language Model ByT5
Google Research has open-sourced ByT5, a natural language processing (NLP) AI model that operates on raw bytes instead of abstract tokens. Compared to baseline models, ByT5 is more accurate on several benchmark tasks and is more robust to misspellings and noise.
-
InterCon 2021 Panel Discussion: Is AI Really Beneficial for End Users?
The recent InterCon in Las Vegas featured a panel discussion titled "Is AI Really Beneficial For End Users?" Some key takeaways were that AI brings benefits by increasing productivity and assisting with problem solving, and that there is a need for governance and ethics in AI and a concern over bias in training datasets.
-
InterCon 2021 Keynote: AI Applications in Business
At the recent InterCon in Las Vegas, the opening keynote session featured two talks focused on AI applications in business. The key takeaways were: identifying the business objective is key, access to good data is a major challenge, and AI has great potential for providing value to end customers.
-
Google Trains Two Billion Parameter AI Vision Model
Researchers at Google Brain announced a deep-learning computer vision (CV) model containing two billion parameters. The model was trained on three billion images and achieved 90.45% top-1 accuracy on ImageNet, setting a new state-of-the-art record.
-
Facebook Open-Sources Expire-Span Method for Scaling Transformer AI
Facebook AI Research (FAIR) open-sourced Expire-Span, a deep-learning technique that learns which items in an input sequence should be remembered, reducing the memory and computation requirements for AI. FAIR showed that Transformer models that incorporate Expire-Span can scale to sequences of tens of thousands of items with improved performance compared to previous models.