InfoQ Homepage Deep Learning Content on InfoQ
-
TensorFlow DTensor: Unified API for Distributed Deep Network Training
Recently released TensorFlow v2.9 introduces a new API for the model, data, and space-parallel (aka spatially tiled) deep network training. DTensor aims to decouple sharding directives from the model code by providing higher-level utilities to partition the model and batch parameters between devices.
-
Amazon Releases 51-Language AI Training Dataset MASSIVE
Amazon Alexa AI's Natural Language Understanding group released Multilingual Amazon SLURP (SLU resource package) for Slot Filling, Intent Classification, and Virtual-Assistant Evaluation (MASSIVE), a dataset for training natural language understanding (NLU) AI models that contains one million annotated samples from 51 languages. The release also includes code and tools for using the data.
-
LAION Releases Five Billion Image-Text Pair Dataset LAION-5B
The Large-scale Artificial Intelligence Open Network (LAION) released LAION-5B, an AI training dataset containing over five billion image-text pairs. LAION-5B contains images and captions scraped from the internet and is 14x larger than its predecessor LAION-400M, making it the largest freely available image-text dataset.
-
DeepMind Trains AI Controller for Nuclear Fusion Research Device
Researchers at Google subsidiary DeepMind and the Swiss Plasma Center at EPFL have developed a deep reinforcement learning (RL) AI that creates control algorithms for tokamak devices used in nuclear fusion research. The system learned control policies while interacting with a simulator, and when used to control a real device was able to achieve novel plasma configurations.
-
Serving Deep Networks in Production: Balancing Productivity vs Efficiency Tradeoff
A recently published work provides an alternative modality for serving deep neural networks. It enables utilizing eager-mode model code directly at production workloads by using embedded CPython interpreters. The goal is to reduce the engineering effort to bring the models from the research stage to the end-user and to create a proof-of-concept platform for migrating future numerical libraries.
-
NVIDIA Announces Next Generation AI Hardware H100 GPU and Grace CPU Superchip
At the recent GTC conference, NVIDIA announced their next generation processors for AI computing, the H100 GPU and the Grace CPU Superchip. Based on NVIDIA's Hopper architecture, the H100 includes a Transformer engine for faster training of AI models. The Grace CPU Superchip features 144 Arm cores and outperforms NVIDIA's current dual-CPU offering on the SPECrate 2017_int_base benchmark.
-
Google Trains 540 Billion Parameter AI Language Model PaLM
Google Research recently announced the Pathways Language Model (PaLM), a 540-billion-parameter AI natural language processing (NLP) model that surpasses average human performance on the BIG-bench benchmark. PaLM outperforms other state-of-the-art systems on many evaluation tasks, and shows strong results on tasks such as logical inference and joke explanation.
-
Google Announces AI-Generated Summaries for Google Docs
Google has announced a new feature for their Docs app that will automatically generate a summary of the document content. The summarization is powered by a natural language processing (NLP) AI model based on the Transformer architecture.
-
Ten Lessons from Three Generations of Tensor Processing Units
A recent report published by Google’s TPU group highlights ten takeaways from developing three generations of tensor processing units. The authors also discuss how their previous experience will affect the development of future tensor processing units.
-
Stanford University Publishes AI Index 2022 Annual Report
Stanford University’s Institute for Human-Centered Artificial Intelligence (HAI) has published its 2022 AI Index annual report. The report identifies top trends in AI, including advances in technical achievements, a sharp increase in private investment, and increasing attention on ethical issues.
-
EleutherAI Open-Sources 20 Billion Parameter AI Language Model GPT-NeoX-20B
Researchers from EleutherAI have open-sourced GPT-NeoX-20B, a 20-billion parameter natural language processing (NLP) AI model similar to GPT-3. The model was trained on 825GB of publicly available text data and has performance comparable to similarly-sized GPT-3 models.
-
Meta Announces Conversational AI Model Project CAIRaoke
Meta AI Research recently announced Project CAIRaoke, an end-to-end deep-learning model for digital assistants. Project CAIRaoke is currently being used in Meta's Portal device and outperforms a previous conversational model when evaluated on a reminder task.
-
University of Washington Open-Sources AI Fine-Tuning Algorithm WISE-FT
A team of researchers from University of Washington (UW), Google Brain, and Columbia University have open-sourced weight-space ensembles for fine-tuning (WiSE-FT), an algorithm for fine-tuning AI models that improves robustness under distribution shift. Experiments on several computer vision (CV) benchmarks show that WISE-FT improves accuracy up to 6 percentage points.
-
Allen Institute Launches Updated Embodied AI Challenge
The Allen Institute for AI (AI2) has announced the 2022 version of their AI2-THOR Rearrangement Challenge. The challenge requires competitors to design an autonomous agent that can move objects in a virtual room and includes several improvements including a new dataset and faster training using the latest release of the AI2-THOR simulation platform.
-
Deep Learning Toolkit Intel OpenVINO Extends API, Improves Performance, and More
The latest release of Intel OpenVINO offers a cleaner API, expands support for natural language processing, and improves performance and portability thanks to its new AUTO plugin. InfoQ has spoken with senior director AI Intel OpenVINO Matthew Formica to learn more.