InfoQ Homepage Neural Networks Content on InfoQ
-
Google Open-Sources Reformer Efficient Deep-Learning Model
Researchers from Google AI recently open-sourced the Reformer, a more efficient version of the Transformer deep-learning model. Using a hashing trick for attention calculation and reversible residual layers, the Reformer can handle text sequences up to 1 million words while consuming only 16GB of memory on a single GPU accelerator.
-
Apple Acquires Edge-Focused AI Startup Xnor.ai
Apple has acquired Xnor.ai, a Seattle-based startup that builds AI models that run on edge devices, for approximately $200 million.
-
Uber's Synthetic Training Data Speeds Up Deep Learning by 9x
Uber AI Labs has developed an algorithm called Generative Teaching Networks (GTN) that produces synthetic training data for neural networks which allows the networks to be trained faster than when using real data. Using this synthetic data, Uber sped up its neural architecture search (NAS) deep-learning optimization process by 9x.
-
Uber Open-Sources Plug-and-Play Language Model for Controlling AI-Generated Text
Uber AI open-sourced the plug-and-play language model (PPLM) which can control the topic and sentiment of AI-generated text. The model's output is evaluated by human judges as achieving 36% better topic accuracy compared to the baseline GPT-2 model.
-
Deep-Learning Framework SINGA Graduates to Top-Level Apache Project
The Apache Software Foundation (ASF) recently announced that SINGA, a framework for distributed deep-learning, has graduated to top-level project (TLP) status, signifying the project's maturity and stability. SINGA has already been adopted by companies in several sectors, including banking and healthcare.
-
PyTorch 1.3 Release Adds Support for Mobile, Privacy, and Transparency
Facebook recently announced the release of PyTorch 1.3. The latest version of the open-source deep learning framework includes new tools for mobile, quantization, privacy, and transparency.
-
Dropbox Predicts What File You Need Next with Content-Specific ML Pipelines
The Dropbox machine learning team shared how the company improved the model behind their content suggestions feature. The enhancements allow Dropbox to deal with different types of content, incorporate folder suggestions into the existing file suggestions model and handle cloud-based documents resulting from relatively recent partnerships.
-
Google Applies NLP Algorithm BERT to Search
BERT, Google's latest NLP algorithm, will power Google search and make it better at understanding user queries in a way more similar to how humans would understand them, writes Pandu Nayak, Google fellow and vice president for Search, with one in 10 queries providing a different set of results.
-
Alexa Research Paper Shows Genetic Algorithms Offer Best Solution for Neural Network Optimization
Amazon's Alexa Science researchers published a paper providing a theoretical basis for neural network optimization. While showing that it is computationally intractable to find a perfect solution, the paper does provide a formulation, the Approximate Architecture Search Problem (a-ASP), that can be solved with genetic algorithms.
-
Facebook Open-Sources CraftAssist Framework for AI Assistants in Minecraft
Facebook AI researchers open-sourced CraftAssist, a framework for building interactive assistants for the Minecraft video game. The bots use natural language understanding (NLU) to parse and execute text commands from human players, such as requests to build houses in the game world. The framework's modular structure can be extended by researchers to perform their own ML experiments.
-
Google Research Use of Concept Vectors for Image Search
Google recently released research about creating a tool for searching Similar Medical Images Like Yours (SMILY). The research uses embeddings for image-based search and allows users to influence the search through the interactive refinement of concepts.
-
New Technique Speeds up Deep-Learning Inference on TensorFlow by 2x
Researchers at North Carolina State University recently presented a paper at the International Conference on Supercomputing (ICS) on their new technique, "deep reuse" (DR), that can speed up inference time for deep-learning neural networks running on TensorFlow by up to 2x, with almost no loss of accuracy.
-
Researchers Develop Technique for Reducing Deep-Learning Model Sizes for Internet of Things
Researchers from Arm Limited and Princeton University have developed a technique that produces deep-learning computer-vision models for internet-of-things (IoT) hardware systems with as little as 2KB of RAM. By using Bayesian optimization and network pruning, the team is able to reduce the size of image recognition models while still achieving state-of-the-art accuracy.
-
Google Releases Post-Training Integer Quantization for TensorFlow Lite
Google announced new tooling for their TensorFlow Lite deep-learning framework that reduces the size of models and latency of inference. The tool converts a trained model's weights from floating-point representation to 8-bit signed integers. This reduces the memory requirements of the model and allows it to run on hardware without floating-point accelerators and without sacrificing model quality.
-
Google Uses Mannequin Challenge Videos to Learn Depth Perception
Google AI Research published a paper describing their work on depth perception from two-dimensional images. Using a training dataset created from YouTube videos of the Mannequin Challenge, researchers trained a neural network that can reconstruct depth information from videos of moving people, taken by moving cameras.