InfoQ Homepage Neural Networks Content on InfoQ
-
Meta Open-Sources 200 Language Translation AI NLLB-200
Meta AI recently open-sourced NLLB-200, an AI model that can translate between any of over 200 languages. NLB-200 is a 54.5B parameter Mixture of Experts (MoE) model that was trained on a dataset containing more than 18 billion sentence pairs. On benchmark evaluations, NLLB-200 outperforms other state-of-the-art models by up to 44%.
-
Ant Group Open Sources Privacy-Preserving Computation Framework
Alibaba financial arm Ant Group has open sourced SecretFlow, its privacy-preserving framework, with a specific focus on data analysis and machine learning.
-
BigScience Releases 176B Parameter AI Language Model BLOOM
The BigScience research workshop released BigScience Large Open-science Open-access Multilingual Language Model (BLOOM), an autoregressive language model based on the GPT-3 architecture. BLOOM is trained on data from 46 natural languages and 13 programming languages and is the largest publicly available open multilingual model.
-
Meta Hopes to Increase Accuracy of Wikipedia with New AI Model
Meta AI's research and advancements team developed a neural-network-based system, called SIDE, that is capable of scanning hundreds of thousands of Wikipedia citations at once and checking whether they truly support the corresponding contents. Wikipedia is a multilingual free online encyclopedia written and maintained by volunteers through open collaboration and a wiki-based editing system.
-
Google's Image-Text AI LIMoE Outperforms CLIP on ImageNet Benchmark
Researchers at Google Brain recently trained Language-Image Mixture of Experts (LIMoE), a 5.6B parameter image-text AI model. In zero-shot learning experiments on ImageNet, LIMoE outperforms CLIP and performs comparably to state-of-the-art models while using fewer compute resources.
-
Adobe Researchers Open-Source Image Captioning AI CLIP-S
Researchers from Adobe and the University of North Carolina (UNC) have open-sourced CLIP-S, an image-captioning AI model that produces fine-grained descriptions of images. In evaluations with captions generated by other models, human judges preferred those generated by CLIP-S a majority of the time.
-
Stanford University Open-Sources Controllable Generative Language AI Diffusion-LM
Researchers at Stanford University have open-sourced Diffusion-LM, a non-autoregressive generative language model that allows for fine-grained control of the model's output text. When evaluated on controlled text generation tasks, Diffusion-LM outperforms existing methods.
-
Google's New Imagen AI Outperforms DALL-E on Text-to-Image Generation Benchmarks
Researchers from Google's Brain Team have announced Imagen, a text-to-image AI model that can generate photorealistic images of a scene given a textual description. Imagen outperforms DALL-E 2 on the COCO benchmark, and unlike many similar models, is pre-trained only on text data.
-
Meta Open-Sources 175 Billion Parameter AI Language Model OPT
Meta AI Research released Open Pre-trained Transformer (OPT-175B), a 175B parameter AI language model. The model was trained on a dataset containing 180B tokens and exhibits performance comparable with GPT-3, while only requiring 1/7th GPT-3's training carbon footprint.
-
New GraphWorld Tool Accelerates Graph Neural-Network Benchmarking
Google AI has recently released GraphWorld, a tool to accelerate performance benchmarking in the area of graph neural networks (GNNs). GraphWorld is a configurable framework to generate graphs with a variety of structural properties like different node degree distributions and Gini index.
-
Google Trains 540 Billion Parameter AI Language Model PaLM
Google Research recently announced the Pathways Language Model (PaLM), a 540-billion-parameter AI natural language processing (NLP) model that surpasses average human performance on the BIG-bench benchmark. PaLM outperforms other state-of-the-art systems on many evaluation tasks, and shows strong results on tasks such as logical inference and joke explanation.
-
Stanford University Publishes AI Index 2022 Annual Report
Stanford University’s Institute for Human-Centered Artificial Intelligence (HAI) has published its 2022 AI Index annual report. The report identifies top trends in AI, including advances in technical achievements, a sharp increase in private investment, and increasing attention on ethical issues.
-
EleutherAI Open-Sources 20 Billion Parameter AI Language Model GPT-NeoX-20B
Researchers from EleutherAI have open-sourced GPT-NeoX-20B, a 20-billion parameter natural language processing (NLP) AI model similar to GPT-3. The model was trained on 825GB of publicly available text data and has performance comparable to similarly-sized GPT-3 models.
-
University of Washington Open-Sources AI Fine-Tuning Algorithm WISE-FT
A team of researchers from University of Washington (UW), Google Brain, and Columbia University have open-sourced weight-space ensembles for fine-tuning (WiSE-FT), an algorithm for fine-tuning AI models that improves robustness under distribution shift. Experiments on several computer vision (CV) benchmarks show that WISE-FT improves accuracy up to 6 percentage points.
-
University Researchers Investigate Machine Learning Compute Trends
A team of researchers from University of Aberdeen, MIT, and several other institutions have released a dataset of historical compute demands for machine learning (ML) models. The dataset contains the compute required for training 123 important models, and an analysis shows that since the year 2010 the trend has significantly increased.