At the recent International Conference on Learning Representations (ICLR), research teams from several tech companies, including Google, Microsoft, IBM, Facebook, and Amazon, presented nearly 250 papers out of a total of 860 on a wide variety of AI topics related to deep learning.
The conference was held online in early May and featured a "round-the-clock" program of live talks and Q&A sessions, in addition to pre-recorded videos for all accepted papers. Each day of the four-day conference featured two Invited Talks from leading deep-learning researchers. Although most of the papers were from academia, many prominent tech companies were well represented by their AI researchers: Google contributed over 100 papers, including several winning Outstanding Paper awards, Microsoft 53, IBM 35, Facebook 23, Salesforce 7, and Amazon 4.
The ICLR conference is the "premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence called representation learning," also known as deep learning. This year's conference, the 9th meeting, was a virtual event, as was last year's, due to the COVID-19 pandemic; the conference organizers noted that this year's event "went quite smoothly" because the organization learned a lot from hosting last year's virtual event. This year's conference had 3,014 submissions, compared to 2,604 in 2020, and 6,194 registrations from more than 60 countries, up from 5,622 last year. During the week-long conference, the livestreams received 16,624 views. Besides paper presentations and workshops, the conference featured ad-hoc virtual mentoring sessions that allowed attendees to discuss specific AI topics or "research life in general."
Researchers from Google AI Research and Google subsidiary DeepMind contributed over 100 papers, including four of the eight winners of Outstanding Paper awards. One of the DeepMind papers, EigenGame: PCA as a Nash Equilibrium, was highlighted on the DeepMind blog. The paper describes a new competitive multi-agent approach to solving old problems, such as principal component analysis (PCA). According to the paper's authors,
[W]e found that the multi-agent perspective allowed us to develop new insights and algorithms which make use of the latest computational resources. This enabled us to scale to massive data sets that previously would have been too computationally demanding, and offers an alternative approach for future exploration.
Several other large tech companies also contributed papers. Facebook employees collaborated on nearly two dozen, including another Outstanding Paper winner, Neural Synthesis of Binaural Speech From Mono Audio, describing work from Facebook Reality Labs. Microsoft researchers contributed 53 papers, including several iterations on the BERT language model and multiple papers on video and image processing. Many of these companies were sponsors of the event in addition to contributing content. Google, Facebook, Microsoft, Amazon, and Intel were "Platinum" level sponsors; Intel also contributed 8 papers and Amazon 4. Apple, IBM, and Salesforce were "Gold" sponsors, with IBM also presenting 35 papers and Salesforce 7.
In a discussion about the conference, Stanford University AI Lab director Christopher Manning tweeted a graph showing the most popular keywords in submitted papers. He noted that the graph showed:
The amazing rise of reinforcement learning! (With graph neural networks and meta-learning in hot pursuit. ConvNets? Tired.)
Many of the conference papers along with the code used in them are available on Papers with Code. Videos of the paper presentations are available on SlidesLive.
