The Stanford University Human-Centered Artificial Intelligence Institute published its AI Index 2019 Report. The 2019 report tracks three times the number of datasets as the previous year's report and contains nearly 300 pages of data and graphs related to several aspects of AI, including research, technical performance, education, and societal considerations.
The report is the result of an effort led by the Institute's AI Index Steering Committee, a team of researchers and industry experts chaired by AI21Labs co-founder Yoav Shoham. This is the report's third year, and it includes updates of previous metrics as well as new ones. In addition to the report, the committee has released two web-based tools: the Global AI Vibrancy Tool for comparing data across countries, and the arXiv Monitor for searching pre-print research papers to track technical metrics. According to the Committee's web site, the Index's mission is:
to provide unbiased, rigorous, and comprehensive data for policymakers, researchers, journalists, executives, and the general public to develop a deeper understanding of the complex field of AI.
The report includes data from dozens of sources categorized into nine broad areas:
- Research and Development - number of publications, citations, adoption of software libraries
- Conferences - conference attendance, topics, milestones achieved
- Technical Performance - progress in AI tasks such as computer vision and natural language processing
- The Economy - jobs, investment, business activity
- Education - trends in training programs, computer science curricula, PhD candidates
- Autonomous Systems - autonomous vehicles (AV’s) and autonomous weapons (AW’s)
- Public Perception - communication and perception of AI use by banks, governments, and businesses
- Societal Considerations - ethical challenges
- National Strategies and Global AI Vibrancy - official strategy documents of national governments
Global AI Startup Funding (Image Source: Stanford HAI AI Index Report)
Because of the large amount of data contained in the report, the Committee has released two web tools for exploring and interacting with several of the metrics. The Global AI Vibrancy Tool allows users to compare over two dozen countries along three of the dimensions: research and development, economy, and inclusion. The tool imports live data directly from the sources used in the report, and it allows users to interactively drill into an individual country's metrics or adjust the weights of the data to produce a custom composite index score. The arXiv Monitor indexes AI-related papers from the popular pre-print repository, and allows users to search by arXiv category, author, and specific AI research topics such as ImageNet or MNIST.
Users on Reddit called particular attention to one of the report's findings about performance improvements in machine learning:
In a year-and-a-half, the time required to train a large image classification system on cloud infrastructure has fallen from about three hours in October 2017 to about 88 seconds in July 2019... Prior to 2012, AI results closely tracked Moore’s Law, with compute doubling every two years. Post-2012, compute has been doubling every 3.4 months.
The report document is available on the HAI website. The raw data used to create the report as well as high-resolution image files of the figures from the report are available on Google Drive. The report is licensed under the Creative Commons Attribution-NoDerivatives 4.0 International license.