Google AI has recently released GraphWorld, a tool to accelerate performance benchmarking in the area of graph neural networks (GNNs). It opens a new direction in GNN architecture experimentation and design by allowing AI engineers and researchers to test new GNN architectures on broader graph datasets.
GraphWorld is a configurable framework to generate graphs with a variety of structural properties like different node degree distributions and Gini index. Researchers use these graph datasets as the inputs to test the behavior of the different GNN architectures. The main features of GraphWorld are its low computational cost and speed while providing a large diversity and variety in generated input graphs. It enables developers and researchers to quickly test the performance of the GNN architectures at scale.
GNNs are powerful deep-learning neural-network architectures for modeling and understanding graph datasets. They mainly use message-passing neural-network layers to update and aggregate each node's information at each stage. This information is finally used as the embedding information to apply different predictive modeling by using different layers. It has been effectively used in drug-molecule discovery, and social-network property prediction, as a few examples.
Much of the research in this area has so far used limited classical datasets like 5-10 benchmark datasets. Most of these datasets are composed of easily labeled paper citations and some molecular networks. This limitation doesn't help with the possible claimed performance of these GNN architectures when used in solving real-world problems. Some of the new datasets try to address this issue like Open Graph Benchmark (OGB). It consists of the datasets like academic paper citations and molecular networks at scale, but still there is a lack of diversity and variety in OGB.
To emphasize more on the motivation behind building GraphWorld, engineers in Google AI used one open large collection of graphs called Network Repository (NR) in comparison to OGB. They considered two simple properties of the graphs: clustering coefficient (how interconnected nodes are to nearby neighbors) and degree distribution Gini coefficient (the inequality among the nodes' connection counts). They showed that OGB is sparsely populated in comparison to the NR and it lacks important graph architectures.
The following figure shows GraphWorld architecture and how this framework generates results. GraphWorld uses parallel processing to produce a large number of GNN datasets by sampling different values with different probability distributions to generate input graphs for GNNs. It conducts tests in parallel with different standards or newly defined GNN models like GCN, GAT, GraphSAGE, etc. It then outputs all the metrics in the metrics repository for evaluation and further analysis.
Image Source: Google AI Blog
The GraphWorld research paper can be found in arXiv database. Also, GraphWorld package is open-sourced on GitHub for public use.