GOOD documentation

GOOD (Graph OOD) is a graph out-of-distribution (OOD) algorithm benchmarking library depending on PyTorch and PyG to make develop and benchmark OOD algorithms easily.

GOOD provides a suite of dataset, GNN, OOD algorithm APIs and constructs an easy-to-use OOD algorithm training and testing bed. All algorithms, GNNs, and datasets can be easily registered by register and automatically embedded into the designed pipeline like a breeze! Using this register and overwriting classes, you can custom your methods and embed them into the test bed conveniently.

Currently, GOOD contains 11 datasets with 17 domain selections. When combined with covariate, concept, and no shifts, we obtain 51 different splits. We provide performance results on 10 commonly used baseline methods (ERM, IRM, VREx, GroupDRO, Coral, DANN, Mixup, DIR, EERM, SRGNN). This results in 510 dataset-model combinations in total. Our results show significant performance gaps between in-distribution and OOD settings. This GOOD benchmark is a growing project and expects to expand in quantity and variety of resources as the area develops. Any contribution is welcomed!

GOOD datasets

Notes

Package reference

GOOD.data

This data module includes 11 GOOD datasets and a dataloader for an organized data loading process.

GOOD.definitions

Only for project usage.

GOOD.kernel

This module contains project kernel configuring, training, testing, and evaluation pipelines.

GOOD.networks

A module that design GNN structures and manage GNN loadings.

GOOD.ood_algorithms

This module is comprised by 7 OOD algorithms and their loader.

GOOD.utils

This module contains various utilities, such as a component register, an argument parser, a configuration reader, and a metric object.

Indices and tables