Benchmark First: Defining Tasks for Graph Transformation Learning
Date:
Learning graph transformations from examples requires datasets for training and benchmarking. To establish which machine learning architectures are suitable for which kinds of problems, we need to experiment on a range of graph-computation tasks. In this paper, we propose a framework for defining such tasks and categorising them along four dimensions: complexity, input-output relation, graph type, and the structural changes they require. We apply the framework to 16 tasks for which we have implemented data generators or provided data sets in a unified interface. Our aim is to work towards a benchmark that supports the training and evaluation of graph transformation models in a framework that can be used by the wider community to support their own research, define more tasks, thereby extending and refining the framework.