The work behind this paper started long ago (2009?), working with Evan Jones and Yang Zhang on some infrastructure to test our DBaaS project (http://relationalcloud.com) and continued with Andy Pavlo and I cursing about how painful it is to build a reasonable testing infrastructure (is like a tax on DB/system PhD students).
We decided to make it easier for future generations, and started to combine Andy’s workloads and mine, and the strength of both infrastructures (at the time a beautiful mess of hacky codes and half-fast scripts). Djellel Difallah and Phil Cudre-Maroux join the effort (and arguably Djellel put in more hours than anyone else on this since then). We polished the infrastructure and added several more workloads, with help and input from many people including Rusty Sears, Ippokratis Pandis, Barzan Mozafari, Dimitri Vorona, Sam Madden, and Mark Callaghan.
The goal was to produce enough critical mass of features and workloads. So that other researchers would prefer to pick up this infrastructure and contribute to it, rather than building from scratch. This seems to be working as we received many requests and contributions from companies and academics all around the world. Andy Pavlo is now heading a revamp of the website, including much needed graphing and comparisons interfaces.
Hopefully our community can rally behind this effort, and drive it in whichever direction seems appropriate (we are open to extensions and changes, even drastic), reducing the repeated work, and fostering some better repeatability and ease of comparison among “scientific” results in papers.
Checkout the paper here:
http://www.vldb.org/pvldb/vol7/p277-difallah.pdf
Our website at:
And get the code from github:
https://github.com/oltpbenchmark/oltpbench?source=cc