Tag: oltp

OLTP-Bench at VLDB 2014

The work behind this paper started long ago (2009?), working with Evan Jones and Yang Zhang on some infrastructure to test our DBaaS project (http://relationalcloud.com) and continued with Andy Pavlo and I cursing about how painful it is to build a reasonable testing infrastructure (is like a tax on DB/system PhD students).

We decided to make it easier for future generations, and started to combine Andy’s workloads and mine, and the strength of both infrastructures (at the time a beautiful mess of hacky codes and half-fast scripts). Djellel Difallah and Phil Cudre-Maroux join the effort (and arguably Djellel put in more hours than anyone else on this since then). We polished the infrastructure and added several more workloads, with help and input from many people including Rusty Sears, Ippokratis Pandis, Barzan Mozafari, Dimitri Vorona, Sam Madden, and Mark Callaghan.

The goal was to produce enough critical mass of features and workloads. So that other researchers would prefer to pick up this infrastructure and contribute to it, rather than building from scratch. This seems to be working as we received many requests and contributions from companies and academics all around the world.  Andy Pavlo is now heading a revamp of the website, including much needed graphing and comparisons interfaces.

Hopefully our community can rally behind this effort, and drive it in whichever direction seems appropriate (we are open to extensions and changes, even drastic), reducing the repeated work, and fostering some better repeatability and ease of comparison among “scientific” results in papers.

Checkout the paper here:

http://www.vldb.org/pvldb/vol7/p277-difallah.pdf

Our website at:

http://oltpbenchmark.com

And get the code from github:

https://github.com/oltpbenchmark/oltpbench?source=cc

 

2010: SCHISM: a Workload-driven Approach to Database Replication and Partitionoing

2010: “Schism: a Workload-Driven Approach to Database Replication and Partitioning”, Carlo Curino, Yang Zhang, Evan Jones, Sam Madden, accepted for publication to Proceedings of Very Large Data Base (VLDB)

ABSTRACT:

We present Schism, a novel workload-aware approach for database partitioning and replication designed to improve scalability of shared-nothing distributed databases.  Because distributed transactions are expensive in  OLTP settings (a fact we demonstrate through a series of experiments), our partitioner attempts to minimize the  number of distributed transactions, while producing balanced  partitions. Schism consists of two phases: i) a  workload-driven, graph-based replication/partitioning phase and ii)  an explanation and validation phase. The first phase creates a  graph with a node per tuple (or group of tuples) and edges between  nodes accessed by the same transaction, and then uses a graph  partitioner to split the graph into k balanced partitions that  minimize the number of cross-partition transactions. The second  phase exploits machine learning techniques to find a predicate-based  explanation of the partitioning strategy (i.e., a set of range  predicates that represent the same replication/partitioning scheme  produced by the partitioner).

The strengths of Schism are: i) independence from the schema  layout, ii) effectiveness on n-to-n relations, typical in social  network databases, iii) a unified and fine-grained approach to  replication and partitioning. We implemented and tested a prototype  of Schism on a wide spectrum of test cases, ranging from classical  OLTP workloads (e.g., TPC-C and TPC-E), to more complex scenarios  derived from social network websites (e.g., Epinions.com), whose  schema contains multiple n-to-n relationships, which are known to be  hard to partition. Schism consistently outperforms simple  partitioning schemes, and in some cases proves superior to the best known manual partitioning, reducing the cost of distributed transactions up to 30%.

 

Contact me to receive a copy of the paper.