BabuDB - efficient key value store for java

The XtreemFS MRC uses a simple key-value store for managing all metadata. We have spent a lot of time to evaluate the different storage backends: HSQLDB (and relational databases in general), BerkeleyDB for Java and simple Java TreeMaps.

Embedded SQL databases like HSQLDB are simply too slow for key-value lookups since SQL parsing is expensive. Stand-alone SQL server require IPC which also rules out that solution. We used BerkeleyDB for a while but due to the lack of documentation and stability issues we had to drop that as well. The Java TreeMaps did their job, but they have real limits in terms of size and serializing to disk is slow and interrups operations.

Finally, we decided to implement our own key-value store called BABUDB. It is based on the LSM-Tree concept and is optimized for applications that need a non-transactional key-value store. If you like to find out more, visit our google code project at http://code.google.com/p/babudb/. In the next release of XtreemFS, the MRC will be ported to the new database which will result in much better performance and a better utilization of multi-core processors.