Friday, December 11, 2009

Luscious Lebkuchen: XtreemFS 1.2 now available

XtreemFS 1.2 ("Luscious Lebkuchen") is now available at http://www.xtreemfs.org.

New features:
  • asynchronous MRC database backups (database dumps are created in the background)
  • new GRID SSL mode: SSL is used for authentication only, data is transferred via plain TCP for a better performance
  • Vivaldi-based replica selection and placement
  • MRC and OSD wait and retry for a configurable time span if they cannot reach the DIR on startup
  • faster file copying of files with 'cp' (by providing correct values for 'st_blksize' with 'getattr')
  • additional binaries with standard Unix names (mount.xtreemfs, mkfs.xtreemfs, lsfs.xtreemfs, rmfs.xtreemfs)
  • experimental Hadoop file system driver for XtreemFS
  • web-based admin console
We also fixed most of the bugs recorded in the tracker.

When upgrading your XtreemFS installation from 1.1 to 1.2, MRC and DIR databases should be converted automatically, without the need for human intervention. However, for safety reasons, we suggest to create an XML dump with the old release before updating; see "Updating" section in the XtreemFS FAQs (http://www.xtreemfs.org/faq.php).

Wednesday, December 9, 2009

A hadoop file system driver for XtreemFS

We have started the development of a FileSystem driver for hadoop. The current version implements the basic functionality and supports parallel I/O for striped files but does not support advanced features like retrieving block locations. The driver is still in an early stage of development!

If you still want to try it, the source is in trunk/contrib/hadoop. To build it, you need the jar from the hadoop release 0.20.1 and the XtreemFS trunk (or release 1.2). Make sure to modify the path to the hadoop.jar in nbproject/project.properties. In order to use XtreemFS with hadoop, you need to load three jar files: the HadoopClient.jar, the XtreemFS.jar and the yidl.jar. You also have to add XtreemFS to your hadoop config:
<property>
  <name>fs.xtreemfs.impl</name>
  <value>org.xtreemfs.common.clients.hadoop.XtreemFSFileSystem</value>
  <description>The FileSystem for xtreemfs: uris.</description>
</property>

The URLs for XtreemFS follow this scheme: xtreemfs://volumeName@dirServiceHost:32638/path

The userID and groupID can be set in the config options xtreemfs.client.userid and xtreemfs.client.groupid

Tuesday, December 8, 2009

web-based admin console

The next release (1.2) will include a web-based admin console. You can easily monitor the status of your installation, edit the UUID mappings, create new volumes, change policies or create MRC backups.
The console also shows the free space per volume and OSD. If you configured the "geographic_coordinates" in your service configuration files, the Google maps overlay will give you a nice overview of your entire installation.

The console .war file is in trunk/contrib/console in our svn.