XtreemFS update 1.2.1

We just released an update for XtreemFS (version 1.2.1). This version contains mainly bug fixes, e.g. for FreeBSD and Fedora 12, and enhanced replica management. The new scrubber will automatically replace failed replicas.

Source code and packages are available for download on http://www.xtreemfs.org.

There is no change in the database or OSD storage, so an update from 1.2 should work out of the box. The usual advice: backup important data before upgrading.

Luscious Lebkuchen: XtreemFS 1.2 now available

XtreemFS 1.2 ("Luscious Lebkuchen") is now available at http://www.xtreemfs.org.

New features:

• asynchronous MRC database backups (database dumps are created in the background)
• new GRID SSL mode: SSL is used for authentication only, data is transferred via plain TCP for a better performance
• Vivaldi-based replica selection and placement
• MRC and OSD wait and retry for a configurable time span if they cannot reach the DIR on startup
• faster file copying of files with 'cp' (by providing correct values for 'st_blksize' with 'getattr')
• additional binaries with standard Unix names (mount.xtreemfs, mkfs.xtreemfs, lsfs.xtreemfs, rmfs.xtreemfs)
• experimental Hadoop file system driver for XtreemFS
• web-based admin console
  

We also fixed most of the bugs recorded in the tracker.

When upgrading your XtreemFS installation from 1.1 to 1.2, MRC and DIR databases should be converted automatically, without the need for human intervention. However, for safety reasons, we suggest to create an XML dump with the old release before updating; see "Updating" section in the XtreemFS FAQs (http://www.xtreemfs.org/faq.php).

A hadoop file system driver for XtreemFS

We have started the development of a FileSystem driver for hadoop. The current version implements the basic functionality and supports parallel I/O for striped files but does not support advanced features like retrieving block locations. The driver is still in an early stage of development!

If you still want to try it, the source is in trunk/contrib/hadoop. To build it, you need the jar from the hadoop release 0.20.1 and the XtreemFS trunk (or release 1.2). Make sure to modify the path to the hadoop.jar in nbproject/project.properties. In order to use XtreemFS with hadoop, you need to load three jar files: the HadoopClient.jar, the XtreemFS.jar and the yidl.jar. You also have to add XtreemFS to your hadoop config:

<property>
  <name>fs.xtreemfs.impl</name>
  <value>org.xtreemfs.common.clients.hadoop.XtreemFSFileSystem</value>
  <description>The FileSystem for xtreemfs: uris.</description>
</property>

The URLs for XtreemFS follow this scheme: xtreemfs://volumeName@dirServiceHost:32638/path

The userID and groupID can be set in the config options xtreemfs.client.userid and xtreemfs.client.groupid

web-based admin console

The next release (1.2) will include a web-based admin console. You can easily monitor the status of your installation, edit the UUID mappings, create new volumes, change policies or create MRC backups.
The console also shows the free space per volume and OSD. If you configured the "geographic_coordinates" in your service configuration files, the Google maps overlay will give you a nice overview of your entire installation.

The console .war file is in trunk/contrib/console in our svn.

Release 2.0 of XtreemOS

The new release 2.0 of XtreemOS was just released here at Supercomputing'09.

XtreemFS @ SC09 (Portland, OR)

You can meet us at Supercomputing in Portland, OR at the XtreemOS booth (#2388) from 11/16/09 to 11/19/09.

Cold Tamales: XtreemFS 1.1 now available

XtreemFS 1.1 is now available at xtreemfs.org.

Note for users upgrading from version 1.0: the metadata database format has changed. You must use dump/restore of the database when updating from 1.0!

New features in release 1.1 include:
- POSIX advisory locks (e.g. fcntl locks)
- Global inode/dev numbers (for dovecot, tar, and other programs that rely on unique inodes in struct stat)
- On-close replication. Replicas can now be automatically created when a file is closed after having been written the first time. This causes the file to become read-only. The behavior of on-close replication can be controlled via the xtfs_repl tool.
- A new "hostname" property in the server configurations, for specifying the host name to advertise to the directory service
- gridmap-file user identification

We also fixed a number of bugs, including a problem with SSL on Windows and issues that prevented the OSD from running on Windows.