Client News: Caching, Win32, OS X, ARM

The client of XtreemFS contains a considerable part of the logic of XtreemFS as it has to coordinate the metadata and object storage servers (OSDs) to provide the user with a POSIX-compliant file system abstraction. 

This setup is great from an architectural view point because it scales well as it takes tasks away from the metadata and storage servers. But it also makes the client a complex beast. 

In the last weeks we have put a lot of work in the client and you will be able to use the features soon as part of the 0.10.0 release. 

The first area of work was caching. While the operating system already caches file data in the page cache, the granularity of the accesses are mostly 4k - too small if you have to fetch each of these over the network. With client-side caching, we can now fetch data with object granularity, which is usually a few hundred k or more. We will follow-up with real measurements soon, but the performance improvements are very good. 

Client-side caching is also laying the foundation for prefetching, RAID, and other things that are on our long-term roadmap. 

We have also implemented a metadata cache that allows the client to retrieve the results of a readdir() together with all the stat()s in one RPC - important for long-latency networks like DSL or installations over the Internet. But it also feels more snappy on the LAN.

We have also started porting the client to other platforms than Linux/FUSE. We have been running on OS X with Mac FUSE for quite a while now (available on our download page), and have now extended our coverage of platforms to Windows and ARM Linux. The Windows client is using the fantastic Dokan library with the fuse4win adapter. It will be available on the download page soon.