Subject: Re: [vserver] Poll: High (ish) availability - how are you doing it?
From: Gordan Bobic <gordan@bobich.net>
Date: Thu, 29 Jul 2010 01:15:53 +0100

JF Straeten wrote:
> On Wed, Jul 28, 2010 at 10:46:07PM +0100, Ed W wrote:
> 
>> Glusterf is the one which most interests me (active-active
>> clustering), but it seems that the public forums are full of people
>> with problems, but then a handful of folks who are very happy.
>> Obviously you always hear most about the outliers, but it does
>> suggest that there are quite a few subtleties with gluster? Anyone
>> got anything to share?
> 
> In April, I've tested glusterfs between VServers (on different hosts),
> as NFS replacement (I don't like the idea of having the host serving
> NFS shares).
> 
> So, it was not real clustering, with replication, etc. (nor any
> torture test), but I must admit it was pretty impressive performance
> wise. Nothing can beat NFS kernel server, but at least glusterfs
> doesn't seem to lag too far behind.

In a single-server setup it isn't too bad, but with multiple replicated 
servers, performance takes a nosedive, since it does away with any 
benefit of caching - it has to check for file locks on all the peers. In 
this sense it comes behind GFS in terms of performance, because GFS 
caches locks, so unless the lock migrates, the lock is 50ns away in RAM, 
rather than 150us away on the peer.

Don't get me wrong - GlusterFS is a great project and I use it 
extensively on several deployments for all sorts of things (including 
some obscure and creative ones, such as transparent mirroring of /boot 
partitions. But make sure you understand what it is and isn't good at. 
Same goes for any other solution. Also note that GlusterFS can do a lot 
more than just mirror volumes (data striping, DHT, etc.).

> And it was without any optimization with some of the translators
> (readahead, writebehind, cache) which should probably improve
> performance a bit.

Or cause crashes and break locking. Mileage varies.

Gordan