Subject: Poll: High (ish) availability - how are you doing it?
From: Ed W <lists@wildgooses.com>
Date: Tue, 27 Jul 2010 14:52:03 +0100

  Hi, I am very pleased with linux-vserver for our needs, which is 
basically a small mail server and associated processes (webserver, mail 
scanner, dns, etc)

I'm now investigating how to scale up from here and really how we can 
move from a simple backup of image/data files and move towards higher 
availability using a small pool of machines (possibly in different data 
centers) and more ability to move vservers between machines in the event 
of problems or downtime.

At present I'm really just looking to achieve a goal of a backup machine 
which has a warm backup of our data and will be periodically (or better) 
synced with the main machine.  Considering rsync or DRBD for this role 
(bit scared of DRBD though).  I would be interested in any systems 
people have experienced that do an async replication between two 
machines, eg some process which uses inotify/dazuko/redirfs to watch for 
changes and then an async queue to merge changes to the second machine - 
the goal being to have a near realtime replica, but without the 
performance penalty associated with cluster filesystems?

We would like to have a plan to move up from there to a hot failover 
option involving multiple machines in the same data center and should a 
machine fail then we can move the services over to the new machine 
(target under 10 mins to failover).  However, there seem to be 
significant issues in automating this and quite a bit of thought seems 
to be needed to fencing dead machines.  Anyone got any experience to share?

A more satisfactory solution seems to be active-active clustering.  This 
is now irrelevant to linux-vserver, but does anyone here have any 
experience of running active-active clusters of mailservers, webservers, 
etc?  I see a GFS question on the list so we have at least on GFS user, 
what is currently the preferred choice for cluster filesystems for low 
cost active-active setups?  NFS, Ceph, Gluster, XtreemFs, DRBD, GFS2, 
OCFS all seem to have pros and conns?

My dream situation would be an active-active mailserver cluster with a 
machine in the USA and a machine in Europe.  Users would reach the 
geographically closest.  This seems like an unsolved problem at present 
and there don't seem to be any cluster filesystems with satisfactory 
performance over long RTT links to achieve this?

Coming back to the near term goals - anyone got any experiences to share 
of simple warm availability solutions?

Thanks

Ed W