Subject: Re: [vserver] SSD/HD hybrids
From: "John A. Sullivan III" <jsullivan@opensourcedevel.com>
Date: Thu, 01 Apr 2010 17:55:55 -0400

On Thu, 2010-04-01 at 11:32 +0200, Eugen Leitl wrote:
> Due to space limitations within the 5015A-PHF rackmounts and my
> budget limitations I'm thinking about using an 80 GByte Intel
> SSD along with a conventional 1-2 TByte SATA drive in a sensible
> way for vserver production systems.
> 
> Linux doesn't have any native/production filesystems like zfs yet.
> Presumably btrfs will be up to snuff sometime, but it is probably
> years away from production still.
Yes - waiting eagerly for btrfs
> 
> Presumably one could mount such a hybrid zfs an iSCSI device, but 
> that would waste the iSCSI host. Also, the GBit Ethernet would
> be a bottleneck (at max 115 MByte/s throughput), so one would
> have to use channel bonding and an additional network card.
We spent a lot of time optimizing a VServer iSCSI environment with a lot
of disappointment.  Linux (at least on AMD64) is limited to a 4KB block
size which makes the iSCSI channel bound by latency rather than
throughput.  Bonding was also a serious disappointment in a VServer
environment.  Because we have lots of servers on a single MAC address,
most bonding schemes fail because they map traffic flows to MAC SA/DA
combinations.  Even when using schemes which can look at a higher layer,
the attaching switches will likely not support it and collapse it back
to MAC SA/DA pairs.  Bonding is great in many-to-many but almost useless
in few-to-few except for failover.  We found ourselves using either
software RAID0 across the iSCSI disks (protected on the backend with
ZFS) or multi-path in multi-bus mode.
> 
> So I'm thinking about how to use such a hybrid SSD/HD setup manually.
> The obvious solutions would be to use the SSD for swap and for /tmp,
> and HD for everything else.
> 
> Another idea is to get the whole (vhashified) servers to live 
> there, and mount /home from the conventional hard disk.
> 
> Any additional ideas?
>