Subject: Re: [vserver] btrfs/hashify/cow....
From: Gordan Bobic <gordan@bobich.net>
Date: Fri, 07 Sep 2012 17:16:17 +0100

On 09/07/2012 04:11 PM, Tor Rune Skoglund wrote:
>
>
> Hi List,
>
> I been trying to read up on the hashify with COW feature of vserver.
> Still some questions...:
>
> - What are the actual requirements for using this feature?
> - Any particular file systems only?

IIRC only ext* file systems are supported/patched (included in the 
vserver patch).

> - Presumably, all hashifed files must reside on the same partition?

Indeed, they must all be on the same file system.

> - What if the process of hashifying is interrupted? Will filesystems
> still be left in good working order?

Yes. The process hard-links all files to a file named by their hash 
under /vservers/.hash/. It then goes and replaces all the files one by 
one with a hard-link to this central hash location. The net result is 
that all the identical files will end up sharing the data blocks. There 
is nothing going on that could trash the file system.

Note: There was a bug in a very old vserver/kernel combo that caused the 
ext4 file system to get trashes. I had the misfortune of being the one 
to discover it (thankfully, on a test system). That patch is no longer 
available, and this was beck in the day before ext4 was considered 
completely production stable (although being subscribed to the ext4 
development list and seeing the number of bugs that get raised against 
it every day, I'm not sure what exactly I mean by "production stable" 
here). Either way, a reasonably recent kernel (2.6.32+) with the 
suitable vserver patch should be perfectly safe. I have been using 
2.6.35.7-vs2.3.0.36.33 for years without any problems.

> - Can the hashify process be run on running vservers, or should they be
> stopped before starting the hashify process?

You can run it on the running server, but you will not gain the full 
benefit of memory deduplication until you re-start all the processes in 
all the guests (i.e. until all the file handles are re-opened).

> lxc seem to prescribe btrfs as the lxc "equivalent"
> (http://www.digipedia.pl/usenet/thread/18777/2045/ ). How does this
> compare to vserver's hashify? (Some answer given in that discussion, but
> still a couple of open ones...)

I started that thread. :)
It is _NOT_ equivalent. The key difference is that it won't give you 
memory deduplication for free. One of the big unique selling point of 
vserver is that by using CoW hard-links it gives you memory 
deduplication for free. Think about how mmap() works and how the memory 
address is picked - it is done by inode number. Hard-links have the same 
inode number. That means that mmap()-ing the same hashified file in 
different guests will all go to the same memory address. Free memory 
dedupe if all your guests run the same OS with the same package versions 
(Note: DISABLE AND UNINSTALL PRELINK!). Imagine having 100 guests and 
instead of needing 100 instances of glibc in RAM you only have one.

Normally memory deduplication is very expensive in terms of CPU time, 
but in vserver with hashify you get it for free.

> - Can btrfs be used with vserver?

Not unless somebody write the CoW hard-link patches recently.

> - What about trebuchet (http://wiki.baserock.org/Trebucket ): Any
> qualified guesses whether it will be compatible for updating a vserver
> guest? (USE-CASE: Update a guest as an atomic operation; keep fallback
> vserver guest ready in case update process incomplete, or in case new
> vserver guest do not perform correctly.)

That link seems to be dead, and I cannot figure out from paragraph 
context alone what it's supposed to achieve. Please elaborate?

Gordan