Subject: Re: [vserver] vhashify not working? - 2.6.27.14-vs2.3.0.36.4
From: Christoph Lukas <christoph.lukas@gmx.net>
Date: Tue, 31 Mar 2009 06:51:04 +0000

Hi John,

> Hello, all.  In our earlier deployments on kernel 2.6.22, we were very
> happy with the results of vhashify.  For some reason, our use in
> 2.6.27.14 using vserver 2.3.0.36.4 does not seem to be working as well.
> This is important to us because we are planning roughly 400 nearly
> identical guests on this one host.
> 
> We currently have 10 Ubuntu 8.0.4 guests running on our CentOS 5.2 based
> host.  Each one is roughly 2GB in size and were all cloned from the same
> template.  Total storage on the vserver partition is roughly 21GB.  The
> only unusual bit about this installation is there is /vservers/vetc
> directory which is then mounted via bind to /etc/vservers.  This was
> originally because there was a single encrypted partition mounted via
> iSCSI holding all the vserver information.
> 
> We've done:
> mkdir /etc/vservers/.defaults/apps/vunify/hash /vservers/.hash
> ln -s /vservers/.hash /etc/vservers/.defaults/apps/vunify/hash/root
> 
> We noticed there is another
> link, /etc/vservers/.defaults/apps/vunify/hash/00
> and hashify complained of a duplicate "root"directory so we deleted the
> root symlink.  With or without it, we have the same results.

are there any hardlinks visible inside the /vservers/.hash directory? If
not the hashify did not work.

You can check which files inside the vservers are not unified by running

find /vserver/<guest> -type f -links 1

and the unified files by running:

find /vserver/<guest> -type f -links +1

This should give you a hint if vhashify worked correctly.

> For each vserver guest, we did:
> mkdir /etc/vservers/<name>/apps/vunify
> 
> Has something changed with 2.6.27.14? 

AFAIK unification is just done in userspace it should not depend on the
kernel version. Just the copy-on-write-link-breakage is done in the
kernel.

> Are these realistic numbers? 

Does not seem realistic to me.

> I
> would think ten identical systems should yield just slightly more than
> the space of one system after running hashify.  

I would guess something with your .hash directories is not setup
correctly and therefore the hashify did nor work as expected. You can
try to fix this and then just run:

vserver <guest> hashify 

on a running guest. I have setup unification successfully here and used
this two wiki pages as howto:

http://linux-vserver.org/Frequently_Asked_Questions#Unification

http://linux-vserver.org/util-vserver:Vhashify

Hope this helps,
Christoph