Subject: Re: [vserver] vhashify not working? - 2.6.27.14-vs2.3.0.36.4
From: "John A. Sullivan III" <jsullivan@opensourcedevel.com>
Date: Tue, 31 Mar 2009 04:08:37 -0400

On Tue, 2009-03-31 at 06:51 +0000, Christoph Lukas wrote:
> Hi John,
> 
> > Hello, all.  In our earlier deployments on kernel 2.6.22, we were very
> > happy with the results of vhashify.  For some reason, our use in
> > 2.6.27.14 using vserver 2.3.0.36.4 does not seem to be working as well.
> > This is important to us because we are planning roughly 400 nearly
> > identical guests on this one host.
> > 
> > We currently have 10 Ubuntu 8.0.4 guests running on our CentOS 5.2 based
> > host.  Each one is roughly 2GB in size and were all cloned from the same
> > template.  Total storage on the vserver partition is roughly 21GB.  The
> > only unusual bit about this installation is there is /vservers/vetc
> > directory which is then mounted via bind to /etc/vservers.  This was
> > originally because there was a single encrypted partition mounted via
> > iSCSI holding all the vserver information.
> > 
> > We've done:
> > mkdir /etc/vservers/.defaults/apps/vunify/hash /vservers/.hash
> > ln -s /vservers/.hash /etc/vservers/.defaults/apps/vunify/hash/root
> > 
> > We noticed there is another
> > link, /etc/vservers/.defaults/apps/vunify/hash/00
> > and hashify complained of a duplicate "root"directory so we deleted the
> > root symlink.  With or without it, we have the same results.
> 
> are there any hardlinks visible inside the /vservers/.hash directory? If
> not the hashify did not work.
> 
> You can check which files inside the vservers are not unified by running
> 
> find /vserver/<guest> -type f -links 1
> 
> and the unified files by running:
> 
> find /vserver/<guest> -type f -links +1
> 
> This should give you a hint if vhashify worked correctly.
> 
> > For each vserver guest, we did:
> > mkdir /etc/vservers/<name>/apps/vunify
> > 
> > Has something changed with 2.6.27.14? 
> 
> AFAIK unification is just done in userspace it should not depend on the
> kernel version. Just the copy-on-write-link-breakage is done in the
> kernel.
> 
> > Are these realistic numbers? 
> 
> Does not seem realistic to me.
> 
> > I
> > would think ten identical systems should yield just slightly more than
> > the space of one system after running hashify.  
> 
> I would guess something with your .hash directories is not setup
> correctly and therefore the hashify did nor work as expected. You can
> try to fix this and then just run:
> 
> vserver <guest> hashify 
> 
> on a running guest. I have setup unification successfully here and used
> this two wiki pages as howto:
> 
> http://linux-vserver.org/Frequently_Asked_Questions#Unification
> 
> http://linux-vserver.org/util-vserver:Vhashify
> 
> Hope this helps,
> Christoph
> 
Thank you, Christoph.  The commands show plenty of both types of files
with the hard links clearly outweighing the regular files.  Strange that
I still show 20GB for 10 servers.  Take care - John
-- 
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan@opensourcedevel.com

http://www.spiritualoutreach.com
Making Christianity intelligible to secular society