On Tue, 2009-03-31 at 08:40 +0000, Christoph Lukas wrote: > Hi John, > > Am Dienstag, den 31.03.2009, 04:08 -0400 schrieb John A. Sullivan III: > > On Tue, 2009-03-31 at 06:51 +0000, Christoph Lukas wrote: > > > Hi John, > > > > > > > Hello, all. In our earlier deployments on kernel 2.6.22, we were very > > > > happy with the results of vhashify. For some reason, our use in > > > > 2.6.27.14 using vserver 2.3.0.36.4 does not seem to be working as well. > > > > This is important to us because we are planning roughly 400 nearly > > > > identical guests on this one host. > > > > > > > > We currently have 10 Ubuntu 8.0.4 guests running on our CentOS 5.2 based > > > > host. Each one is roughly 2GB in size and were all cloned from the same > > > > template. Total storage on the vserver partition is roughly 21GB. The > > > > only unusual bit about this installation is there is /vservers/vetc > > > > directory which is then mounted via bind to /etc/vservers. This was > > > > originally because there was a single encrypted partition mounted via > > > > iSCSI holding all the vserver information. > > > > > > > > We've done: > > > > mkdir /etc/vservers/.defaults/apps/vunify/hash /vservers/.hash > > > > ln -s /vservers/.hash /etc/vservers/.defaults/apps/vunify/hash/root > > > > > > > > We noticed there is another > > > > link, /etc/vservers/.defaults/apps/vunify/hash/00 > > > > and hashify complained of a duplicate "root"directory so we deleted the > > > > root symlink. With or without it, we have the same results. > > > > > > are there any hardlinks visible inside the /vservers/.hash directory? If > > > not the hashify did not work. > > > > > > You can check which files inside the vservers are not unified by running > > > > > > find /vserver/<guest> -type f -links 1 > > > > > > and the unified files by running: > > > > > > find /vserver/<guest> -type f -links +1 > > > > > > This should give you a hint if vhashify worked correctly. > > > > > > > For each vserver guest, we did: > > > > mkdir /etc/vservers/<name>/apps/vunify > > > > > > > > Has something changed with 2.6.27.14? > > > > > > AFAIK unification is just done in userspace it should not depend on the > > > kernel version. Just the copy-on-write-link-breakage is done in the > > > kernel. > > > > > > > Are these realistic numbers? > > > > > > Does not seem realistic to me. > > > > > > > I > > > > would think ten identical systems should yield just slightly more than > > > > the space of one system after running hashify. > > > > > > I would guess something with your .hash directories is not setup > > > correctly and therefore the hashify did nor work as expected. You can > > > try to fix this and then just run: > > > > > > vserver <guest> hashify > > > > > > on a running guest. I have setup unification successfully here and used > > > this two wiki pages as howto: > > > > > > http://linux-vserver.org/Frequently_Asked_Questions#Unification > > > > > > http://linux-vserver.org/util-vserver:Vhashify > > > > > > Hope this helps, > > > Christoph > > > > > Thank you, Christoph. The commands show plenty of both types of files > > with the hard links clearly outweighing the regular files. Strange that > > I still show 20GB for 10 servers. Take care - John > > Are you sure that your measurements of disk usage are correct? > Do the 20 GB disk usage correspond with the df output on the /vservers > partition? > > du -sch /vservers/* > > should notice multiple hardlinks and count them only once. > > Regards, > Christoph > > Yes, the numbers match and they also match the size of the space consumed on the thinly provisioned ZFS zvol on which they reside (28.1 GB): [root@vd01 ~]# du -sch /vservers/* 4.4G /vservers/cle 2.2G /vservers/gssstation 2.2G /vservers/jasstation 1.4G /vservers/jintra 16K /vservers/lost+found 1.4G /vservers/mintra 1.4G /vservers/mlapo 1.4G /vservers/simple 1.4G /vservers/smcc 1.4G /vservers/tkee 1.4G /vservers/tvan 1.4G /vservers/vdb 1.9G /vservers/vdb2 1.4G /vservers/vde 1.1M /vservers/vetc 23G total I am guessing this may also affect my memory usage. Does VServer use the hashify information to determine whether a binary being called into memory already exists? Thanks - John -- John A. Sullivan III Open Source Development Corporation +1 207-985-7880 jsullivan@opensourcedevel.com http://www.spiritualoutreach.com Making Christianity intelligible to secular society