Subject: Re: [vserver] Unification Questions
From: "John Alberts" <john.m.alberts@gmail.com>
Date: Wed, 27 Aug 2008 11:44:38 -0500
Wed, 27 Aug 2008 11:44:38 -0500
Thank you.  That was a very thorough explanation that greatly helped me
understand what the "magic" is behind unification.  Once I get done testing
out unification on a couple of my servers, I'll document on the wiki my
experiences and the explanations both of you have given me.

John Alberts


On Wed, Aug 27, 2008 at 2:44 AM, Herbert Poetzl <herbert@13thfloor.at>wrote:

> On Tue, Aug 26, 2008 at 11:40:37AM -0500, John Alberts wrote:
> > Unification was just brought up on another recent thread and I have
> > some questions about it that google doesn't seem to be finding the
> > answers for me.
>
> > Is there any good documentation for using unification?
> > The linux-vserver site mentions it briefly in the FAQ section,
> > but that's all I could really find.
>
> well, basically it is as simple as hard links
> on unix filesystem (mainly because that is what
> unification is). the main 'property' of hard links
> in unix is, that they refer to the same contents,
> and thus are completely indistinguishable. so there
> is no 'original' and 'copy', there is just two links
> to identical contents.
>
> the evolution in Linux-VServer was like this:
>
>  + hardlinks save disk space, so we use them
>  - ah, damn, evil folks now can replace binaries
>   across guest systems
>  + we make them immutable, so the cross guest
>   issue is actually a non-issue
>  - damn again, now the guest cannot change its
>   files, not even remove them
>  + we introduce a flag to make immutable files
>   unlinkable (i.e. they can be removed)
>  - but that doesn't help with append or any
>   modifications to file attributes
>  + we introduce CoW link breaking, which breaks
>   the link on demand, and creates a writeable
>   copy
>
> that's it up to now, with the following things left
> for the future:
>
>  - how to handle 'guest local' hard links?
>  - how to show unified files to the guest?
>  - how to backup unified guests/files?
>
> today, hard disk space is really cheap, but, the
> hardlink has an indirect effect on memory too, and
> that is because the vfs caches and page mappings
> are based on 'device' and 'inode' numbers, and
> because a hard link uses the same device:inode
> identifier, the data will end up in the same cache
> and even identical page mapping, as long as it is
> read-only data ... which saves a lot on memory too,
> if the guests are reasonably similar ...
>
> > Specifically, I'm looking for:
>
> most of that was already answered in the other
> replies, so I'm only going to add some things ...
>
> >  1. What I would do to initially implement unification on a host with
> >     multiple guests currently running?
>
> create a hash directory, the entry in the config
> and vhashify the guest of your choice ...
>
> sidenote: vhashify is the generalization of the
> original unification (which only compared equal
> pathes between guests, to find identical files)
> which uses a hash to identify duplicates
>
> >  2. Is there anything special when creating a new guest that I would
> >     like to use unification?
>
> the build method 'clone' will take advantage
> of unified guests/templates so that the time
> to create a guest becomes quite small, and the
> creation is efficient
>
> >  3. What is the process for upgrading the guests?
>
> each guest can be upgraded independantly, or
> you can upgrade a bunch of them at once with the
> v* wrappers (vyum, vrpm, vapt-get, ..)
>
> >  3b Can they somehow be upgraded on one guest os and have the other
> >     guests link to those new files?
>
> in theory yes, but you would need to figure out
> what files need to be linked where, and in general
> that is too complicated compared to updating them
> and re-unifying them afterwards
>
> >  4. Are there any gotchas to look out for when using unification?
>
> yes, there are a few things you should always keep
> in mind when using unification:
>
>  - there are special file attributes to mark a
>   file as CoW/unified, you need to handle them
>   carefully when messing with the files on the
>   host system (e.g. backup/restore, scripts, etc)
>  - if you boot with a non Linux-VServer kernel,
>   some of those flags might lose their meaning
>   or get lost when you move them around
>  - the CoW files are _copied_ when written to, so
>   there needs to be enough disk space for that,
>   otherwise the copy will fail
>
> >  4b Special things that need to be done when adding or removing
> >     applications or files from a single guest os that is using
> >     unification for instance?
>
> inside the guest everything is supposed to behave
> like normal. although we do not hide the fact that
> a specific file is CoW/unified, it doesn't look
> different to any application, so that is supposed
> to work fine
>
> usually you want to re-unify (vhashify) the guests
> in regular intervals, or at least after updates, so
> that the entire setup can be optimized
>
> HTC,
> Herbert
>
> > Thanks
> > John Alberts
> >
> > --
> > John Alberts
>



-- 
John Alberts


Thank you.  That was a very thorough explanation that greatly helped me understand what the "magic" is behind unification.  Once I get done testing out unification on a couple of my servers, I'll document on the wiki my experiences and the explanations both of you have given me.

John Alberts


On Wed, Aug 27, 2008 at 2:44 AM, Herbert Poetzl <herbert@13thfloor.at> wrote:
On Tue, Aug 26, 2008 at 11:40:37AM -0500, John Alberts wrote:
> Unification was just brought up on another recent thread and I have
> some questions about it that google doesn't seem to be finding the
> answers for me.

> Is there any good documentation for using unification?
> The linux-vserver site mentions it briefly in the FAQ section,
> but that's all I could really find.

well, basically it is as simple as hard links
on unix filesystem (mainly because that is what
unification is). the main 'property' of hard links
in unix is, that they refer to the same contents,
and thus are completely indistinguishable. so there
is no 'original' and 'copy', there is just two links
to identical contents.

the evolution in Linux-VServer was like this:

 + hardlinks save disk space, so we use them
 - ah, damn, evil folks now can replace binaries
  across guest systems
 + we make them immutable, so the cross guest
  issue is actually a non-issue
 - damn again, now the guest cannot change its
  files, not even remove them
 + we introduce a flag to make immutable files
  unlinkable (i.e. they can be removed)
 - but that doesn't help with append or any
  modifications to file attributes
 + we introduce CoW link breaking, which breaks
  the link on demand, and creates a writeable
  copy

that's it up to now, with the following things left
for the future:

 - how to handle 'guest local' hard links?
 - how to show unified files to the guest?
 - how to backup unified guests/files?

today, hard disk space is really cheap, but, the
hardlink has an indirect effect on memory too, and
that is because the vfs caches and page mappings
are based on 'device' and 'inode' numbers, and
because a hard link uses the same device:inode
identifier, the data will end up in the same cache
and even identical page mapping, as long as it is
read-only data ... which saves a lot on memory too,
if the guests are reasonably similar ...

> Specifically, I'm looking for:

most of that was already answered in the other
replies, so I'm only going to add some things ...

>  1. What I would do to initially implement unification on a host with
>     multiple guests currently running?

create a hash directory, the entry in the config
and vhashify the guest of your choice ...

sidenote: vhashify is the generalization of the
original unification (which only compared equal
pathes between guests, to find identical files)
which uses a hash to identify duplicates

>  2. Is there anything special when creating a new guest that I would
>     like to use unification?

the build method 'clone' will take advantage
of unified guests/templates so that the time
to create a guest becomes quite small, and the
creation is efficient

>  3. What is the process for upgrading the guests?

each guest can be upgraded independantly, or
you can upgrade a bunch of them at once with the
v* wrappers (vyum, vrpm, vapt-get, ..)

>  3b Can they somehow be upgraded on one guest os and have the other
>     guests link to those new files?

in theory yes, but you would need to figure out
what files need to be linked where, and in general
that is too complicated compared to updating them
and re-unifying them afterwards

>  4. Are there any gotchas to look out for when using unification?

yes, there are a few things you should always keep
in mind when using unification:

 - there are special file attributes to mark a
  file as CoW/unified, you need to handle them
  carefully when messing with the files on the
  host system (e.g. backup/restore, scripts, etc)
 - if you boot with a non Linux-VServer kernel,
  some of those flags might lose their meaning
  or get lost when you move them around
 - the CoW files are _copied_ when written to, so
  there needs to be enough disk space for that,
  otherwise the copy will fail

>  4b Special things that need to be done when adding or removing
>     applications or files from a single guest os that is using
>     unification for instance?

inside the guest everything is supposed to behave
like normal. although we do not hide the fact that
a specific file is CoW/unified, it doesn't look
different to any application, so that is supposed
to work fine

usually you want to re-unify (vhashify) the guests
in regular intervals, or at least after updates, so
that the entire setup can be optimized

HTC,
Herbert

> Thanks
> John Alberts
>
> --
> John Alberts



--
John Alberts