Subject: Re: [vserver] Re: [Freedombox-discuss] [vserver] Re: A software architecture for the FreedomBox
From: Martin Fick <mogulguy@yahoo.com>
Date: Thu, 14 Apr 2011 13:40:27 -0700 (PDT)

 Thu, 14 Apr 2011 13:40:27 -0700 (PDT)
--- On Thu, 4/14/11, Gordan Bobic <gordan@bobich.net> wrote:

> Frankly, the only advantage LXC has is that it's in the
> mainline. Having argued a case for a hashify feature in it
> before, my impression that the chances of having it in LXC
> is effectively 0.

...[more talk about other unification solutions]


Actually, I can think of a new approach to unification that
might have some benefits which the current approach does not
have.  Currently, I use vservers with drbd, and each vserver
has its own partition (so that each vserver can failover
independently).  The separate partitions means that I cannot
use unification with my vservers.

Ideally, it would be nice to have a COW like hard link
mechanism that is able to hardlink to files in other
partitions/fses.  That would also help in the case of the
cross snapshotting idea mentioned in the brtfs threads
you linked to.  So, how could cross filesystem COW
hardlinks be implemented? 

What if a unionfs like approach were used instead?  In
a unionfs filessystem, you have 2 layers to the FS,
an upper writeable layer, and a lower readonly (usually)
layer. If each vserver (or lxc) were to mount a
vserver unionfs layer onto a common lower layer, it
might be possible to use this with some fancy management
tools to provide a unification like solution, a solution
which can span partitions/fses!

Picture a setup like this:

/vservers                 mount point for the shared partition /dev/<shared>
/vservers/sha1            holds common files, named after their sha1
/vservers/roots/<vserver> mount point for the vserver unionfs lower layer
/vservers/tops/<vserver>  mount point for the vserver unionfs upper layer

Before startup /vservers/sha1 is populated with the
common (unified) shared files, and shared files for
each vserver are hardlinked to from under
/vservers/roots/<vserver>... Aside from shared files,
/vservers/roots/<vserver>... would be empty. Then, 
it is important to purge the shared files from
/vservers/tops/<vserver>....  So, the top layer 
would contain the entire vserver fs minus the shared
files.

On startup, a vserver mounts its top layer partition:
/dev/<vserverA> at /vservers/tops/A, then using unionfs
mounts this fs onto its lower layer: /vservers/roots/A
masking the lower layer (if I remember correctly how
it works).  And finally, it boots with its root as
/vservers/roots/A (now seeing the stacked filesystem).

Any shared files should be taken from the unionfs lower
layer as long as they are not masked by the upper layer.
On write, the upper layer will likely mask them, this
should protect the lower layer (only if kept readonly?).

What do you think?  I haven't tried this, but I might
at some point.  If it works, it seems like it would
continue to provide the memory sharing benefits of the
current unification method,

-Martin