Subject: Re: [vserver] Re: [Freedombox-discuss] [vserver] Re: A software architecture for the FreedomBox
From: Gordan Bobic <gordan@bobich.net>
Date: Thu, 14 Apr 2011 22:42:28 +0100

On 04/14/2011 09:40 PM, Martin Fick wrote:
> --- On Thu, 4/14/11, Gordan Bobic<gordan@bobich.net>  wrote:
>
>> Frankly, the only advantage LXC has is that it's in the
>> mainline. Having argued a case for a hashify feature in it
>> before, my impression that the chances of having it in LXC
>> is effectively 0.
>
> ...[more talk about other unification solutions]
>
>
> Actually, I can think of a new approach to unification that
> might have some benefits which the current approach does not
> have.  Currently, I use vservers with drbd, and each vserver
> has its own partition (so that each vserver can failover
> independently).  The separate partitions means that I cannot
> use unification with my vservers.

Why do you need separate partitions? Why not have a single partition, 
mirrored between the hosts? All guests are in a seprate /vserver 
subdirectory anyway. What are you gaining from having a separate 
partition per guest?

> Ideally, it would be nice to have a COW like hard link
> mechanism that is able to hardlink to files in other
> partitions/fses.  That would also help in the case of the
> cross snapshotting idea mentioned in the brtfs threads
> you linked to.  So, how could cross filesystem COW
> hardlinks be implemented?

I don't think that's implementable with the current file system model. 
It also wouldn't help you since you would end up with a hard-link 
pointing to a partition that isn't necessarily the one you have 
associated with the guest that is failing over, so you'd end up failing 
over a guest without all it's files being available.

> What if a unionfs like approach were used instead?  In
> a unionfs filessystem, you have 2 layers to the FS,
> an upper writeable layer, and a lower readonly (usually)
> layer. If each vserver (or lxc) were to mount a
> vserver unionfs layer onto a common lower layer, it
> might be possible to use this with some fancy management
> tools to provide a unification like solution, a solution
> which can span partitions/fses!
>
> Picture a setup like this:
>
> /vservers                 mount point for the shared partition /dev/<shared>
> /vservers/sha1            holds common files, named after their sha1
> /vservers/roots/<vserver>  mount point for the vserver unionfs lower layer
> /vservers/tops/<vserver>   mount point for the vserver unionfs upper layer
>
> Before startup /vservers/sha1 is populated with the
> common (unified) shared files, and shared files for
> each vserver are hardlinked to from under
> /vservers/roots/<vserver>... Aside from shared files,
> /vservers/roots/<vserver>... would be empty. Then,
> it is important to purge the shared files from
> /vservers/tops/<vserver>....  So, the top layer
> would contain the entire vserver fs minus the shared
> files.
 >
> On startup, a vserver mounts its top layer partition:
> /dev/<vserverA>  at /vservers/tops/A, then using unionfs
> mounts this fs onto its lower layer: /vservers/roots/A
> masking the lower layer (if I remember correctly how
> it works).  And finally, it boots with its root as
> /vservers/roots/A (now seeing the stacked filesystem).
>
> Any shared files should be taken from the unionfs lower
> layer as long as they are not masked by the upper layer.
> On write, the upper layer will likely mask them, this
> should protect the lower layer (only if kept readonly?).
>
> What do you think?  I haven't tried this, but I might
> at some point.  If it works, it seems like it would
> continue to provide the memory sharing benefits of the
> current unification method,

Same inode on a different file system wouldn't mmap to the same place in 
memory. But I'm still not sure what you are gaining by splitting up the 
file systems.

Gordan