Eugen Leitl wrote: >>>>> One of the first things you have to do when building a >>>>> system like the FreedomBox is figure out the software >>>>> architecture. I have come up with an interesting >>>>> architecture that is made of Linux Containers (Virtual >>>> Using Linux vserver guests for service separation (jails on >>>> steroids) is an excellent idea, actually. >>> I find it interesting, but am sceptical: I believe each >>> jail consume separate memory for their libraries - i.e. >>> cannot benefit from shared libraries. So I worry about memory >>> consumption. >> mainly depends on the guest setup, assuming that the >> guests use identical libraries and the on disk data >> is properly unified, then both, the inode cache and >> the page mappings will use the very same memory > > Vserver unification hasn't been something really solid in the past, often > brokeni, at least in Debian. I've only seen one instance of it being broken, and I posted about it on the list. It was on ext4 fs back before ext4 was actually deemed stable, and all the newer kernels since have worked just fine. I suspect you just had a duff kernel. Did you roll it yourself or did you use a pre-packaged one? Here's a link to the thread on the subject from last year: http://www.paul.sladen.org/vserver/archives/201010/0018.html > Note that the vserver patch support in Debian will stop with wheezy. LXC > might be a better choice in the long term. Frankly, the only advantage LXC has is that it's in the mainline. Having argued a case for a hashify feature in it before, my impression that the chances of having it in LXC is effectively 0. Here's a thread where I brought it up on the LXC list a while back: http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg00433.html From what I can tell, LXC maintainers think file unification is something for the file system to take care of and point at things like snapshots (e.g. in BTRFS) and deduplication (last time I was involved in a discussion on deduping on the BTRFS list I was sufficiently disappointed by the wrongness of the proposed designs I just gave up - thankfully, we now have a fully working ZFS port for Linux so I never have to look at BTRFS again). BTRFS guys neither seemed to comprehend the importance of CoW hard-links nor had any interest in implementing such a feature. Here's the relevant thread on the BTRFS list: http://www.spinics.net/lists/linux-btrfs/msg05224.html Given the lack of interest and understanding from both the LXC camp and the BTRFS camp, I don't actually see any alternative to the efficiency of vserver any time soon. There are other bodge solutions that may be viable, such as marking all memory for same-page merging using LD_PRELOAD intercepts of malloc(), as I mentioned here: http://www.spinics.net/lists/kvm/msg36193.html but that would still be relatively expensive - KSM iterates through and dedupes all the madvise()-ed memory every few seconds. In comparison, the vserver's hashify feature is free in terms of run-time overheads. Gordan