Hi Herbert Thank you for the quick reply. Interesting result testing without aufs: Mounted the ext3 fs in the vserver (--bind -rw) during the initialize script and it now works even better than expected. The hardlink is made correctly and is only broken when the file is touched: Before: ls -li 16428 -rw-r--r-- 8 root root 530 Apr 13 09:37 README.app After: ln README.app README.ajl; ls li 16428 -rw-r--r-- 9 root root 530 Apr 13 09:37 README.ajl 16428 -rw-r--r-- 9 root root 530 Apr 13 09:37 README.app After: touch README.app; ls -li 16428 -rw-r--r-- 8 root root 530 Apr 13 09:37 README.ajl 24622 -rw-r--r-- 1 root root 530 May 1 15:40 README.app Conclusion: aufs feature. About lvm snapshots: I can achieve the same effect (one vserver can keep a private rw copy of this data) by using snapshots but the downside is that a backup of this vserver will include all the data. With aufs the backup contains only the diff from the (large) fixed data. I may have to live with the aufs feature. At least I know what the cause is now! During this testing I also discovered that the 'mv' command also uses hardlinks (no surprise - should have guessed that) so it's not a nice feature and will likely bite more often than just dpkg. All the best and thanks again Allan On 01/05/13 16:18, Herbert Poetzl wrote: > On Wed, May 01, 2013 at 12:13:51PM +0200, Allan Latham wrote: > >> Hi all > >> I am planning on using vservers for an upcoming project and >> I have been doing extensive checking that I can achieve my >> objectives. > >> So far so good - I am very impressed. > >> I have come across something strange (= did not do what I >> expected). > >> First the scenario: > >> A disc contains an ext3 filesystem and is mounted rw on /X >> The top level is: > >> /X/DIR HASH HASHES RO > >> The contents of DIR has been hashified and the hashes are in >> HASHES. > >> /X/DIR is then bind mounted ro on /X/RO > >> This works 100% as I expect - I can mount /X/RO during the >> vserver initialize script and I have an ro version of the data >> for the vserver to use. Good. > >> To give each vserver a rw version of /X/DIR without disturbing >> all the other vservers I use aufs. > >> The ro branch of aufs is /X/RO and the rw branch is outside the >> vserver client tree. (i.e. the vserver has no direct access to >> it). > >> The aufs mount is done in the initialize script at the same >> point where in the previous test I bind mounted /X/RO. > >> This work 99% as I expect. The vserver can use the filesystem >> in rw mode and it is independent of other vservers. > >> The one exception is if I try to hardlink a file which has been >> hashified. I expected the kernel to break the hardlink to the >> hash and give me a fresh copy of the file and then hardlink to >> that. Instead I get EPERM. > >> If I touch the file it does in fact break the hardlink to the >> hash and I can then hardlink the file I have just touched. > >> This is all well and good if I am doing this in a script in >> the vserver but some programs (Debian's dpkg for example) use >> hardlinking in their logic. > >> Is this a bug? >> Is it an enhancement? > > Link breaking happens in various places in the kernel > and it might be simply that this case was never tested > an so just forgotten. > > I wouldn't consider it a bug, more a missing feature > and if we figure out where it happens, it will be quite > simple to add. > >> Is it because of aufs? > > maybe, but that should be rather simple to test, no? > i.e. just use ext3 and let us know if the same thing > happens there ... > >> Is there any reason why the kernel cannot treat the link system >> call the same way as changing the file metadata by touching the >> file? > > The VFS part of the Linux Kernel is quite complicated. > >> This is not a major obstacle to further development of my >> project but it would be nice if I had a solution. > >> Maybe there is a solution using lvm and snapshots but for >> all I know I will hit the same problem there. > > I don't see how LVM and/or snapshots would be related > at all .... > > best, > Herbert > >> I have many other tests to do and I can revisit giving guests >> rw versions of a common filesystem later. > >> Thanks to all involved in vservers. > >> Allan >