Subject: Re: [vserver] Hashify and exclude
From: "Michael S. Zick" <mszick@morethan.org>
Date: Tue, 28 Oct 2008 08:10:23 -0500

On Tue October 28 2008, Ed W wrote:
> Michael S. Zick wrote:
> > On Mon October 27 2008, Ed W wrote:
> >   
> >>>> What is the argument against unifying everything? 
> >>>>     
> >>>>         
> >>> on recente kernels (i.e. in the presence of
> >>> CoW Link Breaking), the only case where a
> >>> unified file might not work is when some
> >>> application checks the file properties and
> >>> requires them to have a link count of 1
> >>>  
> >>>       
> >> Subtle extra one, but as I understand it, the COW stuff breaks ALL
> >> hardlinks on write, so if you have a bunch of hard links within the
> >> vserver for some internal process, then additionally hardlinking those
> >> *across* vservers will cause all the hardlinks (even within the vserver)
> >> to be broken if the file is ever altered.  This may or may not be a
> >> problem in general (seems like a corner case for most people?)
> >>
> >>     
> >
> > It does not work that way here (2.6.27.4+vs2.3)
> >
> > I am using CoW to detect changes to files in replicated development trees -
> > Given a file (inode) with 8 (upto 4000+ in my experience) different appearances
> > in the directory tree - 
> >
> > Changing it creates one, total of (2) inodes, one with a link count of 7 and 
> > the changed one with a link count of 1.
> >   
> 
> 
> Check again. 
> 
> For example on my distro (gentoo), large numbers of my timezone files 
> are just hardlinks of each other.  So imagine file TZ which is hard 
> linked 8 times within a single install and then cross hardlinked across 
> 20 vservers.  When I change that one file on a single vserver I expect 
> it become a hardlink with a count of 8 (again) and the other vservers to 
> all be hardlinked to each other still (link count 19x8). 
> 
> What you are describing is that ALL the hardlinks get broken for that 
> one file.
>

There _is_ only one file - which appears in the device:inode naming system
under different pathnames.  
Change it through one of the eight access points (pathname/filename access)
and "all _other_ hardlinks get broken to the new information" - which is
not quite so, they never came into being, they _others_ point at the old inode,
only the access point gets a new inode with the changed information.

> 
> Hypothetically in my example I would update one timezone file and that 
> would leave all the rest of them in my vserver pointing at the old value 
> which is shared across the other vservers.  
>

That is exactly the behavior I am depending on - it would be quickly
noticed if that behavior failed.

Perhaps an example:

I have a directory used as a content ID reference, outside of the trees
which are replicated.  The "Filename" here is the sha1sum of a file's
content.  I am presuming all files with the same sha1sum are bitwise
identical (there is a very small chance that is wrong but "good enough"
in practice).

Here is the content of one such catalog file:

WRT/WRT_cat/0008f20154b8ca6c6c09b3b17dd22f8fedeec07: (unless I typo'd that name)

900|4292133|8|/WRT/oWRT/trunk/target/linux/brcm63xx/files/include/asm-mips/mach-bcm963xx/.svn/text-base/bcmTag.h.svn-base
900|4292133|8|/WRT/oWRT/trunk/target/linux/brcm63xx/files/include/asm-mips/mach-bcm963xx/bcmTag.h
900|4292133|8|/WRT/openWRT/trunk/target/linux/brcm63xx/files/include/asm-mips/mach-bcm963xx/.svn/text-base/bcmTag.h.svn-base
900|4292133|8|/WRT/openWRT/trunk/target/linux/brcm63xx/files/include/asm-mips/mach-bcm963xx/bcmTag.h
900|4292133|8|/WRT/refWRT/trunk/target/linux/brcm63xx/files/include/asm-mips/mach-bcm963xx/.svn/text-base/bcmTag.h.svn-base
900|4292133|8|/WRT/refWRT/trunk/target/linux/brcm63xx/files/include/asm-mips/mach-bcm963xx/bcmTag.h
900|4292133|8|/WRT/wrkWRT/trunk/target/linux/brcm63xx/files/include/asm-mips/mach-bcm963xx/.svn/text-base/bcmTag.h.svn-base
900|4292133|8|/WRT/wrkWRT/trunk/target/linux/brcm63xx/files/include/asm-mips/mach-bcm963xx/bcmTag.h

Thats: Device|Inode|Link Count|absolute pathname in the filesystem naming system.

Pick one, any one, they are all the same inode, change it, note that the
inode shown by ls -li is now different and the hard link count is 1 of the
changed file, the other seven (any of other seven names for the orginal inode) 
now have a hard link count of 7 and still reference 900:4292133

For my purposes, the scripting that runs the catalog file, looking for
generation changes, has to process about 10,000 files with about 120,000
name appearances in the file system - I would notice real quick if it
wasn't working.  My generation changesets would be wrong.


> This might be a surprise if  
> I was expecting all my local hardlinks to allow me to update one file 
> and see several files change...
> 
> I have tentatively examined the hashify code to get it to ignore files 
> with a link count >1 and +iunlink/+immutable set.  Seems to be workble?  
> Thoughts?
> 

Can't say - I don't use that code.  I am using my own which replicates
an existing tree.  I.E: my multiple directory trees do not pre-exist.
But since that code was intended to be used on pre-existing trees, it
certainly must have an "exclude" feature built in.

Mike
> Ed W
>