Subject: Re: [vserver] NetApp filers and NFS junctions: Operation not permitted [RFC]
From:Cédric Dufour - Idiap Research Institute <cedric.dufour@idiap.ch>
Date: Fri, 31 Jul 2009 09:03:37 +0200
Fri, 31 Jul 2009 09:03:37 +0200
Hello VServer maintainers,

I'm sorry to bump the subject up again, but we'd really need some
feedback about the modifications  we need to have NFS work with our
multi-100k$ filers (and which may certainly benefit other NFS/CIFS/AFS
users in the future); Please see 70~ lines patch attached in previous
message in this ML thread (
http://list.linux-vserver.org/archive?mss:2708:200907:pfggfkbbnhmhomeamdng
).

Are those modifications plain stupid or does they seem acceptable?

Thank you very much for your insight.

Have a good day,

Cédric Dufour @ Idiap Research Institute



On 24/07/09 10:55, Cédric Dufour - Idiap Research Institute wrote:
> Finally, here is the least "invasive" modification - from a security
> point of view - we could come up with (see attached patch). NFS
> junctions now work as expected. As for security, we'd definitively need
> some feedback ;-)
>
> Cheers
>
> Cédric Dufour @ Idiap Research Institute
>
>
>
> On 23/07/09 18:46, Cédric Dufour - Idiap Research Institute wrote:
>   
>> [STRIPPED]
>>
>> On 23/07/09 15:22, Cédric Dufour - Idiap Research Institute wrote:
>>     
>>> Hello again,
>>>
>>> After some research in the kernel tree, it seems that the
>>> 'vfs kern mount' function gets called directly:
>>>  - to handle CIFS DFS (in function 'cifs dfs do refmount')
>>>  - to handle AFS automounts (in function 'afs mntpt do automount')
>>>  - to handle NFS "junctions" (in function 'nfs follow mountpoint')
>>> In other words, for all file systems that require some in-kernel
>>> "sub-mounting", and which should NOT be blocked in anyway (provided the
>>> "root" mount point has been successfully mounted, see below)
>>>
>>> Indirectly (via 'do kern mount'), to performs actual "out-of-kernel"
>>> mount requests.
>>>
>>> Indirectly (via 'kern mount data', with a MS KERNMOUNT flag that is used
>>> subsequently by 'selinux sb kern mount' to relax security) for:
>>>  - '/proc' sub-system
>>>  - IPC sub-system
>>>
>>> This suggests that the capability checks achieved in 'vfs kern mount'
>>> would - maybe - be better fitted and adapted in the calling
>>> functions/contexts:
>>>  - in 'do new mount' for actual file systems mount (where we find an
>>> existing 'capable('CAP SYS ADMIN')' test ;-) )
>>>  - ... I don't know about /proc and IPC ... (but aren't handling the
>>> former with the 'PROC SUPER MAGIC' flag ?)
>>>
>>> What do you think ?
>>>
>>> Cédric Dufour @ Idiap Research Institute
>>>
>>>
>>>
>>> On 23/07/09 13:19, Cédric Dufour - Idiap Research Institute wrote:
>>>   
>>>       
>>>> Hello,
>>>>
>>>> We are currently putting our new NetApp ONTAP GX system into
>>>> production. From the NFS point of view, the particularity of this
>>>> product is that it allows to "assemble" multiple volumes within a
>>>> "unified" namespace, using so-called "junctions", and access this
>>>> "unified" namespace with a (apparently) single mount operation.
>>>>
>>>> Now, when using VServer - even from the host point of view - only the
>>>> root user is able to "roam" through the mounted hierarchy (as separate
>>>> volumes get automatically mounted in the background and appear in
>>>> '/proc/mounts'). Non-root user get a "Operation not permitted" message
>>>> as soon as they try to go pass the initially mounted volume.
>>>>
>>>> This problem seem absolutely similar to what was discussed in this
>>>> thread: http://www.paul.sladen.org/vserver/archives/200709/0046.html
>>>>
>>>> We have verified that removing the VServer-added checks in the
>>>> 'vfs kern mount' function (in '<kernel-source>/fs/super.c') "solves"
>>>> the problem (we used the latest 2.6.30.2 vanilla kernel and
>>>> 2.3.0.36.14pre4 VServer patches to perform this verification).
>>>>
>>>> Now to the questions:
>>>>
>>>>  1. What is the security impact of removing those checks (iow. is it a
>>>> "MUST NOT!!!" or a "well... that's being overkill anyway" ) ?
>>>>
>>>>  2. Can we imagine to perform those checks slightly differently, in
>>>> order to allow non-root users to use such "joined" NFS exports, both
>>>> from the host and guests (since we're no kernel gurus, we don't know
>>>> which function/capability to start looking for) ?
>>>>
>>>> Thank you very much for your help and time.
>>>>
>>>> Cheers
>>>>
>>>> -- 
>>>>
>>>> Cédric Dufour @ Idiap Research Institute
>>>>
>>>> Phone:  +41 27 721 77 40
>>>> Fax:    +41 27 721 77 12
>>>> Mail:   Idiap Research Institute
>>>>         Case postale 592
>>>>         Centre du Parc
>>>>         1920 Martigny (VS)
>>>>         Suisse (Switzerland)
>>>> Web:    www.idiap.ch <http://www.idiap.ch>
>>>>
>>>>     
>>>>         
>>>   
>>>       


Hello VServer maintainers,

I'm sorry to bump the subject up again, but we'd really need some feedback about the modifications  we need to have NFS work with our multi-100k$ filers (and which may certainly benefit other NFS/CIFS/AFS users in the future); Please see 70~ lines patch attached in previous message in this ML thread ( http://list.linux-vserver.org/archive?mss:2708:200907:pfggfkbbnhmhomeamdng ).

Are those modifications plain stupid or does they seem acceptable?

Thank you very much for your insight.

Have a good day,

Cédric Dufour @ Idiap Research Institute

Cédric Dufour @ Idiap Research Institute



On 24/07/09 10:55, Cédric Dufour - Idiap Research Institute wrote:
Finally, here is the least "invasive" modification - from a security
point of view - we could come up with (see attached patch). NFS
junctions now work as expected. As for security, we'd definitively need
some feedback ;-)

Cheers

Cédric Dufour @ Idiap Research Institute



On 23/07/09 18:46, Cédric Dufour - Idiap Research Institute wrote:
  
[STRIPPED]

On 23/07/09 15:22, Cédric Dufour - Idiap Research Institute wrote:
    
Hello again,

After some research in the kernel tree, it seems that the
'vfs_kern_mount' function gets called directly:
 - to handle CIFS DFS (in function 'cifs_dfs_do_refmount')
 - to handle AFS automounts (in function 'afs_mntpt_do_automount')
 - to handle NFS "junctions" (in function 'nfs_follow_mountpoint')
In other words, for all file systems that require some in-kernel
"sub-mounting", and which should NOT be blocked in anyway (provided the
"root" mount point has been successfully mounted, see below)

Indirectly (via 'do_kern_mount'), to performs actual "out-of-kernel"
mount requests.

Indirectly (via 'kern_mount_data', with a MS_KERNMOUNT flag that is used
subsequently by 'selinux_sb_kern_mount' to relax security) for:
 - '/proc' sub-system
 - IPC sub-system

This suggests that the capability checks achieved in 'vfs_kern_mount'
would - maybe - be better fitted and adapted in the calling
functions/contexts:
 - in 'do_new_mount' for actual file systems mount (where we find an
existing 'capable('CAP_SYS_ADMIN')' test ;-) )
 - ... I don't know about /proc and IPC ... (but aren't handling the
former with the 'PROC_SUPER_MAGIC' flag ?)

What do you think ?

Cédric Dufour @ Idiap Research Institute



On 23/07/09 13:19, Cédric Dufour - Idiap Research Institute wrote:
  
      
Hello,

We are currently putting our new NetApp ONTAP GX system into
production. From the NFS point of view, the particularity of this
product is that it allows to "assemble" multiple volumes within a
"unified" namespace, using so-called "junctions", and access this
"unified" namespace with a (apparently) single mount operation.

Now, when using VServer - even from the host point of view - only the
root user is able to "roam" through the mounted hierarchy (as separate
volumes get automatically mounted in the background and appear in
'/proc/mounts'). Non-root user get a "Operation not permitted" message
as soon as they try to go pass the initially mounted volume.

This problem seem absolutely similar to what was discussed in this
thread: http://www.paul.sladen.org/vserver/archives/200709/0046.html

We have verified that removing the VServer-added checks in the
'vfs_kern_mount' function (in '<kernel-source>/fs/super.c') "solves"
the problem (we used the latest 2.6.30.2 vanilla kernel and
2.3.0.36.14pre4 VServer patches to perform this verification).

Now to the questions:

 1. What is the security impact of removing those checks (iow. is it a
"MUST NOT!!!" or a "well... that's being overkill anyway" ) ?

 2. Can we imagine to perform those checks slightly differently, in
order to allow non-root users to use such "joined" NFS exports, both
from the host and guests (since we're no kernel gurus, we don't know
which function/capability to start looking for) ?

Thank you very much for your help and time.

Cheers

-- 

Cédric Dufour @ Idiap Research Institute

Phone:  +41 27 721 77 40
Fax:    +41 27 721 77 12
Mail:   Idiap Research Institute
        Case postale 592
        Centre du Parc
        1920 Martigny (VS)
        Suisse (Switzerland)
Web:    www.idiap.ch <http://www.idiap.ch>