Subject: Re: [vserver] Context disk quotas
From: "Daniel Hokka Zakrisson" <daniel@hozac.com>
Date: Tue, 18 Mar 2008 23:05:28 +0100 (CET)

vitalyb wrote:
> Hello,
>
> I have a problem setting up quotas inside of vservers.
> Looks like kernel ignores XID while updating /aquota.* files or
> quota tools aren't working correctly.
>
> Tested with these setups:
> 1) Linux 2.6.22.19 + 2.6.22.18-vs2.2.0.6-grsec2.1.11-20080212
> 2) Linux 2.6.22.19 + 2.6.22.19-vs2.2.0.7
>
> VServer config:
>
> # CONFIG_VSERVER_LEGACY is not set
> # CONFIG_VSERVER_LEGACYNET is not set
> # CONFIG_VSERVER_REMAP_SADDR is not set
> CONFIG_VSERVER_COWBL=y
> # CONFIG_VSERVER_VTIME is not set
> CONFIG_VSERVER_PROC_SECURE=y
> CONFIG_VSERVER_HARDCPU=y
> CONFIG_VSERVER_IDLETIME=y
> # CONFIG_VSERVER_IDLELIMIT is not set
> # CONFIG_TAGGING_NONE is not set
> # CONFIG_TAGGING_UID16 is not set
> # CONFIG_TAGGING_GID16 is not set
> CONFIG_TAGGING_ID24=y
> # CONFIG_TAGGING_INTERN is not set
> # CONFIG_TAG_NFSD is not set
> # CONFIG_PROPAGATE is not set
> CONFIG_VSERVER_PRIVACY=y
> CONFIG_VSERVER_CONTEXTS=768
> CONFIG_VSERVER_WARN=y
> # CONFIG_VSERVER_DEBUG is not set
> CONFIG_VSERVER=y
> CONFIG_VSERVER_SECURITY=y
> CONFIG_VSERVER_NGNET=y
>
> Test:
> -------------------------------------------------------------------
> # ./testfs.sh -l -t -D /dev/loop0 -M /mnt

You want -x -y -z to actually run some tests too.

> Linux-VServer FS Test [V0.18] Copyright (C) 2005-2007 H.Poetzl
> Linux 2.6.22.19-grsec-vs x86_64/0.30.214
> VCI:  0002:0200 236 030007b1 (ID24)
> ---
> testing ext2 filesystem ...
> [000]. [001]. [002]. [999].
> ---
> testing ext3 filesystem ...
> [000]. [001]. [002]. [999].
> ---
> testing xfs filesystem ...
> [000]* (xfs format failed)
> ---
> testing reiser filesystem ...
> [000]* (reiserfs format failed)
> ---
> testing jfs filesystem ...
> [000]* (jfs format failed)
> -------------------------------------------------------------------
> # ./testfs.sh -t -D /dev/loop0 -M /mnt/tst/
> Linux-VServer FS Test [V0.18] Copyright (C) 2005-2007 H.Poetzl
> Linux 2.6.22.19-grsec-vs x86_64/0.30.214
> VCI:  0002:0200 236 030007b1 (ID24)
> ---
> testing ext2 filesystem ...
> [000]. [001]. [002]. [999].
> ---
> testing ext3 filesystem ...
> [000]. [001]. [002]. [999].
> ---
> testing xfs filesystem ...
> [000]* (xfs format failed)
> ---
> testing reiser filesystem ...
> [000]* (reiserfs format failed)
> ---
> testing jfs filesystem ...
> [000]* (jfs format failed)
> -------------------------------------------------------------------
> # ./testme.sh
> Linux-VServer Test [V0.17] Copyright (C) 2003-2006 H.Poetzl
> chcontext is working.
> chbind is working.
> Linux 2.6.22.19-grsec-vs #3 SMP Sat Mar 15 22:46:19 EET 2008 x86_64
> Ea 0.30.214 236/glibc (DSa) <v13,net,v21,v22,v23,netv2>
> VCI: 0002:0200 236 030007b1 (TbsPHIW)
> ---
> [000]# succeeded.
> [001]# succeeded.
> [011]# succeeded.
> [031]# succeeded.
> [101]# succeeded.
> [102]# succeeded.
> [201]# succeeded.
> [202]# succeeded.
> -------------------------------------------------------------------
>
>
> Problem description:
>
> files are tagged with chxid, context 1001
>  > host# quota -V
>  > Quota utilities version 3.14.
>  > Compiled with EXT2_DIRECT
>  > host# quotacheck /mnt
>  > host# quotaon /mnt
>  > host# repquota /mnt
>  >
>  > *** Report for user quotas on device /dev/mapper/storage-hosting
>  > Block grace time: 7days; Inode grace time: 7days
>  >                         Block limits                File limits
>  > User            used    soft    hard  grace    used  soft  hard  grace
>  > ----------------------------------------------------------------------
>  > root      --  134224       0       0              5     0     0
>  > #50331648 --   15956       0       0             70     0     0
>  > #50341673 --      12       0       0              3     0     0
>  > #50341706 --      12       0       0              3     0     0
>  > #50341739 --      12       0       0              3     0     0
>  > ....... etc,etc,etc
>
> User ids are weird because of tagging, I believe, for example,
> 50331648 = 0x3000000 -> uid 0
>
> Is this the way it supposed to be?

Tagging doesn't make much sense in combination with quotas, as you need a
separate filesystem per guest (see below).

> next, let's create some files
>
>  > vs# dd if=/dev/zero bs=1M count=100 of=file1
>  > vs# chown 10001 file
>  > host# repquota /mnt
>  > Block grace time: 7days; Inode grace time: 7days
>  >                         Block limits                File limits
>  > User            used    soft    hard  grace    used  soft  hard  grace
>  > ----------------------------------------------------------------------
>  > root      --  134224       0       0              5     0     0
>  > #50331648 --   15956       0       0             70     0     0
>  > #50341673 --      12       0       0              3     0     0
>  > ...
>  > #10001    --  102504       0       0              1     0     0
>  > ...
>
> using vroot device everything's the same and
>  > vs# repquota -n /dev/hd
>  > Block grace time: 7days; Inode grace time: 7days
>  >                         Block limits                File limits
>  > User            used    soft    hard  grace    used  soft  hard  grace
>  > ----------------------------------------------------------------------
>  > #0        --  134224       0       0              5     0     0
>  > #50331648 --   15956       0       0             70     0     0
>  > ...
>  > #10001    --  102504       0       0              1     0     0
>  > ...
>
> next,
>  > host# quotaoff /mnt
>  > host# quotacheck /mnt
>  > host# quotao /mnt
>  > host# repquota /mnt
>  > root      --  134220       0       0              4     0     0
>  > #50331648 --   15960       0       0             71     0     0
>  > ...
>  > #50341649 --  102532       0       0              8     0     0
>  > ...
>
> Compiling quota without EXT2_DIRECT support makes quotacheck produce
> real uids without xid part, but, again, quota does not account xid
> values and calculate quotas per uid only.
>
> So, what am I missing? Is there a way to make context quotas work
> on a shared partition without sharing quota between same uids
> on different vservers?

No, current 2.6 kernels do not support per-guest user/group quotas on
shared filesystems. You need to setup separate filesystems for each guest,
and create vroots for them. See
http://linux-vserver.org/Standard_non-shared_quota for more information.

> Thanks!
>
> --
> Regards,
> Vitaly
>


-- 
Daniel Hokka Zakrisson