Subject: Re: [vserver] make the host safe with 10% of ressources :)
From: Herbert Poetzl <herbert@13thfloor.at>
Date: Tue, 4 Sep 2007 17:00:30 +0200

On Tue, Sep 04, 2007 at 09:02:12AM +0200, ADNET Ghislain wrote:
> Hello,
> 
>  I have started to play with vserver limits to set limits on guest so
> the host allways got some cpu/ram for itself. Right now i am failling
> to make it happen. I read the docs on the website but i still have
> problem making it work. 

> The cpu limits seems to not be enforced, the ram limits seems but
> since i used them the server does not use swap anymore at all so i
> wonder if i missconfigured something.

most likely, as the SCHED_HARD (token bucket hard limits)
is exactly what you want, you even can give the 10% you
want to 'reserve' as idle share to the guests, which will
make use of that 10% when the host isn't running anything
(but that is your choice)

> Let's see the settings for the vserver:
> 
> My goal is to limit the physical ram to  something like 80/90% of the 
> ram , same thing for the swap so the host still got:
> 
> - 10% of cpu

as described, simple to do

> - 10% of ram

very hard to do, unless you switch off overcommitment
(btw, this isn't terribly important, as ram can always
be freed by discading buffers and read only mappings

> - 10% of swap

not implemented yet, but having strict limits to the
guest memory will automagically leave some of the swap
space (swap+ram - Sum[guests]) unused by guest processes

but if you really want absolute memory control and a
guarantee that no process will get more memory than 
available, then you have to turn off overcommitment
on your linux system 

> I want to enforce this on a PIV ht 1Go ram on Debian GNU/Linux 4.0r1 
> (etch) with :
> 
> 
> Versions:
>                   Kernel: 2.6.22.2-vs2.2.0.3
>                   VS-API: ???
>             util-vserver: 0.30.212; Dec 31 2006, 12:22:23
> 
> Features:
>                       CC: gcc, gcc (GCC) 3.3.5 (Debian 1:3.3.5-13)
>                      CXX: g++, g++ (GCC) 3.3.5 (Debian 1:3.3.5-13)
>                 CPPFLAGS: ''
>                   CFLAGS: '-g -O2 -std=c99 -Wall -pedantic -W'
>                 CXXFLAGS: '-g -O2 -ansi -Wall -pedantic -W 
> -fmessage-length=0'
>               build/host: i686-pc-linux-gnu/i686-pc-linux-gnu
>             Use dietlibc: yes
>       Build C++ programs: yes
>       Build C99 programs: yes
>           Available APIs: v13,net,v21
>            ext2fs Source: e2fsprogs
>    syscall(2) invocation: alternative
>      vserver(2) syscall#: 273/glibc
> 
> Paths:
>                   prefix: /usr/local
>        sysconf-Directory: ${prefix}/etc
>            cfg-Directory: ${prefix}/etc/vservers
>         initrd-Directory: /etc/init.d
>       pkgstate-Directory: ${prefix}/var/run/vservers
>          vserver-Rootdir: /vservers
> 
> 
> 
>  So i put:
> 
> flags:
> 
> sched_hard
> virt_load
> virt_mem
> 
> (root)> getconf  PAGESIZE
> 4096
> 
> (aqadmin)> more /proc/virtual/40134/limit
> Limit    current             min/max                soft/hard           hits
> PROC:         73               0/     210             -1/      
> -1            0
> VM:       165385               0/  808614             -1/      
> -1            0
> VML:           0               0/       0             -1/      
> -1            0
> RSS:       81226               0/  235929         200000/  
> 235929          180

here you see the memory limit was hit 180 times

> ANON:      56588               0/  220930             -1/      
> -1            0
> RMAP:      24638               0/   77728             -1/      
> -1            0
> FILES:       870               0/    1130             -1/      
> -1            0
> OFD:        1309               0/    1370             -1/      
> -1            0
> LOCKS:        11               0/      50             -1/      
> -1            0
> SOCK:         76               0/     106             -1/      
> -1            0
> MSGQ:          0               0/       0             -1/      
> -1            0
> SHM:         452               0/     452             -1/      
> -1            0
> SEMA:          5               0/       5             -1/      
> -1            0
> SEMS:          5               0/       5             -1/      
> -1            0
> DENT:      66666               0/   66691             -1/      
> -1            0
> 
> (aqadmin)> more /proc/virtual/40134/sched
> FillRate:              9,2
> Interval:             10,10
> TokensMin:            25
> TokensMax:           125
> PrioBias:              0
> cpu 0: 2210383 258188 35969 57648202 1252440 RI 125 25 125 9/10 2/10 0 0
> cpu 1: 1437742 146506 40107 54003732 786430 RI 125 25 125 9/10 2/10 0 0

the token bucket scheduler is also working as expected,
and you have idle time enabled (thus it will use up
idle cpu too, disabling that should bring your guest
down to 90%)

HTH,
Herbert

> /proc/virtual/40134/status
> UseCnt: 170
> Tasks:  50
> Flags:  00000006020b0110
> BCaps:  00000000344c05ff
> CCaps:  0000000000000101
> Spaces: 0c020200
> 
> 
> for the cpu and the limit you saw upside. Which setting i got wrong to 
> have the desired effect ? :)
> 
> _here a sniplet of top on the host :
> 
> _top - 08:53:52 up 3 days,  8:16,  6 users,  load average: 0.83, 0.35, 0.20
> Tasks: 113 total,   4 running, 109 sleeping,   0 stopped,   0 zombie
> Cpu(s):100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  
> 0.0%st
> Mem:   1034404k total,   442080k used,   592324k free,   105128k buffers
> Swap:  1028152k total,       32k used,  1028120k free,   116696k cached
> 
>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 13175 root      25   0    92    8    4 R  101  0.0   0:20.15 burnP6
> 13177 root      25   0    92   12    4 R  100  0.0   0:19.03 burnP6
> 13148 root      15   0  2232 1152  860 R    1  0.1   0:00.26 top
> 
> _and on the vserver:
> 
> _top - 08:56:05 up 24 min,  0 users,  load average: 0.40, 0.19, 0.07
> Tasks:  54 total,   4 running,  50 sleeping,   0 stopped,   0 zombie
> Cpu(s): 99.3% us,  0.7% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,  0.0% si
> Mem:    800000k total,   287368k used,   512632k free,        0k buffers
> Swap:   143716k total,        0k used,   143716k free,        0k cached
> 
>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 13324 root      25   0    92   12    4 R 99.5  0.0   0:16.84 burnP6
> 13326 root      25   0    96   12    4 R 97.9  0.0   0:07.50 burnP6
> 12630 www-data  17   0 19876 7736 4072 S  0.3  1.0   0:00.24 apache2
>    1 root      15   0  1944  640  544 S  0.0  0.1   0:01.06 init
> 11892 root      15   0  1632  540  440 S  0.0  0.1   0:00.03 syslogd
> 
> 
>   Any help appreciated ! :)
> 
> 
> regards,
> Ghislain.
>