Subject: Re: [vserver] Q. about scheduling parameters and HARD_SCHED
From: Herbert Poetzl <herbert@13thfloor.at>
Date: Mon, 30 Mar 2009 21:13:09 +0200

On Sat, Mar 28, 2009 at 01:23:22PM +0100, Markus Fischer wrote:
> Hi,
> 
> [warning: very long email, much config output]
> 
> I'm not sure if I've got things right with the scheduling. I've a few
> vservers runnning with different configurations when comparing the
> information I set in /etc/vservers/*/sched/* and the information I get
> from /proc/virtual/*/sched I'm not sure if everything is fine. I've
> attached the kernel/vserver output at the end. This vservers are solely
> for a development environment, providing dev servers for Apache2, PHP5,
> Tomcat5.5, MySQL5, Samba, MediaWiki, etc., all running on Debian, either
> Etchbut most of them Lenny.
> 
> In /proc/virtual/*/sched, next to FillRate and Interval is always
> reported a second number after a comma, what does that mean? E.g.
> 
> 101
> FillRate:	       1,1
> Interval:	       1,8
> 
> I've configured this context 101 to be run with fill-rate:1 and
> interval:2 in /etc/, however I've re-configured it at runtime with
> 'vsched --xid 101 --interval 1' to boost it. Am I right that this change
> will not last after restart of that vserver? So the vsched tool is just
> for runtime changes?

yep, and it is used at guest startup to implement
the config settings ...

> As can be seen in the configurations below, I've two vservers with
> SCHED_HARD but not actual fill-rate/interval given (context 121 and
> 122). Is it expected that those vservers run at FillRate:1 and
> Interval:4 ? 

yes, that is the kernel default, if nothing is set

> So actually I enabled it, forget to configure it and though
> it won't be cut, but actually it is?

correct, enabling the hard limit will 'by default'
limit it to roughly 25% cpu

> And my last question: when using the hard scheduling and configuring
> it, it permanently cuts the available processing power for that
> vserver. If I give it FillRate:1/Interval:2, it always runs at half
> the power it could have. Now I'm not really sure if this is what I
> really want/need anyway. When I've two vservers both configured to
> 1/2, they always run and half speed and even if both start number
> crunching, no one is taking away processing power from the other. 

> But if only one of them crunches, they cannot use the full CPU
> resources and crunch at half the speed and actually there would be no
> difference when both start crunching. Can that be true?

correct, that is what the idle time (or the CFS in newer
kernels) is for (and the idle skipping)

> Given that, I would probably just remove SCHED_HARD from all vservers
> and just let them take what they need because I'm permanently cutting
> power from vservers for my developers?

another alternative to run on fair scheduling and not
with hard limits, yes

> But when a process within a vserver goes berserk, one vserver can
> stall the whole host/guests, is this assumption right? Are there ways
> to avoid this?

only to some extend .. i.e. if there are many processes
which do nothing but hogging cpu, and the other guest
has just a single process trying to get some work done,
then it might be a problem ...

> Or, re-phrased: under what circumstances to I really want hard CPU
> scheduling?

if you sell 10% per guest, and that's what the customer
pays for, then you want to keep it at 10% :)

> Companioned by it's name "hard scheduling", is there "soft schedulung"
> available too, or is this just used to make it clean it cuts
> processing power for real?

yes, the 'soft' scheduling is called priority scheduling
and uses the TB scheduling extension to adjust the priorities

HTC,
Herbert

> This are my configuration from /etc/vservers/*/sched/* ; I've included
> the context IDs so it can be easier compared to the /proc/ output below:
> 
> context: 101
> SCHED_HARD flag set: yes
> fill-rate: 1
> interval: 2
> 
> context: 102
> SCHED_HARD flag set: yes
> fill-rate: 1
> interval: 2
> 
> context: 121
> SCHED_HARD flag set: yes
> fill-rate: cat: db01/sched/fill-rate: No such file or directory
> interval: cat: db01/sched/interval: No such file or directory
> 
> context: 122
> SCHED_HARD flag set: yes
> fill-rate: cat: db02/sched/fill-rate: No such file or directory
> interval: cat: db02/sched/interval: No such file or directory
> 
> context: 143
> SCHED_HARD flag set: yes
> fill-rate: 1
> interval: 2
> 
> context: 20
> SCHED_HARD flag set: no
> fill-rate: 1
> interval: 3
> 
> context: 4
> SCHED_HARD flag set: yes
> fill-rate: 1
> interval: 4
> 
> context: 50
> SCHED_HARD flag set: yes
> fill-rate: 1
> interval: 2
> 
> 
> Now the information reported by /proc/virtual/*/sched:
> 101
> FillRate:	       1,1
> Interval:	       1,8
> TokensMin:	      15
> TokensMax:	     125
> PrioBias:	       0
> cpu 0: 2308702 336569 1419699 338107884 0 R- 125 15 125 1/1 1/8 0 0
> cpu 1: 3549106 437821 2668054 338016022 0 R- 118 15 125 1/1 1/8 0 0
> cpu 2: 1658762 254835 685292 337991118 0 R- 125 15 125 1/1 1/8 0 0
> cpu 3: 3170241 377575 2119382 338180257 0 R- 125 15 125 1/1 1/8 0 0
> 
> 102
> FillRate:	       1,1
> Interval:	       2,8
> TokensMin:	      15
> TokensMax:	     125
> PrioBias:	       0
> cpu 0: 1477097 310027 24545 416629116 0 R- 125 15 125 1/2 1/8 0 0
> cpu 1: 1593197 331248 29256 415334288 0 R- 125 15 125 1/2 1/8 0 0
> cpu 2: 1471072 308125 49452 422348846 0 R- 125 15 125 1/2 1/8 0 0
> cpu 3: 1220053 246043 44160 420281533 0 R- 125 15 125 1/2 1/8 0 0
> 
> 121
> FillRate:	       1,1
> Interval:	       4,8
> TokensMin:	      15
> TokensMax:	     125
> PrioBias:	       0
> cpu 0: 13779 3018 1543 18678542 0 R- 125 15 125 1/4 1/8 0 0
> cpu 1: 94838 5333 121055 18655019 0 R- 125 15 125 1/4 1/8 0 0
> cpu 2: 60412 4125 83845 18679959 0 R- 125 15 125 1/4 1/8 0 0
> cpu 3: 23436 3815 1660 18678046 0 R- 125 15 125 1/4 1/8 0 0
> 
> 122
> FillRate:	       1,1
> Interval:	       4,8
> TokensMin:	      15
> TokensMax:	     125
> PrioBias:	       0
> cpu 0: 880942 53180 522479 216972295 0 R- 125 15 125 1/4 1/8 0 0
> cpu 1: 1571503 83894 1009672 216955251 0 R- 125 15 125 1/4 1/8 0 0
> cpu 2: 434831 35950 225719 216957323 0 R- 125 15 125 1/4 1/8 0 0
> cpu 3: 361288 35077 108402 216984103 0 R- 125 15 125 1/4 1/8 0 0
> 
> 143
> FillRate:	       1,1
> Interval:	       2,8
> TokensMin:	      15
> TokensMax:	     125
> PrioBias:	       0
> cpu 0: 2054678 267339 417069 414483366 0 R- 125 15 125 1/2 1/8 0 0
> cpu 1: 2196427 278635 457017 413633516 0 R- 125 15 125 1/2 1/8 0 0
> cpu 2: 1826417 251302 340132 415917644 0 R- 125 15 125 1/2 1/8 0 0
> cpu 3: 2170892 295417 416855 414486633 0 R- 125 15 125 1/2 1/8 0 0
> 
> 20
> FillRate:	       1,1
> Interval:	       3,8
> TokensMin:	      15
> TokensMax:	     125
> PrioBias:	       0
> cpu 0: 6130931 338038 0 0 0 R- 62 15 125 1/3 1/8 0 0
> cpu 1: 3036856 210304 0 0 0 R- 62 15 125 1/3 1/8 0 0
> cpu 2: 7850650 311422 0 0 0 R- 62 15 125 1/3 1/8 0 0
> cpu 3: 5333722 220078 0 0 0 R- 62 15 125 1/3 1/8 0 0
> 
> 4
> FillRate:	       1,1
> Interval:	       4,8
> TokensMin:	      15
> TokensMax:	     125
> PrioBias:	       0
> cpu 0: 11972 5788 276 217111999 0 R- 125 15 125 1/4 1/8 0 0
> cpu 1: 15103 8847 1102 217227271 0 R- 125 15 125 1/4 1/8 0 0
> cpu 2: 13151 6134 1004 217132314 0 R- 125 15 125 1/4 1/8 0 0
> cpu 3: 13211 8486 668 217188812 0 R- 125 15 125 1/4 1/8 0 0
> 
> 50
> FillRate:	       1,1
> Interval:	       2,8
> TokensMin:	      15
> TokensMax:	     125
> PrioBias:	       0
> cpu 0: 74230 13049 1711 58120235 0 R- 124 15 125 1/2 1/8 0 0
> cpu 1: 86459 14154 1506 58161889 0 R- 125 15 125 1/2 1/8 0 0
> cpu 2: 71691 12226 1252 58133271 0 R- 125 15 125 1/2 1/8 0 0
> cpu 3: 82949 13151 845 58154866 0 R- 125 15 125 1/2 1/8 0 0
> 
> 
> 
> 
> vserver-info output:
> 
> Versions:
>                    Kernel:
> 2.6.22.19-vs2.3.0.34-20090215-nd4-nd-vserv-bigmem4
>                    VS-API: 0x00020302
>              util-vserver: 0.30.216-pre2772; Jan 13 2009, 12:32:16
> 
> Features:
>                        CC: gcc, gcc (GCC) 4.1.2 20061115 (prerelease)
> (Debian 4.1.1-21)
>                       CXX: g++, g++ (GCC) 4.1.2 20061115 (prerelease)
> (Debian 4.1.1-21)
>                  CPPFLAGS: ''
>                    CFLAGS: '-Wall -g  -O2 -std=c99 -Wall -pedantic -W
> -funit-at-a-time'
>                  CXXFLAGS: '-g -O2 -ansi -Wall -pedantic -W
> -fmessage-length=0 -funit-at-a-time'
>                build/host: i486-pc-linux-gnu/i486-pc-linux-gnu
>              Use dietlibc: yes
>        Build C++ programs: yes
>        Build C99 programs: yes
>            Available APIs: v13,net,v21,v22,v23,netv2
>             ext2fs Source: e2fsprogs
>     syscall(2) invocation: alternative
>       vserver(2) syscall#: 273/glibc
>                crypto api: beecrypt
>    use library versioning: yes
> 
> Paths:
>                    prefix: /usr
>         sysconf-Directory: /etc
>             cfg-Directory: /etc/vservers
>          initrd-Directory: $(sysconfdir)/init.d
>        pkgstate-Directory: /var/run/vservers
>           vserver-Rootdir: /var/lib/vservers
> 
> 
> Any hints are very appreciated, thanks!
> 
> - Markus