Subject: Re: [vserver] High Systemload and many Processes in D state
From: Urban Loesch <bind@enas.net>
Date: Thu, 21 Jun 2012 12:42:02 +0200

 Thu, 21 Jun 2012 12:42:02 +0200
Hi,

for the records.
Disabling "inotify" solved my problem.

Details can be found here:
http://thread.gmane.org/gmane.linux.kernel/1315430

thanks
Urban


On 18.05.2012 17:55, Urban Loesch wrote:
>
>
> On 18.05.2012 15:56, Ghislain wrote:
>> Le 18/05/2012 15:30, Urban Loesch a écrit :
>>> Hi Gishlain,
>>>
>>> thanks for your anwser.
>>>
>>> If I stop Dovecot then I have to wait a couple of minutes (2-3) and all the processes
in D state are disappearing.
>>> Also the kernel thread [fsnotify mark] goes in sleep state and the systemload goes
down to normal values.
>>>
>>
>> lucky one mine was D state process locked and only reboot can cure it.
>
> I tested it again to be sure. After stpping Dovecot and wating 2 minutes all D states
are gone.
>
>
>>
>>> I use the following cgroup limits:
>>>
>>> mount options:
>>> vserver on /dev/cgroup type cgroup (rw,cpuset,cpu,cpuacct,memory,devices,freezer,net
cls,blkio,perf event)
>>>
>>>
>>> In /etc/vservers/$VSNAME/cgroup" if have:
>>>
>>> cpuset.cpus: 0-5
>>> cpu.shares: 512
>>> memory.limit in bytes: 4G
>>> memory.memsw.limit in bytes: 6G
>>
>> perhaps you should remove blkio from the cgroup if you do not use it as it seems
to be from my point of view the culprit as this is what appeared on
>> those versions. As all my servers are vserver patched i really do not know if this
is related to cgroup/vserver i never been ale to have a valuable
>> trace on this issue. 3.0.28 is pretty recent one so this is curious.
>
> First I tried with "blkio" disabled
> mount:
> vserver on /dev/cgroup type cgroup (rw,cpuset,cpu,cpuacct,memory,devices,freezer,net
cls,perf event)
>
> After a couple of minutes the "[fsnotify mark]" was in D state again.
>
>
> Now I unmounted cgroup completly. I don't need it on this machine.
> Now for 20 minutes it runs normaly.
>
> The [fsnotify mark] sometimes comes up in D state, but goes in sleep state some seconds
after:
>
> # vps -ostat,pid,time,wchan='WCHAN-xxxxxxxxxxxxxxxxxxxx',cmd ax |grep D
> STAT PID CONTEXT TIME WCHAN-xxxxxxxxxxxxxxxxxxxx CMD
> D 93 0 MAIN 00:00:12 synchronize sched [fsnotify mark]
> S+ 30193 0 MAIN 00:00:00 pipe wait grep D
> # vps -ostat,pid,time,wchan='WCHAN-xxxxxxxxxxxxxxxxxxxx',cmd ax |grep D
> STAT PID CONTEXT TIME WCHAN-xxxxxxxxxxxxxxxxxxxx CMD
> S+ 30599 0 MAIN 00:00:00 pipe wait grep D
>
> The only thing no is, that vserver-stat gimves me no more status info:
>
> # vserver-stat
> open(memory.stat): No such file or directory
> CTX PROC VSZ RSS userTIME sysTIME UPTIME NAME
> 154 598 15.5G 0 0m00s00 0m00s00 19m54s54 vsmail1
>
>
> Perhaps I should try with 3.2.17?
>
>>
>>>
>>> Currently there is only 1 vServer running on this Host.
>>>
>>
>> this is how i use it most of the time ^^ hardware issue free migration and i keep
ssh access even if the guest is hacked...
>>
>>> Perhaps I should completly disable cgroup? Do you know how I can do this?
>>
>> disbale no but remount it with blkio disabled to see if it helps.
>>
>>>
>>> Sorry, but I have never heard about dlimit?
>>
>> then i guess you do not use them ^^
>
> I googled :-)
> dlimit is for IO throttling, etc. No I don't need it.
>
>
>>
>> Ghislain.
>>
>
> Thanks
> Urban
>
>