Subject: Re: [vserver] throttler script
From:Sandino Araico Sánchez <sandino@sandino.net>
Date: Thu, 17 Jan 2013 17:59:09 -0600
Thu, 17 Jan 2013 17:59:09 -0600
On 17/01/13 08:25, Phil Parris wrote:
>
> Im running into similar issues.  Please provide the script.  One thing
> i thought about was to change cron source and add in a random sleep.
>
>

I wrote a couple of throttler scripts two years ago for a host with 50
vservers and load >250.

The ionice-updatedb looks for known I/O consumming processes in the
vserver, renices them and puts the most agressive ones in the throttler.

The throttler script pauses the process for some seconds (depending on
the system load) and lets it run for one second. Two methods are
avilable SIGSTOP and freezer but I stopped using the freezer on kernels
2.6.32 because some processes used to stay forever in status DN and were
impossible to kill until next host server reboot. I have been using the
SIGSTOP method since then. I have not tried the freezer on recent 3.x
kernels but it might work now. Some minor changes may be needed on the
throttler to use /sys/fs/gcroup instead of the older /dev/cgroup...

You can download all the scripts from http://sandino.araico.net/throttler/

I have put the ionice.cron in /etc/cron.d but you could also use crontab
-e to set the crontab rules on the root user. I run the  ionice-updatedb
every minute all day but you might prefer to run it every 3 or 5 minutes
and just on known high load hours... it's up to you.

For the near future I have plans to rewrite the ionice-updatedb script
using iotop on context 1 to detect the I/O abusers. I was not able to
run iotop on context 1 in the past but some months ago I saw some
recipes on this mailing list to make it work so I will give them a try
some day.


-- 
Sandino Araico Sánchez 
http://sandino.net



On 17/01/13 08:25, Phil Parris wrote:

Im running into similar issues.  Please provide the script.  One thing i thought about was to change cron source and add in a random sleep.



I wrote a couple of throttler scripts two years ago for a host with 50 vservers and load >250.

The ionice-updatedb looks for known I/O consumming processes in the vserver, renices them and puts the most agressive ones in the throttler.

The throttler script pauses the process for some seconds (depending on the system load) and lets it run for one second. Two methods are avilable SIGSTOP and freezer but I stopped using the freezer on kernels 2.6.32 because some processes used to stay forever in status DN and were impossible to kill until next host server reboot. I have been using the SIGSTOP method since then. I have not tried the freezer on recent 3.x kernels but it might work now. Some minor changes may be needed on the throttler to use /sys/fs/gcroup instead of the older /dev/cgroup...

You can download all the scripts from http://sandino.araico.net/throttler/

I have put the ionice.cron in /etc/cron.d but you could also use crontab -e to set the crontab rules on the root user. I run the  ionice-updatedb every minute all day but you might prefer to run it every 3 or 5 minutes and just on known high load hours... it's up to you.

For the near future I have plans to rewrite the ionice-updatedb script using iotop on context 1 to detect the I/O abusers. I was not able to run iotop on context 1 in the past but some months ago I saw some recipes on this mailing list to make it work so I will give them a try some day.


-- 
Sandino Araico Sánchez 
http://sandino.net