Subject: Re: [vserver] oom-killer killapps on hosts not not guest
From: Herbert Poetzl <herbert@13thfloor.at>
Date: Wed, 19 Aug 2009 23:33:19 +0200

On Thu, Aug 20, 2009 at 07:19:43AM +1200, Michael wrote:
> > you can avoid that by making host processes immune
> > to the OOM killer, or by raising the badness_bias of
> > the guests above any host vm usage, or disable
> > overcommit completely

> And how to make host immune to OOM killer?

via oom_adj in /proc/<pid>
the mechanism is a little tricky, as it functions as
shift count to the calculated total, but fortunately
you can check the results with /proc/<pid>/oom_score

for example, if your process gets an oom score of 42,
then writing 2 to oom_adj will result in 168, writing
-1 to oom_adj will give 21 ...

so setting that to e.g. -64 should make sure that
the process will never have any badness and is quite
unlikely to be killed ...

> >> I've tring to setup vm.overcommit_memory=2  which doesen't help
> >
> > probably because you left the overcommit_ratio untouched,
> > but you want to set this to 0 too
> 
> used overcommit 80 , or 0 with almoust the same results.

overcommit_ratio not overcommit_memory, and you still
need to set the former to 2 :)

> >> The only option seems to be
> >> vm.oom_kill_allocating_task = 1
> >
> > this is another way to address this specific case, but
> > it might not be the best for the entire system
> >
> >> Is it a right way to do it?
> >
> > the OOM 'badness' which is the criterion used to choose
> > the 'victim' is calculated like this:
> >
> >  1) total_vm of a process (system wide) plus
> >  2) the badness_bias (if in a guest, default is 0) plus
> >  3) the difference between rss_curr and rss_soft

> How difference rss_curr and rss_soft works? 
> (which settings give more score for a guest?)

the higher the difference is, the more likely it will
be a candidate for the killing

> Added 500000 to /etc/vservers/vserver1/badness
> but still got process killed on host.

again, the more points the more likely to be killed

on my servers, the oom scores are between 10 and 9000,
on my desktop, the values go up to 200000, so you must
have a real memory hog running on your host to still
be above your 500000 ... 

in any case, check with the oom_score ...

> > this is how it is calculated right now, with one exception,
> > if there is a swapoff process running, it gets full score :)
> >
> > we might add another 'difference' based on the vm
> > settings or some kind of multiplicative factor to that
> > (if you like to test it, let me know)
> 
> Yes , I've got this testing comp, so can test it.

okay, so the first thing would be to figure out what
makes your host processes 'win', then we can think
about mechanisms to adjust the likelyness

best,
Herbert

> Using latest kernel 2.6.30.5-vs2.3.0.36.14-pre5
> 
> 
> > HTH,
> > Herbert
> >
> >> --
> >> --
> >> Michael
> >
> 
> 
> 
> -- 
> --
> Michael