Subject: Re: [vserver] Can vservers' RSS limits be decreased dynamically?
From: Matei Zaharia <matei@matei.ca>
Date: Sat, 31 Oct 2009 03:08:06 -0700

Thanks for the clarifications, Herbert. I'll explore using vkill.

> basically you could use vkill to kill whatever process
> you consider 'over limit' with whatever signal you
> consider appropriate ... you could also add a check
> to the function changing the limit, although I'm not
> sure (yet) that this is a good idea (i.e. you have
> to convince me with good arguments first :)

My use case might be a little unique, and I'm not even positive that  
the feature I described would be the best solution. Basically I want  
to share a cluster between multiple parallel computing applications,  
such as multiple instances of Hadoop. Each application will be running  
arbitrary, and possibly buggy, user code. I want to be able to  
schedule work from multiple applications on the same machine, and to  
dynamically change each app's share of each machine as new jobs enter/ 
leave the system. If the apps are well-written, they will use only the  
resources allocated to them. However, I don't want a buggy app to grab  
a large amount of memory when its share of a machine is high, and then  
fail to release it when I reduce its share. Killing processes over the  
RSS limit is a simple way to ensure that I never overcommit memory.  
Another option would be to let swapping do its thing and page out some  
of the offending apps, but this can hurt performance for all the  
applications, and seems difficult to control (I'd want to prioritize  
swapping out data from apps that are over their share).

> won't hurt to update to a newer kernel/patch and
> util-vserver ... but won't change anything in this
> regard ...

Thanks, I'll try that out too.

Matei