Subject: AW: [vserver] Guest hourly/daily/... cron job parallel execution
From: Fiedler Roman <Roman.Fiedler@ait.ac.at>
Date: Thu, 17 Jan 2013 09:07:28 +0000

 Thu, 17 Jan 2013 09:07:28 +0000
> -----Ursprüngliche Nachricht-----
> Von: Bendtsen, Jon [mailto:Jon.Bendtsen@laerdal.dk]
> 
> On 17/01/2013, at 09.43, Fiedler Roman <Roman.Fiedler@ait.ac.at> wrote:
> 
> > Hello List,
> >
> > What would be the most suitable solution to avoid high load/overlong cron
> job execution for hourly/daily/... jobs? When using similar vservers, all of
> them would start those jobs at the same time.
> 
> You could manually reschedule them?

Manual reschedule is annoying, you need to estimate the run-time and make some job-start-plan
to know free slots.

> Or use some kind of vserver <name>
> exec cat /etc/crontab to find out when they are run and change them to run
> at a different time?

From my point of view, the 2 main advantages of this are:
* Easy implementable, easy understandable solution
* Lowest security impact/security-relevance of code

The  problem here ist the run-time estimate: Let assume, you have jobs in every guest,
that need one CPU-core for 2min and 2 cores. So if 2 run, the complete in 2min, if 3
run in parallel they would need 3min, but due to more parallel disk IO, decreasing cache
efficiency execution time could be more like 3:20. If stacking is too narrow, jobs deviating
from estimate or additional jobs on guest or host are likely to make some jobs that
slow, that execution overlaps with the next, thus decreasing the performance even further,
thus starting more jobs in parallel ...

On the other hand, if distance between jobs is quite large, the last guest daily jobs
will run quite long after the first guest jobs have completed. So it might get narrow
if all jobs should be finished between 0:00 and start of business hours.

> But maybe just using nice on all the commands in the guests will raise the
> responsiveness, but not lower the load.

This is sufficient to improve the customer-side performance but will decrease the execution
efficiency of cron jobs even further. Would starting cron daemon with nice make all
jobs run with the same nice-values?

Roman