On Thu, Jan 17, 2013 at 10:35:43AM +0000, Bendtsen, Jon wrote: > On 17/01/2013, at 11.05, Fiedler Roman <Roman.Fiedler@ait.ac.at> wrote: >>> -----Ursprüngliche Nachricht----- >>> Von: Bendtsen, Jon [mailto:Jon.Bendtsen@laerdal.dk] >>> On 17/01/2013, at 10.07, Fiedler Roman <Roman.Fiedler@ait.ac.at> >>> wrote: >>>>> -----Ursprüngliche Nachricht----- >>>>> Von: Bendtsen, Jon [mailto:Jon.Bendtsen@laerdal.dk] >>>>> On 17/01/2013, at 09.43, Fiedler Roman <Roman.Fiedler@ait.ac.at> >>> wrote: > [cuuuuuut] >>> Or maybe just pick some random time, sort of what old >>> ethernet did when there is a collision because 2 or more >>> computers tried to transmit data at the same time. >> Yes. Or use the context-ID modulo something. > good idea > [cuuuuuut] >>> I see your point. If so maybe we should split up >>> daily/weekly/hourly cronjobs even further, such that all the >>> same programs which are hard linked together can use the same >>> cache and ram for instructions. But running locate to update >>> the database might not be a good choice to all run at the >>> same time. >> I would expect, that caching of the application code itself >> is not the main performance boost. It is more about getting >> rid of the bottle-necks. As you noted, too many locate-updates >> in parallel will kill disk performance. But also loosing the >> usual disk-cache benefit might be problematic. If e.g. just a >> few databases run a job, the relevant db-content from disk is >> likely the end up completely in OS RAM cache. All disk-reads >> from db will return immediately. When too many DBs execute in >> parallel, disk content of one process will be put in cache, >> eliminating pages soon need again by another job again. > This is not really any different from running many other kinds > of virtual machines in parallel. Your storage system needs to > perform. if the guest is 'friendly' and basically controlled by the host admin, it should suffice to put crond in the idle I/O class and renice it a little. >>> Maybe we need kind of scheduler in the kernel that notices >>> which processes in the different guests are hard linked and >>> then prioritizing running those? >>> Maybe we need some new kind of scheduler system that is made >>> with virtualization in mind, such that it adjusts to when >>> there is a low load and then tries to run the maintenance >>> scripts, meaning that sometimes scripts are run with only 20 >>> hours between them, other times it might be 36 hours. >> In my opinion, such an intelligent/learning scheduler would >> allow significance increase in execution performance, but >> looking of the simplicity of current cron, I think, that such >> a program is years away, if even written ever. > probably. But I can imagine other ways to do it. I already use > the at system for some tasks. Like my backup scripts. If the > load is too high or the system is other wise not ready, I have > my backup scripts call 'at now + 1 hour' to run the script > itself 1 hour later. If it is still not ready, I call +2 hours, > +4 and finally +8 hours. > But it all hinges on the guest knowing the over all system load > of the virtualization host, just like a process in a normal > stand alone system can see the system load. Maybe we need a new > field in the load data? Or a new syscall to give those data? you can simply provide that information to a guest from a cron job on the host (e.g. via a bind mounted file or simply by echoing the necessary information into the (root) filesystem of each guest. > [cuuuuut] >>> I am not starting cron daemon with nice. I put nice in side >>> the crontab file, like these examples: >>> 37 0 * * * root nice -n 15 >>> /usr/local/sbin/AD_integration/find_disabled_users_from_AD_in_groups.sh >>> 0,15,30,45 * * * * root nice -n 5 >>> /usr/local/sbin/AD_integration/merge_AD_groups_with_unix.sh >> Ah I see. >> My current solution is: >> * Check for each guest, if cron scheduler is installed inside >> * Check if guest cron would run hourly/daily by himself, if >> yes let him do it. A misconfigured/malicious guest can always >> run any process at any time, this has to be addressed via >> other means >> * If guest cron is installed, cron.daily/hourly ... >> directories exist but guest scheduler does NOT run those jobs >> from etc/crontab (cooperative guest) by itself, then start >> those via vserver-exec >> * Make sure to run only a given number of those guest >> processes in parallel >> So all you need to do is to install cron in guest but remove >> run-scripts directives for hourly/daily from /etc/crontab to >> opt in. > Yes, that is a method too, but I would not really like that > outsiders, like hosting providers ran tasks inside my guest, at > least not without my beforehand knowledge. I guess it works if > one administrates both the host and the guests. I think it might be just fine if the procedure is well documented and most importantly well tested. i.e. 95% of all customers will except whatever the provider offers as environment without changing anything at all, the remaining 5% might just setup their cron jobs differently without causing a big problem for the system. as usual, it's all about communication :) best, Herbert > JonB