Subject: Re: [vserver] where are the lost processes?
From: "Daniel Hokka Zakrisson" <daniel@hozac.com>
Date: Thu, 13 Mar 2008 06:49:36 +0100 (CET)

Jason Drage wrote:
> Hi ,
> I'm running kernel 2.6.22.18-vs2.3.0.32 (on x86_64), util-vserver:
> 0.30.214
>
> I'm having trouble finding 'lost' processes attributed to guest.
>
> Basically I have a guest with approx 9 processes (see below) yet
> vserver-stat reports vastly different numbers, 286 after running for
> a week or 60 for a freshly started guest.
>
> Why are the numbers reported by vserver-stat and ps so different?
> Where can I find more details on my machine to help me track down these
> 'lost processes'?

Use ps maux to see all the threads. I suppose this is a bit of a semantic
problem, as vserver-stat calls them processes as opposed to tasks, but I'm
not too inclined to fix it. Either way, it refers to threads, not process
groups.

> Also this issue only happens to some of my guests, on some of them
> process usage reported by vserver-stat and ps match. Can anyone offer an
> explanation?
>
>
> Details...
>
> Below is the output of vserver-stat for a troublesome guest,
>
> A. When guest was discovered 'using too many processes'
> CTX   PROC    VSZ    RSS  userTIME   sysTIME    UPTIME NAME
> 221    286   1.7G  38.2M   0m09s60   0m09s16   8d20h10 aptcache
>
> B. After a restart (vserver aptcache stop; vserver aptcache start)
> CTX   PROC    VSZ    RSS  userTIME   sysTIME    UPTIME NAME
> 221     60 717.9M    13M   0m00s88   0m00s88   0m55s06 aptcache
>
> C. And after then running for almost a day
> CTX   PROC    VSZ    RSS  userTIME   sysTIME    UPTIME NAME
> 221     95   1.6G  20.1M   0m01s31   0m00s91  21h44m04 aptcache
>
> In all cases the actual proccesses reported by either "#ps aux" from
> within the guest or "#vps aux" from the host were wildly different (see
> below)
>
> ======================
> root@aptcache:/#  ps aux
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> root         1  0.0  0.0   5112  1968 ?        Ss   Mar12   0:04
> /sbin/init
> syslog   11627  0.0  0.0  12284   764 ?        Ss   06:25   0:00
> /sbin/syslogd -u syslog
> root     15893  0.0  0.0    120    32 ?        R+   10:27   0:00
> login
>
> root     15920  0.0  0.0  16916  2024 pts/1    Ss   10:27   0:00
> /bin/bash -login
> root     16151  0.0  0.0  13984  1072 pts/1    R+   10:46   0:00 ps aux
> 107      30636  0.0  0.0 1024944 6220 ?        Ssl  Mar12   0:01
> /usr/sbin/apt-cacher-ng -c /etc/apt-cacher-ng
> pidfile:/var/run/apt-cacher-ng/pid SocketPa
> root     30644  0.0  0.0  36264  1240 ?        Ss   Mar12   0:00
> /usr/sbin/sshd
> root     30662  0.0  0.0  17792   832 ?        Ss   Mar12   0:00
> /usr/sbin/cron
> root     30671  0.0  0.0  51240  2788 ?        Ss   Mar12   0:00
> /usr/sbin/apache2 -k start -DSSL
> www-data 30673  0.0  0.0  50980  1936 ?        S    Mar12   0:00
> /usr/sbin/apache2 -k start -DSSL
> www-data 30676  0.0  0.0 274768  3424 ?        Sl   Mar12   0:00
> /usr/sbin/apache2 -k start -DSSL
> www-data 30679  0.0  0.0 274768  3400 ?        Sl   Mar12   0:00
> /usr/sbin/apache2 -k start -DSSL

Apache is your most likely culprit for all the threads.

> [snip]

-- 
Daniel Hokka Zakrisson