Subject: Re: [vserver] Question about Postfix
From: Bruno Galindro da Costa <bruno.galindro@gmail.com>
Date: Tue, 10 Feb 2009 10:32:03 -0200
Tue, 10 Feb 2009 10:32:03 -0200
Ok Nikolay. Thank you very much!


2009/2/10 Nikolay Kichukov <hijacker@oldum.net>

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi Bruno,
> I was reading through the post and just want to add something up:
>
> sshd[1893]: fatal: fork of unprivileged child failed
>
> I am getting the above message from my sshd when someone tries to log on
> to the virtual server. This only happens if sshd is started upon guest
> startup. If I manually restart the sshd, then all starts working fine.
> This all looks similar to the problem you described.
>
> I have spoken to Bertl on irc, however have not found so far the time to
> build the same guest and try to replicate the problem. I will keep you
> posted if I manage to replicate that on another guest. Maybe this will
> bring some clues.
>
> My versions:
>
> 2.6.28-vs2.3.0.36.4 and utils 0.30.216-pre2827.
>
> Cheers,
> - -Nik
>
>
> Bruno Galindro da Costa wrote:
> > Herbert,
> >
> >   Thank you very much for your anwser. I made the modifications showed by
> > you.
> >
> > 1 - I deleted both bcapabilities and ccapabilities files from the dir
> > /etc/vservers/mx001
> > *# ls -l /etc/vservers/mx001
> > total 28
> > drwxr-xr-x 4 root root 4096 2009-02-04 18:12 apps
> > lrwxrwxrwx 1 root root   39 2009-02-04 18:12 cache ->
> > /etc/vservers/.defaults/cachebase/mx001
> > drwxr-xr-x 2 root root 4096 2009-02-04 18:12 cpuset
> > -rw-r--r-- 1 root root  137 2009-02-05 11:24 fstab
> > drwxr-xr-x 3 root root 4096 2009-02-04 18:12 interfaces
> > -rw-r--r-- 1 root root    6 2009-02-04 18:12 name
> > lrwxrwxrwx 1 root root   23 2009-02-04 18:12 run ->
> /var/run/vservers/mx001
> > drwxr-xr-x 2 root root 4096 2009-02-09 22:03 ulimits
> > drwxr-xr-x 2 root root 4096 2009-02-04 18:12 uts
> > lrwxrwxrwx 1 root root   38 2009-02-04 18:12 vdir ->
> > /etc/vservers/.defaults/vdirbase/mx001*
> >
> > 2 - I removed the string '* - nofile 65536' from the file
> > /etc/security/limits.conf on both host and guest
> >
> > 3 - I maked a dir named ulimits in /etc/vservers/mx001 and created two
> files
> > inside them (nofile.hard and nofile.soft) with this contents:
> >
> > *# ls -l /etc/vservers/mx001/ulimits/
> > total 8
> > -rw-r--r-- 1 root root 6 2009-02-09 22:03 nofile.hard
> > -rw-r--r-- 1 root root 6 2009-02-09 22:02 nofile.soft
> >
> > # cat nofile.hard
> > 32768
> >
> > # cat nofile.soft
> > 16384
> >
> > 4 - After those steps above, I rebooted the host machine to make sure the
> > ulimit -n was changed to default in host (1024) and to make sure the
> ulimit
> > -n was changed to the defaults setted in both files above in guest
> machine.
> >
> > Unfortunately, the procedure above not worls. The same behaviour
> continues.
> > It's there a bug of vserver in SPARC64 architheture?
> >
> > Reading the site witch you recommended, I noticed the following option: *
> > rlimits***
> > Can you explain wath's the difference between *rlimits *and *ulimits*?
> >
> > In the next step, I'll use strace to manually tracing the postfix daemon
> > process...
> >
> > 2009/2/7 Herbert Poetzl <herbert@13thfloor.at>
> >
> >> On Sat, Feb 07, 2009 at 12:14:11PM -0300, Bruno Galindro da Costa wrote:
> >>> Hi all
> >>>
> >>>    I'm have a Sun v240 Sparc64. I've installed recently a Debian 4.0
> >>> RC6 (Etch) plus Linux VServer kernel. The following comands i used to
> >>> build VServer:
> >>> Commands on the host machine:
> >>> # apt-get -y install ssh linux-image-vserver-sparc64 util-vserver
> >>> linux-headers-2.6.18-6-vserver-sparc64 debootstrap vserver-debiantools
> >>> # reboot
> >>> # uname -a
> >>> Linux dmzux001 2.6.18-6-vserver-sparc64 #1 SMP Fri Dec 12 19:06:09 UTC
> >> 2008
> >>> sparc64 GNU/Linux
> >>> # vserver mx001 build -n mx001 --hostname
> mx001.domain.com.br--interface
> >>> eth0:172.20.0.21/24 -m debootstrap -- -d etch
> >>> # ulimit -HS -n 65536
> >>> # ulimit -Ha
> >>> core file size          (blocks, -c) unlimited
> >>> data seg size           (kbytes, -d) unlimited
> >>> max nice                        (-e) 0
> >>> file size               (blocks, -f) unlimited
> >>> pending signals                 (-i) unlimited
> >>> max locked memory       (kbytes, -l) unlimited
> >>> max memory size         (kbytes, -m) unlimited
> >>> open files                      (-n) 65536
> >>> pipe size            (512 bytes, -p) 8
> >>> POSIX message queues     (bytes, -q) unlimited
> >>> max rt priority                 (-r) 0
> >>> stack size              (kbytes, -s) unlimited
> >>> cpu time               (seconds, -t) unlimited
> >>> max user processes              (-u) unlimited
> >>> virtual memory          (kbytes, -v) unlimited
> >>> file locks                      (-x) unlimited
> >>> # echo '* - nofile 65536' >> /etc/security/limits.conf
> >>> # echo "default" > /etc/vservers/mx001/apps/init/mark
> >>> # echo 'CAP SYS ADMIN' > /etc/vservers/mx001/bcapabilities
> >>         ~~~~~~~~~~~~~~~~ you definitely do not want to give
> >>                         that to a guest lightly
> >>
> >> why? because the guest can easily mess up the host now, and
> >> that's probably not what you want (see big fat red warning,
> >> here: http://linux-vserver.org/Capabilities and Flags)
> >>
> >>> # echo 'CAP SYS RESOURCE' > /etc/vservers/mx001/bcapabilities
> >>         ~~~~~~~~~~~~~~~~~~~ similar here, usually no need for
> >>
> >>> # echo 'SET RLIMIT' > /etc/vservers/mx001/ccapabilities
> >>> # vserver mx001 start
> >>> # vserver mx001 enter
> >>> Comands on guest machine after postfix installation:
> >>> # uname -a
> >>> Linux mx001.domain.com.br 2.6.18-6-vserver-sparc64 #1 SMP Fri Dec 12
> >>> 19:06:09 UTC 2008 sparc64 GNU/Linux
> >>> # ulimit -HS -n 65536
> >>> # echo '* - nofile 65536' >> /etc/security/limits.conf
> >>> # ulimit -n
> >>> 65536
> >>> # ulimit -Ha
> >>> core file size          (blocks, -c) unlimited
> >>> data seg size           (kbytes, -d) unlimited
> >>> max nice                        (-e) 0
> >>> file size               (blocks, -f) unlimited
> >>> pending signals                 (-i) unlimited
> >>> max locked memory       (kbytes, -l) unlimited
> >>> max memory size         (kbytes, -m) unlimited
> >>> open files                      (-n) 65536
> >>> pipe size            (512 bytes, -p) 8
> >>> POSIX message queues     (bytes, -q) unlimited
> >>> max rt priority                 (-r) 0
> >>> stack size              (kbytes, -s) unlimited
> >>> cpu time               (seconds, -t) unlimited
> >>> max user processes              (-u) unlimited
> >>> virtual memory          (kbytes, -v) unlimited
> >>> file locks                      (-x) unlimited
> >>>
> >>>
> >>> The guest builded above, will be a mail server. The MTA is Postfix
> >>> version 2.3.8.
> >>>
> >>> The followig error is occurring when postfix service is started "ONLY"
> >>> in two situations: on guest / host boot and/or when the ulimit -n is
> >>> 1024. But if you can see above, I've set the ulimit -n to 65536 on the
> >>> host and guest machines.
> >>> *postfix/cleanup[5324]: fatal: setrlimit: Operation not permitted*
> >>> If I restart Postfix service *after* the above error is printed to
> >>> /var/log/mail.log, the error does not ocurr anymore until the next
> >>> restart of the guest machine or host machine.
> >> well, you kind of answered your question yourself,
> >> you are setting the host limit very high, but only
> >> when you logon, the very same is tried inside the
> >> guest, but there it fails, when the host limit is
> >> below the value you try to set
> >>
> >>> Can anyone help me?
> >> the correct solution is to put the guest ulimits
> >> into the guest config tree, to have them applied
> >> when the guest is started.
> >> (see http://www.nongnu.org/util-vserver/doc/conf/configuration.html)
> >>
> >> look for ulimits, adjust them correctly, remove
> >> the additional bcapabilities (to make the guest
> >> safe again) and you should be fine ...
> >>
> >> no need to have the limit raised on the host though,
> >> but it doesn't hurt either ...
> >>
> >>> Sorry my poor english.
> >> english isn't my primary language either, and it
> >> was more than sufficient to understand what you
> >> meant ...
> >>
> >> best,
> >> Herbert
> >>
> >>> --
> >>> Att.
> >>> Bruno Galindro da Costa
> >>> bruno.galindro@gmail.com
> >>> Florianópolis - SC
> >
> >
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iQEcBAEBAgAGBQJJkVfiAAoJEDFLYVOGGjgXnRMH/RiuWauwXAuuinQ5D8zxLT0H
> 4KsHdYYBaY1jYPue3TyloceOhSKU3OolQRfqo3FFMFFpfyXZ3/PMCmJzTbRI4MhO
> WW2eBKLjmLSHLkWESK8AhBAt8D5BK4gVe+okTDi1vpTWUcUVRwRHg9Jae+ZFJ1/b
> soxrIRIfHhASKiZjLNxEDLR+Z5CzlQrifcETj5LRe3Hh4GG3dkcnfTmJYVfI5iw6
> RojtI6KZeVVNFBLGYPOVT3XZi6PoNOo3z3XLGHFk3R8yq9XJgEIMVQJcvHoBQ2Jt
> mYWTgUbStlMEWSfyM94I+7y5eKm+ZHlVsEKe4VePpjfNcCyV2bcMlSUCFKv3jH8=
> =GOAZ
> -----END PGP SIGNATURE-----
>



-- 
Att.
Bruno Galindro da Costa
bruno.galindro@gmail.com
Florianópolis - SC


Ok Nikolay. Thank you very much!


2009/2/10 Nikolay Kichukov <hijacker@oldum.net>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Bruno,
I was reading through the post and just want to add something up:

sshd[1893]: fatal: fork of unprivileged child failed

I am getting the above message from my sshd when someone tries to log on
to the virtual server. This only happens if sshd is started upon guest
startup. If I manually restart the sshd, then all starts working fine.
This all looks similar to the problem you described.

I have spoken to Bertl on irc, however have not found so far the time to
build the same guest and try to replicate the problem. I will keep you
posted if I manage to replicate that on another guest. Maybe this will
bring some clues.

My versions:

2.6.28-vs2.3.0.36.4 and utils 0.30.216-pre2827.

Cheers,
- -Nik


Bruno Galindro da Costa wrote:
> Herbert,
>
>   Thank you very much for your anwser. I made the modifications showed by
> you.
>
> 1 - I deleted both bcapabilities and ccapabilities files from the dir
> /etc/vservers/mx001
> *# ls -l /etc/vservers/mx001
> total 28
> drwxr-xr-x 4 root root 4096 2009-02-04 18:12 apps
> lrwxrwxrwx 1 root root   39 2009-02-04 18:12 cache ->
> /etc/vservers/.defaults/cachebase/mx001
> drwxr-xr-x 2 root root 4096 2009-02-04 18:12 cpuset
> -rw-r--r-- 1 root root  137 2009-02-05 11:24 fstab
> drwxr-xr-x 3 root root 4096 2009-02-04 18:12 interfaces
> -rw-r--r-- 1 root root    6 2009-02-04 18:12 name
> lrwxrwxrwx 1 root root   23 2009-02-04 18:12 run -> /var/run/vservers/mx001
> drwxr-xr-x 2 root root 4096 2009-02-09 22:03 ulimits
> drwxr-xr-x 2 root root 4096 2009-02-04 18:12 uts
> lrwxrwxrwx 1 root root   38 2009-02-04 18:12 vdir ->
> /etc/vservers/.defaults/vdirbase/mx001*
>
> 2 - I removed the string '* - nofile 65536' from the file
> /etc/security/limits.conf on both host and guest
>
> 3 - I maked a dir named ulimits in /etc/vservers/mx001 and created two files
> inside them (nofile.hard and nofile.soft) with this contents:
>
> *# ls -l /etc/vservers/mx001/ulimits/
> total 8
> -rw-r--r-- 1 root root 6 2009-02-09 22:03 nofile.hard
> -rw-r--r-- 1 root root 6 2009-02-09 22:02 nofile.soft
>
> # cat nofile.hard
> 32768
>
> # cat nofile.soft
> 16384
>
> 4 - After those steps above, I rebooted the host machine to make sure the
> ulimit -n was changed to default in host (1024) and to make sure the ulimit
> -n was changed to the defaults setted in both files above in guest machine.
>
> Unfortunately, the procedure above not worls. The same behaviour continues.
> It's there a bug of vserver in SPARC64 architheture?
>
> Reading the site witch you recommended, I noticed the following option: *
> rlimits***
> Can you explain wath's the difference between *rlimits *and *ulimits*?
>
> In the next step, I'll use strace to manually tracing the postfix daemon
> process...
>
> 2009/2/7 Herbert Poetzl <herbert@13thfloor.at>
>
>> On Sat, Feb 07, 2009 at 12:14:11PM -0300, Bruno Galindro da Costa wrote:
>>> Hi all
>>>
>>>    I'm have a Sun v240 Sparc64. I've installed recently a Debian 4.0
>>> RC6 (Etch) plus Linux VServer kernel. The following comands i used to
>>> build VServer:
>>> Commands on the host machine:
>>> # apt-get -y install ssh linux-image-vserver-sparc64 util-vserver
>>> linux-headers-2.6.18-6-vserver-sparc64 debootstrap vserver-debiantools
>>> # reboot
>>> # uname -a
>>> Linux dmzux001 2.6.18-6-vserver-sparc64 #1 SMP Fri Dec 12 19:06:09 UTC
>> 2008
>>> sparc64 GNU/Linux
>>> # vserver mx001 build -n mx001 --hostname mx001.domain.com.br--interface
>>> eth0:172.20.0.21/24 -m debootstrap -- -d etch
>>> # ulimit -HS -n 65536
>>> # ulimit -Ha
>>> core file size          (blocks, -c) unlimited
>>> data seg size           (kbytes, -d) unlimited
>>> max nice                        (-e) 0
>>> file size               (blocks, -f) unlimited
>>> pending signals                 (-i) unlimited
>>> max locked memory       (kbytes, -l) unlimited
>>> max memory size         (kbytes, -m) unlimited
>>> open files                      (-n) 65536
>>> pipe size            (512 bytes, -p) 8
>>> POSIX message queues     (bytes, -q) unlimited
>>> max rt priority                 (-r) 0
>>> stack size              (kbytes, -s) unlimited
>>> cpu time               (seconds, -t) unlimited
>>> max user processes              (-u) unlimited
>>> virtual memory          (kbytes, -v) unlimited
>>> file locks                      (-x) unlimited
>>> # echo '* - nofile 65536' >> /etc/security/limits.conf
>>> # echo "default" > /etc/vservers/mx001/apps/init/mark
>>> # echo 'CAP SYS ADMIN' > /etc/vservers/mx001/bcapabilities
>>         ~~~~~~~~~~~~~~~~ you definitely do not want to give
>>                         that to a guest lightly
>>
>> why? because the guest can easily mess up the host now, and
>> that's probably not what you want (see big fat red warning,
>> here: http://linux-vserver.org/Capabilities and Flags)
>>
>>> # echo 'CAP SYS RESOURCE' > /etc/vservers/mx001/bcapabilities
>>         ~~~~~~~~~~~~~~~~~~~ similar here, usually no need for
>>
>>> # echo 'SET RLIMIT' > /etc/vservers/mx001/ccapabilities
>>> # vserver mx001 start
>>> # vserver mx001 enter
>>> Comands on guest machine after postfix installation:
>>> # uname -a
>>> Linux mx001.domain.com.br 2.6.18-6-vserver-sparc64 #1 SMP Fri Dec 12
>>> 19:06:09 UTC 2008 sparc64 GNU/Linux
>>> # ulimit -HS -n 65536
>>> # echo '* - nofile 65536' >> /etc/security/limits.conf
>>> # ulimit -n
>>> 65536
>>> # ulimit -Ha
>>> core file size          (blocks, -c) unlimited
>>> data seg size           (kbytes, -d) unlimited
>>> max nice                        (-e) 0
>>> file size               (blocks, -f) unlimited
>>> pending signals                 (-i) unlimited
>>> max locked memory       (kbytes, -l) unlimited
>>> max memory size         (kbytes, -m) unlimited
>>> open files                      (-n) 65536
>>> pipe size            (512 bytes, -p) 8
>>> POSIX message queues     (bytes, -q) unlimited
>>> max rt priority                 (-r) 0
>>> stack size              (kbytes, -s) unlimited
>>> cpu time               (seconds, -t) unlimited
>>> max user processes              (-u) unlimited
>>> virtual memory          (kbytes, -v) unlimited
>>> file locks                      (-x) unlimited
>>>
>>>
>>> The guest builded above, will be a mail server. The MTA is Postfix
>>> version 2.3.8.
>>>
>>> The followig error is occurring when postfix service is started "ONLY"
>>> in two situations: on guest / host boot and/or when the ulimit -n is
>>> 1024. But if you can see above, I've set the ulimit -n to 65536 on the
>>> host and guest machines.
>>> *postfix/cleanup[5324]: fatal: setrlimit: Operation not permitted*
>>> If I restart Postfix service *after* the above error is printed to
>>> /var/log/mail.log, the error does not ocurr anymore until the next
>>> restart of the guest machine or host machine.
>> well, you kind of answered your question yourself,
>> you are setting the host limit very high, but only
>> when you logon, the very same is tried inside the
>> guest, but there it fails, when the host limit is
>> below the value you try to set
>>
>>> Can anyone help me?
>> the correct solution is to put the guest ulimits
>> into the guest config tree, to have them applied
>> when the guest is started.
>> (see http://www.nongnu.org/util-vserver/doc/conf/configuration.html)
>>
>> look for ulimits, adjust them correctly, remove
>> the additional bcapabilities (to make the guest
>> safe again) and you should be fine ...
>>
>> no need to have the limit raised on the host though,
>> but it doesn't hurt either ...
>>
>>> Sorry my poor english.
>> english isn't my primary language either, and it
>> was more than sufficient to understand what you
>> meant ...
>>
>> best,
>> Herbert
>>
>>> --
>>> Att.
>>> Bruno Galindro da Costa
>>> bruno.galindro@gmail.com
>>> Florianópolis - SC
>
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQEcBAEBAgAGBQJJkVfiAAoJEDFLYVOGGjgXnRMH/RiuWauwXAuuinQ5D8zxLT0H
4KsHdYYBaY1jYPue3TyloceOhSKU3OolQRfqo3FFMFFpfyXZ3/PMCmJzTbRI4MhO
WW2eBKLjmLSHLkWESK8AhBAt8D5BK4gVe+okTDi1vpTWUcUVRwRHg9Jae+ZFJ1/b
soxrIRIfHhASKiZjLNxEDLR+Z5CzlQrifcETj5LRe3Hh4GG3dkcnfTmJYVfI5iw6
RojtI6KZeVVNFBLGYPOVT3XZi6PoNOo3z3XLGHFk3R8yq9XJgEIMVQJcvHoBQ2Jt
mYWTgUbStlMEWSfyM94I+7y5eKm+ZHlVsEKe4VePpjfNcCyV2bcMlSUCFKv3jH8=
=GOAZ
-----END PGP SIGNATURE-----



--
Att.
Bruno Galindro da Costa
bruno.galindro@gmail.com
Florianópolis - SC