Subject: Re: [vserver] chbind the init process of the host (/sbin/init)
From: Herbert Poetzl <herbert@13thfloor.at>
Date: Thu, 4 Sep 2008 06:19:09 +0200

On Wed, Sep 03, 2008 at 10:28:51PM +0200, Manfred Heubach wrote:
> 
> Herbert Poetzl schrieb:
>> On Wed, Sep 03, 2008 at 09:30:33AM +0200, Manfred Heubach wrote:
>>> Herbert Poetzl schrieb:
>>>> On Tue, Sep 02, 2008 at 10:27:03PM +0200, Manfred Heubach wrote:

>>>>> Hello Gyu, hello Herbert,

>>>>> well - o.k. as I cannot restrict the init process to certain IPs
>>>>> and use different IPs for the guests at the same time - I will
>>>>> think this all over. I have once written a script which checks all
>>>>> init scripts and wraps them if necessary. I just have to adjust
>>>>> this script to wrap them into e.g. --nid 2. netstat -l will not
>>>>> show the sockets in --nid 2 chbind --nid 2 netstat -l will *only*
>>>>> show those in --nid 2 So if I want to get a quick overview what's
>>>>> listening I will end up with 2 commands (I can build an alias for
>>>>> that).

>>>> try chbind --nid 1 netstat -l
>>>> (nid=1 is the spectator context, like xid=1 for processes :)

>>> Then maybe this is a bug.
>>> chbind --nid 1 netstat -l neither shows sockets without any 
>>> nid nor sockets in nid 2

>> all sockets have a nid (host nid=0, spectator=1)
>>
>> [root]$ nc -l -p 100 &
>> [root]$ chbind --nid 2 --ip 127.0.0.1 -- nc -l -p 200 &
>>
>> [root]$ lsof -ni
>> COMMAND PID USER   FD   TYPE DEVICE SIZE NODE NAME
>> nc      259 root    3u  IPv4    102       TCP *:100 (LISTEN)
>> [root]$ chbind --nid 1 -- lsof -ni
>> COMMAND PID USER   FD   TYPE DEVICE SIZE NODE NAME
>> nc      259 root    3u  IPv4    102       TCP *:100 (LISTEN)
>> nc      264 root    3u  IPv4    107       TCP *:200 (LISTEN)
>> [root]$ chbind --nid 2 -- lsof -ni
>> COMMAND PID USER   FD   TYPE DEVICE SIZE NODE NAME
>> nc      264 root    3u  IPv4    107       TCP *:200 (LISTEN)
>>
>> note: if you have CONFIG_VSERVER_PRIVACY=y then
>> the spectator context is disabled in most cases
>> including the network information

> This is how the Debian kernel has been built:

> CONFIG_VSERVER_AUTO_LBACK=y
> # CONFIG_VSERVER_AUTO_SINGLE is not set
> CONFIG_VSERVER_COWBL=y
> # CONFIG_VSERVER_VTIME is not set
> # CONFIG_VSERVER_DEVICE is not set
> CONFIG_VSERVER_PROC_SECURE=y
> CONFIG_VSERVER_HARDCPU=y
> CONFIG_VSERVER_IDLETIME=y
> CONFIG_VSERVER_IDLELIMIT=y
> CONFIG_VSERVER_PRIVACY=y
> CONFIG_VSERVER_CONTEXTS=256
> CONFIG_VSERVER_WARN=y
> # CONFIG_VSERVER_DEBUG is not set
> CONFIG_VSERVER=y
> CONFIG_VSERVER_SECURITY=y
> 
> So CONFIG_VSERVER_PRIVACY=y is why I don't see anything in nid 1.

yep, precisely. this and other privacy aspects are
covered by the 'Respect Guest Privacy' option
(like ptrace, devmapper, devpts, locks, (v)kill)

> Do you see anything which will lead me into other problems with 
> this kernel config?

no, it looks almost like the suggested default
config. after all, enabling guest privacy is an
option :)

> (I for myself see that I'm in desperate need for some RTFM sessions 
> I lost track of the vserver project about 3 years ago and loads 
> of things have changed since then :-))

everything is in flux :) wb!

> >> (Why does the output below show 0.30.215 when actually 0.30.216 is
> >> installed - strange ...)
> >>
> >> ---------------------------
> >> Versions:
> >>                    Kernel: 2.6.26-1-vserver-686
> >>                    VS-API: 0x00020303
> >>              util-vserver: 0.30.215; Jun  1 2008, 21:56:10
> >>     
> >
> > well, I'd say it is 0.30.215 at least at heart :)
> >
> > maybe you installed two competing versions of
> > util-vserver in different locations?
> >   
> hm - maybe I should get some sleep - this is what it looks like today - ...
> 
> 
>                    Kernel: 2.6.26-1-vserver-686
>                    VS-API: 0x00020303
>              util-vserver: 0.30.216-pre2772; Aug 27 2008, 16:39:45
> 
> Good night and thanks for helping!
> Manfred

you're welcome!

best,
Herbert

> > HTH,
> > Herbert
> >
> >   
> >> Features:
> >>                        CC: gcc, gcc (GCC) 4.1.2 20061115 (prerelease)
> >> (Debian 4.                                              1.1-21)
> >>                       CXX: g++, g++ (GCC) 4.1.2 20061115 (prerelease)
> >> (Debian 4.                                              1.1-21)
> >>                  CPPFLAGS: ''
> >>                    CFLAGS: '-Wall -g  -O2 -std=c99 -Wall -pedantic -W
> >> -funit-at-                                              a-time'
> >>                  CXXFLAGS: '-g -O2 -ansi -Wall -pedantic -W
> >> -fmessage-length=0 -                                             
> >> funit-at-a-time'
> >>                build/host: i486-pc-linux-gnu/i486-pc-linux-gnu
> >>              Use dietlibc: yes
> >>        Build C++ programs: yes
> >>        Build C99 programs: yes
> >>            Available APIs: v13,net,v21,v22,v23,netv2
> >>             ext2fs Source: e2fsprogs
> >>     syscall(2) invocation: alternative
> >>       vserver(2) syscall#: 273/glibc
> >>                crypto api: beecrypt
> >>
> >> Paths:
> >>                    prefix: /usr
> >>         sysconf-Directory: /etc
> >>             cfg-Directory: /etc/vservers
> >>          initrd-Directory: $(sysconfdir)/init.d
> >>        pkgstate-Directory: /var/run/vservers
> >>           vserver-Rootdir: /var/lib/vservers
> >>
> >>
> >>     
> >>>> Maybe it's better to forget about the wrapper and configure the services
> >>>> to bind only to the right IPs. I will still have a problem with services
> >>>> that don't offer such an option - but these are hopefully none of which
> >>>> I need :-)
> >>>>     
> >>>>         
> >>> a good way (for certain services) is to use something
> >>> like the v* runlevel wrappers we had (and maybe still
> >>> have?) somewhere ...
> >>>
> >>> i.e. make a separate runlevel script called v<service>
> >>> which only does the chbind and then hand over the
> >>> real stuff to the <service> script
> >>>
> >>> this way, you'll keep them on updates, while benefiting
> >>> from the changes/upgrades to the wrapped scripts
> >>> (which you simply disable)
> >>>
> >>>   
> >>>       
> >>>> Is someone interested in a script which checks that services on the host
> >>>> aren't bound to *:port. I think about this as a sanity check before
> >>>> starting a vserver - so a misconfigured host won't mess up the guests.
> >>>> This won't be bullet proof. You can still stop a service in the guest,
> >>>> then restart the same service on the host and you end up with the host
> >>>> bound to the IP of a guest. But it seems to be a good way not to shoot
> >>>> yourself in the foot in the first place ;-)
> >>>>     
> >>>>         
> >>> I think a warning script might be interesting for some
> >>> folks, but I also think it would be much better to fix
> >>> most of the runlevel scripts, which report 'success'
> >>> while the service actually failed (like klogd, ssh, ...)
> >>>
> >>> best,
> >>> Herbert
> >>>
> >>>   
> >>>       
> >>>> Some comments follow below.
> >>>>
> >>>> Thanks
> >>>> Manfred
> >>>>
> >>>> PÁSZTOR György schrieb:
> >>>>     
> >>>>         
> >>>>> Hi,
> >>>>>
> >>>>> "Manfred Heubach" <entwicklung@heubach-edv.de> írta 2008-09-02 14:12-kor:
> >>>>>   
> >>>>>       
> >>>>>           
> >>>>>> PÁSZTOR György schrieb:
> >>>>>>     
> >>>>>>         
> >>>>>>             
> >>>>>>> Hi!
> >>>>>>>
> >>>>>>> "Manfred Heubach" <entwicklung@heubach-edv.de> írta 2008-09-02 09:07-kor:
> >>>>>>>           
> >>>>>>>               
> >>>>>>>>> well, why not just 'restrict' the host to those two addresses
> >>>>>>>>> and be done?
> >>>>>>>>>               
> >>>>>>>>>                   
> >>>>>>>> Well, i *want* to restrict the host to those two addresses - but
> >>>>>>>> how?
> >>>>>>>>             
> >>>>>>>>                 
> >>>>>>> I don't understand your attitude. Why do you want to run services
> >>>>>>> on the host, if you use vserver?
> >>>>>>>           
> >>>>>>>               
> >>>>>> I use vserver mainly for server consolidation. This is why I have
> >>>>>> some more services on the host itself (syslog-ng, mta, bacula,
> >>>>>> raid-console, ssh, snmp, ...).
> >>>>>>     
> >>>>>>         
> >>>>>>             
> >>>>> Yepp, so do I.
> >>>>>
> >>>>> But with some differences:
> >>>>>
> >>>>> syslog-ng -> I run it on a dedicated management server. If the host
> >>>>> dies we still have its logs.
> >>>>>       
> >>>>>           
> >>>> see below - no extra hardware :-(
> >>>>
> >>>>     
> >>>>         
> >>>>> MTA -> I don't understand why do you want MTA on the host. On
> >>>>> Special vservers, which purpose is not to handle the mails I run
> >>>>> nullmailer to let
> >>>>>       
> >>>>>           
> >>>> Hey - great stuff - I didn't know about nullmailer until now - and
> >>>> this after 10 years of Linux with postfix, courier and all the lot.
> >>>>
> >>>>     
> >>>>         
> >>>>> my error reports reach it's destination. I have some vservers which
> >>>>> runs backround service, and don't have public IP but an internal for
> >>>>> interchange data between servers have. Those nullmailers sends it's
> >>>>> message to the management host, and that delivers to the sysadmins.
> >>>>> For the "corporate" mailing I use a vserver, which handles our
> >>>>> mails, and another vserver which runs webmail things.
> >>>>>
> >>>>> Bacula -> I run bacula-fd on the host. But I don't need any other
> >>>>> bacula things in the guests.
> >>>>>
> >>>>> I don't know raid-console. What it is?
> >>>>>       
> >>>>>           
> >>>> It's just the web management console of e.g. 3ware raid controllers
> >>>> (3dm)
> >>>>
> >>>>     
> >>>>         
> >>>>> ssh -> I wrote it already: Yes, the (almost) only thing you may want
> >>>>> also on the host.
> >>>>>
> >>>>> snmp -> I collect the traps on the mentioned management machine.
> >>>>> What do you want to run on the host? I don't trust in snmp things,
> >>>>> so I won't allow to run snmpd on the host, neither on the guests.
> >>>>>       
> >>>>>           
> >>>> I use snmp for performance and health monitoring - does a great job
> >>>> together with nagios or munin. Read only and restricted to management
> >>>> IPs it cannot do much harm.
> >>>>
> >>>>     
> >>>>         
> >>>>> ... -> So... Basic approach: Just run sshd (and bacula-fd) on the
> >>>>> host. Everything else in the guests. If you do the consolidate like
> >>>>> this you will have easy job, when you have to move a vserver to
> >>>>> another host, because of decrease the load of a host, etc.
> >>>>>       
> >>>>>           
> >>>> I have customers with only one server (hardware) - so all has to go on
> >>>> one machine.
> >>>>     
> >>>>         
> >>>>> Cheers,
> >>>>> gyu
> >>>>>   
> >>>>>       
> >>>>>