On Saturday 22 November 2008, Herbert Poetzl wrote: > On Sun, Nov 16, 2008 at 08:15:40PM -0500, Chuck wrote: > > On Thursday 13 November 2008, Herbert Poetzl wrote: > > > On Thu, Nov 13, 2008 at 10:03:42AM -0500, Chuck wrote: > > > > i stopped and restarted our openfire server and all was well with that > > > > action. however, 2 web servers, one running an older centos and the > > > > other running gentoo64 with openrc/baselayout2 both lost any network > > > > communication. ips showed present on the host but were unusable even > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ that is unusual > > > > > > > from the host. the ip blocks of the various guests and the host use 3 > > > > networks on the same nic. > > > > > > > i restarted each of those affected servers. when they stopped each > > > > gave this error: > > > > > > > RTNETLINK answers: Cannot assign requested address > > > > RTNETLINK answers: Cannot assign requested address > > > > > > > > it appears it gave one line for every ip assigned to the guest. > > > > > > that happens if the IP is already there (maybe with > > > the wrong netmask or so) > > > > > > > restarting those 2 vservers cured the problem. these are the only two > > > > affected out of the others living on that host. > > > > > > this will have removed and re-added the IPs properly > > > > > > > i dont think this has anything to do with it but the centos guest was > > > > moved to this server by simply tarring its /vservers and /etc/vservers > > > > entries. the openfire and gentoo guests were created and configured on > > > > this host. no ip addresses clash. > > > > > > > the host is running gentoo64 baselayout1 with > > > > > > > kernel 2.6.22-vs2.3.0.34-gentoo > > > > util-vserver 0.30.215 > > > > iproute2 version 2.6.22.20070710 > > > > > > > i am planning on updating the kernel to 2.6.26-vs2.3.0.35.6-gentoo and > > > > baselayout2/openrc but it is on my 'when i get to it' priority list. > > > > > > > anyone seen this behavior of stopping a guest and it affecting > > > > networking on other guests before? this is new behavior to me. > > > > > > well, it is kind of expected, if you are using several > > > IPs in the same network (i.e. with a netmask) and without > > > secondary propagation, that when you remove the primary, > > > all secondaries are gone too (that is a mainline 'feature') > > > > > > but I'm not sure that matches what you are seeing, because > > > as I said, the secondaries are gone, so they are not supposed > > > to show up on the host or anywhere (in this case) > > > > > > > hmm that behavior appears to be different than earlier versions then. > > no, you are just misinterpreting what I said, please reread > and/or read some information on how Linux Networking handles > addresses within the same subnet (with emphasis on primary > vs secondary addresses) > > > previously, if i had 5 guests all using the same /24 network but > > of course different ips, shutting one guest down vanished only its > > ips and the others remained functional. > > the very same works now as well, given that your config > is correct and/or you enabled 'promote secondaries' > > > now it appears to destroy all networking within that global network. > > ie: guest1 may have xxx.xxx.34.23 thru 26 assigned.. guest2 may > > be using xxx.xxx.34.56 thru 60. both /24 , each guest may have > > other networks assigned as well depending on their functions. > > the networks for the most part are not subnetted in any way. > > that would create a nightmare. > > nope, works fine and more importantly, as expected if done > properly ... no nightmare, no issues ... > > > i need this to be able to just shut down the ip addresses involved > > with the downed guest leaving everything else intact.. > > then get your setup right > > > is there some setting maybe in the kernel or a capability or > > something that can do this? > > as I mentioned several times, you probably want to enable > 'promote secondaries' for your setup, and make sure that > the config holds valid addresses and prefix/netmasks > ok i think i understand what happens then... this time around in the new hosts, i did not assign a primary ip in each network, only one in one network for the host. this then gave a guest a primary ip in a network the host was not using before starting the guest... if that guest were stopped then it would shut down everything in its network... so... i assigned a primary ip to the host for each network that will be used, and also set up net.ipv4.conf.eth0.promote_secondaries=1 in sysctl.conf for backup safety. i think this combination will cure it. i will know my next scheduled maintenance which is tomorrow. > > maybe with all these upgrades we have done recently i missed > > some setting? > > > > we have 3 /24 blocks assigned to web hosting and they are > > distributed throughout 10 guests according to various > > customers/assignment rules/website purpose etc that we have. > > so one /24 network may be shared in all 10 guests living on > > the same host. this same network may be shared with other > > guests on other hosts as well. > > prefectly fine ... > > > so if i read this right, i must now be very careful not to put > > any 2 guests sharing the same network on the same host? > > without 'promote secondaries' or a dummy base address, yes > > > ugh... literally impossible with our setup. > > (doctor, doctor, it hurts when I poke my eye) > then don't do that :) > > HTH, > Herbert > > > > best, > > > Herbert > > > > > > > -- > > > > > > > > Chuck > > > > > > > > > > > -- > > > > Chuck > -- Chuck