Hi, Thanks a lot for your explanations, Works like a charm after severe battle with hp with (next time I'll buy real switch) Thanks again Le 15 janv. 2010 à 10:59, Laurent Spagnol a écrit : > Olivier BATARD a écrit : >> >> Hi, >> >> >> Thanks again but I have a problem : >> >> >> I've made two table : >> >> >> - rt1 : One route 192.168.2.0/ 24 dev eth1 >> - rt2 : One route 192.168.3.0/34 dev eth0 >> > I suppose you mean /24 ? ;) > >> The rules has been made too : >> >> >> 0: from all lookup local >> 32764: from 192.168.3.0/24 lookup rt2 >> 32765: from 192.168.2.0/24 lookup rt1 >> >> >> The main table is empty. >> >> Works greats, guests are isolated but when I want to add default route with : >> >> ip route add default via 192.168.2.1 table rt1 or ip route add 192.168.3.0/24 via 192.168.2.1 table rt1 >> >> I've the error : >> >> RTNETLINK answers: No such process >> >> Thanks for your help >> > An example is better for a good understanding: > > vsh-13:~# ip rule add from 10.7.0.0/16 table 7 => OK > vsh-13:~# ip route add 10.7.0.0/16 dev eth0.7 table 7 > Cannot find device "eth0.7" => You must create VLAN before > vsh-13:~# vconfig add eth0 7 => OK > Added VLAN with VID == 7 to IF -:eth0:- > vsh-13:~# ip route add 10.7.0.0/16 dev eth0.7 table 7 > RTNETLINK answers: Network is down => You must enable interface.VLAN before > vsh-13:~# ifconfig eth0.7 up => OK > vsh-13:~# ip route add 10.7.0.0/16 dev eth0.7 table 7 => OK > vsh-13:~# ip route add default via 10.7.0.1 table 7 > RTNETLINK answers: No such process => You must have an active IP on this network before > vsh-13:~# ifconfig eth0.7:test1 10.7.0.53 up => OK > vsh-13:~# ip route add default via 10.7.0.1 table 7 => OK > > vsh-13:~# ip rule > 0: from all lookup local > 32765: from 10.7.0.0/16 lookup 7 > 32766: from all lookup main > 32767: from all lookup default > > vsh-13:~# ip route show table 7 > 10.7.0.0/16 dev eth0.7 scope link > default via 10.7.0.1 dev eth0.7 > > So the short answer is "some operations have some prerequisites": > ip rule add from ... table ... => no constraints > ip route add ... dev ... table ... => interface.VLAN MUST be active > ip route add default via ... table ... => The network MUST have at less one active IP => you must have at less on Vserver running with one active IP on this network) > > In the real life, i use the "start-stop-scripts" of Util-Vserver (/etc/vservers/.../scripts/...) > > * /etc/vservers/.../scripts/initialize.d/ > Before doing anything: > - initialisation of VARS (ip, vlan, device, network, prefix, gateway) for all IPs of the Vserver > NOTE: "gateway" is not handled with Util-Vserver, i write this address in /etc/vservers/.../interfaces/.../gateway > - check of IPs (fping) -> abortion of start sequence if one of them is already active elsewhere > > * /etc/vservers/.../scripts/prepre-start.d/10 vlan-up > - loading of dot1q Kernel module if required > - VLAN(s) is (are) added to the network device if required > - interface.VLAN(s) is (are) enabled if required (MTU should be changed at this stage) > > * /etc/vservers/.../scripts/pre-start.d/10 route-up > => At this stage, all the interfaces of Vserver are available > - installation of routing tables if required > > * /etc/vservers/.../scripts/post-start.d/99 start-end > => At this stage, the Vserver is fully functional > - "arping" to gateway and network to force update of ARP tables > > NOTE: > You must be extremely carefull: "/etc/init.d/util-vserver" (Debian way) and "/etc/init.d/vservers-default" (genuine) will start several Vserver at once, so the risk is to create duplicate routing tables. > That why i use Kernel locks (dotlockfile) in my script to avoid this problem. (probably better to use /usr/lib/util-vserver/lockfile ?) > > The handling of routing with Util-Vserver should be a great idea ;) > > Regards, > > L.S. > > >> Le 12 janv. 2010 à 23:56, Laurent Spagnol a écrit : >> >> >>> Hi, >>> >>> Message de Olivier BATARD <obatard@gmail.com>: >>> >>> >>>> Hello, >>>> >>>> We're trying to build a vserver infrastructure on Debian Lenny with >>>> following parameter : >>>> >>>> OS : >>>> >>>> - One host with the kernel 2.6.26 / Debian lenny >>>> >>> The usage of the Lenny Kernel is not a good idea. >>> >>> with the debian tools >>> >>>> - 3 guests, each one runs on a logical volume in reiserfs >>>> >>>> Network : >>>> >>>> - 2 Vlan, one for the the lan, one for the dmz >>>> - some guests's ip are one the dmz, some on the lan >>>> - on the hosts we have an /etc/interface with two interfaces with bonding >>>> (balance-rr mode) >>>> - Vlan have been set up in the network configuration, each guest interfaces >>>> as bond0.vlan as device. >>>> >>>> >>>> Here's my problem, >>>> >>>> On a dmz guest, when I run a ip route show, I see the routes of the host, so >>>> a route to the lan ... >>>> >>> Linux-Vserver doesn't virtualize the network layer, so all the routes are seen by all the guests. >>> >>> which is not good for security ... >>> I think it's not really a problem. You can use iptables on the host side for isolating your networks. >>> >>> >>>> What is the best solution to isolate the guest and controlling routing ? . I >>>> googled it and I found some answers by using ip route with alternate >>>> table ... >>>> >>> Sure. I use 802.1Q VLANs and alternate routing tables. >>> >>> Is it the best and easiest solution now ? >>> In my opinion, it is the most simplest and cleanest way to do that. >>> >>> How can I set up that >>> >>>> correctly in Debian ? >>>> >>> The solution is not specific to Debian. >>> >>> An example: >>> vconfig add eth0 7 >>> ip rule add from 10.7.0.0/16 table 7 >>> ip route add 10.7.0.0/16 dev eth0.7 table 7 >>> ip route add default via 10.7.0.1 table 7 >>> echo eth0.7 > /etc/vservers/vs 7/interfaces/0/dev >>> echo 10.0.7.0.2 > /etc/vservers/vs 7/interfaces/0/ip >>> >>> Google => "presentation-vserver.odp" for the full example. >>> >>> But the author has forgotten two details: >>> - a routing table can be enabled only when at less one IP is available on the matching network >>> - you must force "automatic promotion of secondary addresses to primary" >>> echo "net.ipv4.conf.all.promote secondaries=1">>/etc/sysctl.conf >>> >>> In fact, i wrote a script that handle routes during the start sequence of the Vservers. >>> >>> An example is available here: >>> http://dokuwicri.univ-reims.fr/files/vs-tools2/vs-tools/scripts/ >>> >>> You can get some ideas here for firewalling: >>> http://dokuwicri.univ-reims.fr/wiki/doku.php?id=public:projets:vs-tools:configuration du firewall >>> My old ugly scripts are available here: >>> https://listes.univ-reims.fr/sympa/d read/vs-tools/Sources/ >>> (http://dokuwicri.univ-reims.fr/wiki/doku.php?id=public:projets:vs-tools:start) >>> >>> Regards, >>> >>> L.S. >>> >>> >>> >>> >>> >>>> Thanks by advance ... >>>> >>>> >>> >>> -- >>> Laurent Spagnol >>> Administrateur système Linux >>> >>> CRI - Université de Reims >>> >>> Campus du Moulin de la Housse >>> BP 1039 - 51687 Reims cedex 2 >>> >>> Tel: 03.26.91.88.32 >>> Fax: 03.26.91.31.87 >>> >>> >>> >>> >> >> >> >> > > > -- > Laurent Spagnol > Administrateur Linux > > Université de Reims > Centre de Ressources Informatiques > > Campus du Moulin de la Housse > BP 1039 - 51687 Reims cedex 2 > > Tel: 03.26.91.88.32 > Fax: 03.26.91.31.87
Olivier BATARD a écrit :I suppose you mean /24 ? ;)Hi, Thanks again but I have a problem : I've made two table : - rt1 : One route 192.168.2.0/ 24 dev eth1 - rt2 : One route 192.168.3.0/34 dev eth0
An example is better for a good understanding:The rules has been made too : 0: from all lookup local 32764: from 192.168.3.0/24 lookup rt2 32765: from 192.168.2.0/24 lookup rt1 The main table is empty. Works greats, guests are isolated but when I want to add default route with : ip route add default via 192.168.2.1 table rt1 or ip route add 192.168.3.0/24 via 192.168.2.1 table rt1 I've the error : RTNETLINK answers: No such process Thanks for your help
vsh-13:~# ip rule add from 10.7.0.0/16 table 7 => OK
vsh-13:~# ip route add 10.7.0.0/16 dev eth0.7 table 7
Cannot find device "eth0.7" => You must create VLAN before
vsh-13:~# vconfig add eth0 7 => OK
Added VLAN with VID == 7 to IF -:eth0:-
vsh-13:~# ip route add 10.7.0.0/16 dev eth0.7 table 7
RTNETLINK answers: Network is down => You must enable interface.VLAN before
vsh-13:~# ifconfig eth0.7 up => OK
vsh-13:~# ip route add 10.7.0.0/16 dev eth0.7 table 7 => OK
vsh-13:~# ip route add default via 10.7.0.1 table 7
RTNETLINK answers: No such process => You must have an active IP on this network before
vsh-13:~# ifconfig eth0.7:test1 10.7.0.53 up => OK
vsh-13:~# ip route add default via 10.7.0.1 table 7 => OK
vsh-13:~# ip rule
0: from all lookup local
32765: from 10.7.0.0/16 lookup 7
32766: from all lookup main
32767: from all lookup default
vsh-13:~# ip route show table 7
10.7.0.0/16 dev eth0.7 scope link
default via 10.7.0.1 dev eth0.7
So the short answer is "some operations have some prerequisites":
ip rule add from ... table ... => no constraints
ip route add ... dev ... table ... => interface.VLAN MUST be active
ip route add default via ... table ... => The network MUST have at less one active IP => you must have at less on Vserver running with one active IP on this network)
In the real life, i use the "start-stop-scripts" of Util-Vserver (/etc/vservers/.../scripts/...)
* /etc/vservers/.../scripts/initialize.d/
Before doing anything:
- initialisation of VARS (ip, vlan, device, network, prefix, gateway) for all IPs of the Vserver
NOTE: "gateway" is not handled with Util-Vserver, i write this address in /etc/vservers/.../interfaces/.../gateway
- check of IPs (fping) -> abortion of start sequence if one of them is already active elsewhere
* /etc/vservers/.../scripts/prepre-start.d/10 vlan-up
- loading of dot1q Kernel module if required
- VLAN(s) is (are) added to the network device if required
- interface.VLAN(s) is (are) enabled if required (MTU should be changed at this stage)
* /etc/vservers/.../scripts/pre-start.d/10 route-up
=> At this stage, all the interfaces of Vserver are available
- installation of routing tables if required
* /etc/vservers/.../scripts/post-start.d/99 start-end
=> At this stage, the Vserver is fully functional
- "arping" to gateway and network to force update of ARP tables
NOTE:
You must be extremely carefull: "/etc/init.d/util-vserver" (Debian way) and "/etc/init.d/vservers-default" (genuine) will start several Vserver at once, so the risk is to create duplicate routing tables.
That why i use Kernel locks (dotlockfile) in my script to avoid this problem. (probably better to use /usr/lib/util-vserver/lockfile ?)
The handling of routing with Util-Vserver should be a great idea ;)
Regards,
L.S.
Le 12 janv. 2010 à 23:56, Laurent Spagnol a écrit :Hi, Message de Olivier BATARD <obatard@gmail.com>:Hello, We're trying to build a vserver infrastructure on Debian Lenny with following parameter : OS : - One host with the kernel 2.6.26 / Debian lennyThe usage of the Lenny Kernel is not a good idea. with the debian tools- 3 guests, each one runs on a logical volume in reiserfs Network : - 2 Vlan, one for the the lan, one for the dmz - some guests's ip are one the dmz, some on the lan - on the hosts we have an /etc/interface with two interfaces with bonding (balance-rr mode) - Vlan have been set up in the network configuration, each guest interfaces as bond0.vlan as device. Here's my problem, On a dmz guest, when I run a ip route show, I see the routes of the host, so a route to the lan ...Linux-Vserver doesn't virtualize the network layer, so all the routes are seen by all the guests. which is not good for security ... I think it's not really a problem. You can use iptables on the host side for isolating your networks.What is the best solution to isolate the guest and controlling routing ? . I googled it and I found some answers by using ip route with alternate table ...Sure. I use 802.1Q VLANs and alternate routing tables. Is it the best and easiest solution now ? In my opinion, it is the most simplest and cleanest way to do that. How can I set up thatcorrectly in Debian ?The solution is not specific to Debian. An example: vconfig add eth0 7 ip rule add from 10.7.0.0/16 table 7 ip route add 10.7.0.0/16 dev eth0.7 table 7 ip route add default via 10.7.0.1 table 7 echo eth0.7 > /etc/vservers/vs 7/interfaces/0/dev echo 10.0.7.0.2 > /etc/vservers/vs 7/interfaces/0/ip Google => "presentation-vserver.odp" for the full example. But the author has forgotten two details: - a routing table can be enabled only when at less one IP is available on the matching network - you must force "automatic promotion of secondary addresses to primary" echo "net.ipv4.conf.all.promote secondaries=1">>/etc/sysctl.conf In fact, i wrote a script that handle routes during the start sequence of the Vservers. An example is available here: http://dokuwicri.univ-reims.fr/files/vs-tools2/vs-tools/scripts/ You can get some ideas here for firewalling: http://dokuwicri.univ-reims.fr/wiki/doku.php?id=public:projets:vs-tools:configuration du firewall My old ugly scripts are available here: https://listes.univ-reims.fr/sympa/d read/vs-tools/Sources/ (http://dokuwicri.univ-reims.fr/wiki/doku.php?id=public:projets:vs-tools:start) Regards, L.S.Thanks by advance ...-- Laurent Spagnol Administrateur système Linux CRI - Université de Reims Campus du Moulin de la Housse BP 1039 - 51687 Reims cedex 2 Tel: 03.26.91.88.32 Fax: 03.26.91.31.87
-- Laurent Spagnol Administrateur Linux Université de Reims Centre de Ressources Informatiques Campus du Moulin de la Housse BP 1039 - 51687 Reims cedex 2 Tel: 03.26.91.88.32 Fax: 03.26.91.31.87