Subject: Network confusion
From: Phil Daws <uxbod@splatnix.net>
Date: Mon, 11 Mar 2013 15:05:32 +0000 (GMT)

Hello all,

am wishing to consolidate some of my hardware into a single server that will have a
number of vservers which need to be exposed to the Internet.  I already have a block
of IP addresses from my DSL provider and wish to use them for different services.  What
has been baffling me is how to do this on a single server.  I tried to create two VLANs,
one external and the other internal, but the upstream ISP performs routing based on
the vtag so I was unable to reach their gateway. What I have at the moment is as follows:

3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 90:b1:1c8 brd ff:ff:ff:ff:ff:ff
    inet 212.XXX.XXX.2/24  brd 212.XXX.XXX.255 scope global eth0
    inet 212.XXX.XXX.10/24 brd 212.XXX.XXX.255 scope global eth0
    inet 212.XXX.XXX.20/24 brd 212.XXX.XXX.255 scope global eth0

18: eth0.192@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP

    link/ether 90:b1:1c8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.10/24 brd 192.168.1.255 scope global eth0.192
    inet 192.168.1.20/24 brd 192.168.1.255 scope global eth0.192
    inet 192.168.1.30/24 brd 192.168.1.255 scope global eth0.192

and this is what the routing table shows:

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         212.XXX.XXX.1   0.0.0.0         UG        0 0          0 eth0
212.XXX.XXX.0   0.0.0.0         255.255.255.0   U         0 0          0 eth0
212.XXX.XXX.1   0.0.0.0         255.255.255.255 UH        0 0          0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 eth0.192

for my vservers I have created:

[root@vs1 ~]# vserver-stat
CTX   PROC    VSZ    RSS  userTIME   sysTIME    UPTIME NAME
0      110   3.6G  54.9M  10m16s47   1m59s17   1h55m55 root server
10001   57  94.5G   2.6G   6m12s70   1m03s33  46m48s20 wm01             192.168.1.10
10002    2 540.8M  15.8M   0m00s20   0m00s20  17m01s91 ns01             192.168.1.20
10003    3 857.3M    92M   0m03s99   0m00s33   1h55m33 db01             192.168.1.30

I wish to have 212.XXX.XXX.10 to send its requests to 192.168.1.10 as-well for outbound
traffic to go out on the same IP.  Therefore I created a couple of IP tables rules for
SNAT/DNAT to map:

-I INTI -d 212.XXX.XXX.10  -j DNAT --to-destination 192.168.1.10
-I INTI -d 212.XXX.XXX.20  -j DNAT --to-destination 192.168.1.20
-I INTO -s 192.168.1.10    -j SNAT --to 212.XXX.XXX.10
-I INTO -s 192.168.1.10    -j SNAT --to 212.XXX.XXX.20

This works fine and the two internal servers on 192.168.1.10 and 192.168.1.20 can update
their packages from the Internet.  The problem comes that the server on 192.168.1.30
is unable to get out to the Internet as no SNAT/DNAT is in place.

I guess there must be a way that any other servers would assume the main IP 212.XXX.XXX.2
if no SNAT/DNAT is defined ? Or should I be thinking of making the server more like
a firewall and putting the vservers into their own network namespace so that a separate
IP stack can be created eg. http://linux-vserver.org/util-vserver:SplitSharedNetworks

Any thoughts would be gratefully appreciated.

Thanks.