Subject: VServer and Multicasting
From: Furgerot Julien <julien.furgerot@gmail.com>
Date: Mon, 10 Jan 2011 16:24:47 +0100

 Mon, 10 Jan 2011 16:24:47 +0100
Dear VServer community,

In my VServer application use case, each VM hosts a software peer that
communicates with another peer through multicast. My concern is that I
can send multicast traffic from my VM but I am not able to receive
multicast requests form other peers/machines.

The behaviour of my system is as follows :

- VMs are bound to eth interface properly with assigned IP address.
- I have enabled the following flags : NET BROADCAST, NET ADMIN, NET RAW
- I can ping to/from my VM.
- I can send multicast requests from my VM.
- HOWEVER, when sending multicast from outside the VM, I can see
through TCPDump in a VM that packets arrive to the interface. However,
they are not received as a multicast at the application level.

Details about my configuration and tests are given bellow.

I went through VServer documentation and the different forums and I
could not find a solution to this problem. Do you have any idea about
how to resolve this problem ?

Thank you and kind regards,

Julien Furgerot


vserver-info result :
===================================================
Versions:
  Kernel: 2.6.32-27-vserver
  VS-API: 0x00020305
  util-vserver: 0.30.215; Aug  7 2009, 02:57:43

Features:
  CC: gcc, gcc (Ubuntu 4.4.1-1ubuntu3) 4.4.1
  CXX: g++, g++ (Ubuntu 4.4.1-1ubuntu3) 4.4.1
  CPPFLAGS: ''
  CFLAGS: '-Wall -g  -O2 -std=c99 -Wall -pedantic -W -funit-at-a-time'
  CXXFLAGS: '-g -O2 -ansi -Wall -pedantic -W -fmessage-length=0
-funit-at-a-time'
  build/host: x86 64-pc-linux-gnu/x86 64-pc-linux-gnu
  Use dietlibc: yes
  Build C++ programs: yes
  Build C99 programs: yes
  Available APIs: v13,net,v21,v22,v23,netv2
  ext2fs Source: e2fsprogs
  syscall(2) invocation: alternative
  vserver(2) syscall#: 236/glibc
  crypto api: nss
  python bindings: no
  use library versioning: yes

Paths:
  prefix: /usr
  sysconf-Directory: /etc
  cfg-Directory: /etc/vservers
  initrd-Directory: $(sysconfdir)/init.d
  pkgstate-Directory: /var/run/vservers
  vserver-Rootdir: /var/lib/vservers
===================================================


Then I build a 32 bits guest using debootstrap :
===================================================
    vserver vt001 build -m debootstrap --hostname vt001.lan
--interface eth2:192.168.46.56 -- -d lucid -m
http://packages.ubuntu.com/ubuntu -- --arch i386
===================================================


VM starts without any problem : IP address is properly assigned to vm.
===================================================
VM# ip addr
1: lo: <LOOPBACK,UP,LOWER UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
4: eth2: <BROADCAST,MULTICAST,UP,LOWER UP> mtu 1500 qdisc pfifo fast
state UP qlen 1000
    link/ether 00:1e:67:01:a3:8a brd ff:ff:ff:ff:ff:ff
    inet 192.168.46.56/24 brd 192.168.46.255 scope global secondary eth2
===================================================

I set up routes for multicast :
===================================================
VM# ip route
192.168.46.0/24 dev eth2  proto kernel  scope link  src 192.168.46.55
239.192.30.0/24 dev eth2  scope link
default via 192.168.46.1 dev eth2  metric 100
===================================================
192.168.46.55 is the host IP on eth2, and we use multicast group 239.192.30.0



I tried to ping between VM, host and another PC in the same network
and everything works fine.

Then came multicast tests. I used a very simple example of multicast
programming (http://ntrg.cs.tcd.ie/undergrad/4ba2/multicast/antony/example.html)
to test multicast communication.
These sources are compiled (with "#define HELLO GROUP 239.192.30.0" )
to "mcastSender" and "mcastListener" binaries.

For short, communication tests worked right except for multicast
reception on the VM.


I tried with all network flags combinationn and It does not work.
===================================================
Host# cat /etc/vservers/vt001/bcapabilities
NET BROADCAST
NET ADMIN
NET RAW
===================================================


When NET RAW is used, I can listen on interface "eth2" with tcpdump,
and I saw that datagrams are properly received :
===================================================
VM# tcpdump -n -i eth2 udp port 5555
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth2, link-type EN10MB (Ethernet), capture size 96 bytes
14:39:19.162796 IP 192.168.46.151.5555 > 239.192.30.0.5555: UDP, length 3
===================================================
But nothing appears at the software level.

I run mcastListener with strace :
===================================================
VM# strace ./mcastListener
execve("./mcastListener", ["./mcastListener"], [/* 23 vars */]) = 0
brk(0)                                  = 0x8a12000
access("/etc/ld.so.nohwcap", F OK)      = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT READ|PROT WRITE, MAP PRIVATE|MAP ANONYMOUS, -1,
0) = 0xf771c000
access("/etc/ld.so.preload", R OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O RDONLY)      = 3
fstat64(3, {st mode=S IFREG|0644, st size=9246, ...}) = 0
mmap2(NULL, 9246, PROT READ, MAP PRIVATE, 3, 0) = 0xf7719000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F OK)      = -1 ENOENT (No such file or directory)
open("/lib/libc.so.6", O RDONLY)        = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0
m\1\0004\0\0\0"..., 512) = 512
fstat64(3, {st mode=S IFREG|0755, st size=1335560, ...}) = 0
mmap2(NULL, 1345896, PROT READ|PROT EXEC, MAP PRIVATE|MAP DENYWRITE,
3, 0) = 0xf75d0000
mprotect(0xf7712000, 4096, PROT NONE)   = 0
mmap2(0xf7713000, 12288, PROT READ|PROT WRITE,
MAP PRIVATE|MAP FIXED|MAP DENYWRITE, 3, 0x142) = 0xf7713000
mmap2(0xf7716000, 10600, PROT READ|PROT WRITE,
MAP PRIVATE|MAP FIXED|MAP ANONYMOUS, -1, 0) = 0xf7716000
close(3)                                = 0
mmap2(NULL, 4096, PROT READ|PROT WRITE, MAP PRIVATE|MAP ANONYMOUS, -1,
0) = 0xf75cf000
set thread area({entry number:-1 -> 12, base addr:0xf75cf6c0,
limit:1048575, seg 32bit:1, contents:0, read exec only:0,
limit in pages:1, seg not present:0, useable:1}) = 0
mprotect(0xf7713000, 8192, PROT READ)   = 0
mprotect(0x8049000, 4096, PROT READ)    = 0
mprotect(0xf773a000, 4096, PROT READ)   = 0
munmap(0xf7719000, 9246)                = 0
socket(PF INET, SOCK DGRAM, IPPROTO IP) = 3
setsockopt(3, SOL SOCKET, SO REUSEADDR, [1], 4) = 0
bind(3, {sa family=AF INET, sin port=htons(5555),
sin addr=inet addr("0.0.0.0")}, 16) = 0
setsockopt(3, SOL IP, IP ADD MEMBERSHIP, "\357\300\36\0\0\0\0\0", 8) = 0
recvfrom(3,
===================================================
bind call show socket bind on port 5555, on all interfaces
("0.0.0.0"), without error.
I also tried to force bind on eth2 ip "192.168.46.56", same problem.