Subject: Re: [vserver] tftp server problem / partly solved
From: Corin Langosch <corinl@gmx.de>
Date: Thu, 28 Feb 2008 16:30:00 +0100
Thu, 28 Feb 2008 16:30:00 +0100
Hello,

the mixup of the IPs was caused by my iptables masquerade rule on the host.
I changed this rule and now it works always. Seems like some caches 
internal
to the kernel made it work sometimes, sometimes not. So this is solved.

But I still don't get it to work with the current stable vserver branch, may
be UDP is buggy in there?

This brings me to the next question:
Is the current development branch already stable enough to be used in a
production environment?

I also was able to build kernel 2.6.22.19 using the latest vserver 
development
branch without any changes. Is this safe? May be one could update the
website? :)

Regards,
Corin

Am 28.02.2008 15:39, Corin Langosch schrieb:
> Hello,
>
> I'm still having trouble running a tftp server inside a vserver guest.
>
> I now compiled the kernel myself using the latest development branch
> (Linux system 2.6.22.18-vs2.3.0.32-vs #1 SMP Thu Feb 28 14:01:32 CET 
> 2008 x86_64 GNU/Linux)
> and sometimes tftp works, sometimes not! It's really weierd.
>
> The output of tcpdump is interessting too, because something seems to
> mess around with the IPs:
>
> 15:32:12.020090 IP 192.168.1.32.57091 > 192.168.1.203.69:  51 RRQ 
> "pxelinux.cfg/C0A80120" octet tsize 0 blksize 1408
> 15:32:12.021249 IP 192.168.1.254.32812 > 192.168.1.32.57091: UDP, 
> length 19
> 15:32:33.127937 IP 192.168.1.32.57092 > 192.168.1.203.69:  50 RRQ 
> "pxelinux.cfg/C0A8012" octet tsize 0 blksize 1408
> 15:32:33.129348 IP 192.168.1.254.32812 > 192.168.1.32.57092: UDP, 
> length 19
> 15:32:33.757553 IP 192.168.1.32.57092 > 192.168.1.203.69:  50 RRQ 
> "pxelinux.cfg/C0A8012" octet tsize 0 blksize 1408
> 15:32:33.758866 IP 192.168.1.254.32812 > 192.168.1.32.57092: UDP, 
> length 19
> 15:32:35.077954 IP 192.168.1.32.57092 > 192.168.1.203.69:  50 RRQ 
> "pxelinux.cfg/C0A8012" octet tsize 0 blksize 1408
> 15:32:35.079248 IP 192.168.1.254.32812 > 192.168.1.32.57092: UDP, 
> length 19
> 15:32:37.717157 IP 192.168.1.32.57092 > 192.168.1.203.69:  50 RRQ 
> "pxelinux.cfg/C0A8012" octet tsize 0 blksize 1408
> 15:32:37.719313 IP 192.168.1.254.32812 > 192.168.1.32.57092: UDP, 
> length 19
> 15:32:38.126243 arp who-has 192.168.1.32 tell 192.168.1.254
> 15:32:38.126338 arp reply 192.168.1.32 is-at 00:13:8f:56:02:81
> 15:32:42.995138 IP 192.168.1.32.57092 > 192.168.1.203.69:  50 RRQ 
> "pxelinux.cfg/C0A8012" octet tsize 0 blksize 1408
> 15:32:42.996413 IP 192.168.1.254.32812 > 192.168.1.32.57092: UDP, 
> length 19
>
> As can be seen the packet is sent from the client to the vserver guest 
> (.203). The atftp server replies, but
> not with the IP of the vserver guest it's running in but with the IP 
> of the host (.254)?!
>
> Could anyone try to confirm this? I'm using a plain debian system with 
> iptables and conntrack. No packets
> are dropped, so it's not a firewall issue!
>
> Best Regards,
> Corin
>
> Am 19.02.2008 20:58, Corin Langosch schrieb:
>> Hi!
>>
>> nc shows some interessting behavior.
>>
>> When listeing part running on the host itself I can write as many 
>> lines of text to the server as I want.
>>
>> 20:33:03.489755 IP 192.168.1.24.32831 > 192.168.1.254.33020: UDP, 
>> length 5
>> 20:33:05.352968 IP 192.168.1.24.32831 > 192.168.1.254.33020: UDP, 
>> length 5
>> 20:33:06.345795 IP 192.168.1.24.32831 > 192.168.1.254.33020: UDP, 
>> length 5
>> 20:33:07.393783 IP 192.168.1.24.32831 > 192.168.1.254.33020: UDP, 
>> length 5
>> 20:33:08.513780 IP 192.168.1.24.32831 > 192.168.1.254.33020: UDP, 
>> length 5
>>
>> Running the listeing part inside the vserver I can only write exactly 
>> one line to the server. The second
>> line does not get to the server and the clients exits.
>>
>> 20:32:05.777390 IP 192.168.1.24.32830 > 192.168.1.223.33020: UDP, 
>> length 5
>> 20:32:08.913378 IP 192.168.1.24.32830 > 192.168.1.223.33020: UDP, 
>> length 5
>> 20:32:08.913409 IP 192.168.1.223 > 192.168.1.24: ICMP 192.168.1.223 
>> udp port 33020 unreachable, length 41
>>
>> Something seems to interrupt the "connection", but what is it?
>>
>> The setup is a normal debian testing box without any fancy software. 
>> I also tried on antoher debian
>> testing box, no luck neither :(
>>
>> Corin
>>
>> Thomas Weber schrieb:
>>> Am Dienstag, den 19.02.2008, 16:09 +0100 schrieb Corin Langosch:
>>>   
>>>> Hi!
>>>>
>>>> I added all caps, but it still does not work :-(
>>>>
>>>> buero:/etc/vservers# cat system/bcapabilities
>>>> NET_RAW
>>>> NET_BROADCAST
>>>> SYS_TIME
>>>> SYS_NICE
>>>> SYS_RESOURCE
>>>>
>>>> The caps must be set as the dhcp server inside the vserver is running
>>>> fine, which
>>>> wont without the caps.
>>>>
>>>> No tftp server is not running inside the host, I unsinatlled it there.
>>>> "netstat -l -p" confirms this. 
>>>> Otherwise the tftpd inside the host wouldn't be able to bind to the
>>>> port neither..?
>>>>     
>>>
>>> yep, i'd think so.
>>>
>>>   
>>>> This is my tcpdump from the host:
>>>>
>>>> buero:/etc/vservers# tcpdump -n | grep '\.1\.203'
>>>>     
>>>
>>> you might wanna try tcpdump src or dst 192.168.1.203
>>>  
>>>   
>>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>>> decode
>>>> listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
>>>> 16:02:32.108114 IP 192.168.1.203.67 > 255.255.255.255.68: BOOTP/DHCP,
>>>> Reply, length 300
>>>> 16:02:34.198749 IP 192.168.1.203.67 > 255.255.255.255.68: BOOTP/DHCP,
>>>> Reply, length 300
>>>> 16:02:34.206389 arp who-has 192.168.1.203 tell 192.168.1.32
>>>> 16:02:34.206410 arp reply 192.168.1.203 is-at 00:13:8f:56:02:96
>>>> 16:02:34.206614 IP 192.168.1.32.2070 > 192.168.1.203.69:  27 RRQ
>>>> "pxelinux.0" octet tsize 0
>>>> 16:02:34.207638 IP 192.168.1.32.2070 > 192.168.1.203.32973: UDP,
>>>> length 17
>>>> 16:02:34.207674 IP 192.168.1.203 > 192.168.1.32: ICMP 192.168.1.203
>>>> udp port 32973 unreachable, length 53
>>>> 16:02:34.214165 IP 192.168.1.32.2071 > 192.168.1.203.69:  32 RRQ
>>>> "pxelinux.0" octet blksize 1456
>>>> 16:02:34.215175 IP 192.168.1.32.2071 > 192.168.1.203.32974: UDP,
>>>> length 4
>>>> 16:02:34.215207 IP 192.168.1.203 > 192.168.1.32: ICMP 192.168.1.203
>>>> udp port 32974 unreachable, length 40
>>>> 16:02:37.968200 IP 192.168.1.32.2071 > 192.168.1.203.32972: UDP,
>>>> length 4
>>>> 16:02:37.968234 IP 192.168.1.203 > 192.168.1.32: ICMP 192.168.1.203
>>>> udp port 32972 unreachable, length 40
>>>>     
>>>
>>>   
>>>> For me it seems that the chosen data-port for the transfer by the
>>>> server is somehow not reachable? 
>>>> What could the reason be, how to solve it?
>>>>     
>>>
>>> Firewalling or other security stuff?
>>> Did you try testing with netcat?
>>> Inside the vserver: nc -l -u -p 32974
>>>   
>>>> >From the client: nc -u 192.168.1.32 32974
>>>>     
>>> and type some stuff there, see if it gets through.
>>>
>>> I'm running this setup on various boxes witch different vserver
>>> versions, and i haven't had a problem with it for years.
>>>
>>>   Tom
>>>
>>>
>>>   


Hello,

the mixup of the IPs was caused by my iptables masquerade rule on the host.
I changed this rule and now it works always. Seems like some caches internal
to the kernel made it work sometimes, sometimes not. So this is solved.

But I still don't get it to work with the current stable vserver branch, may
be UDP is buggy in there?

This brings me to the next question:
Is the current development branch already stable enough to be used in a
production environment?

I also was able to build kernel 2.6.22.19 using the latest vserver development
branch without any changes. Is this safe? May be one could update the
website? :)

Regards,
Corin

Am 28.02.2008 15:39, Corin Langosch schrieb:
Hello,

I'm still having trouble running a tftp server inside a vserver guest.

I now compiled the kernel myself using the latest development branch
(Linux system 2.6.22.18-vs2.3.0.32-vs #1 SMP Thu Feb 28 14:01:32 CET 2008 x86_64 GNU/Linux)
and sometimes tftp works, sometimes not! It's really weierd.

The output of tcpdump is interessting too, because something seems to
mess around with the IPs:

15:32:12.020090 IP 192.168.1.32.57091 > 192.168.1.203.69:  51 RRQ "pxelinux.cfg/C0A80120" octet tsize 0 blksize 1408
15:32:12.021249 IP 192.168.1.254.32812 > 192.168.1.32.57091: UDP, length 19
15:32:33.127937 IP 192.168.1.32.57092 > 192.168.1.203.69:  50 RRQ "pxelinux.cfg/C0A8012" octet tsize 0 blksize 1408
15:32:33.129348 IP 192.168.1.254.32812 > 192.168.1.32.57092: UDP, length 19
15:32:33.757553 IP 192.168.1.32.57092 > 192.168.1.203.69:  50 RRQ "pxelinux.cfg/C0A8012" octet tsize 0 blksize 1408
15:32:33.758866 IP 192.168.1.254.32812 > 192.168.1.32.57092: UDP, length 19
15:32:35.077954 IP 192.168.1.32.57092 > 192.168.1.203.69:  50 RRQ "pxelinux.cfg/C0A8012" octet tsize 0 blksize 1408
15:32:35.079248 IP 192.168.1.254.32812 > 192.168.1.32.57092: UDP, length 19
15:32:37.717157 IP 192.168.1.32.57092 > 192.168.1.203.69:  50 RRQ "pxelinux.cfg/C0A8012" octet tsize 0 blksize 1408
15:32:37.719313 IP 192.168.1.254.32812 > 192.168.1.32.57092: UDP, length 19
15:32:38.126243 arp who-has 192.168.1.32 tell 192.168.1.254
15:32:38.126338 arp reply 192.168.1.32 is-at 00:13:8f:56:02:81
15:32:42.995138 IP 192.168.1.32.57092 > 192.168.1.203.69:  50 RRQ "pxelinux.cfg/C0A8012" octet tsize 0 blksize 1408
15:32:42.996413 IP 192.168.1.254.32812 > 192.168.1.32.57092: UDP, length 19

As can be seen the packet is sent from the client to the vserver guest (.203). The atftp server replies, but
not with the IP of the vserver guest it's running in but with the IP of the host (.254)?!

Could anyone try to confirm this? I'm using a plain debian system with iptables and conntrack. No packets
are dropped, so it's not a firewall issue!

Best Regards,
Corin

Am 19.02.2008 20:58, Corin Langosch schrieb:
Hi!

nc shows some interessting behavior.

When listeing part running on the host itself I can write as many lines of text to the server as I want.

20:33:03.489755 IP 192.168.1.24.32831 > 192.168.1.254.33020: UDP, length 5
20:33:05.352968 IP 192.168.1.24.32831 > 192.168.1.254.33020: UDP, length 5
20:33:06.345795 IP 192.168.1.24.32831 > 192.168.1.254.33020: UDP, length 5
20:33:07.393783 IP 192.168.1.24.32831 > 192.168.1.254.33020: UDP, length 5
20:33:08.513780 IP 192.168.1.24.32831 > 192.168.1.254.33020: UDP, length 5

Running the listeing part inside the vserver I can only write exactly one line to the server. The second
line does not get to the server and the clients exits.

20:32:05.777390 IP 192.168.1.24.32830 > 192.168.1.223.33020: UDP, length 5
20:32:08.913378 IP 192.168.1.24.32830 > 192.168.1.223.33020: UDP, length 5
20:32:08.913409 IP 192.168.1.223 > 192.168.1.24: ICMP 192.168.1.223 udp port 33020 unreachable, length 41

Something seems to interrupt the "connection", but what is it?

The setup is a normal debian testing box without any fancy software. I also tried on antoher debian
testing box, no luck neither :(

Corin

Thomas Weber schrieb:
Am Dienstag, den 19.02.2008, 16:09 +0100 schrieb Corin Langosch:
  
Hi!

I added all caps, but it still does not work :-(

buero:/etc/vservers# cat system/bcapabilities
NET_RAW
NET_BROADCAST
SYS_TIME
SYS_NICE
SYS_RESOURCE

The caps must be set as the dhcp server inside the vserver is running
fine, which
wont without the caps.

No tftp server is not running inside the host, I unsinatlled it there.
"netstat -l -p" confirms this. 
Otherwise the tftpd inside the host wouldn't be able to bind to the
port neither..?
    

yep, i'd think so.

  
This is my tcpdump from the host:

buero:/etc/vservers# tcpdump -n | grep '\.1\.203'
    

you might wanna try tcpdump src or dst 192.168.1.203
 
  
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
16:02:32.108114 IP 192.168.1.203.67 > 255.255.255.255.68: BOOTP/DHCP,
Reply, length 300
16:02:34.198749 IP 192.168.1.203.67 > 255.255.255.255.68: BOOTP/DHCP,
Reply, length 300
16:02:34.206389 arp who-has 192.168.1.203 tell 192.168.1.32
16:02:34.206410 arp reply 192.168.1.203 is-at 00:13:8f:56:02:96
16:02:34.206614 IP 192.168.1.32.2070 > 192.168.1.203.69:  27 RRQ
"pxelinux.0" octet tsize 0
16:02:34.207638 IP 192.168.1.32.2070 > 192.168.1.203.32973: UDP,
length 17
16:02:34.207674 IP 192.168.1.203 > 192.168.1.32: ICMP 192.168.1.203
udp port 32973 unreachable, length 53
16:02:34.214165 IP 192.168.1.32.2071 > 192.168.1.203.69:  32 RRQ
"pxelinux.0" octet blksize 1456
16:02:34.215175 IP 192.168.1.32.2071 > 192.168.1.203.32974: UDP,
length 4
16:02:34.215207 IP 192.168.1.203 > 192.168.1.32: ICMP 192.168.1.203
udp port 32974 unreachable, length 40
16:02:37.968200 IP 192.168.1.32.2071 > 192.168.1.203.32972: UDP,
length 4
16:02:37.968234 IP 192.168.1.203 > 192.168.1.32: ICMP 192.168.1.203
udp port 32972 unreachable, length 40
    

  
For me it seems that the chosen data-port for the transfer by the
server is somehow not reachable? 
What could the reason be, how to solve it?
    

Firewalling or other security stuff?
Did you try testing with netcat?
Inside the vserver: nc -l -u -p 32974
  
>From the client: nc -u 192.168.1.32 32974
    
and type some stuff there, see if it gets through.

I'm running this setup on various boxes witch different vserver
versions, and i haven't had a problem with it for years.

  Tom