I had kernel panic on different servers under 3.0.24-vs2.3.2.3-beng lately, these seems to be network related. There is a recurrent message on the kern.log on these before the crashes. Here is what i got on one of the servers : Apr 29 21:42:33 localhost kernel: [823558.569637] ------------[ cut here ]------------ Apr 29 21:42:33 localhost kernel: [823558.569647] WARNING: at net/ipv4/tcp_input.c:3375 tcp_ack+0x8c4/0x19b8() Apr 29 21:42:33 localhost kernel: [823558.569656] Hardware name: X9SCL/X9SCM Apr 29 21:42:33 localhost kernel: [823558.569657] Modules linked in: nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_tcpudp iptable_filter ip_tables x_tables nfs lockd auth_rpcgss nfs_acl sunrpc acpi_cpufreq mperf cpufreq_conservative cpufreq_userspace cpufreq_powersave cpufreq_stats ext3 jbd coretemp ipmi_si ipmi_msghandler loop snd_pcm snd_timer snd soundcore intel_agp tpm_tis tpm snd_page_alloc i2c_i801 i2c_core button joydev ghes processor intel_gtt tpm_bios hed pcspkr evdev thermal_sys psmouse serio_raw ext4 mbcache jbd2 crc16 usbhid hid sd_mod crc_t10dif ahci libahci libata scsi_mod ehci_hcd usbcore e1000e [last unloaded: scsi_wait_scan] Apr 29 21:42:33 localhost kernel: [823558.569686] Pid: 20900, comm: nodeusage.sh Tainted: G W 3.0.24-vs2.3.2.3-beng #1 Apr 29 21:42:33 localhost kernel: [823558.569687] Call Trace: Apr 29 21:42:33 localhost kernel: [823558.569688] <IRQ> [<ffffffff8103f4ca>] ? warn_slowpath_common+0x78/0x8c Apr 29 21:42:33 localhost kernel: [823558.569694] [<ffffffff812b1133>] ? tcp_ack+0x8c4/0x19b8 Apr 29 21:42:33 localhost kernel: [823558.569697] [<ffffffff812b22f4>] ? tcp_rcv_established+0xcd/0x686 Apr 29 21:42:33 localhost kernel: [823558.569699] [<ffffffff812b96ce>] ? tcp_v4_do_rcv+0x1a9/0x3b4 Apr 29 21:42:33 localhost kernel: [823558.569701] [<ffffffff812bab29>] ? T.1236+0x37/0x69 Apr 29 21:42:33 localhost kernel: [823558.569703] [<ffffffff812bb445>] ? tcp_v4_rcv+0x449/0x6e4 Apr 29 21:42:33 localhost kernel: [823558.569705] [<ffffffff8129e6cf>] ? ip_local_deliver_finish+0x13d/0x1aa Apr 29 21:42:33 localhost kernel: [823558.569708] [<ffffffff81278556>] ? __netif_receive_skb+0x466/0x499 Apr 29 21:42:33 localhost kernel: [823558.569710] [<ffffffff812787c6>] ? netif_receive_skb+0x77/0x7d Apr 29 21:42:33 localhost kernel: [823558.569712] [<ffffffff81278d2d>] ? napi_gro_receive+0x1f/0x2d Apr 29 21:42:33 localhost kernel: [823558.569714] [<ffffffff8127889b>] ? napi_skb_finish+0x1c/0x31 Apr 29 21:42:33 localhost kernel: [823558.569720] [<ffffffffa0010941>] ? e1000_clean_rx_irq+0x1f6/0x28c [e1000e] Apr 29 21:42:33 localhost kernel: [823558.569724] [<ffffffffa0010dde>] ? e1000_clean+0x71/0x229 [e1000e] Apr 29 21:42:33 localhost kernel: [823558.569726] [<ffffffff81278e5a>] ? net_rx_action+0xa8/0x206 Apr 29 21:42:33 localhost kernel: [823558.569728] [<ffffffff81044c1b>] ? __do_softirq+0xc4/0x1a0 Apr 29 21:42:33 localhost kernel: [823558.569731] [<ffffffff81092690>] ? handle_irq_event_percpu+0x166/0x184 Apr 29 21:42:33 localhost kernel: [823558.569734] [<ffffffff8133655c>] ? call_softirq+0x1c/0x30 Apr 29 21:42:33 localhost kernel: [823558.569736] [<ffffffff8100408b>] ? do_softirq+0x3f/0x79 Apr 29 21:42:33 localhost kernel: [823558.569738] [<ffffffff810449eb>] ? irq_exit+0x44/0xb5 Apr 29 21:42:33 localhost kernel: [823558.569740] [<ffffffff810039d5>] ? do_IRQ+0x94/0xaa Apr 29 21:42:33 localhost kernel: [823558.569742] [<ffffffff81334c93>] ? common_interrupt+0x13/0x13 Apr 29 21:42:33 localhost kernel: [823558.569743] <EOI> [<ffffffff810ea2b7>] ? kmem_cache_free+0x27/0x62 Apr 29 21:42:33 localhost kernel: [823558.569749] [<ffffffff810d5ebe>] ? remove_vma+0x64/0x6c Apr 29 21:42:33 localhost kernel: [823558.569751] [<ffffffff810d5fa0>] ? exit_mmap+0xda/0xfb Apr 29 21:42:33 localhost kernel: [823558.569753] [<ffffffff8103d496>] ? mmput+0x36/0xd5 Apr 29 21:42:33 localhost kernel: [823558.569756] [<ffffffff810fde0b>] ? flush_old_exec+0x52b/0x60d Apr 29 21:42:33 localhost kernel: [823558.569759] [<ffffffff81135830>] ? load_elf_binary+0x2af/0x1733 Apr 29 21:42:33 localhost kernel: [823558.569762] [<ffffffff810fd3f2>] ? get_arg_page+0x4b/0xd9 Apr 29 21:42:33 localhost kernel: [823558.569764] [<ffffffff810fd760>] ? copy_strings+0x1fe/0x245 Apr 29 21:42:33 localhost kernel: [823558.569766] [<ffffffff810feafd>] ? search_binary_handler+0xa0/0x243 Apr 29 21:42:33 localhost kernel: [823558.569767] [<ffffffff810fee36>] ? do_execve_common+0x196/0x2ca Apr 29 21:42:33 localhost kernel: [823558.569771] [<ffffffff8102a991>] ? sys32_execve+0x3f/0x54 Apr 29 21:42:33 localhost kernel: [823558.569773] [<ffffffff81336c05>] ? ia32_ptregs_common+0x25/0x4b Apr 29 21:42:33 localhost kernel: [823558.569775] ---[ end trace 1cf81a8e50efa328 ]--- And on another : May 1 12:19:23 localhost kernel: [95833.147842] ------------[ cut here ]------------ May 1 12:19:23 localhost kernel: [95833.147844] WARNING: at net/ipv4/tcp_input.c:2999 tcp_ack+0xe42/0x19b8() May 1 12:19:23 localhost kernel: [95833.147845] Hardware name: X8STi May 1 12:19:23 localhost kernel: [95833.147846] Modules linked in: acpi_cpufreq mperf cpufreq_userspace cpufreq_stats cpufreq_conservative cpufreq_powersave bonding ipt_LOG xt_limit xt_multiport iptable_filter ip_tables x_tables ext3 jbd w83627ehf hwmon_vid coretemp ipmi_si ipmi_msghandler loop snd_pcm snd_timer snd soundcore i2c_i801 i2c_core tpm_tis tpm ioatdma tpm_bios snd_page_alloc i7core_edac evdev dca edac_core button processor thermal_sys psmouse pcspkr serio_raw ext4 mbcache jbd2 crc16 dm_mod raid1 raid0 md_mod sd_mod crc_t10dif usbhid hid uhci_hcd ahci libahci libata scsi_mod ehci_hcd usbcore e1000e [last unloaded: scsi_wait_scan] May 1 12:19:23 localhost kernel: [95833.147866] Pid: 1916, comm: php5-cgi Tainted: G W 3.0.24-vs2.3.2.3-beng #1 May 1 12:19:23 localhost kernel: [95833.147867] Call Trace: May 1 12:19:23 localhost kernel: [95833.147868] <IRQ> [<ffffffff8103f4ca>] ? warn_slowpath_common+0x78/0x8c May 1 12:19:23 localhost kernel: [95833.147871] [<ffffffff812b16b1>] ? tcp_ack+0xe42/0x19b8 May 1 12:19:23 localhost kernel: [95833.147873] [<ffffffff812b02d4>] ? tcp_validate_incoming+0x68/0x255 May 1 12:19:23 localhost kernel: [95833.147876] [<ffffffff812b27dc>] ? tcp_rcv_established+0x5b5/0x686 May 1 12:19:23 localhost kernel: [95833.147878] [<ffffffff812b96ce>] ? tcp_v4_do_rcv+0x1a9/0x3b4 May 1 12:19:23 localhost kernel: [95833.147880] [<ffffffff812bab29>] ? T.1236+0x37/0x69 May 1 12:19:23 localhost kernel: [95833.147881] [<ffffffff812bb445>] ? tcp_v4_rcv+0x449/0x6e4 May 1 12:19:23 localhost kernel: [95833.147884] [<ffffffffa02a6a19>] ? bond_alb_initialize+0x159/0x159 [bonding] May 1 12:19:23 localhost kernel: [95833.147886] [<ffffffff8129e6cf>] ? ip_local_deliver_finish+0x13d/0x1aa May 1 12:19:23 localhost kernel: [95833.147888] [<ffffffff81278556>] ? __netif_receive_skb+0x466/0x499 May 1 12:19:23 localhost kernel: [95833.147890] [<ffffffff812787c6>] ? netif_receive_skb+0x77/0x7d May 1 12:19:23 localhost kernel: [95833.147892] [<ffffffff81278d2d>] ? napi_gro_receive+0x1f/0x2d May 1 12:19:23 localhost kernel: [95833.147894] [<ffffffff8127889b>] ? napi_skb_finish+0x1c/0x31 May 1 12:19:23 localhost kernel: [95833.147898] [<ffffffffa0010941>] ? e1000_clean_rx_irq+0x1f6/0x28c [e1000e] May 1 12:19:23 localhost kernel: [95833.147902] [<ffffffffa0010dde>] ? e1000_clean+0x71/0x229 [e1000e] May 1 12:19:23 localhost kernel: [95833.147904] [<ffffffff81278e5a>] ? net_rx_action+0xa8/0x206 May 1 12:19:23 localhost kernel: [95833.147906] [<ffffffff81044c1b>] ? __do_softirq+0xc4/0x1a0 May 1 12:19:23 localhost kernel: [95833.147908] [<ffffffff81092690>] ? handle_irq_event_percpu+0x166/0x184 May 1 12:19:23 localhost kernel: [95833.147910] [<ffffffff81044c94>] ? __do_softirq+0x13d/0x1a0 May 1 12:19:23 localhost kernel: [95833.147912] [<ffffffff8133655c>] ? call_softirq+0x1c/0x30 May 1 12:19:23 localhost kernel: [95833.147913] [<ffffffff8100408b>] ? do_softirq+0x3f/0x79 May 1 12:19:23 localhost kernel: [95833.147915] [<ffffffff810449eb>] ? irq_exit+0x44/0xb5 May 1 12:19:23 localhost kernel: [95833.147917] [<ffffffff810039d5>] ? do_IRQ+0x94/0xaa May 1 12:19:23 localhost kernel: [95833.147919] [<ffffffff81334c93>] ? common_interrupt+0x13/0x13 May 1 12:19:23 localhost kernel: [95833.147920] <EOI> [<ffffffff81336a83>] ? ia32_do_call+0x13/0x13 May 1 12:19:23 localhost kernel: [95833.147922] ---[ end trace 43efaa2da94bb3b9 ]--- It seems to be this bug : http://patchwork.ozlabs.org/patch/144844/ I installed linux-image-3.2.15-vs2.3.2.8-beng on the servers that had this kernel hoping it wont happen again on these. Greetings.