Subject: Re: [vserver] Again: [vserver] Linux vServer: general protection fault with apache2 and kernel 2.6.38.6
From: Urban Loesch <bind@enas.net>
Date: Sat, 06 Aug 2011 14:26:17 +0200

Hi Herbert,

>> [1462583.323969] Call Trace:
>> [1462583.329191]  [<ffffffff8105cfbc>] try_to_wake_up+0x3c/0x410
>> [1462583.340642]  [<ffffffff8105d3e5>] wake_up_process+0x15/0x20
>> [1462583.352091]  [<ffffffff812710e0>] freeary+0x1e0/0x260
>> [1462583.362503]  [<ffffffff812721b1>] T.623+0x71/0xf0
>> [1462583.372223]  [<ffffffff81169685>] ? vfs_write+0x125/0x190
>> [1462583.383326]  [<ffffffff81272299>] sys_semctl+0x69/0xa0
>> [1462583.393911]  [<ffffffff8100bf82>] system_call_fastpath+0x16/0x1b
>> Result of "addr2line -e vmlinux $ADDRESS":
>> ffffffff8105cfbc:
>> sched.c:0
>> ffffffff8105d3e5:
>> ??:0
>> ffffffff812710e0:
>> sem.c:0
>> ffffffff812721b1:
>> sem.c:0
>> ffffffff81169685:
>> ??:0
>> ffffffff81272299:
>> ??:0
>> ffffffff8100bf82:
>> signal.c:0
> your kernel (or at least the vmlinux) file seems to be
> missing the necessary debug information (which isn't
> that unusual, as most folks compile the kernel without,
> which of course makes debugging the kernel a little
> tricker :)

If it helps, I can build one with debug enabled. You must me only give a 
hint which config options
I have to enable.

>> And the same with RIP:
>> [1462583.444912] RIP  [<ffffffff8104ecaa>] task_rq_lock+0x4a/0xa0
>> [1462583.456545]  RSP<ffff88041e259dc8>
>> [1462583.464160] ---[ end trace e26d734810b28493 ]---
>> Result:
>> ffffffff8104ecaa:
>> sched.c:0
>>> might be fixed in .41 or might just be less likely to
>>> happen, after all, 16 days is not something you consider
>>> easily reproduceable ...
>> Not shure, but my other server with .41 runs since 38 days
>> without problems and with a apache installed.
> it might be interesting to find what exactly triggers
> this and maybe test with some more extreme apache
> setups, like for example having a lot more threads or
> handling high load (web test or similar)

On my systems this happens only on webservers under heavy load  (around 
2360 virtualhosts enabled). I have only 2 servers with heavy apache
load. Other webservers only with less than 25 processes runnig aren't 
affected and they are working well.

>> I think you are right.
>> I have running more than 10 servers with kernel 2.6.28.[4-7]
> I presume this should be 2.6.38.x ...
Right, was a typo.

thanks,
Urban