Subject: Re: [vserver] linux 4.4.113-2.3.9.6 kernel crash during "vserver ... stop"
From: Herbert Poetzl <herbert@13thfloor.at>
Date: Tue, 6 Feb 2018 17:30:37 +0100

On Tue, Feb 06, 2018 at 05:25:19AM -0500, user3441 wrote:
> Hello!

> I have tried updating a 3.2.98-vserver kernel to a 4.4.113
> vserver kernel. It seems to start up correctly. But when I
> execute "vserver ... stop" on the command line, the kernel
> crashes with ~50% probability. This did not happen with the
> 3.2.98 kernel. When it crashes, this is printed on the screen
> (the top of the trace has been cut, because it did not fit on
> the screen):

> [ 101.390624] [<ffffffff810cb927>] ? vprintk_emit+0x517/0x560
> [ 101.390788] [<ffffffff810cba8a>] vprintk_default+0x1a/0x20
> [ 101.390951] [<ffffffff81141fab>] printk+0x43/0x4b
> [ 101.391113] [<ffffffff8109c939>] __schedule_bug+0x39/0xb0
> [ 101.391277] [<ffffffff816816b5>] __schedule+0x6d5/0x860
> [ 101.391433] [<ffffffff81681a7e>] preempt_schedule_irq+0x5e/0xa0
> [ 101.391610] [<ffffffff8168834e>] retint_kernel+0x1b/0x2d
> [ 101.391774] [<ffffffff812e567c>] ? dump_stack+0x8d/0xa1
> [ 101.391937] [<ffffffff8109c95e>] __schedule_bug+0x5e/0xb0
> [ 101.392102] [<ffffffff816816b5>] __schedule+0x6d5/0x860
> [ 101.392266] [<ffffffff816816ba>] schedule+0x3a/0x90
> [ 101.392429] [<ffffffff8107b450>] do_exit+0x7f0/0xc70
> [ 101.392594] [<ffffffff810c146d>] ? trace_hardirqs_on+0xd/0x10
> [ 101.392761] [<ffffffff81141488>] ? context_tracking_exit+0x18/0x20
> [ 101.392925] [<ffffffff8107b95b>] do_group_exit+0x4b/0xc0
> [ 101.393089] [<ffffffff8107b9df>] SyS_exit_group+0xf/0x10
> [ 101.393254] [<ffffffff81687748>] entry_SYSCALL_64_fastpath+0x1c/0x91
> [ 101.393429] Code: e8 ea 60 01 00 85 c0 74 09 80 3d 3a 08 8a 00 00 71 65 49 8b
> 56 60 48 c7 c6 40 3d 96 81 48 63 cb 48 8b 82 d8 00 00 00 48 03 04 ce <4c> 00 20
> 48 8b 52 48 48 85 d2 75 e9 e8 03 60 01 00 85 c0 74 0d
> [ 101.396170] RIP  [<ffffffff810bdf0c>] cpuacct_charge+0x8c/0x1a0
> [ 101.396367]  RSP <fff88023fc43ad0>
> [ 101.396520] CR2: 000000000000f0e9
> [ 101.396673] ---[ end trace 365c9bf61aab792c ]---

> I have once seen this when connected via ssh:

> test1:~# reboot

> The system is going down for reboot NOW!
> test1:~#
> Message from syslogd@test1 at Feb  5 13:27:00 ...
> kernel:[ 2326.768581] Kernel panic - not syncing: corrupted stack end detected inside
scheduler

> Message from syslogd@test1 at Feb  5 13:27:00 ...
> kernel:[ 2326.768581]
> Write failed: Broken pipe
> $

> Also, there are these messages when booting (I think they are
> triggered by vservers starting up):

[stuff zapped]

> [   50.082543] BUG: sleeping function called from invalid context at include/linux/vs_context.h:152
> [   50.082832] in_atomic(): 1, irqs_disabled(): 0, pid: 3500, name: exim4
> [   50.082994] no locks held by exim4/3500.
> [   50.083167] Preemption disabled at:[<ffffffff8107b95b>] do_group_exit+0x4b/0xc0
> [   50.083497]
> [   50.083657] CPU: 5 PID: 3500 Comm: exim4 Not tainted 4.4.113-vs2.3.9.6 #1
> [   50.083819] Hardware name: Supermicro X8SIL/X8SIL, BIOS 1.1 05/27/2010
> [   50.083982]  0000000000000000 ffff880233e47e18 ffffffff812e5657 0000000000000000
> [   50.084395]  ffff8800babd2200 ffff880233e47e40 ffffffff810a6069 ffffffff8182ef96
> [   50.084814]  0000000000000098 0000000000000000 ffff880233e47e68 ffffffff810a6194
> [   50.085253] Call Trace:
> [   50.085410]  [<ffffffff812e5657>] dump_stack+0x68/0xa1
> [   50.085576]  [<ffffffff810a6069>] ___might_sleep+0x159/0x240
> [   50.085736]  [<ffffffff810a6194>] __might_sleep+0x44/0x80
> [   50.085898]  [<ffffffff810ddca5>] exit_vx_info+0x45/0x60
> [   50.086058]  [<ffffffff8107b412>] do_exit+0x7b2/0xc70
> [   50.086245]  [<ffffffff810c146d>] ? trace_hardirqs_on+0xd/0x10
> [   50.086407]  [<ffffffff81141488>] ? context_tracking_exit+0x18/0x20
> [   50.086568]  [<ffffffff8107b95b>] do_group_exit+0x4b/0xc0
> [   50.086727]  [<ffffffff8107b9df>] SyS_exit_group+0xf/0x10
> [   50.086887]  [<ffffffff81687748>] entry_SYSCALL_64_fastpath+0x1c/0x91

> Any ideas?

Can you try this patch and see if it fixes the issue for you?

http://vserver.13thfloor.at/Experimental/delta-exit-fix01.diff

Thanks in advance,
Herbert