Subject: Re: [vserver] "Stack smash" error running vserver under hardened AMD64?
From: vitalyb <vitalyvb@ukr.net>
Date: Fri, 21 Mar 2008 20:24:42 +0200

Ed Wildgoose wrote:
> Hi, can someone please give me some tips on vserver under hardened amd64?
> 
> I have a working 32 bit server running the latest 2.2 vserver+grsec 
> patch without any problems.  I have taken the kernel .config and 
> switched it over to 64bit and recompiled on a new machine, then rsynced 
> over the guests to this new machine.  When I try and start them under 
> the 64bit machine (they are 32 bit images obviously) I get the error
> 
> # vserver www2 start
> stack smashed in main (value 0x00000000000kpp00)

Had to track this down recently. I suppose it's Hardened Gentoo?
This error happens because of dietlibc. "diet" wrapper do not
passes "-fno-stack-protector" option to gcc (which has ssp
enabled by default) and strips this parameter if you
try to add it to CFLAGS manually.

Use the following patch. The first part adds no-ssp parameters
to gcc and the second removes problematic function completely
from library (you should get link time error if ssp is used).

You'll need to recompile everything util-vserver depends of
(like beecrypt) with ssp disabled and util-vserver itself, of course.

============================================== CUT
--- dietlibc-0.30/diet.c.orig   2008-02-26 12:52:33.000000000 +0200
+++ dietlibc-0.30/diet.c        2008-02-26 12:55:08.000000000 +0200
@@ -337,6 +336,10 @@
        }
  #endif
        *dest++="-D__dietlibc__";
+
+      *dest++="-fno-stack-protector";
+      *dest++="-fno-stack-protector-all";
+
        if (mangleopts) {
         const char **o=Os;

--- dietlibc-0.30/lib/stack_smash_handler.c.orig        2006-04-04 08:35:14.000000000
+0300
+++ dietlibc-0.30/lib/stack_smash_handler.c     2008-02-26 13:26:50.000000000 +0200
@@ -2,7 +2,7 @@
  #include <unistd.h>

  /* this is only used with ProPolice in gcc 3.x */
-
+#ifdef WANT_SSP
  void __stack_smash_handler(char* func,unsigned int damaged);
  void __stack_smash_handler(char* func,unsigned int damaged) {
    char buf[sizeof(char*)*2+1];
@@ -21,5 +21,5 @@
    __write2(")\n");
    _exit(127);
  }
-
+#endif

--- dietlibc-0.30/lib/stack_smash_handler2.c.orig       2006-04-04 08:35:14.000000000
+0300
+++ dietlibc-0.30/lib/stack_smash_handler2.c    2008-02-26 13:26:56.000000000 +0200
@@ -1,6 +1,7 @@
  #include <unistd.h>
  #include <write12.h>

+#ifdef WANT_SSP
  void __stack_chk_fail(void);

  /* earlier versions of ProPolice actually gave the address and function
@@ -10,3 +11,4 @@
    __write2("smashed stack detected, program terminated.\n");
    _exit(127);
  }
+#endif
============================================== CUT

-- 
Regards,
Vitaly