Subject: Re: [vserver] For a period of 2 months, the server file system has switched three times into the readonly mode
From: Laurent Spagnol <laurent.spagnol@univ-reims.fr>
Date: Thu, 29 Oct 2009 12:36:25 +0100

Before changing your hard-drive ... you can check it with 
"smartmontools" (for non-raid or supported raid devices).

http://smartmontools.sourceforge.net/badblockhowto.html



KLEIN Stéphane a écrit :
> Hi,
>
> I've a vserver installed on Debian Lenny.
>
> For a period of 2 months, the server file system has switched three
> times into the readonly mode.
>
> What do you think ?
> * is it a vserver issue ?
> * is it a hardware issue ?
> * is it a debian issue ?
>
> Last switch time, I've verify my disk with smartctl and fsck… my disk
> seem to be clean.
>
> This is some information about the last switch :
>
> # uptime
> 10:29:53 up 15 days, 37 min,  0 users,  load average: 0.00, 0.00, 0.00
>
> # dmesg | grep "Shorewall" -v
> TTL=120 ID=21306 DF PROTO=TCP SPT=64969 DPT=8443 WINDOW=8192 RES=0x00 SYN URGP=0
> [1366416.782108] EXT3-fs error (device md1): ext3_free_blocks: Freeing
> blocks in system zones - Block = 131072, count = 1
> [1366416.782161] Aborting journal on device md1.
> [1366416.782386] Remounting filesystem read-only
> [1366416.782748] EXT3-fs error (device md1) in
> ext3_reserve_inode_write: Journal has aborted
> [1366416.782792] EXT3-fs error (device md1) in ext3_truncate: Journal
> has aborted
> [1366416.782833] EXT3-fs error (device md1) in
> ext3_reserve_inode_write: Journal has aborted
> [1366416.782875] EXT3-fs error (device md1) in ext3_orphan_del:
> Journal has aborted
> [1366416.782915] EXT3-fs error (device md1) in
> ext3_reserve_inode_write: Journal has aborted
>
> # mdadm --detail /dev/md1
> /dev/md1:
>        Version : 00.90
>  Creation Time : Wed Aug 19 15:56:46 2009
>     Raid Level : raid1
>     Array Size : 487749376 (465.15 GiB 499.46 GB)
>  Used Dev Size : 487749376 (465.15 GiB 499.46 GB)
>   Raid Devices : 2
>  Total Devices : 2
> Preferred Minor : 1
>    Persistence : Superblock is persistent
>
>    Update Time : Thu Oct 29 10:07:11 2009
>          State : clean
> Active Devices : 2
> Working Devices : 2
> Failed Devices : 0
>  Spare Devices : 0
>
>           UUID : 191320ed:72bbf45b:bfe78010:bc810f04
>         Events : 0.6
>
>    Number   Major   Minor   RaidDevice State
>       0       8        2        0      active sync   /dev/sda2
>       1       8       18        1      active sync   /dev/sdb2
>
> # lspci
> 00:00.0 Host bridge: Intel Corporation Mobile 945GM/PM/GMS, 943/940GML
> and 945GT Express Memory Controller Hub (rev 03)
> 00:02.0 VGA compatible controller: Intel Corporation Mobile 945GM/GMS,
> 943/940GML Express Integrated Graphics Controller (rev 03)
> 00:02.1 Display controller: Intel Corporation Mobile 945GM/GMS/GME,
> 943/940GML Express Integrated Graphics Controller (rev 03)
> 00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express
> Port 1 (rev 01)
> 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev e1)
> 00:1f.0 ISA bridge: Intel Corporation 82801GB/GR (ICH7 Family) LPC
> Interface Bridge (rev 01)
> 00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE
> Controller (rev 01)
> 00:1f.2 SATA controller: Intel Corporation 82801GR/GH (ICH7 Family)
> SATA AHCI Controller (rev 01)
> 00:1f.3 SMBus: Intel Corporation 82801G (ICH7 Family) SMBus Controller (rev 01)
> 01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
> RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 01)
>
> # cat /proc/cpuinfo
> processor : 0
> vendor_id : GenuineIntel
> cpu family : 6
> model : 15
> model name : Intel(R) Core(TM)2 CPU         T7200  @ 2.00GHz
> stepping : 6
> cpu MHz : 2000.186
> cache size : 4096 KB
> physical id : 0
> siblings : 2
> core id : 0
> cpu cores : 2
> apicid : 0
> initial apicid : 0
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm
> constant_tsc arch_perfmon pebs bts pni monitor ds_cpl vmx est tm2
> ssse3 cx16 xtpr lahf_lm
> bogomips : 4004.73
> clflush size : 64
> power management:
>
> processor : 1
> vendor_id : GenuineIntel
> cpu family : 6
> model : 15
> model name : Intel(R) Core(TM)2 CPU         T7200  @ 2.00GHz
> stepping : 6
> cpu MHz : 2000.186
> cache size : 4096 KB
> physical id : 0
> siblings : 2
> core id : 1
> cpu cores : 2
> apicid : 1
> initial apicid : 1
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm
> constant_tsc arch_perfmon pebs bts pni monitor ds_cpl vmx est tm2
> ssse3 cx16 xtpr lahf_lm
> bogomips : 4000.55
> clflush size : 64
> power management:
>
> # dpkg -l | grep "vserver"
> ii  linux-image-2.6.26-1-vserver-686   2.6.26-13lenny2          Linux
> 2.6.26 image on PPro/Celeron/PII/PIII/P4, Linux-VServer
> ii  util-vserver                       0.30.216~r2772-6
> user-space tools for Linux-VServer virtual private servers
> ii  vserver-debiantools                0.6.3                    Tools
> to manage debian virtual servers
>
> # vserver-info
> Versions:
>                   Kernel: 2.6.26-1-vserver-686
>                   VS-API: 0x00020303
>             util-vserver: 0.30.216-pre2772; Dec 12 2008, 23:24:33
>
> Features:
>       CC: gcc, gcc (Debian 4.3.2-1) 4.3.2
>                      CXX: g++, g++ (Debian 4.3.2-1) 4.3.2
>                 CPPFLAGS: ''
>                   CFLAGS: '-Wall -g  -O2 -std=c99 -Wall -pedantic -W
> -funit-at-a-time'
>                 CXXFLAGS: '-g -O2 -ansi -Wall -pedantic -W
> -fmessage-length=0 -funit-at-a-time'
>               build/host: i486-pc-linux-gnu/i486-pc-linux-gnu
>             Use dietlibc: yes
>       Build C++ programs: yes
>       Build C99 programs: yes
>           Available APIs: v13,net,v21,v22,v23,netv2
>            ext2fs Source: e2fsprogs
>    syscall(2) invocation: alternative
>      vserver(2) syscall#: 273/glibc
>               crypto api: beecrypt
>   use library versioning: yes
>
> Paths:
>                   prefix: /usr
>        sysconf-Directory: /etc
>            cfg-Directory: /etc/vservers
>         initrd-Directory: $(sysconfdir)/init.d
>       pkgstate-Directory: /var/run/vservers
>          vserver-Rootdir: /var/lib/vservers
>
>
> Assumed 'SYSINFO' as no other option given; try '--help' for more information.
>
> #fsck.ext3 -v /dev/md1
> e2fsck 1.40.8 (13-Mar-2008)
> / was not cleanly unmounted, check forced.
> Pass 1: Checking inodes, blocks, and sizes
> Deleted inode 122881 has zero dtime.  Fix<y>? yes
>
> Deleted inode 3194881 has zero dtime.  Fix<y>? yes
>
> Deleted inode 10838017 has zero dtime.  Fix<y>?
>
> Inode 10838018 is in use, but has dtime set.  Fix<y>? yes
>
> Inode 10838018, i_blocks is 11013, should be 0.  Fix<y>? yes
>
> Deleted inode 10838081 has zero dtime.  Fix<y>? yes
>
> Deleted inode 10838113 has zero dtime.  Fix<y>? yes
>
> Inode 10838114 is in use, but has dtime set.  Fix<y>? yes
>
> Inode 10838114, i_blocks is 3707185669, should be 0.  Fix<y>? yes
>
> Deleted inode 10838177 has zero dtime.  Fix<y>? yes
>
> Deleted inode 10838209 has zero dtime.  Fix<y>? yes
>
> Deleted inode 10838433 has zero dtime.  Fix<y>? yes
>
> Deleted inode 10838465 has zero dtime.  Fix<y>? yes
>
> [snap]
>
> Do you need other information ?
>
> Thanks for your help.
> Stephane
>
>
>   


-- 
Laurent Spagnol
Administrateur Linux

Université de Reims
Centre de Ressources Informatiques

Campus du Moulin de la Housse
BP 1039 - 51687 Reims cedex 2

Tel: 03.26.91.88.32
Fax: 03.26.91.31.87