Re: [zfs-discuss] How to fsck the spacemap?
Am 19.09.2010 um 18:59 schrieb Victor Latushkin: On Sep 19, 2010, at 12:08 AM, Stephan Ferraro wrote: Is there a way to fsck the spacemap? Does scrub helps for this? No, because issues that you see are internal inconsistencies with unclear nature. Though as actual issue varies from one inctance to another this is likely some random corruption. The error repeated in endless loop because the server rebooted in endless loop and each time when it booted up ZFS it showed this ASSERTION error of the corrupt spacemap every time in the same memory location. -- Best regards, Stephan FERRARO Director Member - Ferraro Ltd. - IT Linux UNIX solutions Rechtliche Informationen: - Name der Gesellschaft: FERRARO Ltd. Gesellschafter und Geschäftsführer: Stephan Ferraro Registergericht:Amtsgericht Stuttgart Registernummer: HRB 724764 Umsatzsteuer-Indentifikationsnummer:DE814799566 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to fsck the spacemap?
Am 19.09.2010 um 19:24 schrieb Victor Latushkin: On Sep 19, 2010, at 9:06 PM, Stephan Ferraro wrote: Am 19.09.2010 um 18:59 schrieb Victor Latushkin: On Sep 19, 2010, at 12:08 AM, Stephan Ferraro wrote: Is there a way to fsck the spacemap? Does scrub helps for this? No, because issues that you see are internal inconsistencies with unclear nature. Though as actual issue varies from one inctance to another this is likely some random corruption. The error repeated in endless loop because the server rebooted in endless loop and each time when it booted up ZFS it showed this ASSERTION error of the corrupt spacemap every time in the same memory location. Once it is there you'll trip various assertions each time affected space map is loaded. By random corruption I mean that it is different each time you encounter it another time after fixing it. I admit that hardware is probably the most cheapest you can get. I rent a server at Hetzner hosting service which provide a rescue system of OpenSolaris snv_111b. -- Best regards, Stephan FERRARO Director Member - Ferraro Ltd. - IT Linux UNIX solutions Rechtliche Informationen: - Name der Gesellschaft: FERRARO Ltd. Gesellschafter und Geschäftsführer: Stephan Ferraro Registergericht:Amtsgericht Stuttgart Registernummer: HRB 724764 Umsatzsteuer-Indentifikationsnummer:DE814799566 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to fsck the spacemap?
On Sep 19, 2010, at 12:11 AM, Stephan Ferraro wrote: This is new for me: $ zpool status pool: rpool state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. see: http://www.sun.com/msg/ZFS-8000-9P scrub: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0ONLINE 0 0 2 c7t0d0s0 ONLINE 0 0 2 c7t1d0s0 ONLINE 0 0 2 errors: No known data errors I never saw that before. It means the CRC error is on the two disks. So did ZFS deleted the bad blocks or what did it? This strongly suggests that you need to look for issue in some common component. It may be hardware like memory, caches, CPU, system busses or storage controllers, as you disks are apparently on the same controller. It may also be some software that corrupts memory contents. To troubleshoot HW issues you may consider running memtest86 or Oracle VTS for example. Setting kmem_flags to e.g. 0xf would be helpful to catch software issue.. regards victor ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to fsck the spacemap?
On Sep 19, 2010, at 12:08 AM, Stephan Ferraro wrote: Is there a way to fsck the spacemap? Does scrub helps for this? No, because issues that you see are internal inconsistencies with unclear nature. Though as actual issue varies from one inctance to another this is likely some random corruption. regards victor ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to fsck the spacemap?
On Sep 19, 2010, at 9:06 PM, Stephan Ferraro wrote: Am 19.09.2010 um 18:59 schrieb Victor Latushkin: On Sep 19, 2010, at 12:08 AM, Stephan Ferraro wrote: Is there a way to fsck the spacemap? Does scrub helps for this? No, because issues that you see are internal inconsistencies with unclear nature. Though as actual issue varies from one inctance to another this is likely some random corruption. The error repeated in endless loop because the server rebooted in endless loop and each time when it booted up ZFS it showed this ASSERTION error of the corrupt spacemap every time in the same memory location. Once it is there you'll trip various assertions each time affected space map is loaded. By random corruption I mean that it is different each time you encounter it another time after fixing it. victor ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to fsck the spacemap?
This is new for me: $ zpool status pool: rpool state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. see: http://www.sun.com/msg/ZFS-8000-9P scrub: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0ONLINE 0 0 2 c7t0d0s0 ONLINE 0 0 2 c7t1d0s0 ONLINE 0 0 2 errors: No known data errors I never saw that before. It means the CRC error is on the two disks. So did ZFS deleted the bad blocks or what did it? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How to fsck the spacemap?
On 09/19/10 08:11 AM, Stephan Ferraro wrote: This is new for me: $ zpool status pool: rpool state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. see: http://www.sun.com/msg/ZFS-8000-9P scrub: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0ONLINE 0 0 2 c7t0d0s0 ONLINE 0 0 2 c7t1d0s0 ONLINE 0 0 2 errors: No known data errors I never saw that before. It means the CRC error is on the two disks. So did ZFS deleted the bad blocks or what did it? ZFS handled the errors, some common to both drives in you system caused it. See the thread from an hour before this, ZFS checksum errors (ZFS-8000-8A) -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
