> -----Original Message----- > From: Chris Siebenmann [mailto:c...@cs.toronto.edu] > Sent: 星期三, 三月 23, 2016 23:33 > To: Richard Jahnel > Cc: Chris Siebenmann; Fred Liu; omnios-discuss@lists.omniti.com > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > > It should be noted that using a 512e disk as a 512n disk subjects you > > to a significant risk of silent corruption in the event of power loss. > > Because 512e disks does a read>modify>write operation to modify > > 512byte chunk of a 4k sector, zfs won't know about the other > > 7 corrupted 512e sectors in the event of a power loss during a write > > operation. So when discards the incomplete txg on reboot, it won't do > > anything about the other 7 512e sectors it doesn't know were affected. > > This is true; under normal circumstances you do not want to use a 512e drive > in an ashift=9 vdev. However, if you have a dead 512n drive and you have no > remaining 512n spares, your choices are to run without redundancy, to wedge > in a 512e drive and accept the potential problems on power failure (problems > that can likely be fixed by scrubbing the pool afterwards), or obtain enough > additional drives (and perhaps > server(s)) to entirely rebuild the pool on 512e drives with ashift=12. > > In this situation, running with a 512e drive and accepting the performance > issues and potential exposure to power failures is basically the lesser evil. > (I > wish ZFS was willing to accept this, but it isn't.) > [Fred Liu]: I have a similar test here:
[root@00-25-90-74-f5-04 ~]# zpool status pool: tank state: ONLINE scan: resilvered 187G in 21h9m with 0 errors on Thu Jan 15 08:05:16 2015 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 c2t45d0 ONLINE 0 0 0 c2t46d0 ONLINE 0 0 0 c2t47d0 ONLINE 0 0 0 c2t48d0 ONLINE 0 0 0 c2t49d0 ONLINE 0 0 0 c2t52d0 ONLINE 0 0 0 c2t53d0 ONLINE 0 0 0 c2t44d0 ONLINE 0 0 0 spares c0t5000CCA6A0C791CBd0 AVAIL errors: No known data errors pool: zones state: ONLINE scan: scrub repaired 0 in 2h45m with 0 errors on Tue Aug 12 20:24:30 2014 config: NAME STATE READ WRITE CKSUM zones ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 c0t5000C500584AC07Bd0 ONLINE 0 0 0 c0t5000C500584AC557d0 ONLINE 0 0 0 c0t5000C500584ACB1Fd0 ONLINE 0 0 0 c0t5000C500584AD7B3d0 ONLINE 0 0 0 c0t5000C500584C30DBd0 ONLINE 0 0 0 c0t5000C500586E54A3d0 ONLINE 0 0 0 c0t5000C500586EF0CBd0 ONLINE 0 0 0 c0t5000C50058426A0Fd0 ONLINE 0 0 0 logs c4t0d0 ONLINE 0 0 0 c4t1d0 ONLINE 0 0 0 cache c0t55CD2E404BE9CB7Ed0 ONLINE 0 0 0 errors: No known data errors [root@00-25-90-74-f5-04 ~]# format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c0t55CD2E404BE9CB7Ed0 <ATA-INTEL SSDSC2BW18-DC32-167.68GB> /scsi_vhci/disk@g55cd2e404be9cb7e 1. c0t5000C500584AC07Bd0 <SEAGATE-ST91000640SS-0004-931.51GB> /scsi_vhci/disk@g5000c500584ac07b 2. c0t5000C500584AC557d0 <SEAGATE-ST91000640SS-0004-931.51GB> /scsi_vhci/disk@g5000c500584ac557 3. c0t5000C500584ACB1Fd0 <SEAGATE-ST91000640SS-0004-931.51GB> /scsi_vhci/disk@g5000c500584acb1f 4. c0t5000C500584AD7B3d0 <SEAGATE-ST91000640SS-0004-931.51GB> /scsi_vhci/disk@g5000c500584ad7b3 5. c0t5000C500584C30DBd0 <SEAGATE-ST91000640SS-0004-931.51GB> /scsi_vhci/disk@g5000c500584c30db 6. c0t5000C500586E54A3d0 <SEAGATE-ST91000640SS-0004-931.51GB> /scsi_vhci/disk@g5000c500586e54a3 7. c0t5000C500586EF0CBd0 <SEAGATE-ST91000640SS-0004-931.51GB> /scsi_vhci/disk@g5000c500586ef0cb 8. c0t5000C50058426A0Fd0 <SEAGATE-ST91000640SS-0004-931.51GB> /scsi_vhci/disk@g5000c50058426a0f 9. c0t5000CCA6A0C791CBd0 <ATA-Hitachi HTS54101-A480-931.51GB> /scsi_vhci/disk@g5000cca6a0c791cb 10. c0t50000F0056425331d0 <ATA-SAMSUNG MMCRE28G-AS1Q-119.24GB> /scsi_vhci/disk@g50000f0056425331 11. c2t44d0 <ATA-Hitachi HTS54101-A480-931.51GB> /pci@0,0/pci8086,1c10@1c/pci1000,3140@0/sd@2c,0 12. c2t45d0 <ATA-Hitachi HTS54101-A480-931.51GB> /pci@0,0/pci8086,1c10@1c/pci1000,3140@0/sd@2d,0 13. c2t46d0 <ATA-ST1000LM024 HN-M-0002-931.51GB> /pci@0,0/pci8086,1c10@1c/pci1000,3140@0/sd@2e,0 14. c2t47d0 <ATA-ST1000LM024 HN-M-0002-931.51GB> /pci@0,0/pci8086,1c10@1c/pci1000,3140@0/sd@2f,0 15. c2t48d0 <ATA-WDC WD10JPVT-08A-1A01-931.51GB> /pci@0,0/pci8086,1c10@1c/pci1000,3140@0/sd@30,0 16. c2t49d0 <ATA-WDC WD10JPVT-75A-1A01-931.51GB> /pci@0,0/pci8086,1c10@1c/pci1000,3140@0/sd@31,0 17. c2t52d0 <ATA-ST1000LM024 HN-M-0001-931.51GB> /pci@0,0/pci8086,1c10@1c/pci1000,3140@0/sd@34,0 18. c2t53d0 <ATA-ST1000LM024 HN-M-0001-931.51GB> /pci@0,0/pci8086,1c10@1c/pci1000,3140@0/sd@35,0 19. c4t0d0 <ATA-ANS9010_2NNN2NNN-_200-1.78GB> /pci@0,0/pci15d9,624@1f,2/disk@0,0 20. c4t1d0 <ATA-ANS9010_2NNN2NNN-_200-1.78GB> /pci@0,0/pci15d9,624@1f,2/disk@1,0 [root@00-25-90-74-f5-04 ~]# zpool replace tank c2t44d0 c0t5000CCA6A0C791CBd0 cannot replace c2t44d0 with c0t5000CCA6A0C791CBd0: devices have different sector alignment But in fact "c2t44d0" and "c0t5000CCA6A0C791CBd0" are in the same model -- ATA-Hitachi HTS54101-A480-931.51GB. That is HTS541010A9E680 (https://www.hgst.com/sites/default/files/resources/TS5K1000_ds.pdf) which is a 512e HDD. The *only* difference is that "c2t44d0" is attached to a LSI 1068 HBA and "c0t5000CCA6A0C791CBd0" is attached to a LSI 2308 HBA. [root@00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c2t44d0s0 | grep ashift ashift: 9 ashift: 9 ashift: 9 ashift: 9 [root@00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c0t5000CCA6A0C791CBd0s0 | grep ashift ashift: 12 ashift: 12 ashift: 12 ashift: 12 format> inq Vendor: ATA Product: Hitachi HTS54101 Revision: A480 format> q adding " "ATA Hitachi HTS54101", "physical-block-size:512"," into sd.conf [root@00-25-90-74-f5-04 ~]# update_drv -vf sd Cannot unload module: sd Will be unloaded upon reboot. Forcing update of sd.conf. sd.conf updated in the kernel. Reboot the server for "cfgadm -c unconfigure" can't work here. [root@00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c0t5000CCA6A0C791CBd0s0 | grep ashift [root@00-25-90-74-f5-04 ~]# No ashift in output now. [root@00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c2t44d0s0 | grep ashift ashift: 9 ashift: 9 ashift: 9 ashift: 9 same like before [root@00-25-90-74-f5-04 ~]# zpool replace tank c2t44d0 c0t5000CCA6A0C791CBd0 cannot replace c2t44d0 with c0t5000CCA6A0C791CBd0: devices have different sector alignment Remove the spare: [root@00-25-90-74-f5-04 ~]# zpool remove tank c0t5000CCA6A0C791CBd0 [root@00-25-90-74-f5-04 ~]# Add it back: [root@00-25-90-74-f5-04 ~]# zpool add tank spare c0t5000CCA6A0C791CBd0 [root@00-25-90-74-f5-04 ~]# [root@00-25-90-74-f5-04 ~]# zpool status tank pool: tank state: ONLINE scan: resilvered 187G in 21h9m with 0 errors on Thu Jan 15 08:05:16 2015 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 c2t45d0 ONLINE 0 0 0 c2t46d0 ONLINE 0 0 0 c2t47d0 ONLINE 0 0 0 c2t48d0 ONLINE 0 0 0 c2t49d0 ONLINE 0 0 0 c2t52d0 ONLINE 0 0 0 c2t53d0 ONLINE 0 0 0 c2t44d0 ONLINE 0 0 0 spares c0t5000CCA6A0C791CBd0 AVAIL errors: No known data errors Still not working; [root@00-25-90-74-f5-04 ~]# zpool replace tank c2t44d0 c0t5000CCA6A0C791CBd0 cannot replace c2t44d0 with c0t5000CCA6A0C791CBd0: devices have different sector alignment maybe the sd.conf update is not correct. _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss