[zfs-discuss] Convert from rz2 to rz1
Is it possible to convert a rz2 array to rz1 array? I have a pool with to rz2 arrays. I would like to convert them to rz1. Would that be possible? If not, is it ok to remove one disk from a rz2 array and just let the array keep running with one disk missing? Regards, Lars-Gunnar Persson System Administrator Nansen Environmental and Remote Sensing Center Address: Thormøhlensgate 47, 5006 Bergen Direct: +47 55 20 58 31, switchboard: +47 55 20 58 00, fax: +47 55 20 58 01 Internet: http://www.nersc.no, e-mail: lars-gunnar.pers...@nersc.no ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Nexsan SATABeast and ZFS
I would like to go back to my question for a second: I checked with my Nexsan supplier and they confirmed that access to every single disk in SATABeast is not possible. The smallest entities I can create on the SATABeast are RAID 0 or 1 arrays. With RAID 1 I'll loose too much disk space and I believe that leaves me with RAID 0 as the only reasonable option. But with this unsecure RAID format I'll need higher redundancy in the ZFS configuration. I think I'll go with the following configuration: On the Nexsan SATABeast: * 14 disks configured in 7 RAID arrays with RAID level 0 (each disk is 1 TB which gives me a total of 14 TB raw disk space). * Each RAID 0 array configured as one volume. On the Sun Fire X4100 M2 with Solaris 10: * Add all 7 volumes to one zpool configured in on raidz2 (gives me approx. 8,8 TB available disk space) Any comments or suggestions? Best regards, Lars-Gunnar Persson On 11. mars. 2009, at 02.39, Bob Friesenhahn wrote: On Tue, 10 Mar 2009, A Darren Dunham wrote: What part isn't true? ZFS has a independent checksum for the data block. But if the data block is spread over multiple disks, then each of the disks have to be read to verify the checksum. I interpreted what you said to imply that RAID6 type algorithms were being used to validate the data, rather than to correct wrong data. I agree that it is necessary to read a full ZFS block in order to use the ZFS block checksum. I also agree that a raidz2 vdev has IOPS behavior which is similar to a single disk. From what I understand, a raidz2 with a very large number of disks won't use all of the disks to store one ZFS block. There is a maximum number of disks in a stripe which can be supported by the ZFS block size. -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Nexsan SATABeast and ZFS
How about this configuration? On the Nexsan SATABeast, add all disk to one RAID 5 or 6 group. Then on the Nexsan define several smaller volumes and then add those volumes to a raidz2/raidz zpool? Could that be an useful configuration? Maybe I'll loose too much space with double raid 5 or 6 configuration? What about performance? Regards, Lars-Gunnar Persson On 10. mars. 2009, at 00.26, Kees Nuyt wrote: On Mon, 9 Mar 2009 12:06:40 +0100, Lars-Gunnar Persson lars-gunnar.pers...@nersc.no wrote: 1. On the external disk array, I not able to configure JBOD or RAID 0 or 1 with just one disk. In some arrays it seems to be possible to configure separate disks by offering the array just one disk in one slot at a time, and, very important, leaving all other slots empty(!). Repeat for as many disks as you have, seating each disk in its own slot, and all other slots empty. (ok, it's just hear-say, but it might be worth a try with the first 4 disks or so). -- ( Kees Nuyt ) c[_] ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Nexsan SATABeast and ZFS
I realized that I'll loose too much disk space with the double raid configuration suggested below. Agree? I've done some performance testing with raidz/raidz1 vs raidz2: bash-3.00# zpool status -v raid5 pool: raid5 state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM raid5 ONLINE 0 0 0 raidz1 ONLINE 0 0 0 c7t6000402001FC442C609DC5A3d0 ONLINE 0 0 0 c7t6000402001FC442C609DCA4Ad0 ONLINE 0 0 0 c7t6000402001FC442C609DCA22d0 ONLINE 0 0 0 c7t6000402001FC442C609DCABFd0 ONLINE 0 0 0 c7t6000402001FC442C609DCADBd0 ONLINE 0 0 0 c7t6000402001FC442C609DCAF8d0 ONLINE 0 0 0 c7t6000402001FC442C609F0291d0 ONLINE 0 0 0 errors: No known data errors bash-3.00# zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT raid5 12.6T141K 12.6T 0% ONLINE - bash-3.00# df -h /raid5 Filesystem size used avail capacity Mounted on raid5 11T41K11T 1%/raid5 bash-3.00# echo zfs_nocacheflush/D | mdb -k zfs_nocacheflush: zfs_nocacheflush: 0 bash-3.00# ./filesync-1 /raid5 1 Time in seconds to create and unlink 1 files with O_DSYNC: 9.871197 bash-3.00# echo zfs_nocacheflush/W1 | mdb -kw zfs_nocacheflush: 0 = 0x1 bash-3.00# ./filesync-1 /raid5 1 Time in seconds to create and unlink 1 files with O_DSYNC: 7.363303 Then I destroyed the raid5 pool and created a raid6 pool: bash-3.00# zpool status -v raid6 pool: raid6 state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM raid6 ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c7t6000402001FC442C609DC5A3d0 ONLINE 0 0 0 c7t6000402001FC442C609DCA4Ad0 ONLINE 0 0 0 c7t6000402001FC442C609DCA22d0 ONLINE 0 0 0 c7t6000402001FC442C609DCABFd0 ONLINE 0 0 0 c7t6000402001FC442C609DCADBd0 ONLINE 0 0 0 c7t6000402001FC442C609DCAF8d0 ONLINE 0 0 0 c7t6000402001FC442C609F0291d0 ONLINE 0 0 0 errors: No known data errors bash-3.00# zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT raid6 12.6T195K 12.6T 0% ONLINE - bash-3.00# df -h /raid6 Filesystem size used avail capacity Mounted on raid6 8.8T52K 8.8T 1%/raid6 bash-3.00# echo zfs_nocacheflush/D | mdb -k zfs_nocacheflush: zfs_nocacheflush: 0 bash-3.00# ./filesync-1 /raid6 1 Time in seconds to create and unlink 1 files with O_DSYNC: 9.879219 bash-3.00# echo zfs_nocacheflush/W1 | mdb -kw zfs_nocacheflush: 0 = 0x1 bash-3.00# ./filesync-1 /raid6 1 Time in seconds to create and unlink 1 files with O_DSYNC: 7.560435 My conclusion on raidz1 vs raidz2 would be no difference in performance and big difference in disk space available. On 10. mars. 2009, at 09.13, Lars-Gunnar Persson wrote: How about this configuration? On the Nexsan SATABeast, add all disk to one RAID 5 or 6 group. Then on the Nexsan define several smaller volumes and then add those volumes to a raidz2/raidz zpool? Could that be an useful configuration? Maybe I'll loose too much space with double raid 5 or 6 configuration? What about performance? Regards, Lars-Gunnar Persson On 10. mars. 2009, at 00.26, Kees Nuyt wrote: On Mon, 9 Mar 2009 12:06:40 +0100, Lars-Gunnar Persson lars-gunnar.pers...@nersc.no wrote: 1. On the external disk array, I not able to configure JBOD or RAID 0 or 1 with just one disk. In some arrays it seems to be possible to configure separate disks by offering the array just one disk in one slot at a time, and, very important, leaving all other slots empty(!). Repeat for as many disks as you have, seating each disk in its own slot, and all other slots empty. (ok, it's just hear-say, but it might be worth a try with the first 4 disks or so). -- ( Kees Nuyt ) c[_] ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http
Re: [zfs-discuss] Nexsan SATABeast and ZFS
Test 1: bash-3.00# echo zfs_nocacheflush/D | mdb -k zfs_nocacheflush: zfs_nocacheflush: 0 bash-3.00# ./filesync-1 /raid6 1 Time in seconds to create and unlink 1 files with O_DSYNC: 292.223081 bash-3.00# echo zfs_nocacheflush/W1 | mdb -kw zfs_nocacheflush: 0 = 0x1 bash-3.00# ./filesync-1 /raid6 1 Time in seconds to create and unlink 1 files with O_DSYNC: 288.099066 Test 2: ash-3.00# echo zfs_nocacheflush/D | mdb -k zfs_nocacheflush: zfs_nocacheflush: 0 bash-3.00# ./filesync-1 /raid6 1 Time in seconds to create and unlink 1 files with O_DSYNC: 13.092332 bash-3.00# echo zfs_nocacheflush/W1 | mdb -kw zfs_nocacheflush: 0 = 0x1 bash-3.00# ./filesync-1 /raid6 1 Time in seconds to create and unlink 1 files with O_DSYNC: 9.591622 Test 3: bash-3.00# echo zfs_nocacheflush/D | mdb -k zfs_nocacheflush: zfs_nocacheflush: 0 bash-3.00# ./filesync-1 /raid6 1 Time in seconds to create and unlink 1 files with O_DSYNC: 9.879219 bash-3.00# echo zfs_nocacheflush/W1 | mdb -kw zfs_nocacheflush: 0 = 0x1 bash-3.00# ./filesync-1 /raid6 1 Time in seconds to create and unlink 1 files with O_DSYNC: 7.560435 Thank you for your reply. If I make a raidz or a raidz2 on the Solaris box, then I get enough redundancy? The Nexsan can't do block-level snapshots. On 9. mars. 2009, at 18.27, Miles Nordin wrote: lp == Lars-Gunnar Persson lars-gunnar.pers...@nersc.no writes: lp Ignore force unit access (FUA) bit: [X] lp Any thoughts about this? run three tests (1) write cache disabled (2) write cache enabled, ignore FUA off (3) write cache enabled, ignore FUA [X] if all three are the same, either the test is broken or something is wrong. If (2) and (3) are the same, then ZFS is working as expected. run with either (2) or (3). If (1) and (2) are the same, then ZFS doesn't seem to have the cache-flush changes implemented, or else they aren't sufficient for your array. You could look into it more, file a bug, something like that. If you're not interested in investigating more and filing a bug in the last case, then you could just set (3), and do no testing at all. It might be good to do some of the testing just to catch wtf cases like, ``oh sorry, we didn't sell you a cache.'' or ``the battery is dead? how'd that happen?'' but there are so many places for wtf cases maybe this isn't the one to worry about. but I'm not sure why you are switching to one big single-disk vdev with FC if you are having ZFS corruption problems. I think everyone's been saying you are more likely to have problems by using FC instead of direct attached, and also by using FC without vdev redundancy (the redundancy seems to work around some bugs). At least the people reporting lots of lost pools are the ones using FC and iSCSI, who lose pools during target reboots or SAN outages. I suppose if the NexSAN can do block-level snapshots, you could snapshot exported copies of your pool from time to time, and roll back to a snapshot if ZFS refuses to import your ``corrupt'' pool. In that way it could help you? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Nexsan SATABeast and ZFS
I'm trying to implement a Nexsan SATABeast (an external disk array, read more: http://www.nexsan.com/satabeast.php, 14 disks available) with a Sun Fire X4100 M2 server running Solaris 10 u6 (connected via fiber) and have a couple of questions: (My motivation for this is the corrupted ZFS volume discussion I had earlier with no result, and this time I'm trying to make a more robust implementation) 1. On the external disk array, I not able to configure JBOD or RAID 0 or 1 with just one disk. I can't find any options for my Solaris server to access the disk directly so I have to configure some raids on the SATABeast. I was thinking of striping two disks in each raid and then add all 7 raids to one zpool as a zraid. The problem with this is if one disk breaks down, I'll loose one RAID 0 disk but maybe ZFS can handle this? Should I rather implement RAID5 disks one the SATABeast and then export them to the Solaris machine? 14 disks would give me 4 RAID5 volumes and 2 spare disks? I'll loose a lot of disk space. What about create larger RAID volumes on the SATABeast? Like 3 RAID volumes with 5 disks in 2 RAIDS and 4 disks in one RAID? I'm really not sure what to choose ... At the moment I've striped two disks in one RAID volume. 2. After reading from the ZFS Evil Tuning Guide (http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#Cache_Flushes ) about cache flushes I checked the cache configuration on the SATABaest and I can change these settings: System Admin Configure Cache Cache Configuration Current write cache state: Enabled, FUA ignored - 495 MB Manually override current write cache status: [ ] Force write cache to Disabled Desired write cache state: [X] Enabled [ ] Disabled Allow attached host to override write cache configuration: [ ] Ignore force unit access (FUA) bit: [X] Write cache streaming mode: [ ] Cache optimization setting: [ ] Random access [X] Mixed sequential/random [ ] Sequential access And from the help section: Write cache will normally speed up host writes, data is buffered in the RAID controllers memory when the installed disk drives are not ready to accept the write data. The RAID controller write cache memory is battery backed, this allows any unwritten array data to be kept intact during a power failure situation. When power is restored this battery backed data will be flushed out to the RAID array. Current write cache state - This is the current state of the write cache that the RAID system is using. Manually override current write cache status - This allows the write caching to be forced on or off by the user, this change will take effect immediately. Desired write cache state - This is the state of the write cache the user wishes to have after boot up. Allow attached host to override write cache configuration - This allows the host system software to issue commands to the RAID system via the host interface that will either turn off or on the write caching. Ignore force unit access (FUA) bit - When the force unit access (FUA) bit is set by a host system on a per command basis data is written / read directly to / from the disks without using the onboard cache. This will incur a time overhead, but guarantees the data is on the media. Set this option to force the controller to ignore the FUA bit such that command execution times are more consistent. Write cache streaming mode - When the write cache is configured in streaming mode (check box ticked), the system continuously flushes the cache (it runs empty). This provides maximum cache buffering to protect against raid system delays adversely affecting command response times to the host. When the write cache operates in non-streaming mode (check box not ticked) the system runs with a full write cache to maximise cache hits and maximise random IO performance. Cache optimization setting - The cache optimization setting adjusts the cache behaviour to maximize performance for the expected host I/ O pattern. Note that the write cache will be flushed 5 seconds after the last host write. It is recommended that all host activity is stopped 30 seconds before powering the system off. Any thoughts about this? Regards, Lars-Gunnar Persson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] zpool vs df
I've a interesting situation. I've created two pool now and one pool named Data and another named raid5. Check the details here: bash-3.00# zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT Data 10.7T 9.82T892G91% ONLINE - raid5 10.9T145K 10.9T 0% ONLINE - As you see, the sizes are approximately the same. If I run the df command, it reports: bash-3.00# df -h /Data Filesystem size used avail capacity Mounted on Data11T 108M 154G 1%/Data bash-3.00# df -h /raid5 Filesystem size used avail capacity Mounted on raid5 8.9T40K 8.9T 1%/raid5 You see that the Data has 11 TB when zpool reported 10.7 TB and the raid5 has 10.9TB in zpool but only 8.9 TB when using df. Thats a difference of 2 TB. Where did they go? Any explanation would be find. Regards, Lars-Gunnar Persson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zpool vs df
Here is what zpool status reports:
bash-3.00# zpool status
pool: Data
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
Data ONLINE 0 0 0
c4t5000402001FC442Cd0 ONLINE 0 0 0
errors: No known data errors
pool: raid5
state: ONLINE
scrub: none requested
config:
NAME STATE READ
WRITE CKSUM
raid5 ONLINE 0
0 0
raidz1 ONLINE 0
0 0
c7t6000402001FC442C609DCA22d0 ONLINE 0
0 0
c7t6000402001FC442C609DCA4Ad0 ONLINE 0
0 0
c7t6000402001FC442C609DCAA2d0 ONLINE 0
0 0
c7t6000402001FC442C609DCABFd0 ONLINE 0
0 0
c7t6000402001FC442C609DCADBd0 ONLINE 0
0 0
c7t6000402001FC442C609DCAF8d0 ONLINE 0
0 0
errors: No known data errors
On 9. mars. 2009, at 14.29, Tomas Ögren wrote:
On 09 March, 2009 - Lars-Gunnar Persson sent me these 1,1K bytes:
I've a interesting situation. I've created two pool now and one pool
named Data and another named raid5. Check the details here:
bash-3.00# zpool list
NAMESIZEUSED AVAILCAP HEALTH
ALTROOT
Data 10.7T 9.82T892G91% ONLINE -
raid5 10.9T145K 10.9T 0% ONLINE -
As you see, the sizes are approximately the same. If I run the df
command, it reports:
bash-3.00# df -h /Data
Filesystem size used avail capacity Mounted on
Data11T 108M 154G 1%/Data
bash-3.00# df -h /raid5
Filesystem size used avail capacity Mounted on
raid5 8.9T40K 8.9T 1%/raid5
You see that the Data has 11 TB when zpool reported 10.7 TB and the
raid5 has 10.9TB in zpool but only 8.9 TB when using df. Thats a
difference of 2 TB. Where did they go?
To your raid5 (raidz) parity.
Check 'zpool status' to see how your two pools differ.. zpool list
shows
the disk space you have.. zfs/df shows how much you can store there..
/Tomas
--
Tomas Ögren, st...@acc.umu.se, http://www.acc.umu.se/~stric/
|- Student at Computing Science, University of Umeå
`- Sysadmin at {cs,acc}.umu.se
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
.--.
|Lars-Gunnar
Persson |
|IT-
sjef |
|
|
|Nansen senteret for miljø og
fjernmåling |
|Adresse : Thormøhlensgate 47, 5006
Bergen|
|Direkte : 55 20 58 31, sentralbord: 55 20 58 00, fax: 55 20 58
01|
|Internett: http://www.nersc.no, e-post: lars-
gunnar.pers...@nersc.no |
'--'
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS volume corrupted?
I thought a ZFS file system wouldn't destroy a ZFS volume? Hmm, I'm not sure what to do now ... First of all, this zfs volume Data/subversion1 has been working for a year and suddenly after a reboot of the Solaris server, running of the zpool export and zpool import command, I get problems with this ZFS volume? Today I checked some more, after reading this guide: http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide My main question is: Is my ZFS volume which is part of a zpool lost or can I recover it? If I upgrade the Solaris server to the latest and do a zpool export and zpool import help? All advices appreciated :-) Here is some more information: -bash-3.00$ zfs list -o name,type,used,avail,ratio,compression,reserv,volsize Data/subversion1 NAMETYPE USED AVAIL RATIO COMPRESS RESERV VOLSIZE Data/subversion1 volume 22.5K 511G 1.00x off250G 250G I've also learned the the AVAIL column reports what's available in the zpool and NOT what's available in the ZFS volume. -bash-3.00$ sudo zpool status -v Password: pool: Data state: ONLINE scrub: scrub in progress, 5.86% done, 12h46m to go config: NAME STATE READ WRITE CKSUM Data ONLINE 0 0 0 c4t5000402001FC442Cd0 ONLINE 0 0 0 errors: No known data errors Interesting thing here is that the scrub process should be finished today but the progress is much slower than reported here. And will the scrub process help anything in my case? -bash-3.00$ sudo fmdump TIME UUID SUNW-MSG-ID Nov 15 2007 10:16:38 8aa789d2-7f3a-45d5-9f5c-c101d73b795e ZFS-8000-CS Oct 14 09:31:40.8179 8c7d9847-94b7-ec09-8da7-c352de405b78 FMD-8000-2K bash-3.00$ sudo fmdump -ev TIME CLASS ENA Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6850ff400401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6850ff400401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6897db600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e688d11500401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68926e600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6897db600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68a3d3900401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68bc6741 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68d8bb600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68da5b51 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68da5b51 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68f0c9800401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68da5b51 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6897db600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68e98191 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68f0c9800401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e690a11000401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68f0c9800401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e69038551 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e69038551 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e690a11000401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e692a4ca1 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68bc6741 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e690a11000401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6850ff400401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6850ff400401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68bc6741 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e69038551 Nov 15 2007 09:33:52 ereport.fs.zfs.data 0x915e6850ff400401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68a3d3900401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68a3d3900401 Nov 15 2007 10:16:12 ereport.fs.zfs.vdev.open_failed 0x0533bb1b56400401 Nov 15 2007 10:16:12 ereport.fs.zfs.zpool 0x0533bb1b56400401 Oct 14 09:31:31.6092 ereport.fm.fmd.log_append 0x02eb96a8b6502801 Oct 14 09:31:31.8643 ereport.fm.fmd.mod_init 0x02ec89eadd100401 On 3. mars. 2009, at 08.10, Lars-Gunnar Persson wrote: I've turned off iSCSI sharing at the moment. My first question is: how can zfs report available is larger than reservation on a zfs volume? I also know that used mshould be larger than 22.5 K. Isn't
Re: [zfs-discuss] ZFS volume corrupted?
Thank you for your long reply. I don't believe that will help me get my ZFS volume back though, From my last reply to this list I confirm that I do understand what the AVAIL column is reporting when running the zfs list command. hmm, still confused ... Regards, Lars-Gunnar Persson On 3. mars. 2009, at 11.26, O'Shea, Damien wrote: Hi, The reason zfs is saying that the available is larger is because in Zfs the size of the pool is always available to the all the zfs filesystems that reside in the pool. Setting a reservation will gaurntee that the reservation size is reserved for the filesystem/volume but you can change that on the fly. You can see that if you create another filsystem within the pool that the reservation in use by your volume will have be deducted from the available size. Like below: r...@testfs create -V 10g testpool/test r...@testfs get all testpool NAME PROPERTY VALUE SOURCE testpool type filesystem - testpool creation Wed Feb 11 13:17 2009 - testpool used 10.1G - testpool available124G - testpool referenced 100M - testpool compressratio1.00x - testpool mounted yes- Here the available is 124g as the volume has been set to 10g from a pool of 134g. If we set a reservation like this r...@test1 set reservation=10g testpool/test r...@test1 zfs get all testpool/test NAME PROPERTY VALUE SOURCE testpool/test type volume - testpool/test creation Tue Mar 3 10:13 2009 - testpool/test used 10G- testpool/test available134G - testpool/test referenced 16K- testpool/test compressratio1.00x - We can see that the available is now 134g, which is the avilable size of the rest of the pool + the 10g reservation that we have set. So in theory this volume can grow to the complete size of the pool. So if we have a look at the availble space now in the pool we see r...@test1# zfs get all testpool NAME PROPERTY VALUE SOURCE testpool type filesystem - testpool creation Wed Feb 11 13:17 2009 - testpool used 10.1G - testpool available124G - testpool referenced 100M - testpool compressratio1.00x - testpool mounted yes- 124g with 10g used to account for the size of the volume ! So if we now create another filesystem like this r...@test1# zfs create testpool/test3 r...@test1# zfs get all testpool/test3 NAMEPROPERTY VALUE SOURCE testpool/test3 type filesystem - testpool/test3 creation Tue Mar 3 10:19 2009 - testpool/test3 used 18K- testpool/test3 available124G - testpool/test3 referenced 18K- testpool/test3 compressratio1.00x - testpool/test3 mounted yes- We see that the total amount available to the filesystem is the amount of the space in the pool minus the 10g reservation. Lets set the reservation to something bigger. r...@test1# zfs set volsize=100g testpool/test r...@test1# zfs set reservation=100g testpool/test r...@test1# zfs get all testpool/test NAME PROPERTY VALUE SOURCE testpool/test type volume - testpool/test creation Tue Mar 3 10:13 2009 - testpool/test used 100G - testpool/test available134G - testpool/test referenced 16K- So the available is still 134G, which is the rest of the pool + the reservation set. r...@test1# zfs get all testpool NAME PROPERTY VALUE SOURCE testpool type filesystem - testpool creation Wed Feb 11 13:17 2009 - testpool used 100G - testpool available33.8G - testpool referenced 100M - testpool compressratio1.00x - testpool mounted yes- The pool however now only has 33.8G left, which should be the same for all the other filesystems in the pool. Hope that helps. -Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org]on Behalf Of Lars-Gunnar Persson Sent: 03 March 2009 07:11 To: Richard Elling Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] ZFS volume corrupted? * This e-mail has been
Re: [zfs-discuss] ZFS volume corrupted?
I run a new command now zdb. Here is the current output: -bash-3.00$ sudo zdb Data version=4 name='Data' state=0 txg=9806565 pool_guid=6808539022472427249 vdev_tree type='root' id=0 guid=6808539022472427249 children[0] type='disk' id=0 guid=2167768931511572294 path='/dev/dsk/c4t5000402001FC442Cd0s0' devid='id1,s...@n6000402001fc442c6e1a0e97/a' whole_disk=1 metaslab_array=14 metaslab_shift=36 ashift=9 asize=11801587875840 Uberblock magic = 00bab10c version = 4 txg = 9842225 guid_sum = 8976307953983999543 timestamp = 1236084668 UTC = Tue Mar 3 13:51:08 2009 Dataset mos [META], ID 0, cr_txg 4, 392M, 1213 objects ... [snip] Dataset Data/subversion1 [ZVOL], ID 3527, cr_txg 2514080, 22.5K, 3 objects ... [snip] Dataset Data [ZPL], ID 5, cr_txg 4, 108M, 2898 objects Traversing all blocks to verify checksums and verify nothing leaked ... and I'm still waiting for this process to finish. On 3. mars. 2009, at 11.18, Lars-Gunnar Persson wrote: I thought a ZFS file system wouldn't destroy a ZFS volume? Hmm, I'm not sure what to do now ... First of all, this zfs volume Data/subversion1 has been working for a year and suddenly after a reboot of the Solaris server, running of the zpool export and zpool import command, I get problems with this ZFS volume? Today I checked some more, after reading this guide: http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide My main question is: Is my ZFS volume which is part of a zpool lost or can I recover it? If I upgrade the Solaris server to the latest and do a zpool export and zpool import help? All advices appreciated :-) Here is some more information: -bash-3.00$ zfs list -o name,type,used,avail,ratio,compression,reserv,volsize Data/subversion1 NAMETYPE USED AVAIL RATIO COMPRESS RESERV VOLSIZE Data/subversion1 volume 22.5K 511G 1.00x off250G 250G I've also learned the the AVAIL column reports what's available in the zpool and NOT what's available in the ZFS volume. -bash-3.00$ sudo zpool status -v Password: pool: Data state: ONLINE scrub: scrub in progress, 5.86% done, 12h46m to go config: NAME STATE READ WRITE CKSUM Data ONLINE 0 0 0 c4t5000402001FC442Cd0 ONLINE 0 0 0 errors: No known data errors Interesting thing here is that the scrub process should be finished today but the progress is much slower than reported here. And will the scrub process help anything in my case? -bash-3.00$ sudo fmdump TIME UUID SUNW-MSG-ID Nov 15 2007 10:16:38 8aa789d2-7f3a-45d5-9f5c-c101d73b795e ZFS-8000-CS Oct 14 09:31:40.8179 8c7d9847-94b7-ec09-8da7-c352de405b78 FMD-8000-2K bash-3.00$ sudo fmdump -ev TIME CLASS ENA Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6850ff400401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6850ff400401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6897db600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e688d11500401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68926e600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6897db600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68a3d3900401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68bc6741 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68d8bb600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68da5b51 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68da5b51 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68f0c9800401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68da5b51 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6897db600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68e98191 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68f0c9800401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e690a11000401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68f0c9800401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e69038551 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e69038551 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e690a11000401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e692a4ca1 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68bc6741
Re: [zfs-discuss] ZFS volume corrupted?
And then the zdb process ends with: Traversing all blocks to verify checksums and verify nothing leaked ... out of memory -- generating core dump Abort (core dumped) hmm, what does that mean?? I also ran these commands: -bash-3.00$ sudo fmstat module ev_recv ev_acpt wait svc_t %w %b open solve memsz bufsz cpumem-retire0 0 0.00.1 0 0 0 0 0 0 disk-transport 0 0 0.04.1 0 0 0 0 32b 0 eft 0 0 0.05.7 0 0 0 0 1.4M 0 fmd-self-diagnosis 0 0 0.00.2 0 0 0 0 0 0 io-retire0 0 0.00.2 0 0 0 0 0 0 snmp-trapgen 0 0 0.00.1 0 0 0 0 32b 0 sysevent-transport 0 0 0.0 1520.8 0 0 0 0 0 0 syslog-msgs 0 0 0.00.1 0 0 0 0 0 0 zfs-diagnosis 301 0 0.00.0 0 0 2 0 120b80b zfs-retire 0 0 0.00.3 0 0 0 0 0 0 -bash-3.00$ sudo fmadm config MODULE VERSION STATUS DESCRIPTION cpumem-retire1.1 active CPU/Memory Retire Agent disk-transport 1.0 active Disk Transport Agent eft 1.16active eft diagnosis engine fmd-self-diagnosis 1.0 active Fault Manager Self-Diagnosis io-retire1.0 active I/O Retire Agent snmp-trapgen 1.0 active SNMP Trap Generation Agent sysevent-transport 1.0 active SysEvent Transport Agent syslog-msgs 1.0 active Syslog Messaging Agent zfs-diagnosis1.0 active ZFS Diagnosis Engine zfs-retire 1.0 active ZFS Retire Agent -bash-3.00$ sudo zpool upgrade -v This system is currently running ZFS version 4. The following versions are supported: VER DESCRIPTION --- 1 Initial ZFS version 2 Ditto blocks (replicated metadata) 3 Hot spares and double parity RAID-Z 4 zpool history For more information on a particular version, including supported releases, see: http://www.opensolaris.org/os/community/zfs/version/N Where 'N' is the version number. I hope I've provided enough information for all you ZFS experts out there. Any tips or solutions in sight? Or is this ZFS gone completely? Lars-Gunnar Persson On 3. mars. 2009, at 13.58, Lars-Gunnar Persson wrote: I run a new command now zdb. Here is the current output: -bash-3.00$ sudo zdb Data version=4 name='Data' state=0 txg=9806565 pool_guid=6808539022472427249 vdev_tree type='root' id=0 guid=6808539022472427249 children[0] type='disk' id=0 guid=2167768931511572294 path='/dev/dsk/c4t5000402001FC442Cd0s0' devid='id1,s...@n6000402001fc442c6e1a0e97/a' whole_disk=1 metaslab_array=14 metaslab_shift=36 ashift=9 asize=11801587875840 Uberblock magic = 00bab10c version = 4 txg = 9842225 guid_sum = 8976307953983999543 timestamp = 1236084668 UTC = Tue Mar 3 13:51:08 2009 Dataset mos [META], ID 0, cr_txg 4, 392M, 1213 objects ... [snip] Dataset Data/subversion1 [ZVOL], ID 3527, cr_txg 2514080, 22.5K, 3 objects ... [snip] Dataset Data [ZPL], ID 5, cr_txg 4, 108M, 2898 objects Traversing all blocks to verify checksums and verify nothing leaked ... and I'm still waiting for this process to finish. On 3. mars. 2009, at 11.18, Lars-Gunnar Persson wrote: I thought a ZFS file system wouldn't destroy a ZFS volume? Hmm, I'm not sure what to do now ... First of all, this zfs volume Data/subversion1 has been working for a year and suddenly after a reboot of the Solaris server, running of the zpool export and zpool import command, I get problems with this ZFS volume? Today I checked some more, after reading this guide: http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide My main question is: Is my ZFS volume which is part of a zpool lost or can I recover it? If I upgrade the Solaris server to the latest and do a zpool export and zpool import help? All advices appreciated :-) Here is some more information: -bash-3.00$ zfs list -o name,type,used,avail,ratio,compression,reserv,volsize Data/ subversion1 NAMETYPE USED AVAIL RATIO COMPRESS RESERV VOLSIZE Data/subversion1 volume 22.5K 511G 1.00x off250G 250G I've also learned the the AVAIL column reports what's available in the zpool and NOT what's available in the ZFS volume. -bash-3.00$ sudo zpool status -v Password: pool: Data state: ONLINE scrub: scrub in progress, 5.86
Re: [zfs-discuss] ZFS volume corrupted?
On 3. mars. 2009, at 14.51, Sanjeev wrote: Thank you for your reply. Lars-Gunnar, On Tue, Mar 03, 2009 at 11:18:27AM +0100, Lars-Gunnar Persson wrote: -bash-3.00$ zfs list -o name,type,used,avail,ratio,compression,reserv,volsize Data/ subversion1 NAMETYPE USED AVAIL RATIO COMPRESS RESERV VOLSIZE Data/subversion1 volume 22.5K 511G 1.00x off250G 250G This shows that the volume still exists. Correct me if I am wrong here : - Did you mean that the contents of the volume subversion1 are corrupted ? I'm not 100% sure if it's the content of this volume or if it's the zpool that is corrupted. It was iSCSI exported to a Linux host where it was formatted as an ext3 file system. What does that volume have on it ? Does it contain a filesystem which can can be mounted on Solaris ? If so, we could try mounting it locally on the Solaris box. This is to rule out any iSCSI issues. I don't think that Solaris supports mounting of ext3 file systems or ? Also, do you have any snapshots of the volume ? If so, you could rollback to the latest snapshot. But, that would mean we lose some amount of data. Nope, No snapshots - since this is a subversion repository with versioning built in. I didn't think I'll end up in this situation. Also, you mentioned that the volume was in use for a year. But, I see in the above output that it has only about 22.5K used. Is that correct ? I would have expected it to be higher. You're absolutely right, the 22.5K is wrong. That is why I suspect zfs is doing something wrong ... You should also check what 'zpool history -i ' says. it says: -bash-3.00$ sudo zpool history Data | grep subversion 2008-04-02.09:08:53 zfs create -V 250GB Data/subversion1 2008-04-02.09:08:53 zfs set shareiscsi=on Data/subversion1 2008-08-14.14:13:58 zfs set shareiscsi=off Data/subversion1 2008-08-29.15:08:50 zfs set shareiscsi=on Data/subversion1 2009-03-02.10:37:36 zfs set shareiscsi=off Data/subversion1 2009-03-02.10:37:55 zfs set shareiscsi=on Data/subversion1 2009-03-02.11:37:22 zfs set shareiscsi=off Data/subversion1 2009-03-03.09:37:34 zfs set shareiscsi=on Data/subversion1 and: 2009-03-01.11:26:22 zpool export -f Data 2009-03-01.13:21:58 zpool import Data 2009-03-01.14:32:04 zpool scrub Data Thanks and regards, Sanjeev More info: I just rebooted the SOlaris server and no change in status: -bash-3.00$ zpool status -v pool: Data state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM Data ONLINE 0 0 0 c4t5000402001FC442Cd0 ONLINE 0 0 0 errors: No known data errors The scrubing has stopped and the zdb command crashed the server. I've also learned the the AVAIL column reports what's available in the zpool and NOT what's available in the ZFS volume. -bash-3.00$ sudo zpool status -v Password: pool: Data state: ONLINE scrub: scrub in progress, 5.86% done, 12h46m to go config: NAME STATE READ WRITE CKSUM Data ONLINE 0 0 0 c4t5000402001FC442Cd0 ONLINE 0 0 0 errors: No known data errors Interesting thing here is that the scrub process should be finished today but the progress is much slower than reported here. And will the scrub process help anything in my case? -bash-3.00$ sudo fmdump TIME UUID SUNW-MSG-ID Nov 15 2007 10:16:38 8aa789d2-7f3a-45d5-9f5c-c101d73b795e ZFS-8000-CS Oct 14 09:31:40.8179 8c7d9847-94b7-ec09-8da7-c352de405b78 FMD-8000-2K bash-3.00$ sudo fmdump -ev TIME CLASS ENA Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6850ff400401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6850ff400401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6897db600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e688d11500401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68926e600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6897db600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68a3d3900401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68bc6741 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68d8bb600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68da5b51 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68da5b51 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68f0c9800401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68da5b51 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e6897db600401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68e98191 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68f0c9800401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e690a11000401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e68f0c9800401 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e69038551 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e69038551 Nov 15 2007 09:33:52 ereport.fs.zfs.io 0x915e690a11000401 Nov 15 2007
[zfs-discuss] ZFS volume corrupted?
Hey to everyone on this mailing list (since this is my first post)! We've a Sun Fire X4100 M2 server running Solaris 10 u6 and after some system work this weekend we have a problem with only one ZFS volume. We have a pool called /Data with many file systems and two volumes. The status of my zpool is: -bash-3.00$ zpool status pool: Data state: ONLINE scrub: scrub in progress, 5.99% done, 13h38m to go config: NAME STATE READ WRITE CKSUM Data ONLINE 0 0 0 c4t5000402001FC442Cd0 ONLINE 0 0 0 errors: No known data errors Yesterday I started the scrub process because I read that was a smart thing to do after a zpool export and zpool import procedure. I did this because I wanted to move the zpool to another OS installation but changed my mind and did a zpool import on the same OS as I did an export. After checking as much information as I could find on the web, I was advised to to run the zpool scrub after an import. Well, the problem now is that one volume in this zpool is not working. I've shared it via iscsi to a Linux host (all of this was working on Friday). The Linux host reports that it can't find a partition table. Here is the log from the Linux host: Mar 2 11:09:36 eva kernel: SCSI device sdb: 524288000 512-byte hdwr sectors (268435 MB) Mar 2 11:09:36 eva kernel: SCSI device sdb: drive cache: write through Mar 2 11:09:36 eva kernel: SCSI device sdb: 524288000 512-byte hdwr sectors (268435 MB) Mar 2 11:09:37 eva kernel: SCSI device sdb: drive cache: write through Mar 2 11:09:37 eva kernel: sdb: unknown partition table Mar 2 11:09:37 eva kernel: Attached scsi disk sdb at scsi28, channel 0, id 0, lun 0 So I checked the status on my Solaris server and I found this information a bit strange;: -bash-3.00$ zfs list Data/subversion1 NAME USED AVAIL REFER MOUNTPOINT Data/subversion1 22.5K 519G 22.5K - How can it bed 519GB available on a volume that is 250GB in size? Here are more details: -bash-3.00$ zfs get all Data/subversion1 NAME PROPERTY VALUE SOURCE Data/subversion1 type volume - Data/subversion1 creation Wed Apr 2 9:06 2008 - Data/subversion1 used 22.5K - Data/subversion1 available 519G - Data/subversion1 referenced 22.5K - Data/subversion1 compressratio 1.00x - Data/subversion1 reservation250G local Data/subversion1 volsize250G - Data/subversion1 volblocksize 8K - Data/subversion1 checksum on default Data/subversion1 compressionoffdefault Data/subversion1 readonly offdefault Data/subversion1 shareiscsi offlocal Will this be fixed after the scrub process is finished tomorrow or is this volume lost forever? Hoping for some quick answers as the data is quite important for us. Regards, Lars-Gunnar Persson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS volume corrupted?
That is correct. It's a raid 6 disk shelf with one volume connected via fibre. Lars-Gunnar Persson Den 2. mars. 2009 kl. 16.57 skrev Blake blake.ir...@gmail.com: It looks like you only have one physical device in this pool. Is that correct? On Mon, Mar 2, 2009 at 9:01 AM, Lars-Gunnar Persson lars-gunnar.pers...@nersc.no wrote: Hey to everyone on this mailing list (since this is my first post)! We've a Sun Fire X4100 M2 server running Solaris 10 u6 and after some system work this weekend we have a problem with only one ZFS volume. We have a pool called /Data with many file systems and two volumes. The status of my zpool is: -bash-3.00$ zpool status pool: Data state: ONLINE scrub: scrub in progress, 5.99% done, 13h38m to go config: NAME STATE READ WRITE CKSUM Data ONLINE 0 0 0 c4t5000402001FC442Cd0 ONLINE 0 0 0 errors: No known data errors Yesterday I started the scrub process because I read that was a smart thing to do after a zpool export and zpool import procedure. I did this because I wanted to move the zpool to another OS installation but changed my mind and did a zpool import on the same OS as I did an export. After checking as much information as I could find on the web, I was advised to to run the zpool scrub after an import. Well, the problem now is that one volume in this zpool is not working. I've shared it via iscsi to a Linux host (all of this was working on Friday). The Linux host reports that it can't find a partition table. Here is the log from the Linux host: Mar 2 11:09:36 eva kernel: SCSI device sdb: 524288000 512-byte hdwr sectors (268435 MB) Mar 2 11:09:36 eva kernel: SCSI device sdb: drive cache: write through Mar 2 11:09:36 eva kernel: SCSI device sdb: 524288000 512-byte hdwr sectors (268435 MB) Mar 2 11:09:37 eva kernel: SCSI device sdb: drive cache: write through Mar 2 11:09:37 eva kernel: sdb: unknown partition table Mar 2 11:09:37 eva kernel: Attached scsi disk sdb at scsi28, channel 0, id 0, lun 0 So I checked the status on my Solaris server and I found this information a bit strange;: -bash-3.00$ zfs list Data/subversion1 NAME USED AVAIL REFER MOUNTPOINT Data/subversion1 22.5K 519G 22.5K - How can it bed 519GB available on a volume that is 250GB in size? Here are more details: -bash-3.00$ zfs get all Data/subversion1 NAME PROPERTY VALUE SOURCE Data/subversion1 type volume - Data/subversion1 creation Wed Apr 2 9:06 2008 - Data/subversion1 used 22.5K - Data/subversion1 available 519G - Data/subversion1 referenced 22.5K - Data/subversion1 compressratio 1.00x - Data/subversion1 reservation250G local Data/subversion1 volsize250G - Data/subversion1 volblocksize 8K - Data/subversion1 checksum on default Data/subversion1 compressionoffdefault Data/subversion1 readonly offdefault Data/subversion1 shareiscsi offlocal Will this be fixed after the scrub process is finished tomorrow or is this volume lost forever? Hoping for some quick answers as the data is quite important for us. Regards, Lars-Gunnar Persson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS volume corrupted?
The Linux host can still see the device. I showed you the log from the Linux host. I tried the fdisk -l and it listed the iSCSI disks. Lars-Gunnar Persson Den 2. mars. 2009 kl. 17.02 skrev O'Shea, Damien daos...@revenue.ie: I could be wrong but this looks like an issue on the Linux side A zpool status is returning the healthy pool What does format/fdisk show you on the Linux side ? Can it still see the iSCSI device that is being shared from the Solaris server ? Regards, Damien O'Shea Strategy Unix Systems Revenue Backup Site VPN: 35603 daos...@revenue.ie mailto:daos...@revenue.ie -Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org]on Behalf Of Blake Sent: 02 March 2009 15:57 To: Lars-Gunnar Persson Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] ZFS volume corrupted? * This e-mail has been received by the Revenue Internet e-mail service. (IP) * It looks like you only have one physical device in this pool. Is that correct? On Mon, Mar 2, 2009 at 9:01 AM, Lars-Gunnar Persson lars-gunnar.pers...@nersc.no wrote: Hey to everyone on this mailing list (since this is my first post)! We've a Sun Fire X4100 M2 server running Solaris 10 u6 and after some system work this weekend we have a problem with only one ZFS volume. We have a pool called /Data with many file systems and two volumes. The status of my zpool is: -bash-3.00$ zpool status pool: Data state: ONLINE scrub: scrub in progress, 5.99% done, 13h38m to go config: NAME STATE READ WRITE CKSUM Data ONLINE 0 0 0 c4t5000402001FC442Cd0 ONLINE 0 0 0 errors: No known data errors Yesterday I started the scrub process because I read that was a smart thing to do after a zpool export and zpool import procedure. I did this because I wanted to move the zpool to another OS installation but changed my mind and did a zpool import on the same OS as I did an export. After checking as much information as I could find on the web, I was advised to to run the zpool scrub after an import. Well, the problem now is that one volume in this zpool is not working. I've shared it via iscsi to a Linux host (all of this was working on Friday). The Linux host reports that it can't find a partition table. Here is the log from the Linux host: Mar 2 11:09:36 eva kernel: SCSI device sdb: 524288000 512-byte hdwr sectors (268435 MB) Mar 2 11:09:36 eva kernel: SCSI device sdb: drive cache: write through Mar 2 11:09:36 eva kernel: SCSI device sdb: 524288000 512-byte hdwr sectors (268435 MB) Mar 2 11:09:37 eva kernel: SCSI device sdb: drive cache: write through Mar 2 11:09:37 eva kernel: sdb: unknown partition table Mar 2 11:09:37 eva kernel: Attached scsi disk sdb at scsi28, channel 0, id 0, lun 0 So I checked the status on my Solaris server and I found this information a bit strange;: -bash-3.00$ zfs list Data/subversion1 NAME USED AVAIL REFER MOUNTPOINT Data/subversion1 22.5K 519G 22.5K - How can it bed 519GB available on a volume that is 250GB in size? Here are more details: -bash-3.00$ zfs get all Data/subversion1 NAME PROPERTY VALUE SOURCE Data/subversion1 type volume - Data/subversion1 creation Wed Apr 2 9:06 2008 - Data/subversion1 used 22.5K - Data/subversion1 available 519G - Data/subversion1 referenced 22.5K - Data/subversion1 compressratio 1.00x - Data/subversion1 reservation250G local Data/subversion1 volsize250G - Data/subversion1 volblocksize 8K - Data/subversion1 checksum on default Data/subversion1 compressionoffdefault Data/subversion1 readonly offdefault Data/subversion1 shareiscsi offlocal Will this be fixed after the scrub process is finished tomorrow or is this volume lost forever? Hoping for some quick answers as the data is quite important for us. Regards, Lars-Gunnar Persson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss This message has been delivered to the Internet by the Revenue Internet e-mail service (OP) * ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman
Re: [zfs-discuss] ZFS volume corrupted?
I've turned off iSCSI sharing at the moment. My first question is: how can zfs report available is larger than reservation on a zfs volume? I also know that used mshould be larger than 22.5 K. Isn't this strange? Lars-Gunnar Persson Den 3. mars. 2009 kl. 00.38 skrev Richard Elling richard.ell...@gmail.com : Lars-Gunnar Persson wrote: Hey to everyone on this mailing list (since this is my first post)! Welcome! We've a Sun Fire X4100 M2 server running Solaris 10 u6 and after some system work this weekend we have a problem with only one ZFS volume. We have a pool called /Data with many file systems and two volumes. The status of my zpool is: -bash-3.00$ zpool status pool: Data state: ONLINE scrub: scrub in progress, 5.99% done, 13h38m to go config: NAME STATE READ WRITE CKSUM Data ONLINE 0 0 0 c4t5000402001FC442Cd0 ONLINE 0 0 0 errors: No known data errors Yesterday I started the scrub process because I read that was a smart thing to do after a zpool export and zpool import procedure. I did this because I wanted to move the zpool to another OS installation but changed my mind and did a zpool import on the same OS as I did an export. After checking as much information as I could find on the web, I was advised to to run the zpool scrub after an import. Well, the problem now is that one volume in this zpool is not working. I've shared it via iscsi to a Linux host (all of this was working on Friday). The Linux host reports that it can't find a partition table. Here is the log from the Linux host: Mar 2 11:09:36 eva kernel: SCSI device sdb: 524288000 512-byte hdwr sectors (268435 MB) Mar 2 11:09:36 eva kernel: SCSI device sdb: drive cache: write through Mar 2 11:09:36 eva kernel: SCSI device sdb: 524288000 512-byte hdwr sectors (268435 MB) Mar 2 11:09:37 eva kernel: SCSI device sdb: drive cache: write through Mar 2 11:09:37 eva kernel: sdb: unknown partition table Mar 2 11:09:37 eva kernel: Attached scsi disk sdb at scsi28, channel 0, id 0, lun 0 So I checked the status on my Solaris server and I found this information a bit strange;: -bash-3.00$ zfs list Data/subversion1 NAME USED AVAIL REFER MOUNTPOINT Data/subversion1 22.5K 519G 22.5K - How can it bed 519GB available on a volume that is 250GB in size? Here are more details: -bash-3.00$ zfs get all Data/subversion1 NAME PROPERTY VALUE SOURCE Data/subversion1 type volume - Data/subversion1 creation Wed Apr 2 9:06 2008 - Data/subversion1 used 22.5K - Data/subversion1 available 519G - Data/subversion1 referenced 22.5K - Data/subversion1 compressratio 1.00x - Data/subversion1 reservation 250G local Data/subversion1 volsize 250G - Data/subversion1 volblocksize 8K - Data/subversion1 checksum on default Data/subversion1 compression off default Data/subversion1 readonly off default Data/subversion1 shareiscsi off local It does not appear that Data/subversion1 is being shared via iscsi? -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
