Re: [zfs-discuss] How long should an empty destroy take? snv_134
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Yaverot rpool remains 1% inuse. tank reports 100% full (with 1.44G free), I recommend: When creating your new pool, use slices of the new disks, which are 99% of the size of the new disks instead of using the whole new disks. Because this is a more reliable way of avoiding the problem my new replacement disk for the failed disk is slightly smaller than the failed disk and therefore I can't replace. I also recommend: In every pool, create some space reservation. So when and if you ever hit 100% usage again and start to hit the system crash scenario, you can do a zfs destroy (snapshot) and delete the space reservation, in order to avoid the system crash scenario you just witnessed. Hopefully. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Solaris Express server name broadcast
Le 06/03/2011 00:14, Hung-Sheng Tsao (Lao Tsao 老曹) Ph. D. a écrit : question to solaris forum http://forums.oracle.com/forums/search.jspa?objID=c300q=Solaris Since you have been (and you alone) kind enough to try and help me, I will give you in writing a detailed explanation on how this mess came to be : however, as I explained, I have only known the Solaris OS, or any other *nix for that matter, for three and a half months, with various instances of success and failure. I have tried a lot, read a lot, searched a lot, and I have often failed - things like compiling an app on Solaris, just for one example. When I do not find the necessary information - here, how to have the blasted server broadcast its name so my backup program can catch it - I ask on a list. Aren't they made for just that ? Believe me, I've tried places, to no avail. did you follow Solaris SMB and window intero AG http://download.oracle.com/docs/cd/E19963-01/html/821-1449/index.html I did. Couldn't find what I was looking for. Please donot ask question to OI list http://mail.opensolaris.org/mailman/listinfo I kind of understand I should not have done that. But I did. Please see below. there is zfs list zfs-discuss@opensolaris.org Actually, I have (I had) a subscription to those two lists. What happened is that I intended to ask my question on zfs-discuss, but when I did, I answered a message that was in the zfs-discuss but had actually been cross-posted from OpenIndiana-discuss - something one should *not* do, except on utterly justified occurrences. Consequently, my question appeared on the OpenIndiana list, where it was immediately intercepted by the local cybercops. I then was chastised by a couple of people. Later on, I wrote another message expressing that I failed to see why my question should be considered off-topic. It elicited yet another answer regarding the fact that a fork had occurred and I shouldn't be asking such questions; obviously, I was on the devil's side of the fork, and as one knows, dining with the devil implies using a very long fork - private joke. I then decided to leave that list, that now has very little appeal to me, given the local mores. I did just that. However, I consulted the last messages on the archive, and had the pleasure to see that after I left the list, someone (thanks Dmitry) had found there had been what could be called, for a lack of a better word, a certain lack of courtesy towards my person. But it was too late. There's only so much I can take. Tell more How did you export your ZFS to window I used smb shares; I have read and printed the whole sun (now oracle) book on how configuring smb shares, and I can tell you that I now am proficient with that. It works very well, and seems more that reliable. But I could find nothing even remotely related to my humble request. But I am a stubborn frog, and I'll certainly find out, in due time, how to have a ZFS server using smb shares broadcast its name on the network.. Thanks again for willing to help. Amitiés, Robert PS: since cross-posting seems to be the rage these days, I'll copy that to the zfs-discuss list, in case a noble soul... -- Éditions de l'Âge d'Or — Stanley G. Weinbaum http://www.lulu.com/robert_soubie ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] zfs pool offline and resilvering
Hi all, I have two T5240 with LDOM and last night cause kernel patching on control domain and guest, I preferred put one mirror side in to offline state on the control domain, I tested the offline state with many reboot and was always ok ...one mirror online and one mirror offline...When I installed the kernel patch 142900-17 (with reconfigure reboot), I was surprise to see the rpool in resilvering and disk's online state after boot -r. Is this strange behavior or not ?? What's happened ? Thanks in advance Fred ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Slices and reservations Was: Re: How long should an empty destroy take? snv_134
I questioning these recommendations to increase my understanding. --- opensolarisisdeadlongliveopensola...@nedharvey.com wrote: From: Edward Ned Harvey opensolarisisdeadlongliveopensola...@nedharvey.com From: Yaverot rpool remains 1% inuse. tank reports 100% full (with 1.44G free), I recommend: When creating your new pool, use slices of the new disks, which are 99% of the size of the new disks instead of using the whole new disks. Because this is a more reliable way of avoiding the problem my new replacement disk for the failed disk is slightly smaller than the failed disk and therefore I can't replace. 1. While performance isn't my top priority, doesn't using slices make a significant difference? 2. Doesn't snv_134 that I'm running already account for variances in these nominally-same disks? 3. The market refuses to sell disks under $50, therefore I won't be able to buy drives of 'matching' capacity anyway. I also recommend: In every pool, create some space reservation. So when and if you ever hit 100% usage again and start to hit the system crash scenario, you can do a zfs destroy (snapshot) and delete the space reservation, in order to avoid the system crash scenario you just witnessed. Hopefully. 1. Why would tank being practically full affect management of other pools and start the crash scenario I encountered? rpool rpool/swap remained at 1% use, the apparent trigger was doing a zpool destroy others which is neither the rpool the system runs out of, nor tank. 2. How can a zfs destroy ($snapshot) complete when both zpool destroy and zfs list fail to complete? 3. Assuming I want to do such an allocation, is this done with quota reservation? Or is it snapshots as you suggest? If it is snapshots is this the process: create snapshot @normal-pre-reservation write (reservation size) random data to pool create snapshot @reserved_chunk delete random data create snapshot @normal_post_reservation Now the only unique data (of significance) in @reserved_chunk is just that. And @reserved_chunk should be excluded from backups, and the @normals can be deleted per whatever standard snapshot policy I have is. Would it make more sense to make another filesystem in the pool, fill it enough and keep it handy to delete? Or is there some advantage to zfs destroy (snapshot) over zfs destroy (filesystem)? While I am thinking about the system and have extra drives, like now, is the time to make plans for the next system is full event. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Slices and reservations Was: Re: How long should an empty destroy take? snv_134
On Mon, Mar 7, 2011 at 1:50 PM, Yaverot yave...@computermail.net wrote: 1. While performance isn't my top priority, doesn't using slices make a significant difference? Write caching will be disabled on devices that use slices. It can be turned back on by using format -e 2. Doesn't snv_134 that I'm running already account for variances in these nominally-same disks? It will allow some small differences. I'm not sure what the limit on the difference size is. 3. The market refuses to sell disks under $50, therefore I won't be able to buy drives of 'matching' capacity anyway. You can always use a larger drive. If you think you may want to go back to smaller drives, make sure that the autoexpand zpool property is disabled though. 3. Assuming I want to do such an allocation, is this done with quota reservation? Or is it snapshots as you suggest? I think Edward misspoke when he said to use snapshots, and probably meant reservation. I've taken to creating a dataset called reserved and giving it a 10G reservation. (10G isn't a special value, feel free to use 5% of your pool size or whatever else you're comfortable with.) It's unmounted and doesn't contain anything, but it ensures that there is a chunk of space I can make available if needed. Because it doesn't contain anything, there shouldn't be any concern for de-allocation of blocks when it's destroyed. Alternately, the reservation can be reduced to make space available. Would it make more sense to make another filesystem in the pool, fill it enough and keep it handy to delete? Or is there some advantage to zfs destroy (snapshot) over zfs destroy (filesystem)? While I am thinking about the system and have extra drives, like now, is the time to make plans for the next system is full event. If a dataset contains data, the blocks will have to be freed when it's destroyed. If it's an empty dataset with a reservation, the only change is to fiddle some accounting bits. I seem to remember seeing a fix for 100% full pools a while ago so this may not be as critical as it used to be, but it's a nice safety net to have. -B -- Brandon High : bh...@freaks.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Slices and reservations Was: Re: How long should an empty destroy take? snv_134
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Yaverot I recommend: When creating your new pool, use slices of the new disks, which are 99% of the size of the new disks instead of using the whole new disks. Because this is a more reliable way of avoiding the problem my new replacement disk for the failed disk is slightly smaller than the failed disk and therefore I can't replace. 1. While performance isn't my top priority, doesn't using slices make a significant difference? Somewhere in some guide, it says so. But the answer is no. If you look more closely at that guide (what is it, the best practices guide? or something else?) you'll see what it *really* says is we don't recommend using slices, because sharing the hardware cache across multiple pools hurts performance or sharing cache across zfs and ufs hurts performance or something like that. But if you're only using one big slice for 99% of the whole disk and not using any other slice, then that whole argument is irrelevant. Also, thanks to system ram cache, I contend that disk-based hardware cache is totally useless anyway. The disk hardware cache will never have a hit except in truly weird esoteric cases. In normal cases, all the disk sectors that were read recently enough to be in disk based hardware cache will also be in system ram cache, and therefore the system will not request that sector from the disk again. I know I did some benchmarking, with and without slices, and found no difference. I'd be interested if anyone in the world has a counterexample. I know how to generate such a scenario, but like I said, it's an esoteric corner case that's not important in reality. 2. Doesn't snv_134 that I'm running already account for variances in these nominally-same disks? Yes. (I don't know which build introduced it, so I'm not confirming b134 specifically, but it's in some build and higher.) But as evidenced by a recent thread from Robert Hartzell cannot replace c10t0d0 with c10t0d0: device is too small it doesn't always work. 3. The market refuses to sell disks under $50, therefore I won't be able to buy drives of 'matching' capacity anyway. Normally, when replacing matching capacity drives, it's either something you bought in advance (like a hotspare or coldspare), or received via warranty. Maybe it doesn't matter for you, but it matters for some people. I also recommend: In every pool, create some space reservation. So when and if you ever hit 100% usage again and start to hit the system crash scenario, you can do a zfs destroy (snapshot) and delete the space reservation, in order to avoid the system crash scenario you just witnessed. Hopefully. 1. Why would tank being practically full affect management of other pools and start the crash scenario I encountered? rpool rpool/swap remained at 1% use, the apparent trigger was doing a zpool destroy others which is neither the rpool the system runs out of, nor tank. That wasn't the trigger - That was just the first symptom that you noticed. The actual trigger happened earlier, while tank was 100% full and some operations were still in progress. The precise trigger is difficult to identify, because it only sends the system into a long slow downward spiral. It doesn't cause immediate system failure. Generally by the time you notice any symptoms, it's already been spiraling downward for some time, so even if you know the right buttons to pull it out of the spiral, you won't know that you know the right buttons. Because after you press them, you still have to wait for some time for it to recover. I had the sun support rep tell me someone else had the same problem, and they waited a week and eventually it recovered. I wasn't able to wait that long. I power cycled and fixed it instantly. I don't know the answer to your question, why would it behave that way. And it doesn't always happen. But I've certainly seen it a few times before. Notice how precisely I told you exactly what you should expect to happen next. It's a clear pattern, but not clear enough or common enough to get enough attention to get fixed, apparently. Long ago I opened bug reports with oracle support, but nobody seems to be doing anything about it. 2. How can a zfs destroy ($snapshot) complete when both zpool destroy and zfs list fail to complete? Precisely the problem. The zfs destroy snapshot also hangs. You're hosed until you reboot. But zfs destroy snapshot isn't the only way in the world to free up some space. You can also zfs set reservation=5G tank zfs set reservation=none tank When you're in the failure mode that you experienced, nobody has yet confirmed the ability or inability to set the reservation to none. IF IT WORKS, then you could immediately afterward do a zfs destroy snapshot. But most likely the reservation won't do any good anyway. But it doesn't hurt anything, and it's worth
Re: [zfs-discuss] Slices and reservations Was: Re: How long should an empty destroy take? snv_134
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Brandon High Write caching will be disabled on devices that use slices. It can be turned back on by using format -e My experience has been, despite what the BPG (or whatever) says, this is not true. When I create pools using slices or not using slices, it doesn't seem to make any difference to the cache status of the drives. Also, when I go into format -e, I attempt to toggle the cache status, and toggling also fails. Which brings me back to my former argument: Who cares about the drive cache anyway. The system ram makes it irrelevant. 3. Assuming I want to do such an allocation, is this done with quota reservation? Or is it snapshots as you suggest? I think Edward misspoke when he said to use snapshots, and probably meant reservation. I meant if you are able to reduce or eliminate your reservation, that should free up enough space to enable you to destroy a snapshot, and re-enable your reservation. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Slices and reservations Was: Re: How long should an empty destroy take? snv_134
Edward Ned Harvey wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Brandon High Write caching will be disabled on devices that use slices. It can be turned back on by using format -e My experience has been, despite what the BPG (or whatever) says, this is not true. When I create pools using slices or not using slices, it doesn't seem to make any difference to the cache status of the drives. Also, when I go into format -e, I attempt to toggle the cache status, and toggling also fails. Which brings me back to my former argument: Who cares about the drive cache anyway. The system ram makes it irrelevant. ZFS only uses system RAM for read caching, as all writes must be written to some form of stable storage before acknowledged. If a vdev represents a whole disk, ZFS will attempt to enable write caching. If a device does not support write caching, the attempt to set wce fails silently. As you made reference to above, one would need to use 'format -e' to manually inquire about this capability on a per disk type basis. If a disk does support write caching, and ZFS enables it, there should be some measurable write I/O performance, although how much is unclear. For those interested, one can trace back the ZFS code starting here: http://cvs.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/vdev_disk.c#276 Jim 3. Assuming I want to do such an allocation, is this done with quota reservation? Or is it snapshots as you suggest? I think Edward misspoke when he said to use snapshots, and probably meant reservation. I meant if you are able to reduce or eliminate your reservation, that should free up enough space to enable you to destroy a snapshot, and re-enable your reservation. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
