Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
Are you trying to run one of those usb all-in-one printers off a laptop or something ?? -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
Are you trying to run one of those usb all-in-one printers off a laptop or something ?? No. The application is home NAS server. I'm up to six systems running in my house on a daily basis. Backup, which I worry about, is getting to be a real job, trotting around to the boxes with a USB disk and taking the backups. In addition, there are the obvious issues of data integrity of the data on both the systems being backed and the disks used to to the backup. I need a small (under 10TB) NAS (to eliminate the wandering around with a data-basket in my hands) with some hope of data integrity (therefore the addition of checksummed and recoverable storage) on line system (hence zfs and not DVDs or offline disks). This is so common an application that although I did stumble through finding the keywords and deciding to do on on my own, now that I know the keywords for searching, there are a large number of people smarter than me that have already figured it out and are net-clamoring for substantially the same thing. There are so many of these kinds of posts on the net that I'm mildly astounded that there's not a particular zfs forum supporting just that topic, including something like reference designs for Buy this, put in this software, and it will work. I'm in the position of not particularly wanting to add a new hobby of learning, loving, and polishing Solaris and zfs for fun and professional interest. I'm kind of second wave - I want it for a tool, and am willing to put in some amount of effort to learn it to get it running. I'm not quite as incapable or disinterested as some PC users whose eyes glaze over if they are told that they have to configure the options for some new piece of software. Actually, on a philosophical level, I think that maybe Solaris/Opensolaris may be approaching the threshold that Linux/BSD/etc. is trying to teeter over - the point of acceptance by people primarily interested in using it, not being a wizard of the internals. Over a few decades I've observed that the far larger audience of computer tool-users as opposed to the involved and motivated internals movers and shakers, are necessary for the long term life of a computer/software entity. It would be a Good Thing for the continued existence of Solaris/Opensolaris to go ahead and make some kind of run for that acceptance. There's a user pool fairly screaming for the application. The application is being addressed, poorly, by other solutions. Especially given the uncertain future of Sun and Solaris in the marked, it would be a Good Thing for the folks that enjoy this OS/ file system as an avocation to get it out to as many end-users as possible, if only to extend the life of the playroom. 8-) IMHO, there's a longer term life for Solaris/Opensolaris and zfs if it can grab a fair share of the small NAS server market. It's a good solution. But if another equally good solution emerges first, that's not going to happen, I'm afraid. I see myself as a fair model of the next level of tool adopters. I have had a long term involvement in computers as tools, suffered through the growing pains of the industry, and know enough to figure out what's a good tool for my own needs. That's how I got to zfs. But I was ...um... frustrated beyond belief in trying to get one of these running in a straightforward manner without doing the whole nine yards of building a few (!) test systems and digging through the source code over about a year of spare time. It's better now. This forum and some other places I've received some answers from have helped a lot. I have a paper design for a first server system, and will probably order the parts and build it over the next few months. But Solaris/Opensolaris is not at the point of being readily adoptable by the next larger audience, unfortunately. I'd really like to see it get over that hump. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
Sorry - I posted in zfs discuss also. What I want is a single- or double-failure tolerant network attached file server. I'm willing to build a system and learn Solaris to get it. The rest of the details are of lesser importance and variable. This is as opposed to wanting a specific number of disks, or performance, or such. My storage needs are not huge, a few TB will be fine. Losing data to file crashes or bit rot is not fine. Hence zfs backed up by something like an ever-increasing box of dvdisaster DVDs for cold storage, topped up periodically. That's basically where I was coming from as well. I'm currently running ~5.5TB of usable space in RAIDZ1 and want to add another 4 disks. I think I'm going to see what happens with black friday deals to see if anything interesting shows up. The ability of ZFS to detect bit-rot was one of the biggest draws for me. The system has been working very well for me and, IMO, was worth the time spent learning OpenSolaris and ZFS. I do see what you're saying about the HCL though. I spent a fair bit of time with that and google looking to make sure chipsets and such were supported before I bought a motherboard. In the end, I took a chance on a board that wasn't in the HCL, but the chipset was listed as working on some similar boards. It worked out fine for me, but it was a little worrying. ECC RAM was particularly annoying. I ended up finding a few boards I liked the features on and downloading the user manual from the manufacturer. Searching for ECC in the PDF files found me a couple choices that had the BIOS options for controlling the ECC settings. Kind of a roundabout way to do it, but it works. Cool. As a practical matter, I'm fine with saying I'll use six disks of the latest good compromise density, and a mirrored boot disk of much smaller size. The MBs I looked at seriously have PATA controllers which will run two PATA drives, so I had in mind using that for the mirrored boot disk vdev, and the six SATAs for the stored-data pool in raidz2. The downside to most modern boards when you talk about using the PATA controller for a mirror, is that they generally have only 1 port. While you can hang 2 drives off a single port, performance suffers and you can lose both drives to a bad cable or controller. As it's a mirrored boot drive, the lack of redundancy there is probably OK if you don't mind the down time. Overall, read performance should be OK, writes will be a bit slow. If you find that it's a problem, you could always add a 2 or 4 port SATA card and run some of your drives from there. Or a PATA card I suppose. My older server was running a boot mirror from PATA drives I had sitting around from my old Tivo boxes. It worked great. The single port issue on new motherboards and the fact that it's getting hard to find PATA drives should I need to replace one were the reasons I use SATA boot drives now. For home users, a 6-disk raidz2 seems like a good config. It was actually my first choice, but I ended up a little short on funds so I went with a 4-disk raidz1 and added 4 more disks from my old server as another raidz1. I was a little worried about the older disks, but I have 3 suitable replacements I can use should one fail. Good luck with your build. I found most questions I had were quickly answered by google with a reference to the Solaris documentation online. Of course, if you're having network problems you need another computer. I'm sure people on these lists/forums will also be willing to help. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
Hi, Constantin's blog does come perilously close to being an AMD reference design, and could easily be turned into a useful one according to my internal metrics. About the only things that turned me away from plunging in and replicating it are that - I had a hard time finding an ASUS M3A78-CM a couple of months ago when I was looking - the Athlon II X2 240e is new, and although Constantin got one for review, the retail outlets won't have them for a while, at least that I can tell. thanks for reading my blog. You don't have to use a 240e. In fact, if I had to buy a CPU today, I'd go with the 238e. .1 GHz less but approx. half the price. Any other recent AMD CPU should work fine, by the way, they really only differ by the amount of cache, cores, GHz and HT links. You want an e as part of the model number so the CPU uses less power over a regular one. A 238e may be easier to get because it's manufactured at higher yields than the faster 240e. The most important thing is to make sure you have ECC support in the CPU and on the motherboard. That is easy to get at low cost with AMD. I've seen a couple of posts that recommended an Asus M4A board, so that may help you as well. The only reason I picked an M3 series was purely cost. I've been on anyone-can-say-anything networks since about 1978. In that time I learned to read for content, but to personally verify anything I read before putting real money into it. So these two blogs were a big inspiration, but they started me off on the verification quest. This led me through the Yes, there's no substitute for real-life experience. But you have to be prepared to spend a couple of bucks as learning money. So I had two thoughts - one, I could bombard the bloggers with questions, or I could take my questions to the forum which seemed to be for this kind of issue - here. Feel free to do both :). Cheers, Constantin -- Sent from OpenSolaris, http://www.opensolaris.org/ Constantin Gonzalez Sun Microsystems GmbH, Germany Principal Field Technologisthttp://blogs.sun.com/constantin Tel.: +49 89/4 60 08-25 91 http://google.com/search?q=constantin+gonzalez Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten Amtsgericht Muenchen: HRB 161028 Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Wolf Frenkel Vorsitzender des Aufsichtsrates: Martin Haering ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
[...] my own *nix machine for a couple of years. I regularly build machines from parts and an assortment of corpses found in the workshop. I'm familiar with OS's and hardware from user level right down to the bare metal. But I don't do this every day, and I have [...] I have a few tips: 1) Forget 32bit systems, including the powerful last generation 32bit Xeon; zfs limits the pool size to something around 1TB on 32bit. 2) Forget Sun Sparc, several missing features, only serial port console, no graphics on Sun FB cards. 3) I tried OpenSolaris on several Opteron first generation systems and all worked fine. Motherboards used: Arima HDAMA, Arima HDAMB (this one surprisingly works also with dual core processors, but it is best to use the HE versions, low power), Iwill DK8. I never had a Tyan, but I believe that most would work. 4) When I last used SCSI there wasn't a 64bit Adaptec driver. I do not know with the latest release, so be aware. 5) Fibre Channel for Qlogic works like a charm 6) SATA silicon is OK as JBOD, I haven't tried the RAID mode, but with ZFS that is not recommended. 7) Opteron AMD PowerNow! frequency scaling does not work, which is a real pain in the neck, I hope that this will be fixed in future releases. So much for global warming, looks like the americans still do not care. 8) Graphic cards may need some manual /etc/X11/xorg.conf tweaking, server motherboards have usually embedded ATI Rage, which is picked up at 800x600 only which is limiting, and there is no way to make it go to any higher resolution with the GUI. 9) If you have more than one physical box, forget sharing one keyboard, mouse and video with a KVM switch, or even forget disconnetting the PS2 mouse by mistake! If you do, you will loose control of the mouse until next REBOOT! (This is GROSS isn't it?) -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
7) Opteron AMD PowerNow! frequency scaling does not work, which is a real pain in the neck, I hope that this will be fixed in future releases. So much for global warming, looks like the americans still do not care. Current AMD chips DO support power/frequency scaling in OpenSolaris. I understand that the older chips are not supported from reading around here. 9) If you have more than one physical box, forget sharing one keyboard, mouse and video with a KVM switch, or even forget disconnetting the PS2 mouse by mistake! If you do, you will loose control of the mouse until next REBOOT! (This is GROSS isn't it?) I use a USB KVM and it works fine with OpenSolaris. I switch around and it doesn't cause any issues. I agree that it's irritating that it doesn't work with PS/2 though, but PS/2 wasn't designed for hot-plug. Most PS/2 KVMs I've used make the attached computer think the mouse and keyboard are still there at all times. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
Thanks! That's quite a useful bit of info, much like what I'd hoped for out of the HCL. I'd decided that ASUS had reliable-enough motherboards, AMD processors were certainly usable and all support ECC. But the issues with the on-board ethernet chips and the disk controllers were something I was trying to avoid. An AMD CPU can save you a neat $100 pretty quickly, as can an ASUS versus Supermicro motherboard. On the other hand, an intel-chip NIC is $50 and a Supermicro disk controller is $80-$100, so the savings get eaten up quickly. When I factor in my clumsy, inept fumbling with a new OS, the scales tip to better support quickly. Thanks for the reply - that is very, very useful info. The onboard disk controller does work properly. I'm using it for my boot drives. Just set it to AHCI in the BIOS or OpenSolaris will only see 4 ports. The NIC also works, I just wasn't 100% sure it would so I bought the Intel NIC so I would be sure to be able to get on the network. I believe that the 2009.06 release doesn't know the PCI ID of the onboard NIC, so you do have to edit files a bit to get it working. Later releases do detect it though. If you need more than 6 SATA ports, you will likely need an add-on card for more drives as most motherboards top out around there. The Supermicro boards are expensive, but they are supported well and are one of the very few options for more than 4 ports in PCI Express. Most of the other boards out there are expensive RAID controllers. And I did get what seems to be a good workaround for the MPT driver issue I was seeing. No errors in the past 16 hours or so. I believe the cheaper 4 port Silicon Image boards everyone seems to sell are also supported in OpenSolaris. I had one of those die in my old server though, so I was willing to pay more for reliability. Perhaps if you said how many drives you want and such? It's hard to give good recommendations when we don't know exactly what you're after. I'm using 2 drives for root in a mirror from the onboard controller and 8 drives (soon to be 12) on the SuperMicro controllers for mass storage. As that means I had to design for 16 SATA ports, I was forced to the controller cards as I've never encountered a motherboard with that many onboard SATA ports. If you wanted 4 data drives and 2 boot drives, there are a lot of options for motherboards with 6 SATA ports onboard that work fine with OpenSolaris. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
Travis Tabbal wrote: 7) Opteron AMD PowerNow! frequency scaling does not work, which is a real pain in the neck, I hope that this will be fixed in future releases. So much for global warming, looks like the americans still do not care. Current AMD chips DO support power/frequency scaling in OpenSolaris. I understand that the older chips are not supported from reading around here. The current support for PowerNow! is limited to those in the 10h family and later. I /believe/ this mean the Sempron M1, Phenom, Phenom II, and Athlon II series of desktop processors, and the Barcelona and later Opteron series, plus the Turion II notebook CPUs. I'm not so sure about the Athlon X4 and the Santa Rosa series of dual-core Opterons (i'm pretty sure they're 0Fh). I finally found the completed lists for the 10h and 11h families: http://support.amd.com/us/Processor_TechDocs/41322.pdf http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/41788.pdf I'm not sure of the blocking issues, but it certainly would be nice to have PowerNow! supported through all the Socket F/AM2/AM2+ CPU families (I can see ignoring all the DDR1-era CPUs, as they're at least 4 years out of date now). 9) If you have more than one physical box, forget sharing one keyboard, mouse and video with a KVM switch, or even forget disconnetting the PS2 mouse by mistake! If you do, you will loose control of the mouse until next REBOOT! (This is GROSS isn't it?) I use a USB KVM and it works fine with OpenSolaris. I switch around and it doesn't cause any issues. I agree that it's irritating that it doesn't work with PS/2 though, but PS/2 wasn't designed for hot-plug. Most PS/2 KVMs I've used make the attached computer think the mouse and keyboard are still there at all times. Removing and reattaching PS/2 peripherals on ANY OS is a crapshoot. PS/2 simply isn't designed to allow for reliable hot swapping. Good PS/2 KVMs have little chips inside which emulate a constantly-connected PS/2 device, which is why they work reliably - crappy KVMs are just mechanical switches, which are the equivalent to plugging and unplugging the PS/2 port (that is, take your chances). I've had no more difficulties with OpenSolaris and PS/2 than I have with Linux or Windows. USB does support hot-swapping, which I've found works nicely on pretty much all OSes, with no real issues. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
Perhaps if you said how many drives you want and such? It's hard to give good recommendations when we don't know exactly what you're after. Sorry - I posted in zfs discuss also. What I want is a single- or double-failure tolerant network attached file server. I'm willing to build a system and learn Solaris to get it. The rest of the details are of lesser importance and variable. This is as opposed to wanting a specific number of disks, or performance, or such. My storage needs are not huge, a few TB will be fine. Losing data to file crashes or bit rot is not fine. Hence zfs backed up by something like an ever-increasing box of dvdisaster DVDs for cold storage, topped up periodically. Here's my explanation from the zfs discuss forum: === Given that data integrity drove me to Opensolaris and zfs for a file server instead of to a simple NAS box which would have been cheaper and easier, I reasoned as follows: - yep, raidz2 is going to meet data integrity better than raidz; so I need plug in capacity for six disks minimum, that gets to raidz2 with the lowest number of data integrity issues based on disk failure (I think...) - there are six-SATA and more motherboards which exist, so there is a premium on using one of these MBs if opensolaris supports the SATA controller on the MB, instead of finding and buying one or more disk controller cards. The disk card which seems most supported and cheapest seems to be the Supermicro PCI-x eight-SATA version for $100. One could argue that two-three cheaper cards would get you under $100, but that then begs the issue of complexity, number of slots on the MB, and additional electrical power use, which is a secondary consideration, but one which is an obvious issue with the more cards stuffed into the box. So a six-SATA native controller is a Good Thing - back at data integrity, a motherboard failure is a problem too; a well-trusted and well tested MB is a plus. I took that to mean don't buy a MB with overclocking mentioned as a plus for it, and don't bother with all the fancy onboard widgies you can get. My personal positive experiences have been with ASUS, Gigabyte, and Intel. I have had motheboard deaths and erratic behavior with ECS, FCI, and DFI. Haven't used a Supermicro, but they seem to be highly recommended. - gotta have ECC RAM on the MB - amount of memory is pretty much a don't care these days; 4gb to 8gb are probably fine, maybe even less is OK. - number of slots is only an issue if I have to use external disk cards. This might be an issue if I was trying to get over a few TB of server storage, but I'm pretty happy with under 10TB. If I was trying to fill up a 20-30 disk array, I'd have different answers. I can't afford that many disks. So a 6-SATA MB is a good compromise. - intel vs AMD CPUs is a don't-care to a first approximation. Both support 64bit, both have ECC support in some flavors. This means select based on MB features, not architecture. At a secondary level, AMD seems to be cheaper and perhaps lower power if you get especially the newly announced e versions, notably the Athlon II X2 240e. But saving $100 on a processor or 20W on the power budget isn't worth not having the right number of disks on the MB or hot having chipset drivers be available. === I'm using 2 drives for root in a mirror from the onboard controller and 8 drives (soon to be 12) on the SuperMicro controllers for mass storage. As that means I had to design for 16 SATA ports, I was forced to the controller cards as I've never encountered a motherboard with that many onboard SATA ports. If you wanted 4 data drives and 2 boot drives, there are a lot of options for motherboards with 6 SATA ports onboard that work fine with OpenSolaris. Cool. As a practical matter, I'm fine with saying I'll use six disks of the latest good compromise density, and a mirrored boot disk of much smaller size. The MBs I looked at seriously have PATA controllers which will run two PATA drives, so I had in mind using that for the mirrored boot disk vdev, and the six SATAs for the stored-data pool in raidz2. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
Yes, I've read that. It was, in fact, one of the things I read that started me down the path of doing my own. There are a couple of other such blogs and testimonials on the net that I've found through the thrashing I've given google over this topic. In fact, that particular blog is the one that pushed me over the edge into thinking I can do this. Let's just get on with doing the footwork necessary to get it running. But I'm glad you posted it - I should have done so, and it's very useful to the audience which may be reading here. Constantin's blog does come perilously close to being an AMD reference design, and could easily be turned into a useful one according to my internal metrics. About the only things that turned me away from plunging in and replicating it are that - I had a hard time finding an ASUS M3A78-CM a couple of months ago when I was looking - the Athlon II X2 240e is new, and although Constantin got one for review, the retail outlets won't have them for a while, at least that I can tell. The other blog that someone would no doubt post here in a while is this one: http://www.nerdblog.com/2009/04/good-enough-zfs-nas.html which describes what I actually decided was my best bet for getting something running. I've been on anyone-can-say-anything networks since about 1978. In that time I learned to read for content, but to personally verify anything I read before putting real money into it. So these two blogs were a big inspiration, but they started me off on the verification quest. This led me through the HCL and absolutely thrashing various retail sellers' sites for price/feature/availability comparisons. And while the two blogs in consideration come very close, they opened up more questions than I could find answers to. The biggest questions came from my impression from reading the blogs that the writers were in fact either professionals or semi-professionals in this micro-niche. They were almost certain to have readily at hand skills to solve problems I'd have to spend weeks or months learning. At least that was my meta-suspicion. So I had two thoughts - one, I could bombard the bloggers with questions, or I could take my questions to the forum which seemed to be for this kind of issue - here. This happened over some weeks, you understand. By the time I'd gotten a fair way into my due diligence reading here, I got frustrated (hence Frustrated 8-) ) with the way that the right info seemed so close, but the more I learned, the more it seemed to be just out of reach. The further I explored the woods the more bewildering it got, even allowing that I'd left a trial of bread crumbs. That train of thought culminated in there oughta be a reference design or two. So - yeah, thanks for mentioning that. I should have, didn't think of it. I'll go do the commentary on the blogs. I am perilously unfit for doing anything like a reference design for opensolaris/zfs. In this arena, I'm forced to be a follower, not a leader. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
r.g. I feel your pain. I'd be careful of that board. I have OpenSolaris installed on a box using a similar AMD chipset (780G in a Gigabyte MA78GM-S2H). It works fine with 2009.06, but not with either of the two releases prior to that. And not with the recent development builds of 2010.02. I don't know the cause of the issue with 2010.02 - it won't finish booting and my knowledge of OpenSolaris isn't sufficient to find out what is going wrong. Prior to 2009.06, the issue was http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6764179 (but you could work around it, by booting in 32bit mode). In SNV116 a further change was committed, http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6773433. I speculate that this is the cause of my boot problems. I'll be posting shortly to try and find out how to find out what is causing my problems. I haven't had problems with the inbuilt NIC, at least not since 2008.05. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
What I'm running right now: ASUS M4N82 Deluxe AMD Phenom II X3 720 Kingston 4GB (2 x 2GB) 240-Pin DDR2 SDRAM DDR2 800 (PC2 6400) ECC Unbuffered Desktop Memory Model KVR800D2E6K2/4G - Retail51195154 (N82E16820134936) All from Newegg and appear to still be available. There's the platform. I added 2x SuperMicro AOC-USAS-L8i to control the drives. (there is a bug in the driver for these LSI based cards that causes I/O stuttering under load. Seems to only happen with XvM loaded. I submitted a bug report on it, but I haven't heard anything else about it.) I also added an Intel Gigabit ethernet card. The onboard port seems to work, but I had read that the Intel driver was quite good so I decided it was worth a few extra bucks. If you aren't using too many drives, the onboard SATA ports might get you taken care of. It's all about what you want to do. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
r.g. I feel your pain. I'd be careful of that board. I have OpenSolaris installed on a box using a similar AMD chipset (780G in a Gigabyte MA78GM-S2H). It works fine with 2009.06, but not with either of the two releases prior to that. And not with the recent development builds of 2010.02. I don't know the cause of the issue with 2010.02 - it won't finish booting and my knowledge of OpenSolaris isn't sufficient to find out what is going wrong. Prior to 2009.06, the issue was http://bugs.opensolaris.org/bugdatabase/view_bug.do?bu g_id=6764179 (but you could work around it, by booting in 32bit mode). In SNV116 a further change was committed, http://bugs.opensolaris.org/bugdatabase/view_bug.do?bu g_id=6773433. I speculate that this is the cause of my boot problems. I'll be posting shortly to try and find out how to find out what is causing my problems. I haven't had problems with the inbuilt NIC, at least not since 2008.05. Chris, thanks for replying. Your note is EXACTLY why I got frustrated -I've seen and been party to it almost worked... situations before. I figured that I could spend some time on line and through research find out what did and didn't work. The HCL seems like exactly what I needed... until I started trying to figure does *this* or *that* work, really, no-fooling. And I am NOT skilled enough at Solaris to find the bugs, for certain. I'm pretty much off AMD for this project, for just such reasons. Intel seems to be more widely supported, so there will probably be less chance of wasting my time and money. Of course, that's what got us Wintel... 8-|... R.g. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
What I'm running right now: ASUS M4N82 Deluxe AMD Phenom II X3 720 Kingston 4GB (2 x 2GB) 240-Pin DDR2 SDRAM DDR2 800 (PC2 6400) ECC Unbuffered Desktop Memory Model KVR800D2E6K2/4G - Retail51195154 (N82E16820134936) All from Newegg and appear to still be available. There's the platform. I added 2x SuperMicro AOC-USAS-L8i to control the drives. (there is a bug in the driver for these LSI based cards that causes I/O stuttering under load. Seems to only happen with XvM loaded. I submitted a bug report on it, but I haven't heard anything else about it.) I also added an Intel Gigabit ethernet card. The onboard port seems to work, but I had read that the Intel driver was quite good so I decided it was worth a few extra bucks. If you aren't using too many drives, the onboard SATA ports might get you taken care of. It's all about what you want to do. Thanks! That's quite a useful bit of info, much like what I'd hoped for out of the HCL. I'd decided that ASUS had reliable-enough motherboards, AMD processors were certainly usable and all support ECC. But the issues with the on-board ethernet chips and the disk controllers were something I was trying to avoid. An AMD CPU can save you a neat $100 pretty quickly, as can an ASUS versus Supermicro motherboard. On the other hand, an intel-chip NIC is $50 and a Supermicro disk controller is $80-$100, so the savings get eaten up quickly. When I factor in my clumsy, inept fumbling with a new OS, the scales tip to better support quickly. Thanks for the reply - that is very, very useful info. R.G. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
Rugrat wrote: There are several threads on opensolaris.org on building a NAS box. A quick scan shows this blog: Yes, I've read that one; also the several others in other places on the web which have blogs about building similar stuff. I've read many of those as well. However, many of them are also as vaque about what runs. Notice from this blog: The other thing I found was a recently retired W2100Z workstation It's actually a good illustration of my point. I actually don't happen to have a retired W2100Z workstation, and I'm not sure that I want to trust the data integrity of retired hardware. I guess, put another way, I'd happily pay extra (as well as paying the brain time to learn opensolaris' quirks and operation) to get it set up. The web is full of I used a batch of old hardware and it worked kinda, but this and that didn't work... If my personal life was such that I spent 8-10 hours a day managing servers, I might feel more solid about simply cobbling together an assortment of parts from an assortment of corpses (as I do for linux and windows machines) but this is another objective, and I don't feel that the side effects of my daily experience is enough to answer the questions sufficiently. So, as I said... I think a reference design or two would be a Very Positive Thing for the opensolaris and zfs community. I totally agree with you, but as this is a community project, it requires someone in the community to step up and do it. It sounds like you should volunteer to lead this effort, and I applaud your efforts. If nothing else, you may want to cobble together the information scattered throughout the mailing list discussions on this topic, and break them down based upon goals (low power, performance, etc.) Gary -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
I want, *snip* to put together a zfs file server for my ever-growing home network. My issue is this. The HCL is substantially useless. My response is this: No, there is not enough effort or resources available from the volunteer community to have a well organized and thorough HCL. There are no funds for opensolaris project to buy one of everything and test it. It is difficult to organize all the unknown and random people who do buy all that stuff, to provide their success or unsuccess into a centrally organized location. It's a problem, and the solution is not easy. It probably won't go away anytime soon. That being said, *most* systems do work. I would encourage this behavior: Find some hardware that you think you like for other reasons, and give it a try. Make sure it's returnable or low risk if it doesn't work. So far, I've had 100% success by doing this, which is to say, 4 out of 4. Personally, at home, I eliminate the risk as follows: I built a machine that has a well known HCL. Specifically a dell machine with windows 64bit, or with RedHat 64bit. And then I paid for VMWare Workstation, and run my solaris inside a VM. Yes, I give up some of the awesomeness of zfs's raid capabilities, but for home usage, at least for me, that's fine. In return, I gain the excellent ability to migrate from hardware to hardware effortlessly, as I do upgrades in the future. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
--- On Sat, 11/21/09, R.G. Keen k...@geofex.com wrote: I believe that there are many people like me out in there that would happily follow a cookbook to get a zfs server running. I don't want to enjoy opensolaris as a hobby, I want a server I can't get any other way. I think there are others. (a) how come there's not a reference design for a low power zfs file server/NAS? (b) which section of these fora do I hit to get a review of my seventeenth paper-design for the server I'm trying to put together? You might look into asking the development team over at Nexenta.org that specialize in ZFS-based NAS boxes and have a storage product called NexentaStor (http://www.nexenta.org). Also, there is Andre Lue handling the EON NAS server development. He has some nice information on his blog and can easily help you with a cookbook requirement for file servers and hardware compatibility. (http://eonstorage.blogspot.com) So, you have options. ;o) Ken Mays Blastwave.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
I totally agree with you, but as this is a community project, it requires someone in the community to step up and do it. Yeah, I know. I've worked with network volunteer efforts before. There are quite reasonable limitations on what you can expect. ... 8-) I have gotten over my frustration a bit. It sounds like you should volunteer to lead this effort, and I applaud your efforts. If I would do that, and probably would simply have jumped in if I were not one of of the apparently few people still employed in the USA these days. Wanting to keep up the luxuries I've become accustomed to - eating and living indoors - keeps on interfering with what I'd like to be doing. 8-) nothing else, you may want to cobble together the information scattered throughout the mailing list discussions on this topic, and break them down based upon goals (low power, performance, etc.) I'll post what I run into, and have started doing that. I may be able to do some kind of matrix setup. I feel completely inadequate to any of this because I'm NOT one of the people who could say off the top of my head Oh, that chipset? that won't work because the driver does... I do very much appreciate even the casual comments from people who are, though. Thanks! -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
My response is this: No, there is not enough effort or resources available from the volunteer community to have a well organized and thorough HCL. There are no funds for opensolaris project to buy one of everything and test it. It is difficult to organize all the unknown and random people who do buy all that stuff, to provide their success or unsuccess into a centrally organized location. It's a problem, and the solution is not easy. It probably won't go away anytime soon. Yeah, I know. I'm calmer now. 8-) I've worked with setups like this before. Personally, at home, I eliminate the risk as follows: I built a machine hat has a well known HCL. Specifically a dell machine with windows 64bit, or with RedHat 64bit. And then I paid for VMWare Workstation, and run my solaris inside a VM. Yes, I give up some of the awesomeness of zfs's raid capabilities, but for home usage, at least for me, that's fine. In return, I gain the excellent ability to migrate from hardware to hardware effortlessly, as I do upgrades in the future. That's a possible option, all right. It does offer upgradability, but it also puts me even further behind in learning to get the system to run because then I'd have to pick up, learn and use VMWare well enough to get zfs into a VM and reliably enough to not blow away my data, which is what I was trying to do before I started wading through these alligators... 8-) -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
You might look into asking the development team over at Nexenta.org that specialize in ZFS-based NAS boxes and have a storage product called NexentaStor (http://www.nexenta.org). OK... doing background reading now... Also, there is Andre Lue handling the EON NAS server development. He has some nice information on his blog and can easily help you with a cookbook requirement for file servers and hardware compatibility. (http://eonstorage.blogspot.com) OK... doing background reading now... Thanks, Ken! -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
Just grafting back together a great reply I got to my original posting, which appeared in its own topic for some reason. -- Forwarded message -- From: Al Hopper a...@logical-approach.com Date: Sat, Nov 21, 2009 at 8:23 PM Subject: Re: [zfs-discuss] The 100,000th beginner question about a zfs server To: R.G. Keen k...@geofex.com On Sat, Nov 21, 2009 at 7:08 PM, R.G. Keen k...@geofex.com wrote: With apologies for clogging up the forum with beginner questions - No problem - we've all been there. I'm trying to figure out how to build a home zfs server. Common question. In the last two months of reading the net and here, I've found many answers, none of which would convince me to part with the $800-$1K to do it. So can someone take pity on a beginner and tell me - will this work? It's about my seventeenth paper design for the server. Objectives are You get extra points for putting in this amount of effort before this post! 1 - zfs and other considerations for data preservation It would be helpful to give us a broad description of what type of data you're planning on storing. Small files, large files, required capactity etc. and we can probably make some specific recommendations. 2 - file server operation 3 - units of terabytes, consistent with a few (less than a dozen) disks 4 - as low an electrical power use as practical, given the above I've thrashed through Intel vs AMD, ECC, chipset support, number of ports, adapters, and so on ad nauseum. Here's what I think will work: Supermicro MBD-X7SBL-LN1-O intel Xeon E3110 unregistered ECC, 4GB - 8GB I see that your research has shown the importance of using ECC memory. Hence the Xeon based choice. Good choice. What I can't pick out of the overwhelming flood of raw data I've read is: 1 - does opensolaris offer driver support for the SATA ports resident on the motherboard (Intel® 3200 + ICH9R), or must I get another board to run them? I'm happy with the six SATA ports on the MB to start with Yes the Intel ICH9R is supported. 2 - does opensolaris directly support the LAN chips ( Intel 82573V) on that MB, or must I grab a NIC to stick in a slot? The Intel based NICs are well supported by OpenSolaris and have been historically. Good choice. 3 - does opensolaris support that graphics chipset (XGI Volari Z9S) well enough to let me install and get it bootstrapped into operation, after which I'll make it headless. It'll definately work in a VGA mode. Thats all you'll need for setup. 4 - Are there any gotchas which would keep me from enabling and running the ECC memory functions productively; I think this is a no, but as long as I'm asking questions... None. It's usually a BIOS config option and transparent to the OS. Things of some considerations but lesser importance: 5. Electrical power; the E3110 is a low-ish power chip (nominally 65W) Low power is nice, but not a killer. I'd prefer it to be low, and am willing to take slow to get more of that, because my file server needs are not in any way real time. I just need a large, but reliable, bit bucket. The requirement for ECC limits you from most of the popular motherboard and processor choices. This is one of the few remaining dividing lines used to separate the marketplace into consumer or enterprise categories. And to enable use of a dual pricing schedule. 6. Cost; I've been through several iterations of something with an AMD Athlon 11 X2 240e with an Asus motherboard to get lower power and cost for the same objectives. I can't tell that the silly thing would or would not be something I could make run. And frankly, the data is worth more than the extra $200 or so for the intel solution - iff the intel solution works. But I really am not interested in buying a canned commercial solution for a couple of $K. I'm willing to put in the work setting up and managing the system in lieu of that, so there is a dollar threshold, I guess. Personally I get great satisfaction from rolling my own. And the last silly question. It seems to me that you'd have many, many adopters if there was a real answer to what the HCL tries to be and isn't - an answer to if I buy this stuff, do I have a prayer of making it work, or is there a subtle gotcha that's going to waste my time and money? We used to solve that with reference designs. They don't have to be perfect, they don't have to be optimal, but they should be practical and they should be modestly predictable given moderate skill in the art. Agreed - the HCL has not proved to be as useful, in practice, as most users would like. It's a difficult task - but the typical OpenSolaris builder is unwilling to put the effort in to contribute to the HCL. Current OpenSolaris releases work well enough and support enough current hardware that the risk of something mainstream *not* working
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
One more thing I'm surprised nobody has mentioned yet, that may be of some use: the device detection tool http://hub.opensolaris.org/bin/view/Project+ddtool/ -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
That's a really good suggestion. I'll dig through that. It's a little clumsy, as the assumption is that you already have a system to run the discovery program on, and I'm trying to do the reverse of that, know what the chipsets are and deduce from that if they're supported. But a very useful addition, nonetheless. Thanks. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
or with RedHat 64bit. And then I paid for VMWare Workstation, and run my it also puts me even further behind in learning to get the system to run because then I'd have to pick up, learn and use VMWare well enough to get zfs into a VM and reliably enough to not blow It doesn't have to be vmware. You could use (free) sun virtualbox. And using it is so ridiculously straightforward, there's no learning curve. Well, I exaggerate slightly. Here is your learning curve: Virtualbox allows you to create a window, and run a complete PC inside it. Including BIOS, booting from a CD (which could be a real CD or just an ISO image), configuring an IP address, everything. Since the guest OS doesn't know anything about the host OS, when you focus your keyboard/mouse into it, it's locked until you hit a special key to escape back to your main OS. That is, until you install special drivers later (described in a moment.) Since it's emulating hardware, it runs slower than natural. But after you finish installation, you Install VMWare Tools or Install Virtualbox Additions and it is a special set of drivers that eliminates both of the above problems (speed and mouse/keyboard control.) And so I say: Don't even think about it; just try it today. Once the guest OS is installed, yes it makes sense to think about how to implement things like the CIFS services reliably, on storage that won't fail, and how to do backups and such. There is a learning curve there. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
I want, like about 90% of the more literate home computer users to put together a zfs file server for my ever-growing home network. snip Very recently, one of Sun's bloggers started a blog about building a small and energy-efficient OpenSolaris/zfs-based home server: http://blogs.sun.com/constantin/entry/a_small_and_energy_efficient#comments We are closely following the development. Perhaps you can do the same (at least post a comment to introduce yourself). -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
[osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
Background: I spend about 10-12 hours a day sitting at a computer, and have for probably 20 years. I have written code in about a dozen languages, for processors from 4-bit up through 64 bit, and admin'ed my own *nix machine for a couple of years. I regularly build machines from parts and an assortment of corpses found in the workshop. I'm familiar with OS's and hardware from user level right down to the bare metal. But I don't do this every day, and I have never been a professional admin for more than my own machines. So I've been there with computers, not a beginner. I want, like about 90% of the more literate home computer users to put together a zfs file server for my ever-growing home network. I'm about two months into reading posts of people who have maybe, kinda or even successfully built and run this kind of thing. The net is full of such stuff. I've thrashed through the fora here and other places. I'm willing to put in the time to graft on opensolaris as a new OS I know and use just to get to the object of a zfs file server. My issue is this. The HCL is substantially useless. There is an overwhelming amount of data there suggesting that every known motherboard/cpu/chipset/disk/adapter will run opensolaris and by extension zfs and do what I want. There is an overwhelming amount of expressed misery in other fora about would-be users who may partly get it running but not quite. There are blogs about successful setups which almost but not quite offer a list of this works. I believe that there are many people like me out in there that would happily follow a cookbook to get a zfs server running. I don't want to enjoy opensolaris as a hobby, I want a server I can't get any other way. I think there are others. (a) how come there's not a reference design for a low power zfs file server/NAS? (b) which section of these fora do I hit to get a review of my seventeenth paper-design for the server I'm trying to put together? Sorry if this strikes the cognoscenti here as unenlightened; I've really tried to do the homework, but have been unsuccessful. Please point me to where I can find the info. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
R.G. Keen wrote: ... I believe that there are many people like me out in there that would happily follow a cookbook to get a zfs server running. I don't want to enjoy opensolaris as a hobby, I want a server I can't get any other way. I think there are others. (a) how come there's not a reference design for a low power zfs file server/NAS? (b) which section of these fora do I hit to get a review of my seventeenth paper-design for the server I'm trying to put together? There are several threads on opensolaris.org on building a NAS box. A quick scan shows this blog: http://blogs.sun.com/icedawn/entry/bondin ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] Frustrated beyond belief trying to cobble together a zfs platform
Rugrat wrote: There are several threads on opensolaris.org on building a NAS box. A quick scan shows this blog: Yes, I've read that one; also the several others in other places on the web which have blogs about building similar stuff. I've read many of those as well. However, many of them are also as vaque about what runs. Notice from this blog: The other thing I found was a recently retired W2100Z workstation It's actually a good illustration of my point. I actually don't happen to have a retired W2100Z workstation, and I'm not sure that I want to trust the data integrity of retired hardware. I guess, put another way, I'd happily pay extra (as well as paying the brain time to learn opensolaris' quirks and operation) to get it set up. The web is full of I used a batch of old hardware and it worked kinda, but this and that didn't work... If my personal life was such that I spent 8-10 hours a day managing servers, I might feel more solid about simply cobbling together an assortment of parts from an assortment of corpses (as I do for linux and windows machines) but this is another objective, and I don't feel that the side effects of my daily experience is enough to answer the questions sufficiently. So, as I said... I think a reference design or two would be a Very Positive Thing for the opensolaris and zfs community. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
