Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
For those who are interested in some of the options out there. DIY DAS: Supermicro 36 bay case - $1800 Promise 16 bay JBOD VTrak J610sD - $3700 Promise VTE610sD - $7500 (SAS attached head unit with onboard raid controllers, takes JBOD expansion) The following apply to 1TB SATA drive configurations, dual controllers under federal pricing: HP MD600 - $722/TB DELL MD1000 - $680/TB (780/TB w/ nearline-SAS) Sun J4500 - $1014/TB (federal discount is a mere 13%, not 40) NexSan SATAbeast - $1400/TB (FC attached dual-head) Of the we force you to buy our overinflated drives camp, Dell is the cheapest but also the most inefficient by far on power/space. The HP puts 70 disks in 4U. NexSan 42, and Sun 48. The clear winner here is HP. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, 23 Feb 2010, matthew patton wrote: Of the we force you to buy our overinflated drives camp, Dell is the cheapest but also the most inefficient by far on power/space. The HP puts 70 disks in 4U. NexSan 42, and Sun 48. The clear winner here is HP. What is the performance like with HP? Is there a loss of bandwidth or reliability due to their approach? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced
On Tue, Feb 9, 2010 at 12:55 PM, matthew patton patto...@yahoo.com wrote: . snip Enter the J4500, 48 drives in 4U, what looks to be solid engineering, and redundancy in all the right places. An empty chassis at $3000 is totally justifiable. Maybe as high as $4000. In comparison a naked Dell MD1000 is $2000. If you do the subtraction from SUN's claimed breakthru pricing of $1/GB, the chassis cost works out to $4000. I can live with that. Now look up the price for 24TB and it's 28 freaking thousand! There's your first mistake. You're probably eligible for a very nice Federal Systems discount. My *guess* would be about 40%. . snip ... Regards, -- Al Hopper Logical Approach Inc,Plano,TX a...@logical-approach.com Voice: 972.379.2133 Timezone: US CDT OpenSolaris Governing Board (OGB) Member - Apr 2005 to Mar 2007 http://www.opensolaris.org/os/community/ogb/ogb_2005-2007/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced
On Fri, Feb 12, 2010 at 12:11 PM, Al Hopper a...@logical-approach.com wrote: There's your first mistake. You're probably eligible for a very nice Federal Systems discount. My *guess* would be about 40%. Promise JBOD and similar systems are often the only affordable choice for those of us who can't get sweetheart discounts, don't work at billion dollar corporations, or aren't bankrolled by the Federal Leviathan. Daniel Bakken Systems Administrator Economic Modeling Specialists Inc 1187 Alturas Drive Moscow, Idaho 83843 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced
On Tue, Feb 09, 2010 at 03:44:02PM -0600, Wes Felter wrote: Have you considered Promise JBODs? They officially support bring-your-own-drives. Have you used these yourself, Wes? I've been considering it, but I talked to a colleague at another institution who had some really awful tales to tell about promise FC arrays. They were clearly not ready for prime time. OTOH a SAS jbod is a lot less complicated. danno -- Dan Pritts, Sr. Systems Engineer Internet2 office: +1-734-352-4953 | mobile: +1-734-834-7224 Internet2 Spring Member Meeting April 26-28, 2010 - Arlington, Virginia http://events.internet2.edu/2010/spring-mm/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced
On Thu, Feb 11, 2010 at 2:46 PM, Dan Pritts da...@internet2.edu wrote: I've been considering it, but I talked to a colleague at another institution who had some really awful tales to tell about promise FC arrays. They were clearly not ready for prime time. OTOH a SAS jbod is a lot less complicated. We have used two Promise Vtrak m500f fibre channel arrays in heavy I/O applications for several years. They don't always handle failing disks gracefully-- sometimes requiring a hard reboot to recover. This is partly due to crappy disks with weird failure modes. But a RAID system should never require a reboot to recover from a single disk failure. That defeats the whole purpose of RAID, which is supposed to survive disk failures through redundancy. However, Promise iSCSI and JBOD systems (we own one of each) are more stable. We use them with Linux (XFS) and OpenSolaris (ZFS), and haven't any experienced problems to date. Their JBOD systems, when filled with Western Digital RE3 disks, are an extremely reliable, low-cost, high performance ZFS storage solution. Daniel Bakken Systems Administrator Economic Modeling Specialists Inc 1187 Alturas Drive Moscow, Idaho 83843 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
Eric D. Mudama edmud...@bounceswoosh.org writes: On Tue, Feb 9 at 2:36, Kjetil Torgrim Homme wrote: no one is selling disk brackets without disks. not Dell, not EMC, not NetApp, not IBM, not HP, not Fujitsu, ... http://discountechnology.com/Products/SCSI-Hard-Drive-Caddies-Trays very nice, thanks. unfortunately it probably won't last: [http://lists.us.dell.com/pipermail/linux-poweredge/2010-February/041335.html]: | | In the case of Dell's PERC RAID controllers, we began informing | customers when a non-Dell drive was detected with the introduction of | PERC5 RAID controllers in early 2006. With the introduction of the | PERC H700/H800 controllers, we began enabling only the use of Dell | qualified drives. -- Kjetil T. Homme Redpill Linpro AS - Changing the game ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced
matthew patton patto...@yahoo.com wrote: It might help people to understand how ridiculous they sound going on and on about buying a premium storage appliance without any storage. Since I started this, let me explain to those who can't begin to understand why I proposed something so stupid. At work (branch of a federal gov't big-5 Department) I need 40TB but have next to nothing in budget. (For some reason all you damn citizens think you're entitled to keep most of your paychecks to yourself instead living off what I decide to give you in foodstamps and rent-controled housing.) Therefore, I can't afford let alone justify the preposterous premium demanded by enterprise EMC/Sun/IBM/NetApp. I can really use dedup, (integrity would be nice), and reasonable rack and power footprint since I'm out of that too. Did you ever try to get an offer for a NetApp machine in the same range as a SunFire 4540? I did and the price was twice as much than with the Sun. With the Sun I did even have a 3 year maintenance contract included. This of course was in both cases with disks. It seems to be wrong to look at single component prices if you are looking for a solution. It is also wrong to cry about the price of single components in a solution if you did already decide to build a system from single components. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 9, 2010 at 2:04 AM, Thomas Burgess wonsl...@gmail.com wrote: On Mon, Feb 08, 2010 at 09:33:12PM -0500, Thomas Burgess wrote: This is a far cry from an apples to apples comparison though. As much as I'm no fan of Apple, it's a pity they dropped ZFS because that would have brought considerable attention to the opportunity of marketing and offering zfs-suitable hardware to the consumer arena. Port-multiplier boxes already seem to be targetted most at the Apple crowd, even it's only in hope of scoring a better margin. Otherwise, bad analogies, whether about cars or fruit, don't help. It might help people to understand how ridiculous they sound going on and on about buying a premium storage appliance without any storage. I think the car analogy was dead on. You don't have to agree with a vendors practices to understand them. If you have a more fitting analogy, then by all means lets hear it. Dell joins the party: http://lists.us.dell.com/pipermail/linux-poweredge/2010-February/041335.html -- Giovanni ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced
On Feb 9, 2010, at 1:55 PM, matthew patton patto...@yahoo.com wrote: The cheapest solution out there that isn't a Supermicro-like server chassis, is DAS in the form of HP or Dell MD-series which top out at 15 or 16 3 drives. I can only chain 3 units per SAS port off a HBA in either case. The new Dell MD11XX series is 24 2.5 drives and you can chain 3 of them together off a single controller. If your drives are dual ported you can use both HBA ports for redundant paths. -Ross ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced
On Wed, Feb 10, 2010 at 12:37:46PM -0500, rwali...@washdcmail.com wrote: I don't disagree with any of the facts you list, but I don't think the alternatives are fully described by Sun vs. much cheaper retail parts. We face exactly this same decision with buying RAM for our servers (maybe more so since it is probably even more difficult to argue there is a difference in RAM chip quality when the same manufacturer's part is sourced from Sun vs. elsewhere). The thing we consider is how we'll live with a failure. [..] So we really don't view it as RAM vs. RAM comparison. It's more RAM + easy warranty service vs. RAM + more difficult warranty service. Yes. This is a great example of exactly the problem with disks. With RAM, the choice is clear and available (if not always easy). You decide how much the operational assurance is worth to you, and how/when you want to pay for the potential downtime and hassle of failure. Once you've made your decision, you act, and the parts work together until and unless some fault arises, at which point the cost implications of your decision come into play. Sun (or HP or IBM, etc) don't use some proprietary RAM socket adapter and skewed pricing model, in an attempt to force or bias the decision. If you want a fully supported product down the road where everything that goes wrong is Sun's fault, then buy that from Sun. Sure, that's never been in doubt. People (not even Sun salespeople) are trying to upsell one component, based on the additional benefits of the overall package. The benefits, and even the convincing value proposition to some other customer, of the package are not in dispute. However, that's irrelevant to someone who is not that customer; the upsell is not attractive to someone who does not want, or cannot afford, the extras. If you want something much cheaper but where you will need to negotiate future fixes, assemble it from various sources. I think the OP reached this conclusion long ago, before the original post. He just wants to buy a disk tray, and however much he or anyone else might wish that he could buy more, he's constrained otherwise, including having to accept the possible implications of later issues. The frustration here is that Sun is not a viable source; he's already tightly constrained, and this is a further unwelcome constraint. Ordinarily, that would be fine and he'd take his money elsewhere. In this case, there's an audience here of interested people, Sun customers and employees, who might either know of a way to work around or remove that constraint, or take note of the market feedback about uncatered demand. Apparently not. In this case, there are also not many other sources for shelves with similar density. If that's really what makes the component premium, so be it. I hope other vendors are taking notice of the opportunity Sun is choosing (for reasons however valid) to ignore. -- Dan. pgpeRYt1LZGpG.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced
It might help people to understand how ridiculous they sound going on and on about buying a premium storage appliance without any storage. Since I started this, let me explain to those who can't begin to understand why I proposed something so stupid. At work (branch of a federal gov't big-5 Department) I need 40TB but have next to nothing in budget. (For some reason all you damn citizens think you're entitled to keep most of your paychecks to yourself instead living off what I decide to give you in foodstamps and rent-controled housing.) Therefore, I can't afford let alone justify the preposterous premium demanded by enterprise EMC/Sun/IBM/NetApp. I can really use dedup, (integrity would be nice), and reasonable rack and power footprint since I'm out of that too. I can't exactly march into my boss' office and propose that we build my own at-home special which is 16 WD RE2/3 drives $(60) in a $70 case, $100 power supply, four 4-in-3 modules ($30) and a Chenbro SAS expander ($250) now can I... Aside: I find it laughable for anyone to claim a J4500 is premium anything. IBM DS800, EMC Symetrix, NetApp FAS5xxx, sure. But a glorified JBOD enclosure? Put down the damn cool-aid! The cheapest solution out there that isn't a Supermicro-like server chassis, is DAS in the form of HP or Dell MD-series which top out at 15 or 16 3 drives. I can only chain 3 units per SAS port off a HBA in either case. Enter the J4500, 48 drives in 4U, what looks to be solid engineering, and redundancy in all the right places. An empty chassis at $3000 is totally justifiable. Maybe as high as $4000. In comparison a naked Dell MD1000 is $2000. If you do the subtraction from SUN's claimed breakthru pricing of $1/GB, the chassis cost works out to $4000. I can live with that. Now look up the price for 24TB and it's 28 freaking thousand! I can buy 24TB worth of good SATA drives for $5000 all day long and twice on Sunday. (http://www.cdw.com/shop/products/default.aspx?EDC=1531954) I can buy Dell trays from DELL themselves let alone a bevy of 3rd parties for as low as $12 each. SUN's are like $25 on the aftermarket and much harder to come by. So, does my question make sense now? My only play is Dell at this time. I was TRYING to see if the SUN route could be made possible since it would be the better solution. But I guess I'm not enterprisy enough, ie with so much more money than brains for the likes of SUN to give 2 (@Rts. Dell and HP *WANT* my business by pricing things where I can reasonably get to them. A fully-qualified 500GB SATA drive from DELL is $300, so a 3x multiplier. Still quite a bit more than I think is warranted, but SUN wants 5x? Nothing SUN makes is so much better than DELL/HP, indeed they are essentially indistinguishable that they can get away with pretending to be EMC. Is it any wonder they failed? Spare me the bit about how there is so much expensive and complex engineering invested in something as stupid straight-forward as the J4500 or in qualifying drive firmware. I've worked on qualifying SUN, IBM, and other storage products (firmware, hardware, OS drivers) some of which were of our own designs that the big names simply slapped their label on. They outsource this stuff to a certain company just west of Chicago on route 355 (among others). I know what I was paid. We had 4 guys in that lab and as a overhead/GB we were no bigger a pimple on a gnat's hind end. There is no mysterious voodoo in storage enclosure design that requires an army of highly trained PhD's months to figure out. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On 2/9/10 12:03 PM +1100 Daniel Carosone wrote: Snorcle wants to sell hardware. LOL ... snorcle But apparently they don't. Have you seen the new website? Seems like a blatant attempt to kill the hardware business to me. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On 9-Feb-10, at 2:02 PM, Frank Cusack wrote: On 2/9/10 12:03 PM +1100 Daniel Carosone wrote: Snorcle wants to sell hardware. LOL ... snorcle But apparently they don't. Have you seen the new website? Seems like a blatant attempt to kill the hardware business to me. That's very sad. I love, love to spec the rebooted Bechtolsheim hardware designs. --Toby ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced
matthew patton wrote: It might help people to understand how ridiculous they sound going on and on about buying a premium storage appliance without any storage. Since I started this, let me explain to those who can't begin to understand why I proposed something so stupid. At work (branch of a federal gov't big-5 Department) I need 40TB but have next to nothing in budget. (For some reason all you damn citizens think you're entitled to keep most of your paychecks to yourself instead living off what I decide to give you in foodstamps and rent-controled housing.) Therefore, I can't afford let alone justify the preposterous premium demanded by enterprise EMC/Sun/IBM/NetApp. I can really use dedup, (integrity would be nice), and reasonable rack and power footprint since I'm out of that too. Frankly, that's your budget problem, and has nothing to do with Sun/IBM/HP/etc. Management needs to understand the actual cost (as decided by the open market) for a given service. They should be told NO if they demand something unreasonable, and educated as to the current market price tradeoffs of different solutions. I can't exactly march into my boss' office and propose that we build my own at-home special which is 16 WD RE2/3 drives $(60) in a $70 case, $100 power supply, four 4-in-3 modules ($30) and a Chenbro SAS expander ($250) now can I... Aside: I find it laughable for anyone to claim a J4500 is premium anything. IBM DS800, EMC Symetrix, NetApp FAS5xxx, sure. But a glorified JBOD enclosure? Put down the damn cool-aid! Premium has nothing to do with absolute cost or size. It's about a superior product, at whatever price point is being discussed. The cheapest solution out there that isn't a Supermicro-like server chassis, is DAS in the form of HP or Dell MD-series which top out at 15 or 16 3 drives. I can only chain 3 units per SAS port off a HBA in either case. Enter the J4500, 48 drives in 4U, what looks to be solid engineering, and redundancy in all the right places. An empty chassis at $3000 is totally justifiable. Maybe as high as $4000. In comparison a naked Dell MD1000 is $2000. If you do the subtraction from SUN's claimed breakthru pricing of $1/GB, the chassis cost works out to $4000. I can live with that. You just fell into the trap of component pricing. You're buying a package, not parts. You can't just disassemble the package and declare that part costs X. Solutions are more than the sum of their mechanical parts. Now look up the price for 24TB and it's 28 freaking thousand! I can buy 24TB worth of good SATA drives for $5000 all day long and twice on Sunday. (http://www.cdw.com/shop/products/default.aspx?EDC=1531954) I can buy Dell trays from DELL themselves let alone a bevy of 3rd parties for as low as $12 each. SUN's are like $25 on the aftermarket and much harder to come by. So, does my question make sense now? My only play is Dell at this time. I was TRYING to see if the SUN route could be made possible since it would be the better solution. But I guess I'm not enterprisy enough, ie with so much more money than brains for the likes of SUN to give 2 (@Rts. Dell and HP *WANT* my business by pricing things where I can reasonably get to them. A fully-qualified 500GB SATA drive from DELL is $300, so a 3x multiplier. Still quite a bit more than I think is warranted, but SUN wants 5x? Nothing SUN makes is so much better than DELL/HP, indeed they are essentially indistinguishable that they can get away with pretending to be EMC. Is it any wonder they failed? See below. Don't directly compare List price with Actual price. Spare me the bit about how there is so much expensive and complex engineering invested in something as stupid straight-forward as the J4500 or in qualifying drive firmware. I've worked on qualifying SUN, IBM, and other storage products (firmware, hardware, OS drivers) some of which were of our own designs that the big names simply slapped their label on. They outsource this stuff to a certain company just west of Chicago on route 355 (among others). I know what I was paid. We had 4 guys in that lab and as a overhead/GB we were no bigger a pimple on a gnat's hind end. There is no mysterious voodoo in storage enclosure design that requires an army of highly trained PhD's months to figure out So why are there so many problems with consumer-grade drives in 3rd-party chassis? Obviously, the Name Brand folks are doing some sort of value-add. You are paying for a service you want: reliability, consistency, and support. Can you provide the same level of these by assembling it yourself? Are you sure? If you don't care about these things, then roll-it-yourself. But make sure Management understands that they are getting a lower level of quality for their $. Frankly, I've spent a fair amount of time talking to HUGE storage users (Google, various US Gov't National Labs), and the only way for
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced
On Tue, Feb 9, 2010 at 1:38 PM, Erik Trimble erik.trim...@sun.com wrote: Bottom line here: if someone comes along and provides the same level of service for a better price, the market will flock to them. Or if the market decides the current level of service is unnecessary, it will move to vendors providing the sufficient level of service at the new price point. But for now, there is no indication that the current pricing/service level models aren't correct. You may /want/ and /think/ that a BMW 528i should cost $30k (I mean, it's not really any different than an Accord, right?), but the market has said no, it's $45k. Sorry, the market is correct. From my perspective as an IT pro, Sun is selling BMW's at $200k. It's a great car, but a Mercedes is half the cost. Most hardware consumers have already flocked to the competition, which explains Sun's staggering losses and the Oracle buyout. We can't afford to pay tens of thousands of dollars extra for bragging rights over IBM, HP, or Dell shops. Daniel Bakken Systems Administrator Economic Modeling Specialists Inc Moscow, Idaho ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Feb 8, 2010, at 20:03, Daniel Carosone wrote: Snorcle wants to sell hardware. Larry Ellison wants Oracle to be a systems company, a la T. J. Watson Jr.'s IBM and Cisco: We are not going into the hardware business. We have no interest in the hardware business. We have a deep interest in the systems business. http://www.youtube.com/watch?v=rmrxN3GWHpM#t=26m40s ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 9, 2010 at 2:39 PM, Toby Thain t...@telegraphics.com.au wrote: On 9-Feb-10, at 2:02 PM, Frank Cusack wrote: On 2/9/10 12:03 PM +1100 Daniel Carosone wrote: Snorcle wants to sell hardware. LOL ... snorcle But apparently they don't. Have you seen the new website? Seems like a blatant attempt to kill the hardware business to me. That's very sad. I love, love to spec the rebooted Bechtolsheim hardware designs. --Toby How do you figure that? There are 5 columns on the front page: Database Middleware Applications Server and Storage Systems Industry How much more focus were you hoping for beyond front page status? Were you expecting them to remove all references to that little database thing that their entire business is founded upon? --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced
On Tue, 9 Feb 2010, Daniel Bakken wrote: From my perspective as an IT pro, Sun is selling BMW's at $200k. It's a great car, but a Mercedes is half the cost. Most hardware consumers have already flocked to the competition, which explains Sun's staggering losses and the Oracle buyout. We can't afford to pay tens I am glad to see that you are a pro. Sun's staggering losses are due to the level of sales not being in line with the large number of product offerings combined with a dramatic two year down-turn in the economy that Sun management had not anticipated. Some of us (outside of Moscow) are keenly aware of the economic down-turn. There were also grave errors in judgement from certain people in Sun management. The only winner in the server-wars has been IBM. All the other big players have been losing. Even Dell has been losing. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On 2/9/10 5:19 PM -0600 Tim Cook wrote: On Tue, Feb 9, 2010 at 2:39 PM, Toby Thain t...@telegraphics.com.au wrote: On 9-Feb-10, at 2:02 PM, Frank Cusack wrote: On 2/9/10 12:03 PM +1100 Daniel Carosone wrote: Snorcle wants to sell hardware. LOL ... snorcle But apparently they don't. Have you seen the new website? Seems like a blatant attempt to kill the hardware business to me. That's very sad. I love, love to spec the rebooted Bechtolsheim hardware designs. --Toby How do you figure that? There are 5 columns on the front page: Database Middleware Applications Server and Storage Systems Industry How much more focus were you hoping for beyond front page status? Were you expecting them to remove all references to that little database thing that their entire business is founded upon? I assume you are responding to my comment and not Toby's. Did you try to drill down past the front page? To look at the specs for ANY server? I just thought it was much more difficult to look at and compare specs than it was on Sun's site. Turns out you can go to shop.sun.com and find the same tab interface as before though. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 9, 2010 at 7:37 PM, Frank Cusack frank+lists/z...@linetwo.netwrote: I assume you are responding to my comment and not Toby's. Did you try to drill down past the front page? To look at the specs for ANY server? I just thought it was much more difficult to look at and compare specs than it was on Sun's site. Turns out you can go to shop.sun.com and find the same tab interface as before though. -frank They've been merged for less than a week. I expect Oracle still has plenty of work to do on the site. Day 1 I found about 20 broken links, and it's gotten much better since then. I'd imagine there's only so much integration they could do ahead of time. --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced
Have you considered Promise JBODs? They officially support bring-your-own-drives. Wes Felter ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 9 at 2:36, Kjetil Torgrim Homme wrote: Daniel Carosone d...@geek.com.au writes: In that context, I haven't seen an answer, just a conclusion: - All else is not equal, so I give my money to some other hardware manufacturer, and get frustrated that Sun won't let me buy the parts I could use effectively and comfortably. no one is selling disk brackets without disks. not Dell, not EMC, not NetApp, not IBM, not HP, not Fujitsu, ... http://discountechnology.com/Products/SCSI-Hard-Drive-Caddies-Trays -- Eric D. Mudama edmud...@mail.bounceswoosh.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
This is a long thread, with lots of interesting and valid observations about the organisation of the industry, the segmentation of the market, getting what you pay for vs paying for what you want, etc. I don't really find within, however, an answer to the original question, at least the way I read it. Perhaps that's the issue - that the question was asked without enough specifics and context, and so everyone has their own interpretation and their own answer to their own question. Remembering that a lot of this was branded and marketed as open storage, the desire to mix and match components is not only natural, a clear expectation has been set that it should be possible and easy and open. That's not to say that you can expect to have your cake and eat it too. Certain combinations and permutations are more qualified, tested, supported and therefore expensive than others; these characteristics are part of what you should be able to mix and match, understanding the full implications of each tradeoff choice. Snorcle wants to sell hardware. Sure, they want even more to sell a complete hardware and annual maintenance package with annuity revenue over multiple years with high markups. Some people are simply not customers for all of that, but might still be customers for the hardware. Especially these days, it seems they still would want to sell the hardware even when they can't sell the rest of the package. I read the following context between the lines of the original question: - I have or can source disk drives I'm comfortable using. - I understand that I'm not paying for, and can't expect, commercial support for whatever final combination I wind up with. - I am comfortable relying on standards and specifications for interoperability, enough that it's unlikely I'll have to get into deep debugging for problems. At least, I'm unwilling or unable to pay high premiums ahead of time in the hope of avoiding potential high costs for later problems. - The J4500 seems like nice hardware, and I know that at least it isn't likely to change unexpectedly to some different chipset not recognised by opensolaris, just before purchase. This would give me some comfort. - I like Sun, and am thankful for ZFS, and since I have to buy hardware anyway I'll look at what Sun offers. Perhaps I would even prefer to buy the Sun offering, all else being approximately equal. This would also give me some comfort. In that context, I haven't seen an answer, just a conclusion: - All else is not equal, so I give my money to some other hardware manufacturer, and get frustrated that Sun won't let me buy the parts I could use effectively and comfortably. -- Dan. pgpvZdqvo577p.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
Daniel Carosone d...@geek.com.au writes: In that context, I haven't seen an answer, just a conclusion: - All else is not equal, so I give my money to some other hardware manufacturer, and get frustrated that Sun won't let me buy the parts I could use effectively and comfortably. no one is selling disk brackets without disks. not Dell, not EMC, not NetApp, not IBM, not HP, not Fujitsu, ... -- Kjetil T. Homme Redpill Linpro AS - Changing the game ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
Just like i said way earlier, The entire idea is like asking to buy a Ferrari without the aluminum wheels they sell because you think they are charging too much for them, after all, aluminum is cheap. It's just not done that way. There are OTHER OPTIONS for people who can't afford it. You really can't have both. You can either afford it or you can't. On Mon, Feb 8, 2010 at 8:36 PM, Kjetil Torgrim Homme kjeti...@linpro.nowrote: Daniel Carosone d...@geek.com.au writes: In that context, I haven't seen an answer, just a conclusion: - All else is not equal, so I give my money to some other hardware manufacturer, and get frustrated that Sun won't let me buy the parts I could use effectively and comfortably. no one is selling disk brackets without disks. not Dell, not EMC, not NetApp, not IBM, not HP, not Fujitsu, ... -- Kjetil T. Homme Redpill Linpro AS - Changing the game ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Monday, February 8, 2010, Kjetil Torgrim Homme kjeti...@linpro.no wrote: Daniel Carosone d...@geek.com.au writes: In that context, I haven't seen an answer, just a conclusion: - All else is not equal, so I give my money to some other hardware manufacturer, and get frustrated that Sun won't let me buy the parts I could use effectively and comfortably. no one is selling disk brackets without disks. not Dell, not EMC, not NetApp, not IBM, not HP, not Fujitsu, ... -- Kjetil T. Homme Redpill Linpro AS - Changing the game ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss Although I am in full support of what sun is doing, to play devils advocate: supermicro is. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Mon, Feb 8, 2010 at 9:13 PM, Tim Cook t...@cook.ms wrote: On Monday, February 8, 2010, Kjetil Torgrim Homme kjeti...@linpro.no wrote: Daniel Carosone d...@geek.com.au writes: In that context, I haven't seen an answer, just a conclusion: - All else is not equal, so I give my money to some other hardware manufacturer, and get frustrated that Sun won't let me buy the parts I could use effectively and comfortably. no one is selling disk brackets without disks. not Dell, not EMC, not NetApp, not IBM, not HP, not Fujitsu, ... -- Kjetil T. Homme Redpill Linpro AS - Changing the game ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss Although I am in full support of what sun is doing, to play devils advocate: supermicro is. This is a far cry from an apples to apples comparison though. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
Tim Cook wrote: On Monday, February 8, 2010, Kjetil Torgrim Homme kjeti...@linpro.no wrote: Daniel Carosone d...@geek.com.au writes: In that context, I haven't seen an answer, just a conclusion: - All else is not equal, so I give my money to some other hardware manufacturer, and get frustrated that Sun won't let me buy the parts I could use effectively and comfortably. no one is selling disk brackets without disks. not Dell, not EMC, not NetApp, not IBM, not HP, not Fujitsu, .. Although I am in full support of what sun is doing, to play devils advocate: supermicro is. True, but they're not a systems vendor. They're a parts OEM. You might be able to get larger integrated solutions from them (motherboard/chassis together), but you'll have to buy the rest of the parts yourself (or go to a system integrator to build a system for you). No brand-name system provider allows you to purchase empty disk sleds. About the best I can come up with on that is that eBay often has a selection of various brackets, usually from 3rd-parties which copy the Brand design. In the end, you pay for support and integration testing. Whether it is worth it depends solely on your situation. But don't expect vendors to service all (or even many) niches - they all pick their battles, and if you're not in their zone, it's a huge uphill struggle to get them to add your zone. It's that simple. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
Although I am in full support of what sun is doing, to play devils advocate: supermicro is. They're not the only ones, although the most-often discussed here. Dell will generally sell hardware and warranty and service add-ons in any combination, to anyone willing and capable of figuring out what to order, although that effort might well be more than the result is worth. Many of the others have issues in being further from the retail market, such as support divisions that are only set up to deal with large enterprise full-service customers. Nothing wrong with that if it suits them. Of the others listed, Sun is the one promoting change and the benefits of ZFS and open storage, and which has the opportunity to make sales to an interested community. They, too, are entitled to exclude themselves from sales they don't want, for whatever reason they or their new masters choose. On Mon, Feb 08, 2010 at 09:33:12PM -0500, Thomas Burgess wrote: This is a far cry from an apples to apples comparison though. As much as I'm no fan of Apple, it's a pity they dropped ZFS because that would have brought considerable attention to the opportunity of marketing and offering zfs-suitable hardware to the consumer arena. Port-multiplier boxes already seem to be targetted most at the Apple crowd, even it's only in hope of scoring a better margin. Otherwise, bad analogies, whether about cars or fruit, don't help. -- Dan. pgpFh9oakiUNu.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Mon, Feb 08, 2010 at 09:33:12PM -0500, Thomas Burgess wrote: This is a far cry from an apples to apples comparison though. As much as I'm no fan of Apple, it's a pity they dropped ZFS because that would have brought considerable attention to the opportunity of marketing and offering zfs-suitable hardware to the consumer arena. Port-multiplier boxes already seem to be targetted most at the Apple crowd, even it's only in hope of scoring a better margin. Otherwise, bad analogies, whether about cars or fruit, don't help. It might help people to understand how ridiculous they sound going on and on about buying a premium storage appliance without any storage. I think the car analogy was dead on. You don't have to agree with a vendors practices to understand them. If you have a more fitting analogy, then by all means lets hear it. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Sat, Feb 6, 2010 at 9:51 AM, Kjetil Torgrim Homme kjeti...@linpro.nowrote: matthew patton patto...@yahoo.com writes: true. but I buy a Ferrari for the engine and bodywork and chassis engineering. It is totally criminal what Sun/EMC/Dell/Netapp do charging customers 10x the open-market rate for standard drives. A RE3/4 or NS drive is the same damn thing no matter if I buy it from ebay or my local distributor. Dell/Sun/Netapp buy drives by the container load. Oh sure, I don't mind paying an extra couple pennies/GB for all the strenuous efforts the vendors spend on firmware verification (HA!). I don't know what the J4500 drive sled contains, but for the J4200 and J4400 they need to include quite a bit of circuitry to handle SAS protocol all the way, for multipathing and to be able to accept a mix of SAS and SATA drives. it's not just a piece of sheet metal, some plastic and a LED. the pricing does look strange, and I think it would be better to raise the price of the enclosure (which is silly cheap when empty IMHO) and reduce the drive prices somewhat. but that's just psychology, and doesn't really matter for total cost. -- Kjetil T. Homme Redpill Linpro AS - Changing the game Why exactly would that be better? Then it's a high cost of entry. What if an SMB only needs 6 drives day one? Why charge them an arm and a leg for the enclosure, and nothing for the drives? Again, the idea is that you're charging based on capacity. Generally speaking, an entity that needs tons and tons of storage has the money to pay for it. Home users ripping legitimately or pirating illegitimately movie's and music excluded. --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
Tim Cook t...@cook.ms writes: Kjetil Torgrim Homme kjeti...@linpro.no wrote: I don't know what the J4500 drive sled contains, but for the J4200 and J4400 they need to include quite a bit of circuitry to handle SAS protocol all the way, for multipathing and to be able to accept a mix of SAS and SATA drives. it's not just a piece of sheet metal, some plastic and a LED. the pricing does look strange, and I think it would be better to raise the price of the enclosure (which is silly cheap when empty IMHO) and reduce the drive prices somewhat. but that's just psychology, and doesn't really matter for total cost. Why exactly would that be better? people are looking at the price list and seeing that the J4200 costs 22550 NOK [1], while one sixpack of 2TB SATA disks to go with it costs 82500 NOK. on the other hand you could get six 2TB SATA disks (Ultrastars) from your friendly neighbourhood shop for 14370 NOK (7700 NOK for six Deskstars). and to add insult to injury, the neighbourhood shop offers five years warranty (required by Norwegian consumer laws), compared to Sun's three years... everyone knows the price of a harddisk since they buy them for their home computers. do they know the price of a disk storage array? not so well. yes, it's a matter of perception for the buyer, but perception can matter. Then it's a high cost of entry. What if an SMB only needs 6 drives day one? Why charge them an arm and a leg for the enclosure, and nothing for the drives? Again, the idea is that you're charging based on capacity. see my numbers above. the chassis itself is just 12% of the cost (22% when half full). some middle ground could be found. anyway, we're buying these systems and are very happy with them. when disks fail, Sun replace them very expediently and with a minimum of fuss. [1] all prices include VAT to simplify comparison. prices are current from shop.sun.com and komplett.no. Sun list prices are subject to haggling. -- Kjetil T. Homme Redpill Linpro AS - Changing the game ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 2, 2010 at 1:58 PM, Tim Cook t...@cook.ms wrote: It's called spreading the costs around. Would you really rather pay 10x the price on everything else besides the drives? This is essentially Sun's way of tiered pricing. Rather than charge you a software fee based on how much storage you have, they increase the price of the drives. Seems fairly reasonable to me... it gives a low point of entry for people that don't need that much storage without using ridiculous capacity based licensing on software. Smells like the Razor and Blades business model [1]. I think the industry is in a sad state when you buy enterprise-level drives and they don't work as expected (see that thread about TLER settings on WD enterprise drives) that you have to spend extra on drives that got reviewed by a third-party (Sun/EMC/etc). Just shows how bad the disk vendors are. I would be curious to know how the internal process of testing these drives work at Sun/EMC/etc when they find bugs and performance problems. Do they have access to the firmware's source code to fix it ? Or do they report the bugs back to Seagate/WD and they provide a new firmware for tests ? Do those bugs get fixed in other drives that Seagate/WD sells ? For me it's just hard to objectively point out the differences between Seagate's enterprise drives and the ones provided by Sun except that they were tested more. 1 - http://en.wikipedia.org/wiki/Freebie_marketing -- Giovanni ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 2, 2010 at 9:07 PM, Marc Nicholas geekyth...@gmail.com wrote: I believe magical unicorn controllers and drives are both bug-free and 100% spec compliant. The leprichorns sell them if you're trying to find them ;) Well, perfect and bug free sure don't exist in our industry. The problem is that we see disk firmwares that are stupidly flawed and the revisions that get released aren't making it better. Otherwise people looking for quality would not have to spend extra on third-party reviewed drives from storage vendors. It's all too convenient how the industry is organized. That is, for disk and storage vendors. Not customers. -- Giovanni ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On 2/8/10 12:49 AM -0200 Giovanni Tirloni wrote: I think the industry is in a sad state when you buy enterprise-level drives and they don't work as expected (see that thread about TLER settings on WD enterprise drives) that you have to spend extra on drives that got reviewed by a third-party (Sun/EMC/etc). Just shows how bad the disk vendors are. Or how tough the hard drive market is. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
matthew patton patto...@yahoo.com writes: true. but I buy a Ferrari for the engine and bodywork and chassis engineering. It is totally criminal what Sun/EMC/Dell/Netapp do charging customers 10x the open-market rate for standard drives. A RE3/4 or NS drive is the same damn thing no matter if I buy it from ebay or my local distributor. Dell/Sun/Netapp buy drives by the container load. Oh sure, I don't mind paying an extra couple pennies/GB for all the strenuous efforts the vendors spend on firmware verification (HA!). I don't know what the J4500 drive sled contains, but for the J4200 and J4400 they need to include quite a bit of circuitry to handle SAS protocol all the way, for multipathing and to be able to accept a mix of SAS and SATA drives. it's not just a piece of sheet metal, some plastic and a LED. the pricing does look strange, and I think it would be better to raise the price of the enclosure (which is silly cheap when empty IMHO) and reduce the drive prices somewhat. but that's just psychology, and doesn't really matter for total cost. -- Kjetil T. Homme Redpill Linpro AS - Changing the game ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On 2/6/10 4:51 PM +0100 Kjetil Torgrim Homme wrote: the pricing does look strange, and I think it would be better to raise the price of the enclosure (which is silly cheap when empty IMHO) and reduce the drive prices somewhat. but that's just psychology, and doesn't really matter for total cost. better for whom? :) if the total price is the same, it's better (for Sun) to charge as much for the razor blades as the market will bear. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Wed, Feb 03, 2010 at 03:02:21PM -0800, Brandon High wrote: Another solution, for a true DIY x4500: BackBlaze has schematics for the 45 drive chassis that they designed available on their website. http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ Someone brought it up on the list a few months ago (which is how I know about it) and there was some interesting discussion at that time. IIRC the consensus was that the vibration dampening was inadequate and the interfaces oversubscribed and the disks being not nearline too unreliable, but I might be misremembering. I'm still happy with my 16x WD RE4 drives (linux mdraid RAID 10, CentOS, Oracle, no zfs). Supermicro does 36x drive chassis now http://www.supermicro.com/products/chassis/4U/?chs=847 so budget DIY for zfs is about 72 TByte raw storage with 2 TByte nearline SATA drives. I've had trouble finding internal 2x 2.5 in one 3.5 SSD mounts from Supermicro for hybrid zfs, but no doubt one could improvise something from the usual ricer supplies. On smaller scale http://www.supermicro.com/products/chassis/2U/?chs=216 works well with 2.5 Intel SSDs and VelociRaptors. I hope to be able to use one for a hybrid zfs iSCSI target for VMWare, probably with 10 GBit Ethernet. There's no way I would use something like this for most installs, but there is definitely some use. Now that opensolaris supports sata pmp, you could use a similar chassis for a zfs pool. -- Eugen* Leitl a href=http://leitl.org;leitl/a http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
This seems to miss the point. I presented an argument for why I think the qualified drives are a huge profit-center, not just making a reasonable profit on the work of qualification. In general, I'd much rather pay reasonable costs for each piece, rather than weird costs artificially shoved around to make things come out some strange way somebody favors. Thats easy for you to SAY but the fact of the matter is, in the REAL world it doesn't hold water. Almost all types of products have this exact same model. It's around because it works. It's true for cheetos, cars, and hard drives. Saying you'd rather pay a fair price for things down the line and actually DOING it are 2 different things. I think if all companies followed that mentality you'd see that you'd end up paying more, not less. It works great for me personally -- I'm using the software with other people's hardware, for free. But why should people who need a lot of storage pay proportionally more? I don't get that, that's grossly wrong. -- They don't HAVE to pay more. It's about being realistic. People who buy sun hardware are doing so for more reasons that to have a lot of storage The cost per gb isn't the only factor that should be thought of. For raw storage its certainly cheaper to buy a cheap case, throw in 20-24 1-2tb consumer grade drives in 2-3 raidz2's and have a reasonable level of fault tollerance, but that TOTALLY misses the point. You don't buy a sunfire x4540 or j45xx just for raw storage. If you want the cheapest raw storage possible you can always go build one of those blackblaze storage pods. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On 2-Feb-10, at 10:11 PM, Marc Nicholas wrote: On Tue, Feb 2, 2010 at 9:52 PM, Toby Thain t...@telegraphics.com.au wrote: On 2-Feb-10, at 1:54 PM, Orvar Korvar wrote: 100% uptime for 20 years? So what makes OpenVMS so much more stable than Unix? What is the difference? The short answer is that uptimes like that are VMS *cluster* uptimes. Individual hosts don't necessarily have that uptime, but the cluster availability is maintained for extremely long periods. You can probably find more discussion of this in comp.os.vms. And the 15MB/sec of I/O throughput on that state-of-the-art cluster is something to write home about? ;) Seriously, as someone alluded to earlier, we're not comparing apples to applies. And a 9000 series VAX Cluster was one of the earlier multi-user systems I worked on for reference ;) Making that kind of stuff work with modern expectations and tolerances is a whole new kettle of fish... OpenVMS runs on modern gear (Itanium). --Toby -marc ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Mon, Feb 1, 2010 at 8:58 PM, matthew patton patto...@yahoo.com wrote: what with the home NAS conversations, what's the trick to buy a J4500 without any drives? SUN like every other enterprise storage vendor thinks it's ok to rape their customers and I for one, am not interested in paying 10x for a silly SATA hard drive. Another solution, for a true DIY x4500: BackBlaze has schematics for the 45 drive chassis that they designed available on their website. http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ Someone brought it up on the list a few months ago (which is how I know about it) and there was some interesting discussion at that time. There's no way I would use something like this for most installs, but there is definitely some use. Now that opensolaris supports sata pmp, you could use a similar chassis for a zfs pool. -B -- Brandon High : bh...@freaks.com For sale: One moral compass, never used. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Wed, February 3, 2010 17:02, Brandon High wrote: On Mon, Feb 1, 2010 at 8:58 PM, matthew patton patto...@yahoo.com wrote: what with the home NAS conversations, what's the trick to buy a J4500 without any drives? SUN like every other enterprise storage vendor thinks it's ok to rape their customers and I for one, am not interested in paying 10x for a silly SATA hard drive. Another solution, for a true DIY x4500: BackBlaze has schematics for the 45 drive chassis that they designed available on their website. http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ I'm just coming up on upgrading my 800GB pool to 1.2TB. I know some home NAS setups with 10 times my capacity, but I don't know how much data they actually have on them. And 10 times my capacity actually fits in my current chassis with modern drives; I'm just still using the 400GB drives I put in originally, and have two more sitting around for the upgrade I'm heading towards. Which is to say that 45 drives is really quite a lot for a HOME NAS. Particularly when you then think about backing up that data. -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Wed, Feb 3, 2010 at 3:13 PM, David Dyer-Bennet d...@dd-b.net wrote: Which is to say that 45 drives is really quite a lot for a HOME NAS. Particularly when you then think about backing up that data. The origin of this thread was how to buy a J4500 (48 drive chassis). One thing that I enjoy about this list (and I'm sure the Sun guys get annoyed about) is the discussion of how to build various small systems for home use. After sitting on the sidelines for a while, I assembled an 8TB server for home. Yeah, 8TB is more than I can backup over my home DSL connection. But it's only got about 2.5TB in use, and most of that is our DVD collection that I've ripped to play on the Popcorn Hour in our living room, or CDs that I've ripped. I'd hate to have to re-rip it all, but I can get it back. The rest is photos and important documents which are copied to a VM instance and backed up offsite via Mozy. I'm considering doing a send/receive of a few volumes to a friend's system (as he will do to mine) to have offsite backups of the pools. It's mostly dependent on him buying more disk. ;-) And for what it's worth, my toying with ZFS and discussing it with coworkers has raised interest in Sun's storage line to replace NetApp at the office. -B -- Brandon High : bh...@freaks.com If violence doesn't solve your problem, you're not using enough of it. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
Brandon High wrote: On Wed, Feb 3, 2010 at 3:13 PM, David Dyer-Bennet d...@dd-b.net wrote: Which is to say that 45 drives is really quite a lot for a HOME NAS. Particularly when you then think about backing up that data. The origin of this thread was how to buy a J4500 (48 drive chassis). One thing that I enjoy about this list (and I'm sure the Sun guys get annoyed about) is the discussion of how to build various small systems for home use. We don't get annoyed at all. What do you think we build to run at home? ;-) After sitting on the sidelines for a while, I assembled an 8TB server for home. Yeah, 8TB is more than I can backup over my home DSL connection. But it's only got about 2.5TB in use, and most of that is our DVD collection that I've ripped to play on the Popcorn Hour in our living room, or CDs that I've ripped. I'd hate to have to re-rip it all, but I can get it back. The rest is photos and important documents which are copied to a VM instance and backed up offsite via Mozy. I'm considering doing a send/receive of a few volumes to a friend's system (as he will do to mine) to have offsite backups of the pools. It's mostly dependent on him buying more disk. ;-) And for what it's worth, my toying with ZFS and discussing it with coworkers has raised interest in Sun's storage line to replace NetApp at the office. Absolutely. My homebrew system has come up in many conversations about ZFS, which has ended up with a customer buying a Thumpers or Amber Road systems from Sun. (but that's my job, I guess;-) -- Andrew ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Mon, Feb 1, 2010 at 8:58 PM, matthew patton patto...@yahoo.com wrote: what with the home NAS conversations, what's the trick to buy a J4500 without any drives? SUN like every other enterprise storage vendor thinks it's ok to rape their customers and I for one, am not interested in paying 10x for a silly SATA hard drive. To get the topic back to the original question... There are Supermicro chassis that you can use. This one holds 36 drives, 24 front and 12 rear: http://www.supermicro.com/products/chassis/4U/847/SC847E16-R1400.cfm ($1800) This one holds only 24 drives: http://www.supermicro.com/products/chassis/4U/846/SC846TQ-R900.cfm ($950) Either of those with the CSE-PTJBOD-CB1 ($30), CBL-0166L ($40) and CBL-0167L($40) parts will be a monster JBOD. That being said, we used to get Dell JBODs at a previous job, and I remember the 12 drive shelves being cheap - cheaper than buying bare drives. This was 8 years ago, so I'm not sure if they're discounting their drives as much. -B -- Brandon High : bh...@freaks.com Mistakes are often the stepping stones to utter failure. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
A dumb question: I see 24 drives in an external chassi. I presume that chassis does only hold drives, it does not hold a motherboard. How do you connect all drives to your OpenSolaris server? Do you place them next to each other, and then you have three 8 SATA ports in your OpenSolaris server, and have 24 SATA cables in the air? And the chassis are wide open? Or, am I wrong, does the chassi also hold a motherboard? Is it possible to have the server in one chassi, and the drives in another chassi - how do you connect everything? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
Ok, I see that the chassi contains a mother board. So never mind that question. Another q: Is it possible to have large chassi with lots of drives, and the opensolaris in another chassi, how do you connect them both? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, February 2, 2010 02:24, matthew patton wrote: true. but I buy a Ferrari for the engine and bodywork and chassis engineering. It is totally criminal what Sun/EMC/Dell/Netapp do charging customers 10x the open-market rate for standard drives. A RE3/4 or NS drive is the same damn thing no matter if I buy it from ebay or my local distributor. Dell/Sun/Netapp buy drives by the container load. Oh sure, I don't mind paying an extra couple pennies/GB for all the strenuous efforts the vendors spend on firmware verification (HA!). Tell that to Intel and their SSD firmware team, or Seagate: http://mswhs.com/2009/01/21/seagate-hard-drive-firmware-bug/ Heck, even things as low-end as Netgear's ReadyNAS product specify using only certain versions of the firmware on many drives: http://www.readynas.com/?page_id=82 You buy enterprise drives to make sure they work as advertised and don't drop SYNC commands on the floor and then lie to ZFS about it. Disk may be cheap, but redundancy, iops, backups, and testing are not. As someone else suggested, you may want to skip the enterprise enclosure and go with a consumer one instead if price is a concern. From home workloads it may be sufficient to meet your needs. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
true. but I buy a Ferrari for the engine and bodywork and chassis engineering. It is totally criminal what Sun/EMC/Dell/Netapp do charging its interesting to read this with another thread containing: timeout issue is definitely the WD10EARS disks. replaced 24 of them with ST32000542AS (f/w CC34), and the problem departed with the WD disks. everyone needs to eat, if Ferrari spreads their NRE over the wheels, it might be because they are light and have been tested to not melt from the heat. Sun/EMC/Dell/Netapp tests each of their components and sells the total car. I'm thankful Sun shares their research and we can build on it. (btw, netapp ontap 8 is freebsd, and runs on std hardware after alittle bios work :-) Rob ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, February 2, 2010 01:26, James C. McPherson wrote: The engineering ratings are different to what you can buy from your local corner PC store, and the firmware is different. The qualification is done with the assumption that the disks will be spinning every single second for a number of years, and that they will have a much, *much* higher duty cycle than consumer grade hardware. Please stop assuming that all this only costs a few pennies. It doesn't. I'm pretty doubtful that the hardware differs from what I can buy from Newegg or whatever *IF* I buy the same enterprise-grade drive model (WD S25 or RE-4, say, rather than Caviar Blue) (I don't know what WD drives, if any, are currently qualified for use in any Sun products.) Just to be clear, are you asserting that? Or are you only asserting that the drives that get qualified are not the cheap drive models most easily found at your handy corner PC store? (I have less trouble believing they might have non-standard microcode.) I'm simply not Sun's market (home NAS); $1000 disk drives simply do not exist in my home; my budget doesn't stretch that far. And I do find it somewhat offensive that software, controller, and drives can't find enough common standards to actually be able to work (and I'll pay the money I need for the duty cycle I need; but in fact my home gear has run 24/7 since 1985, and I've had very little problem with disk drives. In fact drives most often die for me when the equipment is power-cycled). (I've still got the corpse of at least one 300MB drive from long ago that I paid $1500 for, come to think of it!) -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, February 2, 2010 01:27, Tim Cook wrote: Except you think the original engineering is just a couple grand, and that's where you're wrong. I hate the prices just as much as the next guy, but they do in fact need to feed their families. In fact, they need to do a hell of a lot more than that, including paying off what is likely bumping up against a 6-figure education for the engineering degrees they hold to design that hardware. If it were easy, everyone would be doing it, and it wouldn't be expensive. I don't think the complaint is mostly about the part Sun engineered, though; it seems to me the complaint is about the price of what looks and smells to many of us like essentially the same disk drive we can buy for 1/10 the price at Newegg. Now, I'm sure not ALL drives offered at Newegg could qualify; but the question is, how much do I give up by buying an enterprise-grade drive from a major manufacturer, compared to the Sun-certified drive? I can easily believe that Sun and the vendors spend many man-months qualifying a drive, and that perhaps there are even custom firmware versions for the Sun-certified drives. For a single drive, what's a not-crazy estimate? 12 man-months? Fully-loaded man-months costing about $250k/year? If they sell 10,000 of that drive, the per-drive cost of qualification would seem to be $25. Of course, I made up all the numbers, and have no idea if the quantity in particular is sane or not. Anyway, it's thinking like this that leads some of us to feel that a $900 premium is not reasonable. I suppose anybody who really knows sales numbers can't talk about them, and the same for some of the other bits. Still, the way to combat misperceptions (if these are in fact misperceptions) is with more accurate information. (I actually worked in the group Thumper came out of for a bit; I was working on the user interface software for the video streaming software that Thumper was developed to provide storage for. I was with Sun 2005-2008.) If you think they're overcharging, you're more than welcome to go into business, undercut the shit out of them, and still make a ton of money, since you think they 're charging 10x market value. I also think I'm not qualified to do that; I'm a software guy, neither management nor marketing nor hardware engineering. Also, if they really are charging 10x, then they can easily cut prices to compete with any upstarts, a fact that potential investors would take note of. -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 2, 2010 at 9:45 AM, David Dyer-Bennet d...@dd-b.net wrote: On Tue, February 2, 2010 01:27, Tim Cook wrote: Except you think the original engineering is just a couple grand, and that's where you're wrong. I hate the prices just as much as the next guy, but they do in fact need to feed their families. In fact, they need to do a hell of a lot more than that, including paying off what is likely bumping up against a 6-figure education for the engineering degrees they hold to design that hardware. If it were easy, everyone would be doing it, and it wouldn't be expensive. I don't think the complaint is mostly about the part Sun engineered, though; it seems to me the complaint is about the price of what looks and smells to many of us like essentially the same disk drive we can buy for 1/10 the price at Newegg. Now, I'm sure not ALL drives offered at Newegg could qualify; but the question is, how much do I give up by buying an enterprise-grade drive from a major manufacturer, compared to the Sun-certified drive? I can easily believe that Sun and the vendors spend many man-months qualifying a drive, and that perhaps there are even custom firmware versions for the Sun-certified drives. For a single drive, what's a not-crazy estimate? 12 man-months? Fully-loaded man-months costing about $250k/year? If they sell 10,000 of that drive, the per-drive cost of qualification would seem to be $25. Of course, I made up all the numbers, and have no idea if the quantity in particular is sane or not. Anyway, it's thinking like this that leads some of us to feel that a $900 premium is not reasonable. I suppose anybody who really knows sales numbers can't talk about them, and the same for some of the other bits. Still, the way to combat misperceptions (if these are in fact misperceptions) is with more accurate information. (I actually worked in the group Thumper came out of for a bit; I was working on the user interface software for the video streaming software that Thumper was developed to provide storage for. I was with Sun 2005-2008.) If you think they're overcharging, you're more than welcome to go into business, undercut the shit out of them, and still make a ton of money, since you think they 're charging 10x market value. I also think I'm not qualified to do that; I'm a software guy, neither management nor marketing nor hardware engineering. Also, if they really are charging 10x, then they can easily cut prices to compete with any upstarts, a fact that potential investors would take note of. It's called spreading the costs around. Would you really rather pay 10x the price on everything else besides the drives? This is essentially Sun's way of tiered pricing. Rather than charge you a software fee based on how much storage you have, they increase the price of the drives. Seems fairly reasonable to me... it gives a low point of entry for people that don't need that much storage without using ridiculous capacity based licensing on software. --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, February 2, 2010 09:58, Tim Cook wrote: It's called spreading the costs around. Would you really rather pay 10x the price on everything else besides the drives? This seems to miss the point. I presented an argument for why I think the qualified drives are a huge profit-center, not just making a reasonable profit on the work of qualification. In general, I'd much rather pay reasonable costs for each piece, rather than weird costs artificially shoved around to make things come out some strange way somebody favors. This is essentially Sun's way of tiered pricing. Rather than charge you a software fee based on how much storage you have, they increase the price of the drives. Seems fairly reasonable to me... it gives a low point of entry for people that don't need that much storage without using ridiculous capacity based licensing on software. It works great for me personally -- I'm using the software with other people's hardware, for free. But why should people who need a lot of storage pay proportionally more? I don't get that, that's grossly wrong. -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
I agree wholeheartedlyyou're paying to make the problem go away in an expedient manner. That said, I see how much we spend on NetApp storage at work and it makes me shudder ;) I think someone was wondering if the large storage vendors have their own microcode on drives? I can tell you that NetApp do...and that's one way they lock you in (if the drive doesn't report NetApp firmware, the filer will reject the drive) and also how they do tricks like soft-failure/re-validation, 520-byte sectors, etc. -marc On Tue, Feb 2, 2010 at 11:12 AM, Bob Friesenhahn bfrie...@simple.dallas.tx.us wrote: On Tue, 2 Feb 2010, David Dyer-Bennet wrote: Now, I'm sure not ALL drives offered at Newegg could qualify; but the question is, how much do I give up by buying an enterprise-grade drive from a major manufacturer, compared to the Sun-certified drive? If you have a Sun service contract, you give up quite a lot. If a Sun drive fails every other day, then Sun will replace that Sun drive every other day, even if the system warranty has expired. But if it is a non-Sun drive, then you have to deal with a disinterested drive manufacturer, which could take weeks or months. My experiences thus far is that if you pay for a Sun service contract, then you should definitely pay extra for Sun branded parts. Hopefully Oracle will do better than Sun at explaining the benefits and services provided by a service contract. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
Marc Nicholas geekyth...@gmail.com wrote: I think someone was wondering if the large storage vendors have their own microcode on drives? I can tell you that NetApp do...and that's one way they lock you in (if the drive doesn't report NetApp firmware, the filer will reject the drive) and also how they do tricks like soft-failure/re-validation, 520-byte sectors, etc. Since IBM started to use SCSI drives more than 20 years ago for their mainframes, you can format most drives with any sector size. IBM used 800 or 8000 byte sectors (10 or 100 punch cards ;-), but you also may reformat a drive with 520 bytes per sector. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, February 2, 2010 10:21, Marc Nicholas wrote: I agree wholeheartedlyyou're paying to make the problem go away in an expedient manner. That said, I see how much we spend on NetApp storage at work and it makes me shudder ;) Yes, exactly. Pricing must be about right, people wince but pay it :-). If they don't wince it's too low. There are certainly places (NASDAQ, say) who have to have absolute reliability (on both their main and disaster-recovery sites). That level of reliability costs money, and they pay it, probably even fairly cheerfully. Since spare parts have to be stocked and field service people trained, it makes sense that service contracts cover limited sets of equipment. And that's a strong argument for staying within that set of equipment. I do think that a lot of companies buy higher up the reliability curve than they need. But that's their choice. I think someone was wondering if the large storage vendors have their own microcode on drives? I can tell you that NetApp do...and that's one way they lock you in (if the drive doesn't report NetApp firmware, the filer will reject the drive) and also how they do tricks like soft-failure/re-validation, 520-byte sectors, etc. Sigh. It makes perfect sense that getting some special tricks in the drives actually pays off. And yet it's inevitable that they ALSO use it as a lockin. I've seen people down extra days while locked-in parts are shipped to them; the parts were essentially identical to what you could buy that day at retail locally, but the locally-available version wouldn't work. -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
This reminds me of this attorney that charged very much for a contract template he copied and gave to a client. To that, he responded: -You dont pay for me finding this template and copying to you, which took me 5 minutes. You pay me because I sat 5 years in the university, and have 15 years of experience. That is why you pay me. I agree that Sun hardware is way too pricey for a home user, but this is Enterprise stuff. And you should look at IBM prices, they are 5-10x higher than Sun's prices. I think the Enterprise customers, do not pay for Sun bringing you a disk from the cellar, which takes 5 minutes. But they pay for all the research, development, and Sun making sure everything works as intended. I love that Sun shares their products for free. Which other big Unix vendor does that? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Feb 2, 2010, at 8:49 AM, David Dyer-Bennet wrote: On Tue, February 2, 2010 10:21, Marc Nicholas wrote: I agree wholeheartedlyyou're paying to make the problem go away in an expedient manner. That said, I see how much we spend on NetApp storage at work and it makes me shudder ;) Yes, exactly. Pricing must be about right, people wince but pay it :-). If they don't wince it's too low. Business 101. The price will be what the market will bear. If the price seems out of line with your market, then perhaps you aren't in the same market. Personally, I think Ferraris are neat. But here on the ranch, I might be able to squeeze a bale of hay into the passenger seat, but the low ground clearance means I'll have to keep the tractor nearby to pull it out when it gets stuck. So a Ferrari has $0 market value here at the ranch. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On February 2, 2010 8:57:32 AM -0800 Orvar Korvar knatte_fnatte_tja...@yahoo.com wrote: I love that Sun shares their products for free. Which other big Unix vendor does that? Who's left? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 2, 2010 at 11:53 AM, Frank Cusack frank+lists/z...@linetwo.netwrote: On February 2, 2010 8:57:32 AM -0800 Orvar Korvar knatte_fnatte_tja...@yahoo.com wrote: I love that Sun shares their products for free. Which other big Unix vendor does that? Who's left? Pretty sure HP and IBM are still alive and well. --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
* On 02 Feb 2010, Orvar Korvar wrote: Ok, I see that the chassi contains a mother board. So never mind that question. Another q: Is it possible to have large chassi with lots of drives, and the opensolaris in another chassi, how do you connect them both? The J4500 and most other storage products being discussed are not servers: they are SATA concentrators with SAS uplinks. You plug in a bunch of cheap SATA disk, and you connect the chassis to a server with SAS. The logic board on the storage tray just converts the SAS signalling to SATA. It is not a computer in the usual sense. In many cases such products also have SAS expander ports, so that you can link multiple storage trays to a single SAS host bus adapter on your server by daisy-chaining them. So you need at least one SAS HBA on your OpenSolaris box, and SAS cables to hook up the trays containing the SATA drives. To the original question: you can purchase a J4x00 with a limited number of drives (empty is generally not an option), but there is no officially-sanctioned way to obtain the drive adapters except to buy Sun disks. You need either a SAS or a SATA drive bracket to adapt the drive to the J4x00 backplane, but they are not sold separately: one ships with each drive. As mentioned there are companies that sell remanufactured or discarded components, or machine their own substitutes. (Re)marketing Sun or compatible drive brackets has always been a lively business for a few small outfits. But Sun has no involvement with this, and may be unwilling to support a frankenstein server. Sun state that their OEM drives are of higher quality than OTS drives from manufacturers or retailers, and that they have custom firmware that improves their performance and reliability in Sun storage trays/arrays. I see no reason to disbelieve that, but it is quite a steep price to pay for that premium edge. When cost is a bigger concern than performance or reliability, I have generally bought the StorEdge product with the smallest drives I can (250 GB or 500 GB) and upgraded them myself to the size I really want. It's cheaper to buy 20 drives from CDW than 10 from Sun even when you account for the tiny throwaway drive, and you can keep the 10 extra as cold spares. At low enough scale the financial savings are worth the time to replace them as they fail. (I wish I could say the same of the StorEdge arrays themselves. Fully half of my 2540 controllers have failed, costing me huge amounts of time in both direct and contractual service, and I'm given up on them completely as a product line. I'll be thrilled to switch to JBOD.) For larger and less fault-tolerant systems, when money is available, I'm happy to pay Sun's premium. However, as others say, the other brands sometimes offer decent enough products to use instead of Sun's enterprise line. As always, it depends on your site's requirements and budget. I assume that a home NAS is comparatively low on both: therefore I wouldn't even shop with Sun unless you have a line on cheap castoffs from an enterprise shop. -- -D.d...@uchicago.eduNSITUniversity of Chicago ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On February 2, 2010 11:58:17 AM -0600 Tim Cook t...@cook.ms wrote: On Tue, Feb 2, 2010 at 11:53 AM, Frank Cusack frank+lists/z...@linetwo.netwrote: On February 2, 2010 8:57:32 AM -0800 Orvar Korvar knatte_fnatte_tja...@yahoo.com wrote: I love that Sun shares their products for free. Which other big Unix vendor does that? Who's left? Pretty sure HP and IBM are still alive and well. Yeah but who would want it, even for free. :P ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 2, 2010 at 12:00 PM, Frank Cusack frank+lists/z...@linetwo.netwrote: On February 2, 2010 11:58:17 AM -0600 Tim Cook t...@cook.ms wrote: On Tue, Feb 2, 2010 at 11:53 AM, Frank Cusack frank+lists/z...@linetwo.netwrote: On February 2, 2010 8:57:32 AM -0800 Orvar Korvar knatte_fnatte_tja...@yahoo.com wrote: I love that Sun shares their products for free. Which other big Unix vendor does that? Who's left? Pretty sure HP and IBM are still alive and well. Yeah but who would want it, even for free. :P Not exactly unix, but there's still VMS clusters running around out there with 100% uptime for over 20 years. I wouldn't mind seeing it opened up. --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 2, 2010 at 5:41 AM, Orvar Korvar knatte_fnatte_tja...@yahoo.com wrote: I see 24 drives in an external chassi. I presume that chassis does only hold drives, it does not hold a motherboard. How do you connect all drives to your OpenSolaris server? Do you place them next to each other, and then you have three 8 SATA ports in your OpenSolaris server, and have 24 SATA cables in the air? And the chassis are wide open? Or, am I wrong, does the chassi also hold a motherboard? Both of the cases I posted can hold a motherboard. There is also a kit available for about $100 that lets you use the chassis with just disk in it. -B -- Brandon High : bh...@freaks.com For sale: One moral compass, never used. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On February 2, 2010 12:08:13 PM -0600 Tim Cook t...@cook.ms wrote: Not exactly unix, but there's still VMS clusters running around out there with 100% uptime for over 20 years. I wouldn't mind seeing it opened up. Agreed, I'd love to see that opened up. Might even give it new life. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
100% uptime for 20 years? So what makes OpenVMS so much more stable than Unix? What is the difference? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
Also, both of those chassis come in SAS expander version and JBOD. the SAS expander version is the E1 version of the case. With the SAS Expander, and a motherboard using the LSI2008 or LSI1068 chipset, you can attach one cable from the SAS port (SFF8087) to the SAS expander and have all the drives online rather than connecting 24/36 drives individually. . . -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
1) SAS HBA seems to be an I/O card which has SAS cable connection. It sits in the OSol server. It is basically just a simple I/O card, right? I hope these cards are cheap? 2) So I can buy a disk chassi with 24 disks, connect all disks to one SAS cable and connect that SAS cable to my OSol server, which has a SAS HBA I/O card? Is this correct? Or do I need one SAS cable for each disk, hence I need 24 SAS cables? 3) Does the SAS HBA card need Solaris drivers? 4) Will there be performance hit if I connect 24 disks to one SAS cable/HBA? Maybe band width in one SAS cable will not suffice for 24 disks? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, February 2, 2010 11:26, Richard Elling wrote: On Feb 2, 2010, at 8:49 AM, David Dyer-Bennet wrote: On Tue, February 2, 2010 10:21, Marc Nicholas wrote: I agree wholeheartedlyyou're paying to make the problem go away in an expedient manner. That said, I see how much we spend on NetApp storage at work and it makes me shudder ;) Yes, exactly. Pricing must be about right, people wince but pay it :-). If they don't wince it's too low. Business 101. The price will be what the market will bear. If the price seems out of line with your market, then perhaps you aren't in the same market. Yes, perhaps. Quite clearly, in this case; I'm not buying enterprise storage myself. If the market bears it for long, then there's definitely an actual market there; otherwise it might have been a mistake by the company. I want the disk companies to come up with a set of specs for an enterprise-grade drive that can be used in stock form in relatively simple hardware to give good results. This concept that their enterprise-grade drives need tweaking in the firmware and price to be useful is annoying. Fair enough for people pushing the edges of the envelope, but most people don't, there should be a good solid mainstream solution available. A Solaris-based ZFS box using SAS controllers with 5, 8, 24, and 48 drive bay options might just about do it, if it could take a range of stock drives officially. Kill off a big chunk of people who get forced into enterprise storage against their will. Personally, I think Ferraris are neat. But here on the ranch, I might be able to squeeze a bale of hay into the passenger seat, but the low ground clearance means I'll have to keep the tractor nearby to pull it out when it gets stuck. So a Ferrari has $0 market value here at the ranch. Yes, my Camry is good for commuting and running across town through unfortunately frequent stop-and-go traffic, and running down to see my mother (about an hour) a lot more than I used to need to. It can carry 4 very comfortably, which we only use every month or so, and it can carry the 3-head studio lighting kit in the trunk very comfortably. Probably still somewhat marginal on your ranch, though better than a Ferrari. The ground clearance is medium, and it's not mainly a cargo-hauler. -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 2, 2010 at 12:54 PM, Orvar Korvar knatte_fnatte_tja...@yahoo.com wrote: 100% uptime for 20 years? So what makes OpenVMS so much more stable than Unix? What is the difference? They had/have clustering software that was/is bulletproof. I don't think anyone in the Unix community has duplicated it to date. As for differences, google is your friend? http://www3.sympatico.ca/n.rieck/docs/vms_vs_unix.html --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 2, 2010 at 2:14 PM, David Dyer-Bennet d...@dd-b.net wrote: On Tue, February 2, 2010 11:26, Richard Elling wrote: On Feb 2, 2010, at 8:49 AM, David Dyer-Bennet wrote: On Tue, February 2, 2010 10:21, Marc Nicholas wrote: I agree wholeheartedlyyou're paying to make the problem go away in an expedient manner. That said, I see how much we spend on NetApp storage at work and it makes me shudder ;) Yes, exactly. Pricing must be about right, people wince but pay it :-). If they don't wince it's too low. Business 101. The price will be what the market will bear. If the price seems out of line with your market, then perhaps you aren't in the same market. Yes, perhaps. Quite clearly, in this case; I'm not buying enterprise storage myself. If the market bears it for long, then there's definitely an actual market there; otherwise it might have been a mistake by the company. I want the disk companies to come up with a set of specs for an enterprise-grade drive that can be used in stock form in relatively simple hardware to give good results. This concept that their enterprise-grade drives need tweaking in the firmware and price to be useful is annoying. Fair enough for people pushing the edges of the envelope, but most people don't, there should be a good solid mainstream solution available. A Solaris-based ZFS box using SAS controllers with 5, 8, 24, and 48 drive bay options might just about do it, if it could take a range of stock drives officially. Kill off a big chunk of people who get forced into enterprise storage against their will. How exactly do you suggest the drive manufacturers make their drives just work with every SAS/SATA controller on the market, and all of the quirks they have? You're essentially saying you want the drive manufacturers to do what the storage vendors are doing today (all of the integration work), only not charge you for it. You can't have your cake and eat it too. Yes, my Camry is good for commuting and running across town through unfortunately frequent stop-and-go traffic, and running down to see my mother (about an hour) a lot more than I used to need to. It can carry 4 very comfortably, which we only use every month or so, and it can carry the 3-head studio lighting kit in the trunk very comfortably. Probably still somewhat marginal on your ranch, though better than a Ferrari. The ground clearance is medium, and it's not mainly a cargo-hauler. And how well does your Camry run when you try to replace the Toyota transmission with one manufactured by Ford? A mechanic who knows what he's doing and has fabrication skills could probably get it to work, and pretty darn well, but it isn't ever going to be the same as buying an integrated product directly from Toyota... --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, February 2, 2010 14:21, Tim Cook wrote: On Tue, Feb 2, 2010 at 2:14 PM, David Dyer-Bennet d...@dd-b.net wrote: On Tue, February 2, 2010 11:26, Richard Elling wrote: On Feb 2, 2010, at 8:49 AM, David Dyer-Bennet wrote: On Tue, February 2, 2010 10:21, Marc Nicholas wrote: I agree wholeheartedlyyou're paying to make the problem go away in an expedient manner. That said, I see how much we spend on NetApp storage at work and it makes me shudder ;) Yes, exactly. Pricing must be about right, people wince but pay it :-). If they don't wince it's too low. Business 101. The price will be what the market will bear. If the price seems out of line with your market, then perhaps you aren't in the same market. Yes, perhaps. Quite clearly, in this case; I'm not buying enterprise storage myself. If the market bears it for long, then there's definitely an actual market there; otherwise it might have been a mistake by the company. I want the disk companies to come up with a set of specs for an enterprise-grade drive that can be used in stock form in relatively simple hardware to give good results. This concept that their enterprise-grade drives need tweaking in the firmware and price to be useful is annoying. Fair enough for people pushing the edges of the envelope, but most people don't, there should be a good solid mainstream solution available. A Solaris-based ZFS box using SAS controllers with 5, 8, 24, and 48 drive bay options might just about do it, if it could take a range of stock drives officially. Kill off a big chunk of people who get forced into enterprise storage against their will. How exactly do you suggest the drive manufacturers make their drives just work with every SAS/SATA controller on the market, and all of the quirks they have? You're essentially saying you want the drive manufacturers to do what the storage vendors are doing today (all of the integration work), only not charge you for it. You can't have your cake and eat it too. I'm suggesting that the standard for the interface ought to be sufficiently standardized and well-enough documented that things meeting it just work, in the way that desktop motherboards and disk drives just work, i.e. well enough for nearly everybody. I understand why people pushing the limits would need custom-tuned hardware, but I don't think the middle of the market should need it. The controllers shouldn't be full of quirks; companies that routinely make them that way need to clean up their act or be driven out of the market. Same for the drives. Yes, my Camry is good for commuting and running across town through unfortunately frequent stop-and-go traffic, and running down to see my mother (about an hour) a lot more than I used to need to. It can carry 4 very comfortably, which we only use every month or so, and it can carry the 3-head studio lighting kit in the trunk very comfortably. Probably still somewhat marginal on your ranch, though better than a Ferrari. The ground clearance is medium, and it's not mainly a cargo-hauler. And how well does your Camry run when you try to replace the Toyota transmission with one manufactured by Ford? A mechanic who knows what he's doing and has fabrication skills could probably get it to work, and pretty darn well, but it isn't ever going to be the same as buying an integrated product directly from Toyota... When I was first in the industry, in 1969, it was fairly normal to only be able to connect DEC disks to a PDP-11; but even then there were third-party manufacturers making products and customers buying them. Now, forty years down the road, computers are constructed from mostly generic components. The disk drive is one of the ones that went generic first. It's absurd that we can't handle small enterprise storage on standards-compliant drives at this point. -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On 2010-Feb-03 00:12:43 +0800, Bob Friesenhahn bfrie...@simple.dallas.tx.us wrote: On Tue, 2 Feb 2010, David Dyer-Bennet wrote: Now, I'm sure not ALL drives offered at Newegg could qualify; but the question is, how much do I give up by buying an enterprise-grade drive from a major manufacturer, compared to the Sun-certified drive? If you have a Sun service contract, you give up quite a lot. If a Sun drive fails every other day, then Sun will replace that Sun drive every other day, even if the system warranty has expired. But if it is a non-Sun drive, then you have to deal with a disinterested drive manufacturer, which could take weeks or months. OTOH, if I'm paying 10x the street drive price upfront, plus roughly the street price annually in support, I can save a fair amount of money by just buying a pile of spare drives - when one fails, just swap it for a spare and it doesn't matter if it takes weeks for the vendor to swap it. Hopefully Oracle will do better than Sun at explaining the benefits and services provided by a service contract. I know that trying to get Sun to renew a service contract is like pulling teeth but Oracle is far worse - as far as I can tell, Oracle contracts are deliberately designed so you can't be certain whether you are compliant or not. -- Peter Jeremy ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Feb 2, 2010, at 10:54 AM, Orvar Korvar wrote: 100% uptime for 20 years? So what makes OpenVMS so much more stable than Unix? What is the difference? Software reliability studies show that the more reliable software is old software that hasn't changed :-) On Feb 2, 2010, at 12:42 PM, David Dyer-Bennet wrote: I'm suggesting that the standard for the interface ought to be sufficiently standardized and well-enough documented that things meeting it just work, in the way that desktop motherboards and disk drives just work, i.e. well enough for nearly everybody. I understand why people pushing the limits would need custom-tuned hardware, but I don't think the middle of the market should need it. Every mobo and disk I own has quirks. The only time things settle down to a widely accepted norm is when innovation stops. For example, recently the SATA TRIM command has received a lot of press. Next quarter, it will be some other feature on the buzzword list. The controllers shouldn't be full of quirks; companies that routinely make them that way need to clean up their act or be driven out of the market. Same for the drives. I think there are only about 5 HDD companies (Hitachi, Seagate, Western Digital, Samsung, Toshiba) and 3 controller companies today (LSI, Marvell, Intel). The remainder are in the process of getting out of the business or being bought. Interesting history here: http://en.wikipedia.org/wiki/List_of_defunct_hard_disk_manufacturers -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On February 2, 2010 4:31:47 PM -0500 Miles Nordin car...@ivy.net wrote: and FCoE is just dumb if you have IB, honestly. by FCoE are you talking about iSCSI? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, 2 Feb 2010, Frank Cusack wrote: On February 2, 2010 4:31:47 PM -0500 Miles Nordin car...@ivy.net wrote: and FCoE is just dumb if you have IB, honestly. by FCoE are you talking about iSCSI? No. They are different. FCoE uses raw ethernet packets and ethernet switches can/should be specially designed to support it whereas iSCSI is a TCP-based protocol. FCoE is basically fiber channel SAN protocol over ethernet. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Feb 2, 2010, at 1:56 PM, Frank Cusack wrote: On February 2, 2010 4:31:47 PM -0500 Miles Nordin car...@ivy.net wrote: and FCoE is just dumb if you have IB, honestly. by FCoE are you talking about iSCSI? FCoE is to iSCSI as Netware (IPX/SPX) is to NFS :-) -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Feb 2, 2010, at 2:56 PM, David Magda wrote: On Feb 2, 2010, at 15:17, Tim Cook wrote: On Tue, Feb 2, 2010 at 12:54 PM, Orvar Korvar wrote: 100% uptime for 20 years? So what makes OpenVMS so much more stable than Unix? What is the difference? They had/have clustering software that was/is bulletproof. I don't think anyone in the Unix community has duplicated it to date. As for differences, google is your friend? http://www3.sympatico.ca/n.rieck/docs/vms_vs_unix.html And by clustering we're not talking about something like Sun Cluster where it restarts an application after a node fails. It's more along the lines of multiple machines acting as a single server (though each runs its own copy of the OS--not a single image system): Did you ever wonder why Solaris Cluster seemed to be overkill for a simple failover service? The original design goals for Solaris Cluster looked a lot more like VMScluster than what you see today in Solaris Cluster. You can see the remnants remain in the code and features like pxfs (no, ZFS won't work with pxfs). The barriers to bringing such technology from a simple process model (like VMS) to a modern OS like Solaris are daunting. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
fc == Frank Cusack frank+lists/z...@linetwo.net writes: fc by FCoE are you talking about iSCSI? FCoE is an L2 design where ethernet ``pause'' frames can be sent specific to one of the seven CoS levels instead of applying to the entire port, which makes PAUSE abuseable for other purposes than their former one. CoS is an L2 priority/QoS tag inside the VLAN header. Before this hack, pause frames are not useful for congestion management because they cause head-of-line blocking, so serious switches only send them in response to backplane congestion, and for example serious hosts might send them for PCI contention, if clever enough. With the hack, the HOL-blocking effect of a PAUSE still spreads further than you might ideally like but can be constrained to one of the seven CoS planes in your fabric (probably, the Storage plane). This lets you have an HOL-blocking, lossless storage fabric in parallel with a buffered TCP fabric that is not lossless (uses packet drops for congestion control like normal Ethernet). You will find some squirrely language from FCoE proponents around these issues because they are trying to convince you that you have every desireable buzzword in every part of your network, while in fact what you're doing is making the same wise trade-off that every other non-Ethernet LAN fabric has always made. My parallel point is that the HOL-blocking lossless fabric is *CHEAPER* to create, not nmore expensive. It is less capable. It has no buffers and therefore no QoS. It just happens to be what's best for storage. so, they want you to pay the prices of a multi-queued QoSed WRED big-buffered non-blocking fabric suitable for transit traffic even though you mostly just need to push storage bits: classic upsell, just like all those ``XL'' PFC's they try to push off to customers who are not even in the DFZ. FCoE also includes a bunch of expensive hocus-pokus to bridge these frames onto a traditional FC-switched network and do a bunch of other things I don't understand like FC zoning and F-SPF. Most of the pitch dwells on this, trying to convince you they've made things ``simpler'' for you because it's once piece of wire. This seems like an anti-feature to me: wire's cheap while understanding things is hard, and now everyone's forced to catch up and learn Fibre Channel before it's safe to touch anything. Good in the long run, absolutely. Cheaper, fuck no. but the legitimate pitch for FCoE over iSCSI, to my view right now, comes from not from this management baloney but from the seven CoS levels, and the possibility some can be blocking and others buffered. Internet *transit* traffic (as opposed to end systems), and anything high-rtt, *must* be buffered, while within the LAN my current thinking is that you're better off with a 10Gbit/s HOL-blocking bufferless link than a 1Gbit/s non-blocking buffered link. The latter applies double for storage traffic which, made up of UDP-like reads and writes where you are stuck trying to perfect TCP to avoid blowing the buffers of normal switches while still getting yourself out of slow-start before the transaction's over andn doing all this in an environment where you cannot even convince thick-skulled netadmins they NEED to provide RED, not this bullshit ``weighted tail drop'' of 3560 u.s.w., and which besides really need backpressure from the fabric so they can be QoS'ed in the initiator's stack ahead of the network so that for example scrubs don't slow down pools (don't you find this happens more over iSCSI than over SAS?). I'm saaying, um...shit...saying, ``You need to think, about what you are trying to accomplish,'' and that Sun might have a suite of protocols based on ancient IB stuff that accomplishes more than FCoE, and does it cheaper (to them) and more simply, so, following their usual annoying plan, step 2 charge FCoE prices minus smallnumber, step 3 profit. meanwhile mellanox, having forseen all this and built open standards to solve it, is out there desperately trying to push some baloney called Etherband or something because all you bank admins are too daft to buy anything that does not have Ether in the name. :( pgpyvz3N8H8Ve.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, 2 Feb 2010, Miles Nordin wrote: fc == Frank Cusack frank+lists/z...@linetwo.net writes: fc by FCoE are you talking about iSCSI? FCoE is an L2 design where ethernet ``pause'' frames can be sent specific to one of the seven CoS levels instead of applying to the entire port, which makes PAUSE abuseable for other purposes than their Please redirect this encyclopedia contribution to WikiPedia, where it belongs. Thanks, Bob former one. CoS is an L2 priority/QoS tag inside the VLAN header. Before this hack, pause frames are not useful for congestion management because they cause head-of-line blocking, so serious switches only send them in response to backplane congestion, and for example serious hosts might send them for PCI contention, if clever enough. With the hack, the HOL-blocking effect of a PAUSE still spreads further than you might ideally like but can be constrained to one of the seven CoS planes in your fabric (probably, the Storage plane). This lets you have an HOL-blocking, lossless storage fabric in parallel with a buffered TCP fabric that is not lossless (uses packet drops for congestion control like normal Ethernet). You will find some squirrely language from FCoE proponents around these issues because they are trying to convince you that you have every desireable buzzword in every part of your network, while in fact what you're doing is making the same wise trade-off that every other non-Ethernet LAN fabric has always made. My parallel point is that the HOL-blocking lossless fabric is *CHEAPER* to create, not nmore expensive. It is less capable. It has no buffers and therefore no QoS. It just happens to be what's best for storage. so, they want you to pay the prices of a multi-queued QoSed WRED big-buffered non-blocking fabric suitable for transit traffic even though you mostly just need to push storage bits: classic upsell, just like all those ``XL'' PFC's they try to push off to customers who are not even in the DFZ. FCoE also includes a bunch of expensive hocus-pokus to bridge these frames onto a traditional FC-switched network and do a bunch of other things I don't understand like FC zoning and F-SPF. Most of the pitch dwells on this, trying to convince you they've made things ``simpler'' for you because it's once piece of wire. This seems like an anti-feature to me: wire's cheap while understanding things is hard, and now everyone's forced to catch up and learn Fibre Channel before it's safe to touch anything. Good in the long run, absolutely. Cheaper, fuck no. but the legitimate pitch for FCoE over iSCSI, to my view right now, comes from not from this management baloney but from the seven CoS levels, and the possibility some can be blocking and others buffered. Internet *transit* traffic (as opposed to end systems), and anything high-rtt, *must* be buffered, while within the LAN my current thinking is that you're better off with a 10Gbit/s HOL-blocking bufferless link than a 1Gbit/s non-blocking buffered link. The latter applies double for storage traffic which, made up of UDP-like reads and writes where you are stuck trying to perfect TCP to avoid blowing the buffers of normal switches while still getting yourself out of slow-start before the transaction's over andn doing all this in an environment where you cannot even convince thick-skulled netadmins they NEED to provide RED, not this bullshit ``weighted tail drop'' of 3560 u.s.w., and which besides really need backpressure from the fabric so they can be QoS'ed in the initiator's stack ahead of the network so that for example scrubs don't slow down pools (don't you find this happens more over iSCSI than over SAS?). I'm saaying, um...shit...saying, ``You need to think, about what you are trying to accomplish,'' and that Sun might have a suite of protocols based on ancient IB stuff that accomplishes more than FCoE, and does it cheaper (to them) and more simply, so, following their usual annoying plan, step 2 charge FCoE prices minus smallnumber, step 3 profit. meanwhile mellanox, having forseen all this and built open standards to solve it, is out there desperately trying to push some baloney called Etherband or something because all you bank admins are too daft to buy anything that does not have Ether in the name. :( -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On 2-Feb-10, at 1:54 PM, Orvar Korvar wrote: 100% uptime for 20 years? So what makes OpenVMS so much more stable than Unix? What is the difference? The short answer is that uptimes like that are VMS *cluster* uptimes. Individual hosts don't necessarily have that uptime, but the cluster availability is maintained for extremely long periods. You can probably find more discussion of this in comp.os.vms. --Toby -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
what with the home NAS conversations, what's the trick to buy a J4500 without any drives? SUN like every other enterprise storage vendor thinks it's ok to rape their customers and I for one, am not interested in paying 10x for a silly SATA hard drive. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Mon, Feb 1, 2010 at 10:58 PM, matthew patton patto...@yahoo.com wrote: what with the home NAS conversations, what's the trick to buy a J4500 without any drives? SUN like every other enterprise storage vendor thinks it's ok to rape their customers and I for one, am not interested in paying 10x for a silly SATA hard drive. Good luck. Unless you find a third party to buy drive sled's from, you're buying them from s/Sun/Oracle. --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
+-- | On 2010-02-01 23:01:33, Tim Cook wrote: | | On Mon, Feb 1, 2010 at 10:58 PM, matthew patton patto...@yahoo.com wrote: | | what with the home NAS conversations, what's the trick to buy a J4500 | without any drives? SUN like every other enterprise storage vendor thinks | it's ok to rape their customers and I for one, am not interested in paying | 10x for a silly SATA hard drive. | | Good luck. Unless you find a third party to buy drive sled's from, you're | buying them from s/Sun/Oracle. http://www.memoryx.net/5410456.html I've bought sleds for X4150s and X2270s from them. Cheers. -- bda cyberpunk is dead. long live cyberpunk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
http://www.memoryx.net/5410456.html I've bought sleds for X4150s and X2270s from them. interesting mis-description on the web page. thumper doesn't use SCA drives. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Mon, Feb 1, 2010 at 11:58 PM, matthew patton patto...@yahoo.com wrote: what with the home NAS conversations, what's the trick to buy a J4500 without any drives? SUN like every other enterprise storage vendor thinks it's ok to rape their customers and I for one, am not interested in paying 10x for a silly SATA hard drive. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss I think this is a pretty insane thing to say. First of all, sun doesn't sell home NAS equipment. Second of all, I'm not going to pretend sun doesn't charge a premium for their products but they ARE a enterprise vendor. You wouldn't say something like hey, where can i buy a Ferrari without any wheels...i'm not paying x amount for a silly aluminum wheel ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
charge a premium for their products but they ARE a enterprise vendor. You wouldn't say something like hey, where can i buy a Ferrari without any wheels...i'm not paying x amount for a silly aluminum wheel true. but I buy a Ferrari for the engine and bodywork and chassis engineering. It is totally criminal what Sun/EMC/Dell/Netapp do charging customers 10x the open-market rate for standard drives. A RE3/4 or NS drive is the same damn thing no matter if I buy it from ebay or my local distributor. Dell/Sun/Netapp buy drives by the container load. Oh sure, I don't mind paying an extra couple pennies/GB for all the strenuous efforts the vendors spend on firmware verification (HA!). I will happily pay a couple grand for the chassis, backplane, power supplies, and original engineering. Like my g'papy used to say; don't p*ss down my back and tell me it's raining. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
charge a premium for their products but they ARE a enterprise vendor. You wouldn't say something like hey, where can i buy a Ferrari without any wheels...i'm not paying x amount for a silly aluminum wheel true. but I buy a Ferrari for the engine and bodywork and chassis engineering. It is totally criminal what Sun/EMC/Dell/Netapp do charging customers 10x the open-market rate for standard drives. A RE3/4 or NS drive is the same damn thing no matter if I buy it from ebay or my local distributor. Dell/Sun/Netapp buy drives by the container load. Oh sure, I don't mind paying an extra couple pennies/GB for all the strenuous efforts the vendors spend on firmware verification (HA!). I will happily pay a couple grand for the chassis, backplane, power supplies, and original engineering. Like my g'papy used to say; don't p*ss down my back and tell me it's raining. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On 2/02/10 05:17 PM, matthew patton wrote: charge a premium for their products but they ARE a enterprise vendor. You wouldn't say something like hey, where can i buy a Ferrari without any wheels...i'm not paying x amount for a silly aluminum wheel true. but I buy a Ferrari for the engine and bodywork and chassis engineering. It is totally criminal what Sun/EMC/Dell/Netapp do charging customers 10x the open-market rate for standard drives. A RE3/4 or NS drive is the same damn thing no matter if I buy it from ebay or my local distributor. Dell/Sun/Netapp buy drives by the container load. Oh sure, I don't mind paying an extra couple pennies/GB for all the strenuous efforts the vendors spend on firmware verification (HA!). I will happily pay a couple grand for the chassis, backplane, power supplies, and original engineering. Like my g'papy used to say; don't p*ss down my back and tell me it's raining. Your belief about the amount of time, money and general effort that goes into qualifying disk drives by storage vendors is at odds with reality. The engineering ratings are different to what you can buy from your local corner PC store, and the firmware is different. The qualification is done with the assumption that the disks will be spinning every single second for a number of years, and that they will have a much, *much* higher duty cycle than consumer grade hardware. Please stop assuming that all this only costs a few pennies. It doesn't. James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 2, 2010 at 1:17 AM, matthew patton patto...@yahoo.com wrote: charge a premium for their products but they ARE a enterprise vendor. You wouldn't say something like hey, where can i buy a Ferrari without any wheels...i'm not paying x amount for a silly aluminum wheel true. but I buy a Ferrari for the engine and bodywork and chassis engineering. It is totally criminal what Sun/EMC/Dell/Netapp do charging customers 10x the open-market rate for standard drives. A RE3/4 or NS drive is the same damn thing no matter if I buy it from ebay or my local distributor. Dell/Sun/Netapp buy drives by the container load. Oh sure, I don't mind paying an extra couple pennies/GB for all the strenuous efforts the vendors spend on firmware verification (HA!). I will happily pay a couple grand for the chassis, backplane, power supplies, and original engineering. Like my g'papy used to say; don't p*ss down my back and tell me it's raining. Except you think the original engineering is just a couple grand, and that's where you're wrong. I hate the prices just as much as the next guy, but they do in fact need to feed their families. In fact, they need to do a hell of a lot more than that, including paying off what is likely bumping up against a 6-figure education for the engineering degrees they hold to design that hardware. If it were easy, everyone would be doing it, and it wouldn't be expensive. If you think they're overcharging, you're more than welcome to go into business, undercut the shit out of them, and still make a ton of money, since you think they 're charging 10x market value. /rant --Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives
On Tue, Feb 2, 2010 at 2:17 AM, matthew patton patto...@yahoo.com wrote: charge a premium for their products but they ARE a enterprise vendor. You wouldn't say something like hey, where can i buy a Ferrari without any wheels...i'm not paying x amount for a silly aluminum wheel true. but I buy a Ferrari for the engine and bodywork and chassis engineering. It is totally criminal what Sun/EMC/Dell/Netapp do charging customers 10x the open-market rate for standard drives. A RE3/4 or NS drive is the same damn thing no matter if I buy it from ebay or my local distributor. Dell/Sun/Netapp buy drives by the container load. Oh sure, I don't mind paying an extra couple pennies/GB for all the strenuous efforts the vendors spend on firmware verification (HA!). I will happily pay a couple grand for the chassis, backplane, power supplies, and original engineering. Like my g'papy used to say; don't p*ss down my back and tell me it's raining. You are just wrong. totally wrong. Let's look at the facts: Fact: Sun makes a wonderful software product Fact: Sun gives this away for free Fact: Sun does this for many reasons but one is so they can sell hardware Fact: Sun hardware is a premium product Fact When you buy a premium product you are paying for a lot, the actual cost of the tech is just a small fraction of this cost. You are paying for the reputation of that company and the service they provide. Fact: you are free to build your own white box server and throw in as many cheap newegg hard drives as you want Fact: It will not be built with the same standards as a sun product I'm sorry, you want to have your cake and eat it to. I can't afford a sun chassis either, but luckily sun DOES give us the software for free. You can do what everyone else does and build your own server. Don't complain that they charge to much. Be happy that they CAN charge enough to subsidize your cheap servers. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
