Re: Re[2]: [zfs-discuss] Limit ZFS Memory Utilization

2007-01-10 Thread Jason J. W. Williams
Hi Robert, Thank you! Holy mackerel! That's a lot of memory. With that type of a calculation my 4GB arc_max setting is still in the danger zone on a Thumper. I wonder if any of the ZFS developers could shed some light on the calculation? That kind of memory loss makes ZFS almost unusable for a d

Re: Re[2]: [zfs-discuss] Re: Adding disk to a RAID-Z?

2007-01-10 Thread Jason J. W. Williams
Hi Robert, I read the following section from http://blogs.sun.com/roch/entry/when_to_and_not_to as indicating random writes to a RAID-Z had the performance of a single disk regardless of the group size: Effectively, as a first approximation, an N-disk RAID-Z group will behave as a single

Re: Re[4]: [zfs-discuss] Limit ZFS Memory Utilization

2007-01-10 Thread Jason J. W. Williams
Hi Robert, We've got the default ncsize. I didn't see any advantage to increasing it outside of NFS serving...which this server is not. For speed the X4500 is showing to be a killer MySQL platform. Between the blazing fast procs and the sheer number of spindles, its perfromance is tremendous. If

Re: [zfs-discuss] Limit ZFS Memory Utilization

2007-01-10 Thread Jason J. W. Williams
o the Solaris VM system a Solaris Nevada goal? Or would that be the next major release after Nevada? Best Regards, Jason On 1/10/07, Mark Maybee <[EMAIL PROTECTED]> wrote: Jason J. W. Williams wrote: > Hi Robert, > > Thank you! Holy mackerel! That's a lot of memory. With that t

Re: [zfs-discuss] Limit ZFS Memory Utilization

2007-01-10 Thread Jason J. W. Williams
Hello all, I second Al's motion. Even a little script a-la the CoolTools for tuning Solaris for the T2000 would be great. -J On 1/10/07, Al Hopper <[EMAIL PROTECTED]> wrote: On Wed, 10 Jan 2007, Mark Maybee wrote: > Jason J. W. Williams wrote: > > Hi Robert, > > &g

Re: [zfs-discuss] Limit ZFS Memory Utilization

2007-01-11 Thread Jason J. W. Williams
Hi Mark, That does help tremendously. How does ZFS decide which zio cache to use? I apologize if this has already been addressed somewhere. Best Regards, Jason On 1/11/07, Mark Maybee <[EMAIL PROTECTED]> wrote: Al Hopper wrote: > On Wed, 10 Jan 2007, Mark Maybee wrote: > &

Re: [zfs-discuss] Solid State Drives?

2007-01-11 Thread Jason J. W. Williams
Hello all, Just my two cents on the issue. The Thumper is proving to be a terrific database server in all aspects except latency. While the latency is acceptable, being able to add some degree of battery-backed write cache that ZFS could use would be phenomenal. Best Regards, Jason On 1/11/07,

Re: [zfs-discuss] Replacing a drive in a raidz2 group

2007-01-12 Thread Jason J. W. Williams
Hi Robert, We've experienced luck with flaky SATA drives in our STK array by unseating and reseating the drive to cause a reset of the firmware. It may be a bad drive, or the firmware may just have hit a bug. Hope its the latter! :-D I'd be interested why the hot-spare didn't kick in. I thought

Re: Re[2]: [zfs-discuss] Replacing a drive in a raidz2 group

2007-01-13 Thread Jason J. W. Williams
Hi Robert, Will build 54 offline the drive? Best Regards, Jason On 1/13/07, Robert Milkowski <[EMAIL PROTECTED]> wrote: Hello Jason, Saturday, January 13, 2007, 12:06:57 AM, you wrote: JJWW> Hi Robert, JJWW> We've experienced luck with flaky SATA drives in our STK array by JJWW> unseating a

Re: [zfs-discuss] Eliminating double path with ZFS's volume manager

2007-01-15 Thread Jason J. W. Williams
Hi Torrey, I think it does if you buy Xsan. Its still a separate product isn't it? Thought its more like QFS + MPXIO. Best Regards, Jason On 1/15/07, Torrey McMahon <[EMAIL PROTECTED]> wrote: Robert Milkowski wrote: > > 2. I belive it's definitely possible to just correct your config under > M

Re: [zfs-discuss] Re: ZFS direct IO

2007-01-15 Thread Jason J. W. Williams
Hi Roch, You mentioned improved ZFS performance in the latest Nevada build (60 right now?)...I was curious if one would notice much of a performance improvement between 54 and 60? Also, does anyone think the zfs_arc_max tunable-support will be made available as a patch to S10U3, or would that wai

Re: [zfs-discuss] Eliminating double path with ZFS's volume manager

2007-01-15 Thread Jason J. W. Williams
nsport multipathing - Like Mpxio, DLM, VxDMP, etc. - is usually separated from the filesystem layers. Jason J. W. Williams wrote: > Hi Torrey, > > I think it does if you buy Xsan. Its still a separate product isn't > it? Thought its more like QFS + MPXIO. > > Best Regards, &

Re: [zfs-discuss] Re: Eliminating double path with ZFS's volume manager

2007-01-16 Thread Jason J. W. Williams
Hi Philip, I'm not an expert, so I'm afraid I don't know what to tell you. I'd call Apple Support and see what they say. As horrid as they are at Enterprise support they may be the best ones to clarify if multipathing is available without Xsan. Best Regards, Jason On 1/16/07, Philip Mötteli <[

Re: [zfs-discuss] Re: Heavy writes freezing system

2007-01-17 Thread Jason J. W. Williams
Hi Anantha, I was curious why segregating at the FS level would provide adequate I/O isolation? Since all FS are on the same pool, I assumed flogging a FS would flog the pool and negatively affect all the other FS on that pool? Best Regards, Jason On 1/17/07, Anantha N. Srirama <[EMAIL PROTECTE

Re: Re[2]: [zfs-discuss] Re: Heavy writes freezing system

2007-01-17 Thread Jason J. W. Williams
Hi Robert, I see. So it really doesn't get around the idea of putting DB files and logs on separate spindles? Best Regards, Jason On 1/17/07, Robert Milkowski <[EMAIL PROTECTED]> wrote: Hello Jason, Wednesday, January 17, 2007, 11:24:50 PM, you wrote: JJWW> Hi Anantha, JJWW> I was curious w

Re: [zfs-discuss] Re: What SATA controllers are people using for ZFS?

2007-01-18 Thread Jason J. W. Williams
Hi Frank, Sun doesn't support the X2100 SATA controller on Solaris 10? That's just bizarre. -J On 1/18/07, Frank Cusack <[EMAIL PROTECTED]> wrote: THANK YOU Naveen, Al Hopper, others, for sinking yourselves into the shit world of PC hardware and [in]compatibility and coming up with well qualif

Re: [zfs-discuss] Re: What SATA controllers are people using for ZFS?

2007-01-18 Thread Jason J. W. Williams
18/07, Frank Cusack <[EMAIL PROTECTED]> wrote: On January 18, 2007 4:45:49 PM -0700 "Jason J. W. Williams" <[EMAIL PROTECTED]> wrote: > Sun doesn't support the X2100 SATA controller on Solaris 10? That's > just bizarre. Not only that, their marketing is

Re: [zfs-discuss] Re: What SATA controllers are people using for ZFS?

2007-01-18 Thread Jason J. W. Williams
Hi Toby, Thanks for the links. That's interesting. I assume this goes forward to the M2s. Glad hot-swap isn't a requirement where we use them. Best Regards, Jason On 1/18/07, Toby Thain <[EMAIL PROTECTED]> wrote: On 18-Jan-07, at 9:55 PM, Jason J. W. Williams wrote: > Hi

Re: [zfs-discuss] External drive enclosures + Sun Server for mass storage

2007-01-19 Thread Jason J. W. Williams
Hi David, I don't know if your company qualifies as a startup under Sun's regs but you can get an X4500/Thumper for $24,000 under this program: http://www.sun.com/emrkt/startupessentials/ Best Regards, Jason On 1/19/07, David J. Orman <[EMAIL PROTECTED]> wrote: Hi, I'm looking at Sun's 1U x64

Re: [zfs-discuss] External drive enclosures + Sun Server for mass storage

2007-01-20 Thread Jason J. W. Williams
Hi Shannon, The markup is still pretty high on a per-drive basis. That being said, $1-2/GB is darn low for the capacity in a server. Plus, you're also paying for having enough HyperTransport I/O to feed the PCI-E I/O. Does anyone know what problems they had with the 250GB version of the Thumper

Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage

2007-01-22 Thread Jason J. W. Williams
Hi Frank, I'm sure Richard will check it out. He's a very good guy and not trying to jerk you around. I'm sure the hostility isn't warranted. :-) Best Regards, Jason On 1/22/07, Frank Cusack <[EMAIL PROTECTED]> wrote: On January 22, 2007 10:03:14 AM -0800 Richard Elling <[EMAIL PROTECTED]> wro

Re: [zfs-discuss] Re: External drive enclosures + Sun Server for mass

2007-01-22 Thread Jason J. W. Williams
Hi David, Depending on the I/O you're doing the X4100/X4200 are much better suited because of the dual HyperTransport buses. As a storage box with GigE outputs you've got a lot more I/O capacity with two HT buses than one. That plus the X4100 is just a more solid box. The X2100 M2 while a vast im

Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage

2007-01-22 Thread Jason J. W. Williams
Hi Guys, The original X2100 was a pile of doggie doo-doo. All of our problems with it go back to the atrocious quality of the nForce 4 Pro chipset. The NICs in particular are just crap. The M2s are better, but the MCP55 chipset has not resolved all of its flakiness issues. That being said Sun des

Re: [zfs-discuss] Re: Re: External drive enclosures + Sun Server for mass

2007-01-22 Thread Jason J. W. Williams
Hi David, Glad to help! I don't want to bad-mouth the X2100 M2s that much, because they have been solid. I believe the M2s are made/designed just for Sun by Quanta Computer (http://www.quanta.com.tw/e_default.htm) whereas the mobos in the original X2100 was Tyan Tiger with some slight modificatio

[zfs-discuss] Understanding ::memstat in terms of the ARC

2007-01-22 Thread Jason J. W. Williams
Hello all, I have a question. Below are two ::memstat outputs about 5 days apart. The interesting thing is the "anonymous" memory shows 2GB, though the two major hogs of that memory (two MySQL instances) claim to be consuming about 6.2GB (checked via pmap). Also, it seems like the ARC keeps cree

Re: [zfs-discuss] need advice: ZFS config ideas for X4500 Thumper?

2007-01-23 Thread Jason J. W. Williams
Hi Neal, We've been getting pretty good performance out of RAID-Z2 with 3x 6-disk RAID-Z2 stripes. More stripes mean better performance all around...particularly on random reads. But as a file-server that's probably not a concern. With RAID-Z2 it seems to me 2 hot-spares is very sufficient, but I

Re: [zfs-discuss] Re: Re: Re: Re: External drive enclosures + Sun

2007-01-23 Thread Jason J. W. Williams
I believe the SmartArray is an LSI like the Dell PERC isn't it? Best Regards, Jason On 1/23/07, Robert Suh <[EMAIL PROTECTED]> wrote: People trying to hack together systems might want to look at the HP DL320s http://h10010.www1.hp.com/wwpc/us/en/ss/WF05a/15351-241434-241475-241475 -f79-3232017

Re: [zfs-discuss] need advice: ZFS config ideas for X4500 Thumper?

2007-01-23 Thread Jason J. W. Williams
Hi Peter, Perhaps I'm a bit dense, but I've been befuddled by the x+y notation myself. Is it X stripes consisting of Y disks? Best Regards, Jason On 1/23/07, Peter Tribble <[EMAIL PROTECTED]> wrote: On 1/23/07, Neal Pollack <[EMAIL PROTECTED]> wrote: > Hi: (Warning, new zfs user question

Re: [zfs-discuss] need advice: ZFS config ideas for X4500 Thumper?

2007-01-23 Thread Jason J. W. Williams
Hi Peter, Ah! That clears it up for me. Thank you. Best Regards, Jason On 1/23/07, Peter Tribble <[EMAIL PROTECTED]> wrote: On 1/23/07, Jason J. W. Williams <[EMAIL PROTECTED]> wrote: > Hi Peter, > > Perhaps I'm a bit dense, but I've been befuddled by the

[zfs-discuss] Thumper Origins Q

2007-01-23 Thread Jason J. W. Williams
Hi All, This is a bit off-topic...but since the Thumper is the poster child for ZFS I hope its not too off-topic. What are the actual origins of the Thumper? I've heard varying stories in word and print. It appears that the Thumper was the original server Bechtolsheim designed at Kealia as a mas

Re: [zfs-discuss] Synchronous Mount?

2007-01-23 Thread Jason J. W. Williams
Hi Prashanth, My company did a lot of LVM+XFS vs. SVM+UFS testing in addition to ZFS. Overall, LVM's overhead is abysmal. We witnessed performance hits of 50%+. SVM only reduced performance by about 15%. ZFS was similar, though a tad higher. Also, my understanding is you can't write to a ZFS sna

Re: [zfs-discuss] Thumper Origins Q

2007-01-23 Thread Jason J. W. Williams
Wow. That's an incredibly cool story. Thank you for sharing it! Does the Thumper today pretty much resemble what you saw then? Best Regards, Jason On 1/23/07, Bryan Cantrill <[EMAIL PROTECTED]> wrote: > This is a bit off-topic...but since the Thumper is the poster child > for ZFS I hope its no

Re: [zfs-discuss] Synchronous Mount?

2007-01-23 Thread Jason J. W. Williams
Hi Prashanth, This was about a year ago. I believe I ran bonnie++ and IOzone tests. Tried also to simulate an OLTP load. The 15-20% overhead for ZFS was vs. UFS on a raw disk...UFS on SVM was almost exactly 15% lower performance than raw UFS. UFS and XFS on raw disk were pretty similar in terms o

Re: [zfs-discuss] Re: Thumper Origins Q

2007-01-24 Thread Jason J. W. Williams
'Cept the 3511 is highway robbery for what you get. ;-) Best Regards, Jason On 1/24/07, Richard Elling <[EMAIL PROTECTED]> wrote: Peter Eriksson wrote: >> too much of our future roadmap, suffice it to say that one should expect >> much, much more from Sun in this vein: innovative software and i

Re: [zfs-discuss] Thumper Origins Q

2007-01-24 Thread Jason J. W. Williams
Hi Wee, Having snapshots in the filesystem that work so well is really nice. How are y'all quiescing the DB? Best Regards, J On 1/24/07, Wee Yeh Tan <[EMAIL PROTECTED]> wrote: On 1/25/07, Bryan Cantrill <[EMAIL PROTECTED]> wrote: > ... > after all, what was ZFS going to do with that expensive

Re: [zfs-discuss] ZFS or UFS - what to do?

2007-01-26 Thread Jason J. W. Williams
Hi Jeff, We're running a FLX210 which I believe is an Engenio 2884. In our case it also is attached to a T2000. ZFS has run VERY stably for us with data integrity issues at all. We did have a significant latency problem caused by ZFS flushing the write cache on the array after every write, but t

Re: [zfs-discuss] ZFS or UFS - what to do?

2007-01-26 Thread Jason J. W. Williams
Correction: "ZFS has run VERY stably for us with data integrity issues at all." should read "ZFS has run VERY stably for us with NO data integrity issues at all." On 1/26/07, Jason J. W. Williams <[EMAIL PROTECTED]> wrote: Hi Jeff, We're running a FLX210 whi

Re: [zfs-discuss] Re: Re: How much do we really want zpool remove?

2007-01-26 Thread Jason J. W. Williams
To be fair, you can replace vdevs with same-sized or larger vdevs online. The issue is that you cannot replace with smaller vdevs nor can you eliminate vdevs. In other words, I can migrate data around without downtime, I just can't shrink or eliminate vdevs without send/recv. This is where the ph

Re: [zfs-discuss] multihosted ZFS

2007-01-26 Thread Jason J. W. Williams
You could use SAN zoning of the affected LUN's to keep multiple hosts from seeing the zpool. When failover time comes, you change the zoning to make the LUN's visible to the new host, then import. When the old host reboots, it won't find any zpool. Better safe than sorry Or change the LUN

Re: [zfs-discuss] Project Proposal: Availability Suite

2007-01-26 Thread Jason J. W. Williams
Could the replication engine eventually be integrated more tightly with ZFS? That would be slick alternative to send/recv. Best Regards, Jason On 1/26/07, Jim Dunham <[EMAIL PROTECTED]> wrote: Project Overview: I propose the creation of a project on opensolaris.org, to bring to the community

Re: [zfs-discuss] Re: ZFS or UFS - what to do?

2007-01-29 Thread Jason J. W. Williams
Hi Jeff, Maybe I mis-read this thread, but I don't think anyone was saying that using ZFS on-top of an intelligent array risks more corruption. Given my experience, I wouldn't run ZFS without some level of redundancy, since it will panic your kernel in a RAID-0 scenario where it detects a LUN is

Re: [zfs-discuss] Project Proposal: Availability Suite

2007-01-29 Thread Jason J. W. Williams
Thank you for the detailed explanation. It is very helpful to understand the issue. Is anyone successfully using SNDR with ZFS yet? Best Regards, Jason On 1/26/07, Jim Dunham <[EMAIL PROTECTED]> wrote: Jason J. W. Williams wrote: > Could the replication engine eventually be integr

Re: [zfs-discuss] hot spares - in standby?

2007-01-29 Thread Jason J. W. Williams
Hi Guys, I seem to remember the Massive Array of Independent Disk guys ran into a problem I think they called static friction, where idle drives would fail on spin up after being idle for a long time: http://www.eweek.com/article2/0,1895,1941205,00.asp Would that apply here? Best Regards, Jason

Re: [zfs-discuss] Project Proposal: Availability Suite

2007-01-29 Thread Jason J. W. Williams
Hi Jim, Thank you very much for the heads up. Unfortunately, we need the write-cache enabled for the application I was thinking of combining this with. Sounds like SNDR and ZFS need some more soak time together before you can use both to their full potential together? Best Regards, Jason On 1/2

Re: [zfs-discuss] hot spares - in standby?

2007-01-29 Thread Jason J. W. Williams
ork. Would dramatically cut down on the power. What do y'all think? Best Regards, Jason On 1/29/07, Toby Thain <[EMAIL PROTECTED]> wrote: On 29-Jan-07, at 11:02 PM, Jason J. W. Williams wrote: > Hi Guys, > > I seem to remember the Massive Array of Independent Disk guys ran

Re: [zfs-discuss] Zfs best practice for 2U SATA iSCSI NAS

2007-02-17 Thread Jason J. W. Williams
Hi Nicholas, ZFS itself is very stable and very effective as fast FS in our experience. If you browse the archives of the list you'll see that NFS performance is pretty acceptable, with some performance/RAM quirks around small files: http://www.opensolaris.org/jive/message.jspa?threadID=19858 ht

Re: [zfs-discuss] Zfs best practice for 2U SATA iSCSI NAS

2007-02-19 Thread Jason J. W. Williams
Hi Nicholas, Actually Virtual Iron, they have a nice system at the moment with live migration of windows guest. Ah. We looked at them for some Windows DR. They do have a nice product. 3. Which leads to: coming from Debian, how easy is system updates? I remember with OpenBSD system updates u

<    1   2