Re: Linux vs. Solaris file IO performance
On Mon, Oct 28, 2002 at 03:10:41PM -0500, [EMAIL PROTECTED] wrote: > > In a message dated: Mon, 28 Oct 2002 14:52:29 EST > Mark Komarinski said: > > >Sun is standardizing on FC drives (yep, Fibre Channel). And > >you thought SCSI was expensive > > Actually, FC drives are being produced in such large quantities now > that they're cheaper than SCSI drives in many cases. It's the > controllers that'll really set you back! SCSI controllers are cheap! Really?? Sun charges $2300 for a 36GB FC drive. Unless FC-AL is different from FC. Which may be why their stock is in the toilet... -Mark ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Linux vs. Solaris file IO performance
steveo wrote: >Just curious. Are you aware of IOZONE? I think I've used it in the past but I'd forgotten about it and steveo's reminder caused me a moment of consternation when I realized that I might have reinvented the wheel. It turns out, however, that iozone doesn't tweak many of the things that I'm tweaking, so what I'll probably do is (A) say thanks for the tip and (B) look into incorporating iozone into my tests. ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Linux vs. Solaris file IO performance
In a message dated: Mon, 28 Oct 2002 14:52:29 EST Mark Komarinski said: >Sun is standardizing on FC drives (yep, Fibre Channel). And >you thought SCSI was expensive Actually, FC drives are being produced in such large quantities now that they're cheaper than SCSI drives in many cases. It's the controllers that'll really set you back! SCSI controllers are cheap! -- Seeya, Paul -- It may look like I'm just sitting here doing nothing, but I'm really actively waiting for all my problems to go away. If you're not having fun, you're not doing it right! ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Linux vs. Solaris file IO performance
On Mon, Oct 28, 2002 at 02:40:00PM -0500, Derek Martin wrote: > At some point hitherto, Tom Varga hath spake thusly: > > I made the comparisons on an 8-way Sun-Fire-880 with 900Mhz processors > > against my 700 Mhz IBM a20 linux laptop. I'm fairly sure that the differences > > that I'm seeing have nothing to do with hardware. > > Despite the specs you quote, hardware could still be a factor. What > kind of disk(s)/controllers does the SunFire have, and how are you > using them? Is it in a RAID configuration, or no? Certain RAID > configurations can be slower than a single disk, in my experience. Sun is standardizing on FC drives (yep, Fibre Channel). And you thought SCSI was expensive > You failed to mention anything about memory usage, which probably has > the most impact here. If there's a lot of free memory for caching, > you'll see the kind of performance you're describing with the Linux > machine. If there's not, you'll see slower behavior like the Solaris > machine. As there won't be enough RAM to cache the I/O, all the I/O > will be forced to go directly to disk. Note that I said "free memory" > above, not just memory. Your server could have 10x as much memory as > your laptop, but still have none free... > > Of course, there could still be other factors, some of which could > be hardware related, and some not. And of course, there could be a > combination of factors as well... I think the real issue is that you're not looking at loaded systems. One user on a PC will always smoke (performance-wise) one user on a SPARC-based box. But what happens when you really start to load the system with users and processes? The PC will start to choke and sputter, while the SPARC is probably plugging along at the same rate it was before. Hence the reason why you build clusters of PCs instead of buying a 32-way PC (aside form cost). -Mark ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Linux vs. Solaris file IO performance
In a message dated: Mon, 28 Oct 2002 13:40:32 EST Tom Varga said: >I made the comparisons on an 8-way Sun-Fire-880 with 900Mhz processors >against my 700 Mhz IBM a20 linux laptop. I'm fairly sure that the differences >that I'm seeing have nothing to do with hardware. Okay, given those specs, I agree. > We're talking 40 seconds on the solaris box vs. 1/10 second on linux. > >From my observations, the main difference is that Linux will complete the >operation, be it rm or tar without any disk IO at all. It seems to me that >it's doing this to file cache without immediately flushing the changes to disk. >The solaris box, on the other hand, immediately flushes to disk each and every >change. I can hear the disk kerchunking like crazy. It sounds like the Solaris box is doing a sync() on write() whereas Linux probably isn't. >So, what I'd like to know is if the Solaris OS has a switch or >configuration that enables it to behave like Linux WRT file IO. I don't know. I don't have any of my Solaris books here, and it's been so long since I've touched Solaris, that I've moved that section of knowledge out my little brain to make room for more important things :) -- Seeya, Paul -- It may look like I'm just sitting here doing nothing, but I'm really actively waiting for all my problems to go away. If you're not having fun, you're not doing it right! ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Linux vs. Solaris file IO performance
>Does anybody know what settings could be tweaked >on a Solaris box so that it would do file IO >more efficiently like a linux box? It happens that I am currently involved in some fairly intensive Linux filesystem testing. It's one of those situations where I decided we needed to get some "quick numbers" describing filesystem performance and sat down to write a cheap'n'nasty test script. The numbers turned out to be more interesting than we expected, though, and people kept saying, "Hey, that's cool, but I wonder what happens when you vary this here other thingy?" and before long my script was running almost 200 different tests. And there's still plenty of attributes that I haven't instrumented... It's been an eye-opening experience. One tends to forget how many things can be tweaked, and some of them can make surprisingly large differences in I/O throughput from an app's point of view; I can't believe the same isn't true for Slolaris. I've got several different kinds of boxes rigged up to test ext2, ext3, reiserfs and xfs over several kinds of storage. I'm varying attributes like filesystem blocksize, filesystem type, I/O transfer size, file append versus rewrite, writes with and without OSYNC, mounting with sync versus async option, etc, etc... I was interested to see that all four filesystem types appear near both the top and bottom of the results (some of my best numbers are over 300 times better than my worst ones) and I'd have to say that there's no single "best" filesystem, at least according to my preliminary results. I guess if I wanted to make an "average" Linux box's disk I/O feel really snappy from an interactive user's perspective (ignoring various issues that some people [like me] don't have the luxury of ignoring) I'd make sure it had lots of RAM, fast disks (enable the on-disk write caches if SCSI) and mount my filesystems with async option. And I'd probably use ReiserFS or XFS. And, of course, it's possible that Solaris (or Sun's HW) have fundamental design differences that will keep them from ever going as fast as Linux on a PeeCee... . ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
OT: New Community
Ok, and now for something completely offtopic. Some of you may shoot me the next time I am at a meeting, but call me a messenger who is trying to spread the word. :) I saw a posting on a newsgroup I was reading regarding a new, online community at: forums.devplanet.org I examined it and decided to sign up. It is quite new and it trying to build a member base. If you are interested in things like web development, why not stop by and check it out. Please keep in mind how new this community is. No, in case you are wondering, I don't have a stake in it, and I get nothing out of this plug. Again, sorry for such an off topic posting. Regards, Jeff Kirkland ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Linux vs. Solaris file IO performance
Paul, I made the comparisons on an 8-way Sun-Fire-880 with 900Mhz processors against my 700 Mhz IBM a20 linux laptop. I'm fairly sure that the differences that I'm seeing have nothing to do with hardware. We're talking 40 seconds on the solaris box vs. 1/10 second on linux. From my observations, the main difference is that Linux will complete the operation, be it rm or tar without any disk IO at all. It seems to me that it's doing this to file cache without immediately flushing the changes to disk. The solaris box, on the other hand, immediately flushes to disk each and every change. I can hear the disk kerchunking like crazy. So, what I'd like to know is if the Solaris OS has a switch or configuration that enables it to behave like Linux WRT file IO. -Tom > In a message dated: Mon, 28 Oct 2002 12:11:26 EST > Tom Varga said: > > >Does anybody know what settings could be tweaked on a Solaris box so that it > >would do file IO more efficiently like a linux box? I don't think that this > >is NFS related because we're dealing with local partitions. I'm trying to > >convince our IT people that we're dealing with an incredible inefficiency with > >file IO on Solaris boxes, but I don't know what to suggest that they do to > >improve the situation. > > Well, it would be nice to know the specs of the hardware you're > comparing. It's tough to do the Solaris vs. Linux comparisson > without using similar hardware specs in that comparisson. > > If you're Solaris box is one the SBus-based systems, and you're > comparing that to a recent X86 system, you're comparing a 40Mhz Sbus > to what's probably a PC-100 or 133Mhz based system. > > Also, the IDE drives in a recent PC peak at a much higher rate than > the older SCSI2 based Sun systems. Though SCSI can sustain higher > rates as compared to IDE, IDE's peak is a lot higher, and this is > what you'll probably see in something like an 'rm' or 'tar' > operation. > > The amount of memory and swap in each system also plays a huge part > in performance, especially for Solaris. > > So, tell us the hw specs, and we can probably better help identify > what your bottleneck is. > -- > > Seeya, > Paul > -- > It may look like I'm just sitting here doing nothing, >but I'm really actively waiting for all my problems to go away. > >If you're not having fun, you're not doing it right! > > > ___ > gnhlug-discuss mailing list > [EMAIL PROTECTED] > http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss > ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Linux vs. Solaris file IO performance
In a message dated: Mon, 28 Oct 2002 12:11:26 EST Tom Varga said: >Does anybody know what settings could be tweaked on a Solaris box so that it >would do file IO more efficiently like a linux box? I don't think that this >is NFS related because we're dealing with local partitions. I'm trying to >convince our IT people that we're dealing with an incredible inefficiency with >file IO on Solaris boxes, but I don't know what to suggest that they do to >improve the situation. Well, it would be nice to know the specs of the hardware you're comparing. It's tough to do the Solaris vs. Linux comparisson without using similar hardware specs in that comparisson. If you're Solaris box is one the SBus-based systems, and you're comparing that to a recent X86 system, you're comparing a 40Mhz Sbus to what's probably a PC-100 or 133Mhz based system. Also, the IDE drives in a recent PC peak at a much higher rate than the older SCSI2 based Sun systems. Though SCSI can sustain higher rates as compared to IDE, IDE's peak is a lot higher, and this is what you'll probably see in something like an 'rm' or 'tar' operation. The amount of memory and swap in each system also plays a huge part in performance, especially for Solaris. So, tell us the hw specs, and we can probably better help identify what your bottleneck is. -- Seeya, Paul -- It may look like I'm just sitting here doing nothing, but I'm really actively waiting for all my problems to go away. If you're not having fun, you're not doing it right! ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Linux vs. Solaris file IO performance
I've been a linux user for about a decade and have always been amazed at how much faster file IO is on my linux box than on Solaris boxes that I have to use at work. For example, if I have a large directory structure on a local partition with say thousands of files that I need to delete, I do the following : rm -r directory On my linux box, this happens nearly instantaneously whereas on a Solaris box, it can take minutes or more. I can hear the disk head going crazy as if each and every file needs to be individually deleted. Another example would be doing a 'tar xvfz' of a large tarball. On the linux machine it happens as fast as the listing can go by on the terminal. Later I can hear the disk being efficiently flushed. On the Solaris box, again I hear the disk head going crazy and I don't get back my prompt until each and every file has been written to disk individually. Does anybody know what settings could be tweaked on a Solaris box so that it would do file IO more efficiently like a linux box? I don't think that this is NFS related because we're dealing with local partitions. I'm trying to convince our IT people that we're dealing with an incredible inefficiency with file IO on Solaris boxes, but I don't know what to suggest that they do to improve the situation. Thanks, -Tom ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Update Hell
On Mon, 28 Oct 2002, Thomas M. Albright messed up: > Now that we all know my situation, that's for everyone's help up to now. ^^ Should be "thanks" > :) > -- TARogue (Linux user number 234357) When you have an efficient government, you have a dictatorship. -- Harry Truman ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Update Hell
On Mon, 28 Oct 2002, mike ledoux wrote: > My guess is that he's either still booting his old kernel, or that > 'depmod -a' is failing for some reason. > The first answer is: nope, I booted into 2.2.22-6.2.2 The second answer is: no idea. I've run depmod -a a couple of times with no errors reported. Running 'depmod -v | grep fail' gives me a bunch of output, all starting with xftw. (Whatever that is.) -- TARogue (Linux user number 234357) (from the .sig file of Rodrigo Henriquez M.) (o_ //\ V_/_ May the penguin be with you! ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Update Hell
On Mon, 28 Oct 2002 [EMAIL PROTECTED] wrote: > Please post the output of the following commands: > > rpm -qa | grep kernel > kernel-2.2.19-6.2.16 kernel-pcmcia-cs-2.2.19-6.2.16 kernel-2.2.22-6.2.2 kernel-pcmcia-cs-2.2.22-6.2.2 kernel-utils-2.2.22-6.2.2 > uname -a > Linux machine.tarogue.net 2.2.22-6.2.2 #1 Mon Ser 23 07:15:28 EDT 2002 i586 unknown -- TARogue (Linux user number 234357) As you and I both know, the software may be free, but the beer isn't. --Jon "maddog" Hall ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Update Hell
I need to make a few points, I guess. kernel-2.2.22 may be an old kernel, but 2.2.22-6.2.2 is the latest patched version of the kernel. If not for the local exploit, I'd happily still be running 2.2.19-6.2.16 I never have, and God willing, I never will, build a kernel. If there is no rpm, then I hope I don't need it. 'make modules' and 'make modules install' won't work since my laptop is also my firewall/router and has pretty much nothing but the kernel, networking, pcmcia, and ipchains installed. (And of course the relevent dependencies.) Upgrading to a newer kernel/distribution is not a valid option either. I tried that at home, and I am now getting more error messages, and missing files, than I ever had before. [What the hell is libpthreads.so.0 (GLIBC_PRIVATE version)?] All I wanted to do was rpm -ivh new.kernel and reboot. I have since rebooted back into the old kernel, and now I'm having network problems. (I'm still troubleshooting, I may ask about them later.) Now that we all know my situation, that's for everyone's help up to now. :) -- TARogue (Linux user number 234357) "There is no right to fair use." -- Preston Padden, head of government relations for Walt Disney Corp. ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Update Hell
On Mon, 28 Oct 2002, at 10:55am, [EMAIL PROTECTED] wrote: > Well, Red Hat 6.2 is fairly old now, but it appears that you > may need to rebuild your kernel. He's running a pre-compiled binary kernel; that really should not apply. Unless Red Hat screwed up royally (which is certainly possible), I suspect the configuration of his installed kernel and supporting modules is slightly wacked. It might be a user mistake, or maybe a script guessed wrong somewhere. > 2.2.22 is a VERY old kernel ... Not really. It was released the 16th of September of this year. The 2.2 series is still being actively maintained. We are not seeing many new features being added to it, yes, but many of us consider that to be a Good Thing -- it contributes to a stable operating environment. Stable may be boring, but boring is often what one wants. :) -- Ben Scott <[EMAIL PROTECTED]> | The opinions expressed in this message are those of the author and do not | | necessarily represent the views or policy of any other person, entity or | | organization. All information is provided without warranty of any kind. | ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Update Hell
On Mon, 28 Oct 2002, at 9:50am, [EMAIL PROTECTED] wrote: > I just updated the kernel on my (Red Hat 6.2) laptop. The only kernel > packages needing updating were kernel, kernel-pcmcia-cs, and kernel-utils. > I did them, rewrote lilo, reran lilo, then rebooted. > Starting PCMCIA services: > modules/lib/modules/2.2.22-6.2.2/pcmcia/pcmcia_core.o: unresolved symbol > request_mem_region This means the system tried to load a kernel module, but it failed, because the "pcmcia_core" module is requesting some kernel symbols that cannot be found. This is almost always a configuration error of some kind, although it might not be your error (i.e., Red Hat might have screwed up). Please post the output of the following commands: rpm -qa | grep kernel uname -a -- Ben Scott <[EMAIL PROTECTED]> | The opinions expressed in this message are those of the author and do not | | necessarily represent the views or policy of any other person, entity or | | organization. All information is provided without warranty of any kind. | ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Update Hell
Well, Red Hat 6.2 is fairly old now, but it appears that you may need to rebuild your kernel. If you don't know how to do that, let us know, and I'm sure there will be some folks on the list who can walk you through it. If you do already know how, it appears to me that you may have forgotten to do a 'make modules; make modules_install' at the end as the kernel is trying to load modules that are incompatible with it, as reflected in the various 'unresolved symbols' messages. 2.2.22 is a VERY old kernel, and if at all possible, you might consider moving up to 2.4.19 kernel. However, should you do so, take a long look at /usr/src/linux/Documentation/Changes to see what other system components will require updating - there are a lot. HTH, Bayard ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: LyX, LaTeX, PS, PDF
On Sat, Oct 26, 2002 at 09:45:26PM -0400, [EMAIL PROTECTED] wrote: > On Fri, 25 Oct 2002, at 4:13pm, [EMAIL PROTECTED] wrote: > > And then use htmldoc to convert from HTML for really nice looking PDF. > > That would be doubly bad, since I would lose both the nice appearance of a > "proper" PDF conversion, and I would also lose the even more portable nature > of HTML! :-) I disagree. The LDP is using htmldoc to make its PDFs. I've used it to make very good-looking printouts (ToCs respecitve to the printout). The HTML is just the base you would put in the web page. No modifications required. -Mark ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Update Hell
I just updated the kernel on my (Red Hat 6.2) laptop. The only kernel packages needing updating were kernel, kernel-pcmcia-cs, and kernel-utils. I did them, rewrote lilo, reran lilo, then rebooted. Now pcmcia won't start. Argh! Can anyone help? This is what is being reported when I try to start pcmcia: Starting PCMCIA services: modules/lib/modules/2.2.22-6.2.2/pcmcia/pcmcia_core.o: unresolved symbol request_mem_region /lib/modules/2.2.22-6.2.2/pcmcia/pcmcia_core.o: unresolved symbol release_mem_region /lib/modules/2.2.22-6.2.2/pcmcia/i82365.o: unresolved symbol pci_set_power_state /lib/modules/2.2.22-6.2.2/pcmcia/i82365.o: unresolved symbol pci_enable_device /lib/modules/2.2.22-6.2.2/pcmcia/i82365.o: unresolved symbol unregister_ss_entry /lib/modules/2.2.22-6.2.2/pcmcia/i82365.o: unresolved symbol pci_irq_mask /lib/modules/2.2.22-6.2.2/pcmcia/i82365.o: unresolved symbol register_ss_entry /lib/modules/2.2.22-6.2.2/pcmcia/i82365.o: unresolved symbol CardServices /lib/modules/2.2.22-6.2.2/pcmcia/ds.o: unresolved symbol proc_pccard /lib/modules/2.2.22-6.2.2/pcmcia/ds.o: unresolved symbol CardServices cardmgr -- TARogue (Linux user number 234357) You can always tell a Texan, but you can't tell him much. - Chris Wall ___ gnhlug-discuss mailing list [EMAIL PROTECTED] http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
