Linux-Advocacy Digest #29, Volume #27 Sun, 11 Jun 00 17:13:03 EDT
Contents:
Re: Open Source Programmers Demonstrate Incompetence ([EMAIL PROTECTED])
Re: Dealing with filesystem volumes (Michael Marion)
Re: Open Source Programmers Demonstrate Incompetence (mlw)
Re: Dealing with filesystem volumes ("Sam Morris")
Re: Linux faster than Windows? (Michael Marion)
Re: Dealing with filesystem volumes ("Sam Morris")
Re: Linux faster than Windows? (Michael Marion)
Re: Why Linux, and X.11 when MacOS 'X' is around the corner? (hogu)
Re: Linux faster than Windows? (Pete Goodwin)
----------------------------------------------------------------------------
From: [EMAIL PROTECTED] ([EMAIL PROTECTED])
Subject: Re: Open Source Programmers Demonstrate Incompetence
Reply-To: [EMAIL PROTECTED]
Date: Sun, 11 Jun 2000 20:12:21 GMT
On Sun, 11 Jun 2000 14:29:23 -0400, Gary Hallock <[EMAIL PROTECTED]> wrote:
>Your version has some serious problems.
You are corect. It has a memory leak if you run out of virtual memory
after some has already been allocated.
>It relies on being able to allocate space for the buffer.
Whether you allocate it on the stack of the heap, you are using from the
same pool of memory (almost - virtual address space notwithstanding). If
you can't allocate 1 meg from the heap, you won't be able to get it on the
stack either.
In fact it is bad form to allocate large space on the stack. If there's no
more memory, your program will be able to handle problems with dynamic
memory allocation, but not with running out of stack.
My program has the additional advantage of using no more space than
necessary. It allocates only exactly how much space is needed, while the
stack allocation would necessarily allocate an arbitrary amount which is
much more or much less than what is actually needed.
>Under other circumstances this might be ok, but for a debug routine, it
>may not be.
My routine is a not a debug routine. It reads in a file. I have never
heard of doing this as debug.
>What if there is no more storage available and that is why this function
>got called in the first place.
Then it would return PROGRAM_E_INSVIRMEM - as God intended. What should it
do?
Actually, you just found the only bug. In addition to that, it should free
the buffer before it returns.
>And then there is the fact that it always calls realloc() for every
>BUFLEN block, even if there is enough storage already.
But realloc is always grown with _exactly_ the amount which is being read
in. There is never more allocated than what it began with. The function
overwrites the pointer which is passed to it with NULL; the caller of the
function is assumed to give it an uninitialized pointer.
>Then there is the double buffering causing more performance problems.
There is no "double buffer". There is a small buffer which holds the most
recently read 4k, and a big buffer which holds everything. When you
read in a file you normally make some changes to your internal
representation so you would have to copy it anyways. You also need to
dynamically allocate memory because you can't pass a stack variable back
to the caller.
If you think this can be done, show me the code. Write a function which
dynamically allocates a buffer of exactly the right size for a file, and
returns it the pointer. You can do it successfully using too much memory
and wasting memory, but I do not believe you can do it portably (i.e.
without using select) without doing the extra copy.
>Of course, there is also the fact that your program reads from a
>file while the original one writes to a file.
Um, my code wasn't a replacement for the original function. It has nothing
to do with the original function, but was merely an example to show a
fixed size buffer used in a scenario where the function didn't have a
size limit.
>Your program uses a file pointer while the original used the lower level
>file descriptor and write().
This is not what my program was supposed to do, but if you want that,
replace FILE * with int, change fread to read, and rearrange the arguments
of read so they are correct. Hardly a challenge.
>And you never free up the storage when done.
I don't think you understand what the code is supposed to do. It is
supposed to return a copy of the buffer for a program. If it free'd the
buffer, there would be no data to return. It couldn't use stack allocation
because it would dissappear when the function returned. If you think a
function called read_in_file is supposed to read in the bytes of the file
-- and then throw them away -- you have a very strange definition of the
word "read".
>You didn't document that it is the callers responsibility to free the
>storage.
Neither does the man page for malloc. Most people tend to have a clue
about these things, and know that this function couldn't be done without
dynamic memory allocation, and therefore needed to be freed. I'm sorry
that you don't.
I write no documentation. That is the job of the technical writer. If I
wrote a document for it -- and left that out -- you may have a valid
point, but I didn't write anything at all for it.
In fact, this would be filed as a "documentation bug" not a product bug
_if_ that was indeed left out.
>It is rare that I find so many bugs in such a small piece of code.
At least the code wasn't in a production system, or publically offered
for free under the guise of usefullness. :-) It was written directly to
slrn to show an example. Sheesh.
------------------------------
From: Michael Marion <[EMAIL PROTECTED]>
Crossposted-To: comp.sys.mac.advocacy,comp.os.ms-windows.advocacy,comp.unix.advocacy
Subject: Re: Dealing with filesystem volumes
Date: Sun, 11 Jun 2000 20:14:05 GMT
Full Name wrote:
> We recently had a Mandrake box rendered unusable when the machine that
> was used as a backup failed to answer the mount request. It filled
> it's own hard disk with copies of local files every 24 hours when the
> backup cron job was triggered.
And? Why don't you configure it properly to not run if the mount isn't
completed? Or perhaps checks for disk space usage that mail admins when
a threshold is crossed? Admins are supposed to monitor boxes (no matter
the OS) to try to notice things like this before they become unusable.
--
Mike Marion - Unix SysAdmin/Engineer, Qualcomm Inc.
Homer Simpson: "To alcohol! The cause of, and solution to, all of life's
problems." -- Simpsons
------------------------------
From: mlw <[EMAIL PROTECTED]>
Subject: Re: Open Source Programmers Demonstrate Incompetence
Date: Sun, 11 Jun 2000 16:34:21 -0400
"[EMAIL PROTECTED]" wrote:
>
> On Sun, 11 Jun 2000 12:59:09 -0400, mlw <[EMAIL PROTECTED]> wrote:
>
> >Hold on sport, maybe so, but what's the point? It is unlikely a point of
> >failure. 4K is a pretty good length for debugging info. It does not call
> >malloc. It is a debug function, it should be as simple as possible.
>
> What makes you think it is a debug function? If it was a debug function,
> it would write directly to stderr. It is a general purpose version of
> fprintf which works with a file handle instead of a descriptor. fprintf is
> definitely not a debug routine.
Not true, many times a debugging function should accept a file so that
information can be redirected.
>
> >You are missing the big picture in that it is a "debug" function, which
> >by necessity, should work under all but the most catastrophic
> >circumstances.
>
> But this doesn't work for big buffers - let alone catastrophic
> circumstances. I see no evidence that this is a debug function. If you
> wanted to debug like this, you would just use fprintf.
Perhaps, but then again, fprintf would go one token at a time and on an
unbuffered file, this would be slow.
>
> >Also, the choice of vsnprintf over vfprintf makes sense if you
> >understand how printf works, and assume a file stream used for debugging
> >should not be buffered.
>
> I think you need to go back and look at the original code. It didn't use
> vfprintf because it was writing to a file handle not a descriptor. You
> cannot use vfprintf to write to a file handle, it only works with a
> descriptor. Thus, it used vsnprintf, in order to pass it to write(),
> which takes a file handle, not a descriptor.
OK, my mistake. No big deal.
>
> Furthermore, this has nothing to with buffering. stdout is buffereed
> indepedently of the stdio library. In fact, if you pass stdout as the file
> handle, it will indeed be buffered.
One should turn off buffering in such a circumstance. Any one who has
done any debugging should know this.
>
> >BTW: I have managed and coded many projects, well over 250k lines.
>
> Such as? What are they? Where can I see your work? I went to
> www.mohawksoft.com and saw no evidence of the sophisticated programs you
> have developed. Where are they being used?
Mohawksoft.com is my personal website, not that of people that have
hired me.
>
> >I would call this routine badly written. Why would you not read directly
> >into buffer? Why do you need to go through the data twice? Why would you
> >not make a few function calls to get file position and length and
> >reallocate the buffer upfront?
>
> How do you know the size of the file up front? What if you are reading
> from a pipe?
Then of course, it would fail, but that is a reasonable condition to
code for if this is a general purpose routine.
>
> >Why would you even consider using a FILE
> >* when you want the whole file, an experienced developer would try to
> >use mmap.
>
> For starters, you cannot mmap a pipe. Since you are a Unix developer, you
> are used to going for the "90% solution" - but the rest of us wants code
> which will work for every case, including pipes. True you can attempt to
> mmap it, but think of this as the failover function for when mmap fails.
OK, you can't mmap a pipe. That is a reasonable exception and should be
coded for, however, realloc is an expensive function, a general purpose
function should not suffer performance for a single bad condition.
>
> Furthermore is mmap portable? Does it work in VMS? OS/2? Windows? Some old
> VAX with an ancient version of Ultrix installed? My code is portable and
> will work anywhere ANSI C is present.
mmap is portable as a concept. No major system in use today does not
support this ability. And note, "try to use mmap."
>
> Finally, if you need to make internal modifications to work with
> the data (which you almost always need to do), you need to make a copy of
> it _anyways_. Better now than never.
But no changes are made in your example. You read into a buffer and then
memcpy into another buffer. That is bad programming no matter how you
look at it.
>
> >In fact, this function is poorly designed to do its job. If its job is
> >to manage the memory into which a file will be read (the calls to
> >realloc mandate this). It should issue the original malloc call, or call
> >mmap and avoid using the heap.
>
> I cannot use the original malloc buffer because I do not know the size of
> the what is left to read ahead of time. I get the size as a return value
> from fread, and do not know it ahead of time. That could be worked around
> with select, granted, but then you run into portability problems.
>
You should not be assuming the nature of the buffer passed in to you.
Period. If you are going to manipulate the pointer in any way, you must
allocated it.
> >What if the called passes a stack based buffer to this function? Talk
> >about unexpected results.
>
> As can any function which returns a pointer.
Not true. You make an assumption of the nature of the pointer. This is
out and out bad.
>
> You cannot use the return value for the return pointer; using the return
> value for anything but the status is bad manners.
That's OK, but you did pass in the address of the pointer which you
manipulate. There is no need for it to be a valid pointer. You could
easily have done the initial malloc. In fact, should have.
>
> >The example you gave as bad, only cuts off characters on a debug string,
> >your's could corrupt the program's heap if used incorrectly.
>
> So could strcpy. Would you fire the author of strcpy?
Yes, but strcpy has a documented behavior, yours does not advertise, nor
have the ability to check, that a pointer must have been allocated from
the heap. The closest C library function to your routine is strdup,
which allocates the memory itself.
>
> >In short, if you showed this to me as an example of code you would
> >write, I would not hire you.
>
> And miss those lucrative MohwakSoft options. The horror!
No work at a company where I get pulled into the interview process.
Believe me, you've heard of them.
>
> Le's get several things straight here.
>
> 1. What makes you think I would want to work for you? I wouldn't
> even consider working for anything but a Fortune 500 computer company, or
> possibly a well funded, high profile startup. What are your annual sales
> and profits? How much cash do you have on reserve? Who are your main
> investors? What is your compensation? Most importantly, what exciting,
> innovative, important projects are you working on? It's an employee's job
> market, and first you have to sell the prospective employee; I doubt
> anybody reading your ranting and raving on this newsgroup would want to
> work for you. How are you as a manager? Do you look out for your
> employees? Do you stand up for them? Do you understand what they are
> working on? Do you trust them? Or do you micromanage them, and play
> politics? In short, what makes you think you are manager which a talented
> engineer would want to work for?
For the sake of argument, I would not hire an engineer that would
require micromanagement. Once I have buy-in, I don't want to hear a
thing until it is done, or news of a problem, far in advance of it being
serious. Short of that, I want to see sandals, Frisbees, and long
lunches on fridays.
>
> 2. The code I wrote was written as an example of a specific type of
> programming. It was not intended to be production code.
Obviously. It is a bad design. There can be no argument. If there is,
you are arguing for the sake of the argument.
>
> 3. Judge me by my real code, not what I write in the newsgroups. I
> will be releasing one of my projects as open source imminently, and I'd
> love to see what you think of that.
>
> >If I saw this in a production product, I'd fire you.
>
> Nice. Care to disclose your turnover rate?
0, I hire right. None of the people I have ever hired at a company have
ever been fired. Conversely, people that have worked for me have come to
other companies to work for me again. People that have hired me in one
company, have hired me in other companies.
--
Mohawk Software
Windows 9x, Windows NT, UNIX, Linux. Applications, drivers, support.
Visit http://www.mohawksoft.com
Have you noticed the way people's intelligence capabilities decline
sharply the minute they start waving guns around?
------------------------------
From: "Sam Morris" <[EMAIL PROTECTED]>
Crossposted-To: comp.sys.mac.advocacy,comp.os.ms-windows.advocacy,comp.unix.advocacy
Subject: Re: Dealing with filesystem volumes
Date: Sun, 11 Jun 2000 21:42:08 +0100
> CD sessions (volumes as you call them) are NOT the same as partitions on
a
> hard drive. When you insert a multi-session CD into a CDROM drive, you get
> ONE drive letter. Since there is only one data track on a multi-session
CD,
> that's all that is required. The CD player can read the audio tracks from
> the other sessions, and so can the [windows] file manager, but they are
> displayed as being on the same drive letter.
I'm sure this isn't how it works. If I burn some files onto a CD on my PC,
and then add another session later, the files still show up in the same
volume/partition/whatever on the PC and the Mac. But many CDs I have
knocking around my house have multiple volumes/partitions/whatever on them,
of which my PC only sees the FAT one (heh), but which my Mac mounts both the
HFS and FAT volumes/partitions/whatevers.
> Windows first assigns drive letters (besides a and b) first to primary
> partitions on all fixed drives, then to logical drives within extended
> partitions; and finally to removable drives and CD-ROMs. You can easily
add
> a hard drive without shaking things up if you do a little planning.
Just out of interest - how? I remember doing this on Win2k, but my
System/Device Manager/Disk Drives/*/Settings' properties all have the
Start/End drive letter assignments greyed out.
> -- Rich C.
> "Great minds discuss ideas.
> Average minds discuss events.
> Small minds discuss people."
--
Sam Morris
[EMAIL PROTECTED]
...7/6/00: 3rd installation of Windows since March took 6h30m, and that's
without a working modem...
...you can have my Mac when you pry it from my cold, dead fingers...
------------------------------
From: Michael Marion <[EMAIL PROTECTED]>
Subject: Re: Linux faster than Windows?
Date: Sun, 11 Jun 2000 20:52:01 GMT
"[EMAIL PROTECTED]" wrote:
> What I actually meant was _writing_ the mailbox file. If you are using
> Pine or ELM (the two standard Unix mailers), and you delete a message from
> a large mailbox, the system has to rewrite the whole mailbox to cover
> this, since Unix does not support indexed files. This is dog slow.
You are so full of it.. I just opened my bugtraq folder again using pine
on my U2. It had 1916msgs (10676115 byte file size).. deleted a message
and expunged it. The open process took a whole 2 seconds, and the
deletion took a whole 3 seconds!
--
Mike Marion - Unix SysAdmin/Engineer, Qualcomm Inc.
"I am Beavis of Borg. Resistance, like uh... sucks <heh heh-heh-heh>!"
------------------------------
From: "Sam Morris" <[EMAIL PROTECTED]>
Crossposted-To: comp.sys.mac.advocacy,comp.os.ms-windows.advocacy,comp.unix.advocacy
Subject: Re: Dealing with filesystem volumes
Date: Sun, 11 Jun 2000 21:51:16 +0100
> > On my PC: Look in manual to discover jumper settings for Master/Slave
(IDE
> > only; I haven't tried to add a SCSI card to my PC). Connect up cables.
Boot
> > up. Discover that my CD is now E:, not D: and every damn thing that
expects
> > it's files to be on a CD now needs to be told differently.
>
> Um.. I'ld point out that on a modern mac with IDE/ATA drives.. you still
have Master/Slave settings.. but just about ANY computer
> system shipping in the last 3 ro 4 years use the "cable select" setting..
Our Performa 5300 and the beige G3 and the StarMaxxen I that care for only
have 1 device per channel, so you don't need to bother setting the jumpers
on them. I presume other IDE-enabled Macs are the same.
> This has NOTHING to do with Mac versus PC.. it has everything to do with
the IDE architecture, and it's no different than having to
> set a SCSI drive number.. say 2 instead of 3.
The point I was trying to make was relevant to the moronic Windows system of
drive letters, and that:
>> Cons: [of drive letters] They reaassign themselves at the slightest
>> excuse; add a new drive, and all bets are off as to which of your
>> existing drive letter assignments will stay the same.
>> Verdict: Stupid 1970s way of doing things that should be ashamed to be
>> still showing itself in the 21st century.
>
>The same goes for all the other filesystems in use today.
/^\
This sentence | is not true.
> And, changing drive letters is trivially easy for all except for the
floppies ( A: and B: ).. but I'll let you continue with your
> "hard way"..
The settings are greyed out on my hard disks' property pages. I don't know
why, and Windows Help is no use. Pray tell me how it is done.
--
Sam Morris
[EMAIL PROTECTED]
...7/6/00: 3rd installation of Windows since March took 6h30m, and that's
without a working modem...
...you can have my Mac when you pry it from my cold, dead fingers...
------------------------------
From: Michael Marion <[EMAIL PROTECTED]>
Subject: Re: Linux faster than Windows?
Date: Sun, 11 Jun 2000 20:58:58 GMT
"[EMAIL PROTECTED]" wrote:
> How long would it take to delete message #3219 (including time to write
> the new mailbox back to disk)?
Lets test this shall we?
I just did a
cat bugtraq bugtraq bugtraq bugtraq bugtraq bugtraq bugtraq bugtraq
bugtraq > bugtraq.test
which made a 96Meg spool with 17,000+ messages:
PINE 4.10 MESSAGE INDEX <Mai> lists/bugtraq.test Msg 1 of 17,243
NEW
It took another whopping 6 seconds to open.
Let's delete message... 6,500 8,900 and 16,500. A couple of random
ones.
Ok now that took a _really_ long time. 18 whole seconds to expunge
those.
--
Mike Marion - Unix SysAdmin/Engineer, Qualcomm Inc.
[Bart & Lisa are reading a magazine at the Kwik-E-Mart.]
Apu: "Hey, hey, this is not a lending library. If you're
not going to buy that thing put it down or I'll blow your heads off!"
-- Simpsons
------------------------------
From: hogu <[EMAIL PROTECTED]>
Crossposted-To: comp.os.linux.x
Subject: Re: Why Linux, and X.11 when MacOS 'X' is around the corner?
Date: Sun, 11 Jun 2000 23:03:45 +0200
On Sun, 11 Jun 2000, Spidey wrote:
> > will Mac OSX be free sourced? will it run of something other than
> > the mac??
>
> Yes, it is entirely open with the exception of the GUI.
> Rumour has it that an Intel port is in the works.
> http://www.publicsource.apple.com/
An Intel port ? That's good news. But what do you mean by 'entirely open
with the exception of the GUI'? Will MacOs be free downloadable like
Linux?
Gr. Hogu
------------------------------
Subject: Re: Linux faster than Windows?
From: [EMAIL PROTECTED] (Pete Goodwin)
Date: Sun, 11 Jun 2000 21:04:05 GMT
[EMAIL PROTECTED] (Charlie Ebert) wrote in
<[EMAIL PROTECTED]>:
>Well... But for the sake of GOOD SCIENCE, you must run an extended test
>which covers several hours of duration!
>
>See NT was about 3/4 the speed of Linux under an extended TEST!
>And Windows 2000 is 1/2 the speed under the same test.
>
>All these damn predictive results based on short term tests are probably
>true if the test lasted between a minute to 2-3 minutes.
>
>But when you get past that and start using virtual memory to it's
>fullest, disk IO to it's fullest and so on, you get to see the true
>nature of the OS.
>
>Now, Pete took my comments to heart and ran another test using some
>commercial software written for windows. He then took the same software
>and probably wined it over to Linux pronouncing it was twice as slow
>under Linux.
>
>Well. Is that a fair fight in the sand box or what?
What commercial software are we talking about here? That's your invention
BTW. The first piece of software was something I wrote, the second was
Dhrystone V1.0. The third was POVray which is free software. I compared the
Windows version against the Linux version. I did not run WINE at all.
Pete
------------------------------
** FOR YOUR REFERENCE **
The service address, to which questions about the list itself and requests
to be added to or deleted from it should be directed, is:
Internet: [EMAIL PROTECTED]
You can send mail to the entire list (and comp.os.linux.advocacy) via:
Internet: [EMAIL PROTECTED]
Linux may be obtained via one of these FTP sites:
ftp.funet.fi pub/Linux
tsx-11.mit.edu pub/linux
sunsite.unc.edu pub/Linux
End of Linux-Advocacy Digest
******************************
