Re: vfprintf() has a 4096-byte memory leak?

[David Schultz]

On Sat, Aug 02, 2003, Ryan T. Dean wrote:
> "Poul-Henning Kamp" wrote:
> >In message <3F2B9C59.3060209 at cytherianage.net 
> >>, "Ryan T. 
> >Dean" writes:
> >>/Hey all-
> /> >/I was doing some app debugging tonight, and noticed what appears 
> to /> >/be a memory leak in vfprintf().
> /> 
> >This is probably the buffer which stdio uses for all I/O.
> >
> >Try calling 
> > setbuf(stdout, NULL);
> > setbuf(stderr, NULL);
> >
> >as the very first thing in your program, and you will probably see
> >that it does not allocate the 4k buffer, you may also be able to
> >measure the performance change due to this.
> >
> >In other words, what you see is perfectly normal, and to be expected,
> >but if it is a problem for you, the above is the workaround.
> 
> Aha.  setbuf(stdout, NULL); does prevent the buffer from being allocated.
> However, in the case of stdout and stderr, if you don't setbuf() it to 
> null, a buffer is malloc'd.  The corresponding free() is in fclose.  So, if 
> you [f]printf() to stdout/stderr, and fclose(stdout/stderr), you are fine.  
> If, however, you don't know that buffers are being allocated for 
> stdin/stdout and don't fclose() or setbuf() to NULL, a 4096-byte chunk of 
> memory is allocated and never freed.  For shits and giggles, I checked a 
> DeadRat 8 box - no buffering by default.  I guess the only reason I'm 
> worried is the
> potential number of programs in the ports tree originally written on a 
> system
> where stdout/stderr behave differently, or people (like myself) who didn't
> have a clue any sort of output buffering was enabled on stdout/stderr, and 
> as a result have memory leaks.  If the porter did their job, it shouldn't 
> be an issue (was caught, patched, and the patch submitted upstream), but, 
> then, we never know, right?

stdio's buffering routines make a one-time allocation for their
buffers, and this memory does not get freed because it can
potentially get used during the entire time the program is
running.  This isn't a bug.  See the archives for details.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: vfprintf() has a 4096-byte memory leak?

[Dan Nelson]

In the last episode (Aug 02), Ryan T. Dean said:
> Aha.  setbuf(stdout, NULL); does prevent the buffer from being
> allocated. However, in the case of stdout and stderr, if you don't
> setbuf() it to null, a buffer is malloc'd.  The corresponding free()
> is in fclose.  So, if you [f]printf() to stdout/stderr, and
> fclose(stdout/stderr), you are fine.  If, however, you don't know
> that buffers are being allocated for stdin/stdout and don't fclose()
> or setbuf() to NULL, a 4096-byte chunk of memory is allocated and
> never freed.  For shits and giggles, I checked a DeadRat 8 box - no
> buffering by default.  I guess the only reason I'm worried is the
> potential number of programs in the ports tree originally written on
> a system where stdout/stderr behave differently, or people (like
> myself) who didn't have a clue any sort of output buffering was
> enabled on stdout/stderr, and as a result have memory leaks.  If the
> porter did their job, it shouldn't be an issue (was caught, patched,
> and the patch submitted upstream), but, then, we never know, right?

A memory leak is memory allocated, used, then discarded by the program
without freeing, possibly repeatedly so you build up memory that the
program never uses and cannot reclaim.  These buffers /are/ used, every
time you use stdio to stdout or stderr, so it's not a leak.  They're
simply not freed on exit.

> int main (void)
> {
>   printf("Test\n");
>   return 0;
> }
> test1 memory debug output:
> 1059825156: 1: *** alloc: at 'ra=0x2816ee86' for 4096 bytes, got '0x804b008|s1'
> 1059825156: 1: top 10 allocations:
> 1059825156: 1:  total-size  count in-use-size  count  source
> 1059825156: 1:4096  14096  1  ra=0x2816ee86
> 1059825156: 1:4096  14096  1  Total of 1
> 1059825156: 1: dumping not-freed pointers changed since 0:
> 1059825156: 1:  not freed: '0x804b008|s1' (4096 bytes) from 'ra=0x2816ee86'
> 1059825156: 1:  total-size  count  source
> 1059825156: 1:4096  1  ra=0x2816ee86
> 1059825156: 1:4096  1  Total of 1
> 1059825156: 1:  unknown memory: 1 pointer, 4096 bytes

The stdio buffers are correctly flushed by exit(), but fclose() is not
called to free the buffers.  Why bother when the memory will be
deallocated in .0001 seconds when the process exits? :)  If you want,
you can swap the comment on the two lines in
src/lib/libc/stdio/findfp.c to close all open files on exit:

void
_cleanup()
{   
/* (void) _fwalk(fclose); */
(void) _fwalk(__sflush);/* `cheating' */
}


-- 
Dan Nelson
[EMAIL PROTECTED]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: vfprintf() has a 4096-byte memory leak?

[Ryan T. Dean]

"Poul-Henning Kamp" wrote:
In message <3F2B9C59.3060209 at cytherianage.net 
>, "Ryan T. Dean" writes:
>/Hey all-
/> >/I was doing some app debugging tonight, and noticed what appears to 
/> >/be a memory leak in vfprintf().
/> 
This is probably the buffer which stdio uses for all I/O.

Try calling 
	setbuf(stdout, NULL);
	setbuf(stderr, NULL);

as the very first thing in your program, and you will probably see
that it does not allocate the 4k buffer, you may also be able to
measure the performance change due to this.
In other words, what you see is perfectly normal, and to be expected,
but if it is a problem for you, the above is the workaround.
Aha.  setbuf(stdout, NULL); does prevent the buffer from being allocated.
However, in the case of stdout and stderr, if you don't setbuf() it to null, 
a buffer is malloc'd.  The corresponding free() is in fclose.  So, if you 
[f]printf() to stdout/stderr, and fclose(stdout/stderr), you are fine.  If, 
however, you don't know that buffers are being allocated for stdin/stdout 
and don't fclose() or setbuf() to NULL, a 4096-byte chunk of memory is 
allocated and never freed.  For shits and giggles, I checked a DeadRat 8 
box - no buffering by default.  I guess the only reason I'm worried is the
potential number of programs in the ports tree originally written on a system
where stdout/stderr behave differently, or people (like myself) who didn't
have a clue any sort of output buffering was enabled on stdout/stderr, and as 
a result have memory leaks.  If the porter did their job, it shouldn't be an 
issue (was caught, patched, and the patch submitted upstream), but, then, we 
never know, right?

I don't know.  I'm tired and not thinking the straightest.  I'd like to 
apologize for any incoherence in my thoughts.  I appreciate the prompt reply.

	-Ryan T. Dean

(Code tested below on FBSD)
test1.c:
#include 
int main (void)
{
  printf("Test\n");
  return 0;
}
test1 memory debug output:
1059825156: 1: *** alloc: at 'ra=0x2816ee86' for 4096 bytes, got '0x804b008|s1'
1059825156: 1: top 10 allocations:
1059825156: 1:  total-size  count in-use-size  count  source
1059825156: 1:4096  14096  1  ra=0x2816ee86
1059825156: 1:4096  14096  1  Total of 1
1059825156: 1: dumping not-freed pointers changed since 0:
1059825156: 1:  not freed: '0x804b008|s1' (4096 bytes) from 'ra=0x2816ee86'
1059825156: 1:  total-size  count  source
1059825156: 1:4096  1  ra=0x2816ee86
1059825156: 1:4096  1  Total of 1
1059825156: 1:  unknown memory: 1 pointer, 4096 bytes
test2.c:
#include 
int main (void)
{
  printf("Test\n");
  fclose(stdout);
  return 0;
}
test2 memory debug output:
1059825216: 1: *** alloc: at 'ra=0x2816ee86' for 4096 bytes, got '0x804b008|s1'
1059825216: 2: *** free: at 'ra=0x2815c431' pnt '0x804b008|s2': size 4096, alloced at 
'ra=0x2816ee86'
1059825216: 2: top 10 allocations:
1059825216: 2:  total-size  count in-use-size  count  source
1059825216: 2:4096  1   0  0  ra=0x2816ee86
1059825216: 2:4096  1   0  0  Total of 1
1059825216: 2: dumping not-freed pointers changed since 0:
1059825216: 2:  memory table is empty
test3.c:
#include 
int main (void)
{
  setbuf(stdout, NULL);
  printf("Test\n");
  return 0;
}
test3 memory debug output:
(no output generated)
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: vfprintf() has a 4096-byte memory leak?

[Poul-Henning Kamp]

In message <[EMAIL PROTECTED]>, "Ryan T. Dean" writes:
>Hey all-
>I was doing some app debugging tonight, and noticed what appears to 
>be a memory leak in vfprintf().

This is probably the buffer which stdio uses for all I/O.

Try calling 
setbuf(stdout, NULL);
setbuf(stderr, NULL);

as the very first thing in your program, and you will probably see
that it does not allocate the 4k buffer, you may also be able to
measure the performance change due to this.

In other words, what you see is perfectly normal, and to be expected,
but if it is a problem for you, the above is the workaround.




-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

vfprintf() has a 4096-byte memory leak?

[Ryan T. Dean]

Hey all-
   I was doing some app debugging tonight, and noticed what appears to 
be a memory leak in vfprintf().  I've tested it on -CURRENT and -STABLE; 
any program that makes use of vfprintf() (ie, uses printf) appears to 
have a 4096 byte memory leak.  The memory is allocated on the first 
vfprintf() call and is never deallocated.  I've observed the memory leak 
on -CURRENT as of 30 Jul and -STABLE as of 29 Jul.  It was also 
reported, indirectly, on ports@ on 21 Jul.  The memory leak has been 
observed using dmalloc (devel/dmalloc; dmalloc.com) and NJAMD 
(sourceforge.net/projects/njamd).  I suspect that there may be a similar 
leak in vsprintf() as well.  I did file a PR (misc/55181), but I thought 
I should maybe give a shout-out to the list as well.
 -Ryan T. Dean

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[EMAIL PROTECTED]"