Re: lzo2 shows insane speed gap

2008-12-29 Thread Alex Kozlov
Christian Weisgerber  wrote:

> Oh, and everybody is invited to run   
> 
> $ cd /usr/ports/archivers/lzo2 && make
$cd /usr/ports/archivers/lzo2 && time sudo make
[...]
All tests passed. Now you are ready to install LZO.

real1m1.041s
user0m38.087s
sys 0m17.613s

This is Intel q6600.


--
Adios
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: lzo2 shows insane speed gap

2008-12-29 Thread Kevin Day



Oh, and everybody is invited to run

$ cd /usr/ports/archivers/lzo2 && make

and check for themselves.




I've used lzo2 quite a bit in the past and never saw this, so I  
thought I'd try this on a few boxes we have... Output is from "make  
fetch ; time make"


8-core Opteron 2350 2.0ghz, 64GB RAM, FreeBSD 7.1-PRERELEASE (just  
before RC1 was tagged), amd64

41.464u 20.671s 1:02.04 100.1%  2430+1556k 0+0io 377pf+0w

4-core Opteron 280 2.4ghz, 4GB RAM, FreeBSD 7.0-RELEASE-p6, amd64
40.907u 18.638s 1:03.08 94.3%   2339+603k 182+91io 681pf+0w

Dual Athlon MP 2100+ 1.73ghz, 1GB RAM, FreeBSD 6.3-RELEASE, i386
82.812u 44.963s 2:06.89 100.6%  959+37724k 32+82io 46pf+0w

Dual P3 850mhz, 1GB RAM, FreeBSD 7.0-RELEASE-p4, i386
208.494u 84.935s 8:07.23 60.2%  2270+990k 17+87io 60pf+0w

4-core Opteron 2218 2.6ghz, 16GB RAM, FreeBSD 7.0-RELEASE-p4, amd64
38.893u 16.623s 0:55.53 99.9%   2290+591k 96+99io 48pf+0w

Dual Xeon 3.06GHz, 4GB RAM, FreeBSD 7.0-RELEASE-p4, i386
60.910u 24.667s 1:22.54 103.6%  2143+988k 146+134io 105pf+0w

Dual P3 866mhz, 2GB RAM, FreeBSD 7.0-RELEASE-p4, i386
169.135u 58.198s 3:52.71 97.6%  2443+1002k 160+99io 368pf+0w

2-core Core 2 Duo 2.33ghz, 2GB RAM, Mac OS X 10.5.6, i386
48.155u 29.896s 1:25.14 91.6%   0+0k 30+222io 1845pf+0w

4-core Xeon 2.66ghz, 6GB RAM, Mac OS X 10.5.6, i386
real1m17.024s user  0m44.373s sys   0m34.249s


None of these boxes were idle, so relative times are pretty useless,  
but i'm not seeing anything on the order of tens of minutes or hours.  
Is the source .tar.gz identical on all your systems?


-- Kevin

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: lzo2 shows insane speed gap

2008-12-29 Thread Bruce Cran
On Tue, 30 Dec 2008 01:47:47 +0100
Christian Weisgerber  wrote:

> Nate Eldredge:
> 
> > It might be good first to rule out compiler / library differences.
> 
> Sure.  Let's cut this short:
> 
> "Slow"
> Athlon 64 X2 5200+ 2.6 GHz,  FreeBSD 8.0-CURRENT amd64   ~60
> min Phenom 9350e 2.0 GHz,OpenBSD 4.4-CURRENT amd64
> ~80 min UltraSPARC-IIe 500 MHz (Blade 100),  OpenBSD 4.4-CURRENT
> sparc64  10 h++
> 
> "Fast"
> Pentium 4 3.0 GHz,   FreeBSD 6.4-RELEASE i386 36 s
> Xeon E5405 2.0 GHz (PowerEdge 1950), OpenBSD 4.4-CURRENT amd6447 s
> Alpha 21164A 500 MHz (AlphaPC164),   OpenBSD 4.4-CURRENT alpha 9
> min
> 
> Let me draw your attention to the fact that the two amd64 systems
> that run different operating systems are both slow, whereas the two
> amd64 systems that run the same operating system (compiler, libraries)
> diverge in speed.
> 
> 
> Oh, and everybody is invited to run
> 
> $ cd /usr/ports/archivers/lzo2 && make

I'm running 8.0-CURRENT amd64 here on a Turion64 X2 machine. Without
malloc debugging (malloc.conf -> aj) 'make test' takes 25s; after
removing malloc.conf thus turning on debugging, it takes over 10
minutes.

-- 
Bruce Cran
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: lzo2 shows insane speed gap

2008-12-29 Thread Christian Weisgerber
Nate Eldredge:

> It might be good first to rule out compiler / library differences.

Sure.  Let's cut this short:

"Slow"
Athlon 64 X2 5200+ 2.6 GHz,  FreeBSD 8.0-CURRENT amd64   ~60 min
Phenom 9350e 2.0 GHz,OpenBSD 4.4-CURRENT amd64   ~80 min
UltraSPARC-IIe 500 MHz (Blade 100),  OpenBSD 4.4-CURRENT sparc64  10 h++

"Fast"
Pentium 4 3.0 GHz,   FreeBSD 6.4-RELEASE i386 36 s
Xeon E5405 2.0 GHz (PowerEdge 1950), OpenBSD 4.4-CURRENT amd6447 s
Alpha 21164A 500 MHz (AlphaPC164),   OpenBSD 4.4-CURRENT alpha 9 min

Let me draw your attention to the fact that the two amd64 systems
that run different operating systems are both slow, whereas the two
amd64 systems that run the same operating system (compiler, libraries)
diverge in speed.


Oh, and everybody is invited to run

$ cd /usr/ports/archivers/lzo2 && make

and check for themselves.


PS: The Blade 100 is still crunching as I write this...
-- 
Christian "naddy" Weisgerber  na...@mips.inka.de
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: lzo2 shows insane speed gap

2008-12-29 Thread Christian Weisgerber
Mel  wrote:

> If the program itself doesn't directly cause the system time, do interrupt 
> rates give any hint as to what does?

systat -vmstat shows a conspicuously large number of traps, I think.
(I'm short on comparable FreeBSD machines.)

> And to rule out the obvious, you did check swapping?

No swapping.

-- 
Christian "naddy" Weisgerber  na...@mips.inka.de

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: lzo2 shows insane speed gap

2008-12-29 Thread Artem Belevich
I see this performance difference on my boxes.

First one has Core2Duo(E5-something), 4GB and  runs RELENG_7/i386.
lzotest is very fast.
Second box is Core2Quad (Q9450), 8GB RAM and runs -current as of about
a week ago. lzo2 binary built from ports is *slow*. However, 32-bit
binary from the first box runs very fast.

The only interesting difference I can see in ktrace is that read and
munmap take much much longer in case of 64-bit lzotest.

Here are two excerpts from ktrace on the second box:
### 32-bit app - runs fast on both boxes.

 59657 lzotest  0.10 CALL  open(0xd91b,O_RDONLY,0x1b6)
 59657 lzotest  0.07 NAMI  "./src/lzo1_d.ch"
 59657 lzotest  0.12 RET   open 3
 59657 lzotest  0.05 CALL  fstat(0x3,0xd504)
 59657 lzotest  0.07 STRU  struct stat {dev=102, ino=544718,
mode=-rw-r--r-- , nlink=1, uid=0, gid=0, rdev=2169160,
atime=1230595144, stime=1209559909, ctime=1230588212,
birthtime=1209559909, size=4563, blksize=4096, blocks=12, flags=0x0 }
 59657 lzotest  0.05 RET   fstat 0
 59657 lzotest  0.06 CALL  lseek(0x3,0,SEEK_SET,0x1)
 59657 lzotest  0.05 RET   lseek 0
 59657 lzotest  0.05 CALL  lseek(0x3,0x400,SEEK_SET,0)
 59657 lzotest  0.05 RET   lseek 67108864/0x400
 59657 lzotest  0.06 CALL  lseek(0x3,0,SEEK_SET,0)
 59657 lzotest  0.05 RET   lseek 0
 59657 lzotest  0.05 CALL
mmap(0,0x400,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,0x,0,0)
 59657 lzotest  0.07 RET   mmap 673185792/0x2820
 59657 lzotest  0.06 CALL  read(0x3,0x28196000,0x1000)
 59657 lzotest  0.10 GIO   fd 3 read 4096 bytes
 59657 lzotest  0.29 RET   read 4096/0x1000
 59657 lzotest  0.28 CALL  read(0x3,0x28196000,0x1000)
 59657 lzotest  0.10 GIO   fd 3 read 467 bytes
 59657 lzotest  0.05 RET   read 467/0x1d3
 59657 lzotest  0.10 CALL  read(0x3,0x28196000,0x1000)
 59657 lzotest  0.07 GIO   fd 3 read 0 bytes
 59657 lzotest  0.06 RET   read 0
 59657 lzotest  0.05 CALL  close(0x3)
 59657 lzotest  0.10 RET   close 0
 59657 lzotest  0.25 CALL  getrusage(0,0xd60c)
 59657 lzotest  0.06 RET   getrusage 0
 59657 lzotest  0.05 CALL  getrusage(0,0xd628)
 59657 lzotest  0.06 RET   getrusage 0
 59657 lzotest  0.05 CALL  getrusage(0,0xd60c)
 59657 lzotest  0.06 RET   getrusage 0
 59657 lzotest  0.64 CALL  getrusage(0,0xd60c)
 59657 lzotest  0.06 RET   getrusage 0
 59657 lzotest  0.05 CALL  getrusage(0,0xd60c)
 59657 lzotest  0.06 RET   getrusage 0
 59657 lzotest  0.29 CALL  getrusage(0,0xd60c)
 59657 lzotest  0.06 RET   getrusage 0
 59657 lzotest  0.12 CALL  getrusage(0,0xd60c)
 59657 lzotest  0.36 RET   getrusage 0
 59657 lzotest  0.10 CALL  write(0x1,0x28194000,0x4f)
 59657 lzotest  0.10 GIO   fd 1 wrote 79 bytes
 59657 lzotest  0.06 RET   write 79/0x4f
 59657 lzotest  0.06 CALL  munmap(0x2820,0x400)
 59657 lzotest  0.17 RET   munmap 0

### same file. 64-bit app (slow). Look at read/munmap

 59158 lzotest  0.15 CALL  open(0x7fffe760,O_RDONLY,0x1b6)
 59158 lzotest  0.14 NAMI  "./src/lzo1_d.ch"
 59158 lzotest  0.24 RET   open 3
 59158 lzotest  0.11 CALL  fstat(0x3,0x7fffe2d0)
 59158 lzotest  0.11 STRU  struct stat {dev=102, ino=544718,
mode=-rw-r--r-- , nlink=1, uid=0, gid=0, rdev=2169160,
atime=1230588427, stime=1209559909, ctime=1230588212,
birthtime=1209559909, size=4563, blksize=4096, blocks=12, flags=0x0 }
 59158 lzotest  0.07 RET   fstat 0
 59158 lzotest  0.15 CALL  lseek(0x3,0,SEEK_CUR)
 59158 lzotest  0.07 RET   lseek 0
 59158 lzotest  0.06 CALL  lseek(0x3,0x400,SEEK_SET)
 59158 lzotest  0.07 RET   lseek 67108864/0x400
 59158 lzotest  0.07 CALL  lseek(0x3,0,SEEK_SET)
 59158 lzotest  0.06 RET   lseek 0
 59158 lzotest  0.08 CALL
mmap(0,0x400,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,0x,0)
 59158 lzotest  0.10 RET   mmap 11534336/0x800b0
 59158 lzotest  0.074126 CALL  read(0x3,0x800a9e000,0x1000)
 59158 lzotest  0.54 GIO   fd 3 read 4096 bytes
 59158 lzotest  0.10 RET   read 4096/0x1000
 59158 lzotest  0.07 CALL  read(0x3,0x800a9e000,0x1000)
 59158 lzotest  0.12 GIO   fd 3 read 467 bytes
 59158 lzotest  0.06 RET   read 467/0x1d3
 59158 lzotest  0.07 CALL  read(0x3,0x800a9e000,0x1000)
 59158 lzotest  0.09 GIO   fd 3 read 0 bytes
 59158 lzotest  0.06 RET   read 0
 59158 lzotest  0.08 CALL  close(0x3)
 59158 lzotest  0.20 RET   close 0
 59158 lzotest  0.29 CALL  getrusage(0,0x7fffe3d0)
 59158 lzotest  0.10 RET   getrusage 0
 59158 lzotest  0.07 CALL  getrusage(0,0x7fffe3e0)
 59158 lzotest  0.07 RET   getrusage 0
 59158 lzotest  0.07 CALL  getrusage(0,0x7fffe3d0)
 59158 lzotest  0.07 RET   getrusage 0
 59158 lzotest  0.69 CALL  getrusage(0,0x7fffe3d0)
 59158 lzotest  0.07 RET   getrusage 0
 59158 lzotest  0.06 CALL  getrusage(0,0x7fffe3

Re: lzo2 shows insane speed gap

2008-12-29 Thread Yuri

Christian Weisgerber wrote:

My best guess at this time is that lzo2 somehow manages to induce
crazy cache thrashing on some CPU models.

Ideas and explanations welcome



Try running single command that is different on different machines under
valgrind (callgrind) on these machines and see that at least number of 
instructions

executed is the same.

Lzo2 documentation says that there are a lot of algorithms implemented.
It might be choosing the algorithm based on the CPU and the choice it's 
making might be bad.


Yuri
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: lzo2 shows insane speed gap

2008-12-29 Thread Mel
On Monday 29 December 2008 12:25:00 Christian Weisgerber wrote:
> On the "slow" machines, the tests also consume a lot of system time.
> I've seen figures from 20 to 50%.  However, ktrace shows nothing
> out of the ordinary.

If the program itself doesn't directly cause the system time, do interrupt 
rates give any hint as to what does?
And to rule out the obvious, you did check swapping?
-- 
Mel
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: lzo2 shows insane speed gap

2008-12-29 Thread Yuri

Christian Weisgerber wrote:



My best guess at this time is that lzo2 somehow manages to induce
crazy cache thrashing on some CPU models.

Ideas and explanations welcome.
  


Did you ask the author? He might be the best person to ask.

Yuri

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: lzo2 shows insane speed gap

2008-12-29 Thread Nate Eldredge

On Mon, 29 Dec 2008, Christian Weisgerber wrote:


The archivers/lzo2 port runs a series of regression tests after the
actual build.  These tests show extremely divergent behavior on
different machines.  There are two types of machines:

Type #1:
 Running the tests takes roughly the same time as configure and
 compile did, whether it's 30 seconds on a fast machine or 10
 minutes on an old slow one.

Type #2:
 Running the tests takes much, much, MUCH longer.

I've tried this across alpha, amd64, i386, and sparc64, partially
on FreeBSD, partially on OpenBSD.  The operating system doesn't
matter and there is no pattern related to endianness or 32/64 bits.

You can find machines that are the same architecture (e.g. amd64)
and are of similar overall speed (e.g. an Intel Xeon Xeon E5405 and
an AMD Phenom 9350e) and one of these machines will be type #1 and
the other will be #2 and take _a hundred_ times longer to run the
tests.  A hundred times.

I have never seen anything like this before.


It might be good first to rule out compiler / library differences.

First, can you isolate a single lzo command / input combination whose time 
differs dramatically?  This would simplify tests compared to running the 
whole test suite.  (It should be easy because it looks like the test suite 
prints the time for each test.)  It might also simplify things to work on 
one "fast" and one "slow" machine.


Then try copying the lzo binary from the "fast" machine to the "slow" 
machine (and vice versa) and see if the same test speeds up with the 
copied binary.  If not, try again with the binary statically linked.  If 
still not, it would be good to have a copy of the binary made available, 
along with more information about the "fast" and "slow" machines (CPU, 
amount of memory, load on the machine, kernel version, disk, etc).


If the copied binary isn't faster than the natively produced one, then it 
would be good to have information about the compiler options, versions, 
etc.


--

Nate Eldredge
neldre...@math.ucsd.edu
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: lzo2 shows insane speed gap

2008-12-29 Thread Dimitry Andric
On 2008-12-30 00:17, Dimitry Andric wrote:
> What's up with the memory on these machines?  Lzo tends to take insane 
> amounts

Duh, nevermind... I'm confusing this with lzma. :)  Sorry for the noise.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: lzo2 shows insane speed gap

2008-12-29 Thread Dimitry Andric
On 2008-12-29 22:25, Christian Weisgerber wrote:
> On the "slow" machines, the tests also consume a lot of system time.
> I've seen figures from 20 to 50%.  However, ktrace shows nothing
> out of the ordinary.

What's up with the memory on these machines?  Lzo tends to take insane 
amounts
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


lzo2 shows insane speed gap

2008-12-29 Thread Christian Weisgerber
The archivers/lzo2 port runs a series of regression tests after the
actual build.  These tests show extremely divergent behavior on
different machines.  There are two types of machines:

Type #1:
  Running the tests takes roughly the same time as configure and
  compile did, whether it's 30 seconds on a fast machine or 10
  minutes on an old slow one.

Type #2:
  Running the tests takes much, much, MUCH longer.

I've tried this across alpha, amd64, i386, and sparc64, partially
on FreeBSD, partially on OpenBSD.  The operating system doesn't
matter and there is no pattern related to endianness or 32/64 bits.

You can find machines that are the same architecture (e.g. amd64)
and are of similar overall speed (e.g. an Intel Xeon Xeon E5405 and
an AMD Phenom 9350e) and one of these machines will be type #1 and
the other will be #2 and take _a hundred_ times longer to run the
tests.  A hundred times.

I have never seen anything like this before.

On the "slow" machines, the tests also consume a lot of system time.
I've seen figures from 20 to 50%.  However, ktrace shows nothing
out of the ordinary.

My best guess at this time is that lzo2 somehow manages to induce
crazy cache thrashing on some CPU models.

Ideas and explanations welcome.

-- 
Christian "naddy" Weisgerber  na...@mips.inka.de

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: How process size is calculated? Is it always based on the current highest available address in memory space?

2008-12-29 Thread Robert Watson

On Mon, 29 Dec 2008, Dan Nelson wrote:


In the last episode (Dec 28), Yuri said:
malloc(3) can be controlled by MALLOC_OPTIONS to use mmap-based allocation 
as opposed to sbrk-based allocation. But allocations/deallocations with 
mmaps can eventually lead to non-continuously mmapped memory (having some 
non-mmapped gaps).


Are these gaps excluded from the process size or size is always linked to 
the current highest available address in memory space?


It looks like only mapped memory is counted in process size.  The test 
program below shows that the reported size goes down, even if a memory range 
inbetween two others is unmapped:


You can use procstat(8), or on older versions of FreeBSD procfs, to explore 
the layout of process memory, which may also lend some insight to what's going 
on.


Robert N M Watson
Computer Laboratory
University of Cambridge



$ ./mmap
Before mmap:
 UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT   TIME COMMAND
1000 48058 62165   0   8  0  1928   824 wait   S+ph0:00.01 ./mmap
mmap 64MB A=0x2828
 UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT   TIME COMMAND
1000 48058 62165   0   8  0 67464   824 wait   S+ph0:00.01 ./mmap
mmap 64MB B=0x2c28
 UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT   TIME COMMAND
1000 48058 62165   0   8  0 133000   824 wait   S+ph0:00.01 ./mmap
mmap 64MB C=0x3028
 UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT   TIME COMMAND
1000 48058 62165   0   8  0 198536   824 wait   S+ph0:00.01 ./mmap
munmap B
 UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT   TIME COMMAND
1000 48058 62165   0   8  0 133000   824 wait   S+ph0:00.01 ./mmap


#include 
#include 
#include 
#include 

int main(void)
{
 char *cmd;
 void *a, *b, *c;
 asprintf(&cmd, "ps axlp %d", getpid());
 printf("Before mmap:\n");
 system(cmd);
 a = mmap(NULL, 64 * 1024 * 1024, PROT_READ|PROT_WRITE, MAP_ANON, -1, 0);
 printf("mmap 64MB A=%p\n", a);
 system(cmd);
 b = mmap(NULL, 64 * 1024 * 1024, PROT_READ|PROT_WRITE, MAP_ANON, -1, 0);
 printf("mmap 64MB B=%p\n", b);
 system(cmd);
 c = mmap(NULL, 64 * 1024 * 1024, PROT_READ|PROT_WRITE, MAP_ANON, -1, 0);
 printf("mmap 64MB C=%p\n", c);
 system(cmd);
 printf("munmap B\n");
 munmap(b, 64 * 1024 * 1024);
 system(cmd);
 return 0;
}


--
Dan Nelson
dnel...@allantgroup.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: How process size is calculated? Is it always based on the current highest available address in memory space?

2008-12-29 Thread Dan Nelson
In the last episode (Dec 28), Yuri said:
> malloc(3) can be controlled by MALLOC_OPTIONS to use mmap-based
> allocation as opposed to sbrk-based allocation. But
> allocations/deallocations with mmaps can eventually lead to
> non-continuously mmapped memory (having some non-mmapped gaps).
> 
> Are these gaps excluded from the process size or size is always
> linked to the current highest available address in memory space?

It looks like only mapped memory is counted in process size.  The test
program below shows that the reported size goes down, even if a memory
range inbetween two others is unmapped:

$ ./mmap
Before mmap:
  UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT   TIME COMMAND
 1000 48058 62165   0   8  0  1928   824 wait   S+ph0:00.01 ./mmap
mmap 64MB A=0x2828
  UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT   TIME COMMAND
 1000 48058 62165   0   8  0 67464   824 wait   S+ph0:00.01 ./mmap
mmap 64MB B=0x2c28
  UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT   TIME COMMAND
 1000 48058 62165   0   8  0 133000   824 wait   S+ph0:00.01 ./mmap
mmap 64MB C=0x3028
  UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT   TIME COMMAND
 1000 48058 62165   0   8  0 198536   824 wait   S+ph0:00.01 ./mmap
munmap B
  UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT   TIME COMMAND
 1000 48058 62165   0   8  0 133000   824 wait   S+ph0:00.01 ./mmap


#include 
#include 
#include 
#include 

int main(void)
{
  char *cmd;
  void *a, *b, *c;
  asprintf(&cmd, "ps axlp %d", getpid());
  printf("Before mmap:\n");
  system(cmd);
  a = mmap(NULL, 64 * 1024 * 1024, PROT_READ|PROT_WRITE, MAP_ANON, -1, 0);
  printf("mmap 64MB A=%p\n", a);
  system(cmd);
  b = mmap(NULL, 64 * 1024 * 1024, PROT_READ|PROT_WRITE, MAP_ANON, -1, 0);
  printf("mmap 64MB B=%p\n", b);
  system(cmd);
  c = mmap(NULL, 64 * 1024 * 1024, PROT_READ|PROT_WRITE, MAP_ANON, -1, 0);
  printf("mmap 64MB C=%p\n", c);
  system(cmd);
  printf("munmap B\n");
  munmap(b, 64 * 1024 * 1024);
  system(cmd);
  return 0;
}


-- 
Dan Nelson
dnel...@allantgroup.com
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


Re: debugging mbuf allocation/dealocation

2008-12-29 Thread Nikola Knežević


You can use the KTR(4) facility to trace memory allocations and  
deallocations, logging them to memory, disk, etc.  Unfortunately  
interpreting the data can be fairly tricky, as network leaks tend to  
happen over a long period of time, be stored in sockets, etc, but  
it's definitely possible and has been done. :-) Processing the  
results with a perl script goes a long way, as the allocated/freed  
pointers are included, etc.


Hi Robert,

thanks for the pointers :)
I discovered that root cause of the leakage - Click::Packet had a bad  
destructor, which wasn't free-ing mbufs all the time. Since there are  
some other forms of leaking in Click, I'll use KTR do discover them.


Cheers,
Nikola
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"