Re: [mythtv-users] Re: Is this a memory leak?
It could be a driver bug *if* your talking about a few connections with low message rates (sub 100 connections with message rates of < 5/sec). This specifically references high message rates (> 8/sec) and/or large numbers of virtual connections (>1). Under these circumstances with packet sizes being anywhere from 128 to 4k in size and being sent over slow lines (generally 9600 baud and slower), message buffering can easily use up memory. Besides, the same streams drivers on Solaris, QNX, and other OS's don't have this problem. It is specifically a problem on linux that is a direct result of linux's kernel memory release policy. The point is, that people should be aware that this occurs on linux and could explain strange problems with suddenly not being able to aquire memory. Either way, this really isn't the place for this discussion and I'll no longer post replies for this thread. Simon Kenyon wrote: On Tuesday 11 January 2005 15:57, Michael J. Lynch wrote: Except from whitepaper: This poses a problem for anyone wanting to use Linux as a communications server handling a large number of connections or queueing a large number of messages. The reason is that calling the kernel memory allocator from a STREAMS driver will not cause disk buffers to be released to satisfy the allocation request. Thus, if memory is nearly full of disk buffers and the like there is only a small amount of memory left for control blocks and messages. sounds like a bug in the driver to me -- simon ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users -- Michael J. Lynch What if the hokey pokey IS what it's all about -- author unknown ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] Re: Is this a memory leak?
On Tuesday 11 January 2005 15:57, Michael J. Lynch wrote: > Except from whitepaper: > > This poses a problem for anyone wanting to use Linux as a communications > server handling a large number of connections or queueing a large number > of messages. The reason is that calling the kernel memory allocator > from a STREAMS driver will not cause disk buffers to be released to > satisfy the allocation request. Thus, if memory is nearly full of disk > buffers and the like there is only a small amount of memory left for > control blocks and messages. sounds like a bug in the driver to me -- simon ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] Re: Is this a memory leak?
Except from whitepaper: This poses a problem for anyone wanting to use Linux as a communications server handling a large number of connections or queueing a large number of messages. The reason is that calling the kernel memory allocator from a STREAMS driver will not cause disk buffers to be released to satisfy the allocation request. Thus, if memory is nearly full of disk buffers and the like there is only a small amount of memory left for control blocks and messages. This means that an application that manages a large number of connections or sends a large number of messages, or both, can run fine if started just after a reboot, but will fail due to memory allocation failures when started later once memory has been committed to disk buffers and other cached items. Simon Kenyon wrote: On Tuesday 11 January 2005 13:10, Michael J. Lynch wrote: I don't know if this has anything to do with what you are seeing but recently I had occasion to look at the disk caching code for linux and it turns out that linux will use ALL available memory for disk cache and only release it back to the system under very specific circumstances. what else should it do with it - leave it unused? that would be a waste -- simon ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users -- Michael J. Lynch What if the hokey pokey IS what it's all about -- author unknown ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] Re: Is this a memory leak?
On Tuesday 11 January 2005 13:10, Michael J. Lynch wrote: > I don't know if this has anything to do with what you are seeing but > recently I had occasion to look at the disk caching code for linux and > it turns out that linux will use ALL available memory for disk cache > and only release it back to the system under very specific circumstances. what else should it do with it - leave it unused? that would be a waste -- simon ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] Re: Is this a memory leak?
I don't know if this has anything to do with what you are seeing but recently I had occasion to look at the disk caching code for linux and it turns out that linux will use ALL available memory for disk cache and only release it back to the system under very specific circumstances. I can post a whitepaper describing exactly what happens and the mechanisms involved in the context of LiS (linux streams) if you are interested. Don Brett wrote: OK, maybe I sent everyone on a wild goose chase. Somebody tell me if this makes sense: - Start X, Mythbackend, Mythfrontend, and start to play live TV, all is fine. - Watch memory and it grows to almost max, but mythfrontend doesn't increase. - Stop movie and memory returns to normal. - Start live TV again and let it run. Memory grows to max, then swapping starts. Is the memory growth due to buffered-up video (which is used for jumping back)? If that's the case, then this is perfectly normal operation. Or, is there a limit on the amount of ram that mythfrontend should consume during playback? It's starting to sound like my freezing problem is caused by something else? Comments? Thanks, Don ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users -- Michael J. Lynch What if the hokey pokey IS what it's all about -- author unknown ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
[mythtv-users] Re: Is this a memory leak?
OK, maybe I sent everyone on a wild goose chase. Somebody tell me if this makes sense: - Start X, Mythbackend, Mythfrontend, and start to play live TV, all is fine. - Watch memory and it grows to almost max, but mythfrontend doesn't increase. - Stop movie and memory returns to normal. - Start live TV again and let it run. Memory grows to max, then swapping starts. Is the memory growth due to buffered-up video (which is used for jumping back)? If that's the case, then this is perfectly normal operation. Or, is there a limit on the amount of ram that mythfrontend should consume during playback? It's starting to sound like my freezing problem is caused by something else? Comments? Thanks, Don ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
Re: [mythtv-users] Re: Is this a memory leak?
On Monday 10 January 2005 08:26 am, Don Brett wrote: > Here's some additional data. Actually, I don't see mythbackend or > mythfrontend growing at all. It only happens while playing (movie or > live tv). Thanks for the help, can I provide any more data? Nope. If the processes aren't growing, it's not a memory leak. Isaac ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
[mythtv-users] Re: Is this a memory leak?
Forgot to mention this: yes, I have about a gig of swap drive setup on the box. Is that the swap you meant? Thanks, Don ___ mythtv-users mailing list mythtv-users@mythtv.org http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
[mythtv-users] Re: Is this a memory leak?
Here's some additional data. Actually, I don't see mythbackend or mythfrontend growing at all. It only happens while playing (movie or live tv). Thanks for the help, can I provide any more data? Don Several captures from Top: TopAtStart.txt - X started with no other programs running TopWithMythbackendOnly.txt - X started - Mythbackend started manually TopWithMythfrontend.txt - X started - Mythbackend started manually - Mythbackend started manually TopAfterPlayingOneMinute.txt - X started - Mythbackend started manually - Mythbackend started manually - Live TV playing for one minute TopAfterPlayingTenMinute.txt - X started - Mythbackend started manually - Mythbackend started manually - Live TV playing for ten minutes 07:59:14 up 7 min, 5 users, load average: 0.16, 0.27, 0.14 104 processes: 103 sleeping, 1 running, 0 zombie, 0 stopped CPU states: 1.1% user 0.7% system 0.0% nice 0.0% iowait 98.0% idle Mem: 385784k av, 245692k used, 140092k free, 0k shrd,9604k buff 45236k active, 147032k inactive Swap: 979956k av, 0k used, 979956k free 97448k cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND 376 root 9 0 35860 35M 11412 S 0.0 9.2 0:03 0 mythfrontend 223 root 14 -1 32636 23M 2696 S < 0.7 6.2 0:21 0 X 279 root 9 0 15280 14M 13652 S 0.0 3.9 0:00 0 kdeinit 322 root 11 0 14856 14M 13348 S 0.3 3.8 0:02 0 kdeinit 285 root 9 0 14684 14M 13232 S 0.0 3.8 0:00 0 kdeinit 287 root 9 0 14684 14M 13232 S 0.0 3.8 0:00 0 kdeinit 268 root 9 0 14324 13M 12912 S 0.0 3.7 0:00 0 kdeinit 276 root 9 0 14308 13M 13064 S 0.0 3.7 0:00 0 kdeinit 164 mysql 9 0 14216 13M 2904 S 0.0 3.6 0:00 0 mysqld 272 root 9 0 13200 12M 12112 S 0.0 3.4 0:00 0 kdeinit 283 root 9 0 12728 12M 11680 S 0.0 3.2 0:00 0 kdeinit 288 root 9 0 12568 12M 10792 S 0.0 3.2 0:00 0 korgac 274 root 8 0 12268 11M 11420 S 0.0 3.1 0:00 0 kdeinit 256 root 9 0 11896 11M 11124 S 0.0 3.0 0:00 0 kdeinit 271 root 9 0 11428 11M 10748 S 0.0 2.9 0:00 0 kdeinit 320 root 9 0 11088 10M 9480 S 0.0 2.8 0:00 0 kalarmd 254 root 9 0 10404 10M 1 S 0.0 2.6 0:00 0 kdeinit 341 root 9 0 10376 10M 7992 S 0.0 2.6 0:00 0 mythbackend 339 root 9 0 9744 9740 9496 S 0.0 2.5 0:00 0 kdeinit 251 root 9 0 9312 9308 9032 S 0.0 2.4 0:00 0 kdeinit 248 root 8 0 9308 9304 9156 S 0.0 2.4 0:00 0 kdeinit 174 root 9 0 4284 4284 4196 S 0.0 1.1 0:00 0 httpd 128 root 9 0 2072 2072 1524 S 0.0 0.5 0:00 0 sendmail 131 smmsp 8 0 1964 1956 1492 S 0.0 0.5 0:00 0 sendmail 176 root 9 0 1824 1824 1268 S 0.0 0.4 0:00 0 smbd 178 root 9 0 1708 1708 1568 S 0.0 0.4 0:00 0 nmbd 113 root 9 0 1416 1416 1200 S 0.0 0.3 0:00 0 sshd 292 root 9 0 1328 1328 1032 S 0.0 0.3 0:00 0 bash 323 root 9 0 1324 1324 1032 S 0.0 0.3 0:00 0 bash 08:10:11 up 18 min, 5 users, load average: 0.10, 0.12, 0.11 111 processes: 110 sleeping, 1 running, 0 zombie, 0 stopped CPU states: 0.9% user 0.9% system 0.0% nice 0.0% iowait 98.0% idle Mem: 385784k av, 381160k used,4624k free, 0k shrd,1412k buff 238480k active, 109724k inactive Swap: 979956k av,1544k used, 978412k free 254012k cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND 376 root 9 0 42732 41M 12712 S 0.0 11.0 0:03 0 mythfrontend 223 root 14 -1 32764 23M 2696 S < 0.3 6.2 0:31 0 X 279 root 9 0 15288 14M 13652 S 0.0 3.9 0:01 0 kdeinit 164 mysql 9 0 14352 14M 2920 S 0.0 3.7 0:00 0 mysqld 268 root 9 0 14324 13M 12912 S 0.0 3.7 0:00 0 kdeinit 276 root 9 0 14308 13M 13064 S 0.0 3.7 0:00 0 kdeinit 322 root 9 0 14808 13M 12788 S 0.0 3.7 0:03 0 kdeinit 285 root 9 0 14632 13M 12660 S 0.0 3.6 0:00 0 kdeinit 287 root 9 0 14632 13M 12660 S 0.0 3.6 0:00 0 kdeinit 272 root 9 0 13200 12M 12112 S 0.0 3.4 0:01 0 kdeinit 341 root 9 0 13192 12M 8376 S 0.0 3.4 0:00 0 mythbackend 283 root 9 0 12744 12M 11680 S 0.0 3.3 0:00 0 kdeinit 288 root 9 0 12548 12M 10772 S 0.0 3.2 0:00 0 korgac 274 root 8 0 12268 11M 11420 S 0.0 3.1 0:00 0 kdeinit 256 root 9 0 11896 11M 11124 S 0.0 3.0 0:00 0 kdeinit 271 root 9 0 11432 11M 10748 S 0.0
