RE: [expert] Netscape wish using up memory
Cheers Laurent - perfectly acceptable answer from Jeff, and I can see the logic behind it. Thanks for looking into it for me. Satisfied my curiosity anyway! Steve Flynn NOP Data Migration Ops Analyst * 01603 687386 -Original Message- From: Laurent Duperval [SMTP:[EMAIL PROTECTED]] Sent: Thursday, September 06, 2001 5:04 PM To: [EMAIL PROTECTED] Subject:Re: [expert] Netscape wish using up memory On 6 Sep, To: [EMAIL PROTECTED] wrote: On 5 Sep, FLYNN, Steve wrote: Exactly what is the point in that? I've never done any Tcl programming (yet) but I can't see the reasoning behind this. I don't remember what the exact design decision was. It may have been somnething to do with performance. I'll ask and give you the answer later. It's a multi-tasking OS - everything should be handed back in a friendly manner. It sounds like a deliberate design ploy on the Tcl interpreter authors, presumably for performance increases, but it appears to fail as it causes swapping itself! Have I got the wrong end of the stick? In this case, probably. I suspect the the tkppoe script keeps large amounts of data in memory and never releases it. For example, maybe it sets an array aor a list and constantly adds data to it. In that case, no matter what Tcl does with memory, it will just keep on growing and growing. On the other hand, if the script allocates large amounts of data to do temporary datamanipulation, then releases the handle to that data, the memory growth will be a result of the way Tcl manages memory. Until I see the script, I can't say for sure which of these situations is at work. L As promised, here is Jeff Hobbs' answer, which you can find on comp.lang.tcl (and I think any subsequent discussions on the memory allocation aspect of Tcl should be taken there): Yes, it is in large part for performance. Yes, it still makes a noticable difference in that latest machines, and likely always will since the concept is fairly basic. This is exactly what Tcl does: It allocates Tcl_Obj's in blocks of 100. Ask for the first one, you get 100 and it pops one off to you. The next 99 are already allocated and just popped off to you. When you free a Tcl_Obj, it goes back on to a stack of free objects which can be reused. Any memory associated with the Tcl_Obj (string, list rep, other object data) is all freed, it is just the Tcl_Obj shell that is kept on the stack for reuse. This means if you have 1001 objects in use at once, you will have allocated at least 1100 Tcl_Obj's. However, if you used 1001 objects, but never more than 50 at a time, you only ever allocated 100 Tcl_Obj (others were reused). This goes on for the lifetime of the process. Thus, the objects used high-water mark is maintained - with the philosophy that if you used it before, you may need it again. The way in which they are maintained on the stack is done in a very efficient manner (linking to each other), which unfortunately makes it impossible to deallocate. To free, you would have to have the entire block of 100 Tcl_Obj's ready to free and remember which one was the first. What the person may have been seeing is some mem leak in Tcl. There were (*were*) a few that we hard to exercise, but 8.3.2 had a fairly easy to hit one in channels (unfortunately) that was fixed in 8.3.3. So, while it is true that certain parts of Tcl don't free mem back, it is just the Tcl_Obj shell allocator, everything else gets freed (structures, dstrings, strings, ...). --- Since tkpppoe most likely uses channels, it may be exercising the bug that Jeff mentions. You could try 8.3.3 to see if your script still ill-behaves. L -- Laurent Duperval mailto:[EMAIL PROTECTED] Ne vous moquez pas maître. C'est le titillement de la recherche scientifique qui me fit sortir du lit, malgré l'aube matinale de l'après-midi! -Le disciple File: message.footer ** This email and any files sent with it are intended only for the named recipient. If you are not the named recipient please telephone/email the sender immediately. You should not disclose the content or take/retain/distribute any copies
Re: [expert] Netscape wish using up memory
On Thu, Sep 06, 2001 at 09:59 -0400, Laurent Duperval wrote: On the other hand, if the script allocates large amounts of data to do temporary datamanipulation, then releases the handle to that data, the memory growth will be a result of the way Tcl manages memory. Until I see the script, I can't say for sure which of these situations is at work. Want me to send it to you as .bz2 (24kb)? wobo -- ... and anyway, an html can't carry a virus. (Aug 2001, Usenet) --- [EMAIL PROTECTED] ICQ: #128612867 GPG-ID: A69882EE --- ISDN4LINUX-FAQ -- Deutsch: http://www.wolf-b.de/i4l/i4lfaq-de.html Want to buy your Pack or Services from MandrakeSoft? Go to http://www.mandrakestore.com
Re: [expert] Netscape wish using up memory
On Thu, Sep 06, 2001 at 14:04 -0400, Laurent Duperval wrote: On 6 Sep, To: [EMAIL PROTECTED] wrote: As promised, here is Jeff Hobbs' answer, which you can find on comp.lang.tcl (and I think any subsequent discussions on the memory allocation aspect of Tcl should be taken there): I got another reply from Jeff: If they are using 8.3.2 (then they should upgrade), just grep for open or exec. If it's there, they have a leak. That simple. :( IIRC, it's 19 bytes leaked per call. Not much, but it adds up. OK, d'led tk-8.3.3-5mdk from Cooker and upgraded. # rpm -q --whatprovides /usr/bin/wish tk-8.3.3-5mdk Startet tkpppoe and startet top (sorted on memory useage). wish startet out at 2200. Now, after 5:30 hours it shows like this (I did not connect to the internet during that time, so tkpppoe was just startet but did not do anything!): 12:20am up 5:35, 3 users, load average: 0,03, 0,06, 0,02 78 processes: 74 sleeping, 4 running, 0 zombie, 0 stopped CPU states: 6,1% user, 11,7% system, 0,0% nice, 82,0% idle Mem: 247696K av, 197080K used, 50616K free, 0K shrd, 12108K buff Swap: 401584K av, 0K used, 401584K free 107636K cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 2149 wobo 9 0 27844 27M 1940 S 1,1 11,2 3:07 wish 1959 root 10 0 13828 13M 1824 S 3,9 5,5 1:32 X [] The update of tk from 8.3.2 to 8.3.3 hasn't changed anything. But anyway, thanks for trying. wobo -- ... and anyway, an html can't carry a virus. (Aug 2001, Usenet) --- [EMAIL PROTECTED] ICQ: #128612867 GPG-ID: A69882EE --- ISDN4LINUX-FAQ -- Deutsch: http://www.wolf-b.de/i4l/i4lfaq-de.html Want to buy your Pack or Services from MandrakeSoft? Go to http://www.mandrakestore.com
Re: [expert] Netscape wish using up memory
On 6 Sep, Wolfgang Bornath wrote: On Thu, Sep 06, 2001 at 09:59 -0400, Laurent Duperval wrote: On the other hand, if the script allocates large amounts of data to do temporary datamanipulation, then releases the handle to that data, the memory growth will be a result of the way Tcl manages memory. Until I see the script, I can't say for sure which of these situations is at work. Want me to send it to you as .bz2 (24kb)? wobo Sure but no promises until next week. L -- Laurent Duperval mailto:[EMAIL PROTECTED] If nothing ever sticks to TEFLON, how do they make TEFLON stick to the pan? Want to buy your Pack or Services from MandrakeSoft? Go to http://www.mandrakestore.com
Re: [expert] Netscape wish using up memory
On Thu, Sep 06, 2001 at 11:51 -0400, Laurent Duperval wrote: On 6 Sep, Wolfgang Bornath wrote: Want me to send it to you as .bz2 (24kb)? wobo Sure but no promises until next week. Great. I'll send it direct to you. Take your time, I want to use that proggie for a long time and for now I can live with shutting it off and on again. Pity is, I have to cut the connection each time. If nothing ever sticks to TEFLON, how do they make TEFLON stick to the pan? B/c the pan sticks to TEFLON. ;-) wobo -- ... and anyway, an html can't carry a virus. (Aug 2001, Usenet) --- [EMAIL PROTECTED] ICQ: #128612867 GPG-ID: A69882EE --- ISDN4LINUX-FAQ -- Deutsch: http://www.wolf-b.de/i4l/i4lfaq-de.html Want to buy your Pack or Services from MandrakeSoft? Go to http://www.mandrakestore.com
Re: [expert] Netscape wish using up memory
On 6 Sep, To: [EMAIL PROTECTED] wrote: On 5 Sep, FLYNN, Steve wrote: Exactly what is the point in that? I've never done any Tcl programming (yet) but I can't see the reasoning behind this. I don't remember what the exact design decision was. It may have been somnething to do with performance. I'll ask and give you the answer later. It's a multi-tasking OS - everything should be handed back in a friendly manner. It sounds like a deliberate design ploy on the Tcl interpreter authors, presumably for performance increases, but it appears to fail as it causes swapping itself! Have I got the wrong end of the stick? In this case, probably. I suspect the the tkppoe script keeps large amounts of data in memory and never releases it. For example, maybe it sets an array aor a list and constantly adds data to it. In that case, no matter what Tcl does with memory, it will just keep on growing and growing. On the other hand, if the script allocates large amounts of data to do temporary datamanipulation, then releases the handle to that data, the memory growth will be a result of the way Tcl manages memory. Until I see the script, I can't say for sure which of these situations is at work. L As promised, here is Jeff Hobbs' answer, which you can find on comp.lang.tcl (and I think any subsequent discussions on the memory allocation aspect of Tcl should be taken there): Yes, it is in large part for performance. Yes, it still makes a noticable difference in that latest machines, and likely always will since the concept is fairly basic. This is exactly what Tcl does: It allocates Tcl_Obj's in blocks of 100. Ask for the first one, you get 100 and it pops one off to you. The next 99 are already allocated and just popped off to you. When you free a Tcl_Obj, it goes back on to a stack of free objects which can be reused. Any memory associated with the Tcl_Obj (string, list rep, other object data) is all freed, it is just the Tcl_Obj shell that is kept on the stack for reuse. This means if you have 1001 objects in use at once, you will have allocated at least 1100 Tcl_Obj's. However, if you used 1001 objects, but never more than 50 at a time, you only ever allocated 100 Tcl_Obj (others were reused). This goes on for the lifetime of the process. Thus, the objects used high-water mark is maintained - with the philosophy that if you used it before, you may need it again. The way in which they are maintained on the stack is done in a very efficient manner (linking to each other), which unfortunately makes it impossible to deallocate. To free, you would have to have the entire block of 100 Tcl_Obj's ready to free and remember which one was the first. What the person may have been seeing is some mem leak in Tcl. There were (*were*) a few that we hard to exercise, but 8.3.2 had a fairly easy to hit one in channels (unfortunately) that was fixed in 8.3.3. So, while it is true that certain parts of Tcl don't free mem back, it is just the Tcl_Obj shell allocator, everything else gets freed (structures, dstrings, strings, ...). --- Since tkpppoe most likely uses channels, it may be exercising the bug that Jeff mentions. You could try 8.3.3 to see if your script still ill-behaves. L -- Laurent Duperval mailto:[EMAIL PROTECTED] Ne vous moquez pas maître. C'est le titillement de la recherche scientifique qui me fit sortir du lit, malgré l'aube matinale de l'après-midi! -Le disciple Want to buy your Pack or Services from MandrakeSoft? Go to http://www.mandrakestore.com
Re: [expert] Netscape wish using up memory
On 5 Sep, FLYNN, Steve wrote: Exactly what is the point in that? I've never done any Tcl programming (yet) but I can't see the reasoning behind this. I don't remember what the exact design decision was. It may have been somnething to do with performance. I'll ask and give you the answer later. It's a multi-tasking OS - everything should be handed back in a friendly manner. It sounds like a deliberate design ploy on the Tcl interpreter authors, presumably for performance increases, but it appears to fail as it causes swapping itself! Have I got the wrong end of the stick? In this case, probably. I suspect the the tkppoe script keeps large amounts of data in memory and never releases it. For example, maybe it sets an array aor a list and constantly adds data to it. In that case, no matter what Tcl does with memory, it will just keep on growing and growing. On the other hand, if the script allocates large amounts of data to do temporary datamanipulation, then releases the handle to that data, the memory growth will be a result of the way Tcl manages memory. Until I see the script, I can't say for sure which of these situations is at work. L -- Laurent Duperval mailto:[EMAIL PROTECTED] All of you are sick! All of you should be hospitalized!! -Shaul Yahalom of the Israeli parliament, responding to members who suggest King David was gay Want to buy your Pack or Services from MandrakeSoft? Go to http://www.mandrakestore.com
RE: [expert] Netscape wish using up memory
Exactly what is the point in that? I've never done any Tcl programming (yet) but I can't see the reasoning behind this. It's a multi-tasking OS - everything should be handed back in a friendly manner. It sounds like a deliberate design ploy on the Tcl interpreter authors, presumably for performance increases, but it appears to fail as it causes swapping itself! Have I got the wrong end of the stick? Steve Flynn NOP Data Migration Ops Analyst * 01603 687386 -Original Message- From: Laurent Duperval [SMTP:[EMAIL PROTECTED]] Sent: Wednesday, September 05, 2001 4:13 PM To: Mandrake Expert List Subject:Re: [expert] Netscape wish using up memory Wish (and Tcl in general) does not release memory to the OS. Instead, when memory is released, it is maintianed in a pool to be reused by the application. Most likely, the Tcl (wish) script you are using is using up a lot of memory and never releasing it. One example is setting a new variable or an array and never unsetting it after it is no longer needed. You have to look at the code for tkpppoe to see where that happens. L -- Laurent Duperval mailto:[EMAIL PROTECTED] Bourgeois, ne niez pas: le frisson de l'envie chante au fond de votre prunelle! -Le commerçant d'Achille Talon File: message.footer ** This email and any files sent with it are intended only for the named recipient. If you are not the named recipient please telephone/email the sender immediately. You should not disclose the content or take/retain/distribute any copies. ** Norwich Union Life Pensions Limited Registered Office 2 Rougier Street York YO90 1UU Registered in England Number 3253947 A member of the Norwich Union Marketing Group which is regulated by the Personal Investment Authority. Member of the Association of British Insurers. For further Enquires 01603 622200 Want to buy your Pack or Services from MandrakeSoft? Go to http://www.mandrakestore.com
Re: [expert] Netscape wish using up memory
On Wed, Sep 05, 2001 at 12:12 -0400, Laurent Duperval wrote: On 14 Aug, Wolfgang Bornath wrote: A similar thing happened when I used to work with tkpppoe, a front end for pppoe for my DSL connention. This tkpppoe works with wish and when I saw xosview showing memory useage of more than 400M (ram + swap) I looked at top and top showed wish using more than 260M of memory. As soon as I exited tkpppoe those 260M were immediately released. How come wish is taking so much memory? wobo Wish (and Tcl in general) does not release memory to the OS. Instead, when memory is released, it is maintianed in a pool to be reused by the application. Most likely, the Tcl (wish) script you are using is using up a lot of memory and never releasing it. One example is setting a new variable or an array and never unsetting it after it is no longer needed. You have to look at the code for tkpppoe to see where that happens. You can see it in the 'top' list. Starting tkpppoe it's listed with PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 23664 wobo 9 0 2712 2712 1868 S 0,0 1,0 0:00 wish After running some time I have it like this: PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 22982 wobo 15 0 40416 39M 1924 S 1,1 16,3 4:53 wish You can even watch it consuming memory when you look at 'top' running for a minute. As I'm not too good in scripting and/or coding I just close it now and then and open it again. wobo -- ... and anyway, an html can't carry a virus. (Aug 2001, Usenet) --- [EMAIL PROTECTED] ICQ: #128612867 GPG-ID: A69882EE --- ISDN4LINUX-FAQ -- Deutsch: http://www.wolf-b.de/i4l/i4lfaq-de.html Want to buy your Pack or Services from MandrakeSoft? Go to http://www.mandrakestore.com
Re: [expert] Netscape wish using up memory
On 14 Aug, Wolfgang Bornath wrote: Hi, Just posting this to give an example for Netscape's still unbelievable wasting of memory: I use MDK 8.0 with Netscape 4.77 as browser. My machine got 256M of physical ram and a swap partition of 300M. Now I had Netscape running for 2 days because i kept working on several translations of MandrakeForum articles. The last one was a piece by Civileme about what gets testet by Mandrake QA. I took some breaks in between and in the meantime I read and wrote some mails with Mutt and there was just one console window open and xosview running. All of this took place on KDE. At the end my harddisk startet to run frequently and xosview showed that it startet swapping out. In the end I had a memory assessment of 240M physical ram (just about 10% of it that was cache) and more than 280M of swap space was in use. Reaction of the system startet to be sluggish. Now I sent the finished translation up to the Forum and when I closed Netscape xosview showed a memory occupation of 73M of ram and 52 of swap! That means Netscape has taken more than 380M of memory! A similar thing happened when I used to work with tkpppoe, a front end for pppoe for my DSL connention. This tkpppoe works with wish and when I saw xosview showing memory useage of more than 400M (ram + swap) I looked at top and top showed wish using more than 260M of memory. As soon as I exited tkpppoe those 260M were immediately released. How come wish is taking so much memory? wobo Wish (and Tcl in general) does not release memory to the OS. Instead, when memory is released, it is maintianed in a pool to be reused by the application. Most likely, the Tcl (wish) script you are using is using up a lot of memory and never releasing it. One example is setting a new variable or an array and never unsetting it after it is no longer needed. You have to look at the code for tkpppoe to see where that happens. L -- Laurent Duperval mailto:[EMAIL PROTECTED] Bourgeois, ne niez pas: le frisson de l'envie chante au fond de votre prunelle! -Le commerçant d'Achille Talon Want to buy your Pack or Services from MandrakeSoft? Go to http://www.mandrakestore.com
Re: [expert] Netscape wish using up memory
On 15-Aug-2001 Jaime Herazo B . wrote: * Naka Gadjov ([EMAIL PROTECTED]) wrote: Netscape is using much memory everywhere. In Windows, and strange NS suffers by same problems in Linux too. I am waiting for a 5 years for a new version that do not have memory leakage, but at the moment there is not. Strange Mozilla have the same problem. I prefer Opera, but usually i stick with lynx :) You could check out konqueror too I also prefer Opera. It's amazingly fast it's much more stable then netscape. -- E-Mail: Gregor Maier [EMAIL PROTECTED] Date: 16-Aug-2001 Time: 09:23:36 -- Want to buy your Pack or Services from MandrakeSoft? Go to http://.mandrakestore.com
Re: [expert] Netscape wish using up memory
Wolfgang Bornath wrote: Hi, Just posting this to give an example for Netscape's still unbelievable wasting of memory: Netscape is using much memory everywhere. In Windows, and strange NS suffers by same problems in Linux too. I am waiting for a 5 years for a new version that do not have memory leakage, but at the moment there is not. Strange Mozilla have the same problem. _ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com Want to buy your Pack or Services from MandrakeSoft? Go to http://.mandrakestore.com
Re: [expert] Netscape wish using up memory
At 06:03 AM 08/14/2001 +0200, Wolfgang Bornath wrote: Hi, [snip] At the end my harddisk startet to run frequently and xosview showed that it startet swapping out. In the end I had a memory assessment of 240M physical ram (just about 10% of it that was cache) and more than 280M of swap space was in use. Reaction of the system startet to be sluggish. Now I sent the finished translation up to the Forum and when I closed Netscape xosview showed a memory occupation of 73M of ram and 52 of swap! That means Netscape has taken more than 380M of memory! A similar thing happened when I used to work with tkpppoe, a front end for pppoe for my DSL connention. This tkpppoe works with wish and when I saw xosview showing memory useage of more than 400M (ram + swap) I looked at top and top showed wish using more than 260M of memory. As soon as I exited tkpppoe those 260M were immediately released. How come wish is taking so much memory? Most likely, this is caused by memory leaks -- bugs, to the rest of us ;-) Dave Want to buy your Pack or Services from MandrakeSoft? Go to http://.mandrakestore.com
