[AOLSERVER] new AOL-based thread-happy mem allocator in 8.4
I have finally commited the improved memory allocator from the AOLServer Tcl mods to the 8.4 branch. Some will recall that this alternative allocator has been around since AOL started modifying 7.6. We took a wack at putting this in earlier, but purify barfed or some special condition didn't work. Taking the latest changes from Jim Davidson, I massaged the Windows code to work and fixed it all up to be easily switchable (needs USE_THREAD_ALLOC to be defined - this is now the default on Unix with --enable-threads). The performance of this change can be seem simply here (this is a tclbench run using about 6 threads in the threaded Tcls): Benchmark 1:8.4a5 /home/jeffh/install/linux-ix86/bin/tclsh8.4-thread-aol 00:00:54 elapsed Benchmark 2:8.4a5 /home/jeffh/install/linux-ix86/bin/tclsh8.4-thread 00:06:28 elapsed Benchmark 3:8.4a5 /home/jeffh/install/linux-ix86/bin/tclsh8.4 00:00:46 elapsed Yes, under heavy thread load, the new allocator is 6.5 times faster. Still slightly slower than pure-non-threaded Tcl. The main facet is that lock contention is significantly reduced by having per-thread allocators. This should all be transparent to the end user, as it plugs in very low under ckalloc. I'll be doing more work on Windows to satisfy myself that it cleans up OK and all that and then this will likely be the default allocator when threaded on Windows as well. Jeff
Re: [AOLSERVER] new AOL-based thread-happy mem allocator in 8.4
On Tuesday 23 April 2002 08:00, you wrote: > I have finally commited the improved memory allocator from the AOLServer > Tcl mods to the 8.4 branch. > Hey, I call this *very* good news ! This may also be the point to reconsider improving the AOLserver's way of dealing with Tcl-only extensions and copyiny command-sets from startup to connection threads. There are couple of design issues there which require AOLserver core patch to get some extensions (like XOTcl) to get loaded and used properly. I have the patch for that but I'd like to see wether it can be introduced in the core server, since it will ease the task of using arbitrary (thread-safe) Tcl extensions within AOLserver. Zoran Vasiljevic Archiware
Re: [AOLSERVER] new AOL-based thread-happy mem allocator in 8.4
> On Tuesday 23 April 2002 08:00, you wrote: > > I have finally commited the improved memory allocator from the AOLServer > > Tcl mods to the 8.4 branch. ... > This may also be the point to reconsider improving the > AOLserver's way of dealing with Tcl-only extensions and > copyiny command-sets from startup to connection threads. > There are couple of design issues there which require > AOLserver core patch to get some extensions (like XOTcl) > to get loaded and used properly. > I have the patch for that but I'd like to see wether it > can be introduced in the core server, since it will > ease the task of using arbitrary (thread-safe) Tcl > extensions within AOLserver. AFAIK, what I commited takes care of all the known patches that AOLServer had against the Tcl core. I believe you should be able to use the AOLServer head with the Tcl core head without any mods to the Tcl core, or any funny bugs and the like. However, if there are some "design" level issues that may need improvement, do tell me. Jeff Hobbs The Tcl Guy Senior Developer http://www.ActiveState.com/
Re: [AOLSERVER] new AOL-based thread-happy mem allocator in 8.4
On Tuesday 23 April 2002 09:37, you wrote: > > AFAIK, what I commited takes care of all the known patches that AOLServer > had against the Tcl core. I believe you should be able to use the > AOLServer head with the Tcl core head without any mods to the Tcl core, or > any funny bugs and the like. > > However, if there are some "design" level issues that may need improvement, > do tell me. > Not of that sort, though. The AOLserver does seed the the connection interpreter with commands introspected from within loaded initialized startup interpreter. It just copies (blindly) all command structures found in the initial interp, which causes quite a few problems with Tcl packages planting commands with associated delete callbacks. So the problem is not AOLserver<->Tcl integration. The problem is how the AOLserver does handle Tcl extensions and their registered commands. There is nothing Tcl (you) can do about it. But there is something AOLserver (Kris/Jim/community) can do about. I've made a patch for 3.4 which corrects above problems. I'm just qurious to find out if/how/when can it be included in the core server distro. Hey, thanks for this memory allocator implementation! These (speed) issues have been bugging me for months. Zoran
Re: [AOLSERVER] new AOL-based thread-happy mem allocator in 8.4
Hi, can you upload the patch to Source Forge? Is this patch something that makes it so AOLserver uses the traditional Tcl way of doing dlopen() and running Modulename_Init instead of us doing Ns_ModuleInit jobbage or are we talking about something else? Thanks, Kris On Tue, 23 Apr 2002 08:45:20 +0100, Zoran Vasiljevic <[EMAIL PROTECTED]> wrote: >On Tuesday 23 April 2002 09:37, you wrote: >> >> AFAIK, what I commited takes care of all the known patches that AOLServer >> had against the Tcl core. I believe you should be able to use the >> AOLServer head with the Tcl core head without any mods to the Tcl core, or >> any funny bugs and the like. >> >> However, if there are some "design" level issues that may need improvement, >> do tell me. >> > >Not of that sort, though. The AOLserver does seed the the connection >interpreter with commands introspected from within loaded initialized >startup interpreter. It just copies (blindly) all command structures >found in the initial interp, which causes quite a few problems with >Tcl packages planting commands with associated delete callbacks. > >So the problem is not AOLserver<->Tcl integration. The problem is >how the AOLserver does handle Tcl extensions and their registered >commands. There is nothing Tcl (you) can do about it. But there is >something AOLserver (Kris/Jim/community) can do about. >I've made a patch for 3.4 which corrects above problems. I'm just >qurious to find out if/how/when can it be included in the core >server distro. > >Hey, thanks for this memory allocator implementation! >These (speed) issues have been bugging me for months. > >Zoran
Re: [AOLSERVER] new AOL-based thread-happy mem allocator in 8.4
On Tuesday 23 April 2002 19:22, you wrote: > Hi, can you upload the patch to Source Forge? Of course. > Is this patch something that > makes it so AOLserver uses the traditional Tcl way of doing dlopen() and > running Modulename_Init instead of us doing Ns_ModuleInit jobbage or are we > talking about something else? > No, I'm talking about something else... What I'm talking about is the mechanism the AOLserver is using to copy introspected Tcl command structures from the initial to the connection interpreter(s). The problem is: some Tcl commands may register delete callbacks, which are naively copied to every connection-thread interpreter by the AOLserver code in nsd/tclinit.c This of course results in memory trashing when all those (copied) commands are getting deleted on thread-exit. At that time, all their delete-callbacks are triggered, which may (and do) step on each other toes = memory trash. One of the solutions (which I made in the patch) was to *disable* auto-copying of commands with registered delete-callbacks. In order to handle those, however, I allowed the Tcl exension writer to register his *_Init procedure to be ran each time when the connection interpreter gets initialized. This is how the current Tcl thread extension works. IIRC, Jim (Davidson) made a note about this behaviour/problem in his presentations on the 7th Annual Tcl/Tk Conference (I may be wrong, though). I will post the patch for 3.4-series AOLserver to the SF repository. I'll try to get it for the 4.x as well. Cheers Zoran
