Re: [naviserver-devel] ns_set ... -persistent
On 09.10.12 22:43, Stephen Deasey wrote: Separately, I was experimenting with a performance enhancement to ns_sets, creating a hashtable mapping the keys to indexes. There is a slight cost in memory and on the first lookup, but it should get to breakeven after only around 3 lookups on average, with decreasing amortized cost after that. The difference is measurable and can be significant, as much as 4x faster on a large set. However, that's measured on a very large set, 1000 keys; and the real savings is pretty small, around 15 microseconds per lookup on average hardware. Is that kind of micro-optimization worth the additional complexity? Sets are used in a few core places where this sounds like a disadvantage: - http headers (in/out) - html forms - database rows All low numbers of rows with few lookups. Even config values rarely have huge numbers of elements, or duplicates, and often are looked up only once at startup and then stashed away in a struct. also, in our experience ns_sets are typically quite small. Since the ns_sets are currently used just per thread, the question is for what kind of purposes needing large amounts of keys are these used. Tcl has already arrays and dicts. Handling multiple values per key can be realized with dict lappend. -gustaf neumann -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] ns_set ... -persistent
On Wed, Oct 10, 2012 at 10:09 AM, Gustaf Neumann neum...@wu.ac.at wrote: Using a _ns_set capable of handling -shared which just calls existing ns_set with the unlocked semantics would not work, using a lock with for all _ns_set commands is not good either without modifying the existing ns_set semantics (one should not be able to access variables created by _ns_set from ns_set). If we check for the shared flag in ns_set, we are essentially at the old ns_set implementation. I was thinking that the 2 or 3 people in the world that need backward compatibility would load the nsshare module and then: rename _ns_set ns_set You wouldn't use them both, you just want your old code to work the way it did in 1999. -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] ns_set ... -persistent
On 10.10.2012, at 12:11, Stephen Deasey wrote: I was thinking that the 2 or 3 people in the world that need backward compatibility would load the nsshare module and then: rename _ns_set ns_set You wouldn't use them both, you just want your old code to work the way it did in 1999. I find this to be a good compromise. The nsshare module we can deliver as-is and declare unsupported and deprecated. People that really depend on it can then take the responsibility of keeping it up-to-date. And since it is not in the main server code, we can (mostly) forget about it. Cheers, Zoran -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] ns_set ... -persistent
Am 10.10.12 18:33, schrieb Zoran Vasiljevic: On 10.10.2012, at 18:15, Jeff Rogers wrote: could that renaming of the core tcl command be done as part of the module sure it can there is no need for an explicit rename, one can register the new command as ns_set. once the command is overloaded, all handlings of potentially shared ns_sets have to go through the new command which provides the locking if there is a place in naviserver, where a ns-set handle is passed to a c-function other than ns_set, it could break. but since i guess, this does not happen, this should be fine. -gustaf -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] ns_set ... -persistent
On Tue, Oct 9, 2012 at 11:11 AM, Gustaf Neumann neum...@wu.ac.at wrote: Stephen, do you remember why you took out the -shared flag from ns_set? https://bitbucket.org/naviserver/naviserver/changeset/1cbaf1acc09436f2a1c56102269a8b7fab0be168 it seems that some people love it. We have either to take it out of the documentation (and give sensible explanation) or reintroduce it in the code... If I remember rightly, the code has long been depreciated in AOLserver, and as no one used it and it complicates the arg parsing and locking we removed it, along with the ns_share and ns_var command. There's Tcl wrapper for ns_var because it was trivial. I've created a new module nsshare which reimplements the ns_share command using the old C code: https://bitbucket.org/naviserver/nsshare/overview Haven't put much effort into testing it, but the skeleton of the module is ready to go, it compiles and the sanity tests work. I think the way to add back the -shared switch to ns_set would be to add a new file to this module: setcmd.c, add the C code that was removed, export an _ns_set ?-shared? ?args ...? command which, if the -shared flag is present does it's thing, otherwise calls the underlying ns_set command. If you want to do this, that would be great. As there is now somewhere to put it, the ns_var code could be moved here too. There's some compatibility info here: http://wiki.tcl.tk/22567 -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] ns_set ... -persistent
I don't know that people love it so much as that there is no exact replacement for the functionality. I'm a big proponent of compatibility but shared sets as they previously existed are problematic. I wouldn't be at all surprised if they are thread-unsafe to the point of being able to crash the server. An alternative might be to provide equivalent functionality (order-preserving, multiple entries for keys, access by index or key or case-insensitive key) as a new nsv subcommand, perhaps 'nsv_multiset'. Separately, I was experimenting with a performance enhancement to ns_sets, creating a hashtable mapping the keys to indexes. There is a slight cost in memory and on the first lookup, but it should get to breakeven after only around 3 lookups on average, with decreasing amortized cost after that. The difference is measurable and can be significant, as much as 4x faster on a large set. However, that's measured on a very large set, 1000 keys; and the real savings is pretty small, around 15 microseconds per lookup on average hardware. Is that kind of micro-optimization worth the additional complexity? -J Gustaf Neumann wrote: Stephen, do you remember why you took out the -shared flag from ns_set? https://bitbucket.org/naviserver/naviserver/changeset/1cbaf1acc09436f2a1c56102269a8b7fab0be168 it seems that some people love it. We have either to take it out of the documentation (and give sensible explanation) or reintroduce it in the code... -gustaf -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] ns_set ... -persistent
On 09.10.2012, at 20:49, Jeff Rogers wrote: creating a hashtable mapping the keys to indexes I assume you used Tcl hash-tables with TCL_STRING_KEYS? You know that ns_set keys can be case-insensitive? Cheers, Zoran -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] ns_set ... -persistent
Zoran Vasiljevic wrote: On 09.10.2012, at 20:49, Jeff Rogers wrote: creating a hashtable mapping the keys to indexes I assume you used Tcl hash-tables with TCL_STRING_KEYS? You know that ns_set keys can be case-insensitive? Yes - I've only half-implemented it so far, but the full implementation would need to create two hashtables, one with case preserved and one with all keys tolower-ed. I expect the costs and benefits would be similar. -J -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] ns_set ... -persistent
On 09.10.2012, at 20:49, Jeff Rogers wrote: Is that kind of micro-optimization worth the additional complexity? For me personally: not really. I would not have anything against, though, if somebody (you) give it a try ;-) Cheers, Zoran -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
