Dear friends of NaviServer, on sourceforge the release of NaviServer 4.99.15 is available [1]. As always, the newest release can be as well obtained from bitbucket [2].
In short: this release supports a few new features such as multiple driver threads for listening on the same port (using SO_REUSEPORT when available), allows dynamic connection thread pool mapping, provides better introspection (e.g. statistics from each driver how it was used), provides improved scalability, comes with better documentation and improved portability. I'll prepare as well an announcement for c.l.tcl and OpenACS.org. Below are the changes since 4.99.14: best regards -gustaf [1] https://sourceforge.net/projects/naviserver/files/naviserver/4.99.15/ [2] https://bitbucket.org/naviserver/naviserver/overview ======================================= NaviServer 4.99.15, released 2017-01-10 ======================================= 103 files changed, 3497 insertions(+), 1577 deletions(-) New Features: - Multiple driver threads for listening on the same port: When OS kernels support SO_REUSEPORT (which is the case on Linux since Kernel 3.9, Mac OS 10.3, Solaris 11, *BSD), one can specify now multiple driver threads for exactly the same listening address (protocol/IP-address/port) via the new config option "driverthreads" with a value larger than 1. With this new feature, NaviServer supports now for all stages of a request multi-threaded execution (driverthreads, spoolerthreads, connection threads, and writerthreads). Tests show that even with high number of concurrent clients, one can improve the throughput for high numbers of concurrent clients slightly (e.g. for 2 driverthreads and 250 concurrent clients, the throughput increases by ~9%, see https://www.mail-archive.com/naviserver-devel@lists.sourceforge.net/msg03445.html) Note: When using multiple driver threads on privileged ports, one has to add "#" followed by the number of necessary bind operations to the prebind address. Otherwise, prebind will bind to the address only once, and only one driver thread can be used. - Dynamic connection thread pool mapping: The new version of NaviServer extends the mapping of requests to connection thread pools by making it (a) introspectible and (b) dynamic (changeable at runtime). Dynamic thread pool mapping can avoid overlong blocking of connection threads by an unexpected peak of slow requests by moving these to a separate connection pool. For a detailed motivation, see https://www.mail-archive.com/naviserver-devel@lists.sourceforge.net/msg03446.html. The following new subcommands of "ns_server" implement the dynamic mapping to connection pools: * ns_server ?-server /s/? ?-pool /p/? map ?-noinherit? ?/mapspec/? Map HTTP-method and path (mapspec) to a certain pool, or list mappings from a pool (when no mapspec is given) * ns_server ?-server /s/? mapped ?-exact? ?-noinherit? /mapspec/ Obtain the pool for a mapspec (HTTP method and path) * ns_server ?-server /s/? ?-pool /p/? unmap ?-noinherit? /mapspec/ Undo previous mapping - New commands to obtain network driver information: * ns_driver info Return per driver detailed configuration information (name of module, type of driver, associated server, configured location, protocol, address, sendwait recvwait, and extraheaders) * ns_driver names Return a list of the names of the loaded driver modules. * ns_driver threads Return a list of the names of the driver threads * ns_driver stats Return for every driver thread the name of the driver module, the number of received requests, the number of spooled requests, the partial requests (received via multiple receive operations), and the number of errors. - Obtain detailed timing results from currently running request * ns_conn partialtimes Returns detailed timing snapshot about the current requests. The result is a dict containing the keys "accepttime", "queuetime", "filtertime", and "runtime" (similar to the per-pool timing statistics in "ns_server stats") - Added debug severity "Debug(nsproxy)" for separate debugging of nsproxy. Performance Improvements: - Move JoinConnThread() out of lock to reduce locking duration and to improve scalability. Bug Fixes: - Better error behavior, when prebind fails. - Fixed underspecified config file (simple-config does not set address, fallback to NS_IP_UNSPECIFIED. - Make sure, that URLs produced by urlspace's walk callbacks are proper list elements. - Make startup more robust, when called with invalid arguments. - nsproxy: * Don't reset timeout for for reaper when slaves already expired * Improve debug messages - Improve forward compatibility to forthcoming Tcl 8.7 (fixed bug in Tcl's Memory allocator; Zoran). - configure: Align behavior when "--with-openssl" is specified without arguments vs. when it is not specified (--with-openssl is activated per default). Previously, pkg-config was only looked at when "--with-openssl" was specified. Documentation improvements: - Improved sample config files * added new config parameters * document parameter "listenbacklog" * make simple-config working with default installed tree (activate .adp parsing) - Improved inline documentation about usage of Tcl_FindExecutable(argv[0]) (before fork vs. after fork) - Improved/new documentation pages of * admin-maintenance * ns_cgi * ns_conn * ns_connchan * ns_driver * ns_filestat * ns_http * ns_httpopen * ns_itcl * ns_parseargs * ns_register * ns_sendmail * ns_server * nssock * nsssl * nsv - more consistent capitalization: * use Tcl instead of TCL * use HTTP (or HTTPS) instead of http (or https). C API Changes: - Additional argument for reuseport for Ns_SockBind(), Ns_DriverListenProc(), Ns_SockListenEx(), Ns_SockBindUdp(), and Ns_SockListenUdp(). Configuration Changes: - New parameter for drivers (e.g. nssock) * Boolean Parameter "reuseport": When the parameter is set and SO_REUSEPORT is supported by the os, NaviServer will allow multiple listen operations on the same port (default: false) * Integer Parameter "driverthreads": set to a value > 1 when multiple driver threads should be used (default: 1). When multiple driver threads are configured, parameter "reuseport" is automatically set to "true". - New Boolean parameter "mutexlocktrace" for section "ns/parameters": When this parameter is set to true, timing of long mutex calls are printed to stderr. - Improve configurability: Allow to pass alternative implementation of CPPCHECK to "make" Command Line Changes: - Value provided to prebind option (-b) can be extended by a suffix "#n", where n is the number of pre-bound sockets (when e.g. multiple driver threads are used on privileged ports) Code Changes: - Extended regression tests * ns_conn.test * ns_driver.test * ns_server.test * ns_adp_compress.test (fixed compatibility with lib version > 1.2.8, which leads to different encoding) - Driver reform: make distinction explicit between driver type (e.g. "nssock"), driver name (e.g. "nssock1"), and driver thread name (e.g. "nssock1:1") - Moved definition of type "bool" from ns.h to nsthread.h to make boolean types as well available in nsthread implementation. - Improve compilation under windows: * don't pass "const char*" as member dest of Ns_ObjvSpec * add compatibility type "suseconds_t" for windows * define NS_NAVISERVER fro windows (no configure script) * move conditional definition of P_tmpdir after the .h files are loaded * fix type in macro PTR2UINT in windows code - Further Tcl cleanup * Replaced remaining Tcl_CmdProcs into Tcl_ObjCmdProcs * Reduced usage of Tcl_AppendResult() - Don't mess around with FD_SETSIZE - Don't use implementation defined NULL as last argument of variadic functions - Reduced size of huge switch statements - Reduced variable scopes - Reduce number of return statements before end of function - Added const declarations - Use "unsigned short" consistently for ports - Align argument names in prototypes and function definitions - Silenced static checker Modules: - nsdhcpd, nsdns, nsldapd, nsradiusd, nssip, nssmtpd, nssnmp, nssyslogd, nstftpd, and nsudp: Aligned function definitions with ns.h due to changes in Ns_SockListenUdp() Ns_SockListenEx() Note: Not all of these drivers are aligned yet with IPv6 interfaces. - nssyslogd: Added support for IPv6. - nsstats: * Added statistics for driver usage * Added Page for showing urls mapped to connection threads pools * Added information about driver threads ------------------------------------------------------------------------------ Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi _______________________________________________ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
