Re: [naviserver-devel] reverseproxymode with multiple X-Forwarded-For values

2024-05-02 Thread David Osborne
Thanks again Gustaf. I've been running some tests and it's looking good. There's one case I'm not sure this can cater for, and this is probably more of an issue in development environments... When the trusted cidr is a subnet which also contains the client. Say ns_section

Re: [naviserver-devel] reverseproxymode with multiple X-Forwarded-For values

2024-04-29 Thread David Osborne
Hi Gustaf, >From your description it sounds like we could certainly work round our issue using the ReverseProxyTrustedServers config. Thank you very much for your time on this. On Fri, 26 Apr 2024 at 14:09, Gustaf Neumann (sslmail) wrote: > Hi David, > > I have now implemented the following

Re: [naviserver-devel] reverseproxymode with multiple X-Forwarded-For values

2024-04-25 Thread David Osborne
On Wed, 24 Apr 2024 at 18:53, Gustaf Neumann (sslmail) wrote: > This is the case where proxy 1 should not accept the x-forwarded-for > header from an untrusted upstream server. In case, proxy1 is a nginx > server, it should use > proxy_set_header X-Forwarded-For $remote_addr; > for untrusted

Re: [naviserver-devel] reverseproxymode with multiple X-Forwarded-For values

2024-04-23 Thread David Osborne
Thanks, The situation we were looking at is where NaviServer is behind 2 proxies. Client: IP 1.1.1.1 | Proxy 1: sends X-Forwarded-For: 1.1.1.1 | Proxy 2: sends X-Forwarded-For: 1.1.1.1,2.2.2.2 | Naviserver: peeraddr -source forwarded = 1.1.1.1 Which is fine. But the Client can initiate

[naviserver-devel] reverseproxymode with multiple X-Forwarded-For values

2024-04-22 Thread David Osborne
ramreverseproxymode "true" ns_paramreverseproxytrust [list 192.168.1.21 192.168.2.0/24] Any suggestions on what is best to do here? -- *David Osborne | Software Engineer* Qcode Software *Email:* da...@qcode.co.uk | *Phone:* 01463 896 484 ww

Re: [naviserver-devel] nswebpush & "invalid JWT provided"

2023-08-09 Thread David Osborne
Thanks Gustaf - replies inline... On Wed, 9 Aug 2023 at 10:38, Gustaf Neumann wrote: > Hi David, > > We do not have nswebpush somewhere in production. Can you tell more > precisely, what "suddenly" means? > About lunchtime on 2nd Aug! > Does this mean, that you have not changed anything in

[naviserver-devel] nswebpush & "invalid JWT provided"

2023-08-08 Thread David Osborne
Hi there, We have a chat implementation based on the Naviserver nswebpush module which recently stopped working with Google endpoints (eg. https://fcm.googleapis.com/fcm/send...). Suddenly it's complaining about invalid JWTs. We went back to reference the nswebpush code.

Re: [naviserver-devel] Towards NaviServer 4.99.24

2022-06-14 Thread David Osborne
Thanks again Gustaf. We've been happily running rc2 for a few days now without problem. On Tue, 7 Jun 2022 at 11:13, Gustaf Neumann wrote: > Dear all, > > i've uploaded RC2 to sourceforge. > > In short, the changes relative to RC1 are primarily the fallback charsets > (required to ease the

Re: [naviserver-devel] Towards NaviServer 4.99.24

2022-05-30 Thread David Osborne
Thanks Gustaf - the errorCode will be very handy to trap these encoding errors. We're using the per-server "formfallbackcharset" without issue at the moment. Working well. One problem we encountered is the Warnings logged when invalid UTF-8 is encountered. Some of the POSTs causing encoding

Re: [naviserver-devel] Towards NaviServer 4.99.24

2022-05-20 Thread David Osborne
Thanks Gustaf - I've run some quick tests against the per-server and global fallback and it seems to work well in the cases we're looking at - thanks for the fast work! I've yet to try the ns_getform option but will do so shortly. I've submitted a pull req for the ns_getform doc which I'm hoping

Re: [naviserver-devel] Towards NaviServer 4.99.24

2022-05-16 Thread David Osborne
Hi Gustaf, I spotted that *ns_getform *takes a charset argument from looking at the source code. The options for overriding charsets at the moment seem to be: *ns_getform iso8859-1* *ns_urlcharset iso8859-1* *ns_getform * *ns_conn urlencoding iso8859-1* *ns_getform * We experimented with

Re: [naviserver-devel] Towards NaviServer 4.99.24

2022-05-13 Thread David Osborne
Thanks Gustaf, I didn't pick up that your latest commit makes it possible to catch and handle an encoding error now. Thanks - we'll try to address the issue that way. Regards, Dave On Thu, 12 May 2022 at 12:27, Gustaf Neumann wrote: > Dear David, > > NaviServer is less strict than the

Re: [naviserver-devel] Towards NaviServer 4.99.24

2022-05-12 Thread David Osborne
Thanks again Gustaf, I can see the W3C spec you reference seems quite unequivocal in saying an error message should be sent back when decoding invalid UTF-8 form data. But I was curious why other implementations appear to use the UTF-8 replacement character (U+FFFD) instead, and found a bit of

Re: [naviserver-devel] Towards NaviServer 4.99.24

2022-04-28 Thread David Osborne
Hi Gustaf, We've been testing *4.99.24 rc1* and it seems pretty solid so far. Thanks for all the work that went into it. One change of behaviour that is causing us issues is the handling of invalid UTF8 characters. We have a system which regularly POSTs data to NaviServer - sometimes (for

Re: [naviserver-devel] v4.99.23 Tests failing building on Debian Buster Pt1: encoding_ns_http-1.1

2022-04-26 Thread David Osborne
Thanks Gustaf - we've successfully built 4.99.24 rc1 and are in the process of testing it. Much appreciated! On Sun, 10 Apr 2022 at 13:04, Gustaf Neumann wrote: > There are now two changes committed to bitbucket: > > a) Provide an error message when the configured locale is not installed > on

[naviserver-devel] v4.99.23 Tests failing building on Debian Buster Pt2: ns_strcoll-1.0.0

2022-04-06 Thread David Osborne
Still building of the official NaviServer v4.99.23 release on Debian Buster, we are also seeing some SIGSEGVs starting from ns_strcoll-1.0.0 test ns_strcoll-1.0.0 {ns_strcoll without locale (assuming en_US.UTF-8)} \ -constraints localeCollate -body { return [expr {[ns_strcoll Bär Bor]

[naviserver-devel] v4.99.23 Tests failing building on Debian Buster Pt1: encoding_ns_http-1.1

2022-04-06 Thread David Osborne
Hi there, We're trying to do a build of the official NaviServer v4.99.23 release (from the sourceforge tarball) on Debian Buster (10.12) but we're getting some failed tests. First one is encoding_ns_http-1.1 Seems it's serving an Emoji but the expected content-length is wrong upon receiving it.

Re: [naviserver-devel] What's the replacement of [curl -u] in [ns_http]

2021-02-22 Thread David Osborne
naviserver/files/ns_http.html > > > Best wishes, > I > ___ > naviserver-devel mailing list > naviserver-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- *David Osborne | Sof

Re: [naviserver-devel] ns_param location - change of behaviour for ns_returnredirect

2021-01-29 Thread David Osborne
__________ > naviserver-devel mailing list > naviserver-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- *David Osborne | Software Engineer* Qcode Software, Castle House, Fairways Business Park, Inverness, IV2 6

Re: [naviserver-devel] ns_proxy segmentation fault under Debian Stretch

2021-01-25 Thread David Osborne
ourceforge.net > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- *David Osborne | Software Engineer* Qcode Software, Castle House, Fairways Business Park, Inverness, IV2 6AA *Email:* da...@qcode.co.uk | *Phone:* 01463 896 484 www.qcode.co.uk ___

[naviserver-devel] ns_param location - change of behaviour for ns_returnredirect

2021-01-25 Thread David Osborne
ening. So the procedure drops back to the final option which is to query the driver protocol, address and port. But I'm failing to work out what change has stopped connPtr->location being populated at this point. Any help would be appreciated! Regards, -- *David Osborne | Software Engi

Re: [naviserver-devel] ns_proxy segmentation fault under Debian Stretch

2021-01-21 Thread David Osborne
users" section unless we make this type of change: https://bitbucket.org/naviserver/naviserver/pull-requests/10/illustrative-fix-for-nscp-loadusers Hope this might help. Regards, David On Tue, 19 Jan 2021 at 16:58, David Osborne wrote: > Hi again, > > One issue I'm noticing

Re: [naviserver-devel] ns_proxy segmentation fault under Debian Stretch

2021-01-19 Thread David Osborne
odule/nscp" { ns_param port 4080 #ns_param address 0.0.0.0 } ns_section "ns/server/default/module/nscp/users" { ns_param user "::" } Regards, David On Tue, 12 Jan 2021 at 11:07, David Osborne wrote: > Thank you for a very quick turnarou

Re: [naviserver-devel] ns_proxy segmentation fault under Debian Stretch

2021-01-12 Thread David Osborne
___ > naviserver-devel mailing list > naviserver-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- *David Osborne | Software Engineer* Qcode Software, Castle House, Fairways Business Park, Invernes

[naviserver-devel] ns_proxy segmentation fault under Debian Stretch

2021-01-11 Thread David Osborne
nsproxy.so } Then.. # sudo /usr/local/ns/bin/nsd -u nsadmin -t /usr/local/ns/conf/nsd-config.tcl -c % ns_proxy get test_pool Segmentation fault -- *David Osborne | Software Engineer* Qcode Software, Castle House, Fairways Business Park, Inverness, IV2 6AA *Email:* da...@qcode.co.uk | *Phone:* 01463

Re: [naviserver-devel] revproxy and virtual servers

2020-12-23 Thread David Osborne
Thanks for this Gustaf. We'll reintroduce the 2 virtual servers. Many thanks for all your help and hope you have a good Christmas. Regards, David On Wed, 23 Dec 2020 at 13:17, Gustaf Neumann wrote: > > > Is there a defined way to cancel an upstream proxy request from within > > the

Re: [naviserver-devel] revproxy and virtual servers

2020-12-21 Thread David Osborne
} > > In this callback, one can e.g. query the incoming port and redirect e.g. > the request to a different machine with a different path, etc. > > -g > > > > ___ > naviserver-devel mailing list > naviserver-devel@lists.

Re: [naviserver-devel] revproxy and virtual servers

2020-12-21 Thread David Osborne
Thanks Gustaf, As far as I can remember, the multiple servers are to make routing more convenient. For example, on port 443 we don't want to proxy /somefolder* to the backend, but on port 8443 we always want to proxy to a backend. Plus, /tcl* requests on port 443 would be proxied to a different

[naviserver-devel] revproxy and virtual servers

2020-12-18 Thread David Osborne
Hi, Thanks for your help on the segfaults last week. I've now built and installed NaviServer at HEAD and things are much more stable now. A knock on effect is our reverse proxy code no longer works. It was a custom version of your revproxy code which wasn't as up-to-date. So we're now attempting

Re: [naviserver-devel] Signal 11 crashes

2020-12-15 Thread David Osborne
tbucket.org/naviserver/naviserver/commits/896a4e3765f91b048ccbf570e5afe21b1bb1a41f > [2] https://github.com/gustafn/install-ns > [3] > https://bitbucket.org/naviserver/naviserver/commits/caab40365f0429a44740db1927e9f459d733db3f > On 14.12.20 18:07, David Osborne wrote: > > Hi, > > We

[naviserver-devel] Signal 11 crashes

2020-12-14 Thread David Osborne
Hi, We're building some Naviserver instances (4.99.19) on Debian Buster (v10.7). One of the instances is a revproxy instance which uses connchans to speak to a back end. We're seeing very frequent signal 11 crashes of NaviServer with this combination. (We also see this infrequently with 4.99.18

Re: [naviserver-devel] nsssl: unexpected result SOCK_READERROR

2020-10-19 Thread David Osborne
Thanks very much Gustaf - we'll try to confirm this. On Sat, 17 Oct 2020 at 13:46, Gustaf Neumann wrote: > There were OpenSSL releases after the release of 4.99.19, which report > more states. Probably, the messages are harmless (premature connection > closes from clients). Can you check with

Re: [naviserver-devel] nsssl: unexpected result SOCK_READERROR

2020-10-16 Thread David Osborne
Hi, Just a note that, in this case, these errors appear to be as a result of failing TLS handshakes. On Fri, 16 Oct 2020 at 09:55, David Osborne wrote: > Hi, > > We've recently been upgrading some debian instances to Naviserver 4.99.19. > Operationally everything appears f

[naviserver-devel] nsssl: unexpected result SOCK_READERROR

2020-10-16 Thread David Osborne
:09:43:56][9708.7ff94effd700][-driver:nsssl:0-] Warning: sockread returned unexpected result SOCK_READERROR (err ); close socket (20) [16/Oct/2020:09:43:57][9708.7ff94effd700][-driver:nsssl:0-] Warning: sockread returned unexpected result SOCK_READERROR (err ); close socket (20) -- *David

Re: [naviserver-devel] ns_conn status

2020-09-10 Thread David Osborne
Thank you very much Gustaf On Tue, 8 Sep 2020 at 19:12, Gustaf Neumann wrote: > Dear David, > > You assumption is right, ... AFIKT there is no problem with the case, > when "ns_conn status" is called without additional argument in a trace. > i have added code to bitbucket handle the case with

[naviserver-devel] ns_conn status

2020-09-07 Thread David Osborne
Hi, I notice that "ns_conn status" is flagged in conn.c as requiring an active connection (NS_CONN_REQUIRE_CONNECTED rather than NS_CONN_REQUIRE_CONFIGURED). The http status code is useful to be able to query in a trace after the connection is closed. Obviously that use-case is now incompatible

Re: [naviserver-devel] Tcllib's uuids repeating after ns_eval -sync source ..

2020-04-07 Thread David Osborne
there was a good reason to introduce the extra complexity. On Mon, 6 Apr 2020 at 18:26, Gustaf Neumann wrote: > Dear David, > > is there any reason not using "ns_uuid"? > > -g > On 06.04.20 17:02, David Osborne wrote: > > Hi, > Hope everyone is well. > > We en

[naviserver-devel] Tcllib's uuids repeating after ns_eval -sync source ..

2020-04-06 Thread David Osborne
Hi, Hope everyone is well. We encountered a situation where uuids generated by tcllib's uuid::uuid generate can repeat sequences under naviserver which has caused some issues in our codebase with collisions. We wondered if you would expect this behaviour? Using Debian 9.7 with Naviserver 4.99.19

Re: [naviserver-devel] Memory exhaustion and multiple nsd processes

2019-11-06 Thread David Osborne
icantly by using SYSTEM_MALLOC together with > TCMalloc (see e.g. > https://next-scripting.org/2.3.0/doc/misc/thread-mallocs). > The following chart show the effects on openacs.org, when i switched > to SYSTEM_MALLOC + TCMalloc around August (same code, some > number of request

[naviserver-devel] Memory exhaustion and multiple nsd processes

2019-11-01 Thread David Osborne
Hi, I was wondering if anyone could point us in the right direction with an intermittent problem we're seeing. Apologies for the wooly problem description - we don't have a firm test case as of yet. These instances are running NaviServer/4.99.16d10 on Debian Jessie 8.10 The problem usually

Re: [naviserver-devel] Tcl8.6 tdbc::mysql under NaviServer

2019-05-08 Thread David Osborne
Thank you for that Gustaf! On Wed, 8 May 2019 at 11:26, Gustaf Neumann wrote: > On 07.05.19 18:03, David Osborne wrote: > > Thanks again. > > Does this config snippet look OK? > > The snippet looks OK, but something went wrong during compilation > (you should have seen

Re: [naviserver-devel] Tcl8.6 tdbc::mysql under NaviServer

2019-05-07 Thread David Osborne
ns_paramconnections 5 ns_paramuserdbuser ns_parampassword ns_paramdatasource hostname:3306:databasename ns_paramverbose on } On Tue, 7 May 2019 at 15:20, Gustaf Neumann wrot

Re: [naviserver-devel] Tcl8.6 tdbc::mysql under NaviServer

2019-05-07 Thread David Osborne
int initialised = (int) arg; ^ At top level: nsdbmysql.c:34:24: warning: ‘RCSID’ defined but not used [-Wunused-variable] static const char *RCSID = "$Id$"; ^ On Fri, 3 May 2019 at 21:59, Gustaf Neumann wrote: > On 03.05.19 16:35, David Osborne wr

[naviserver-devel] Tcl8.6 tdbc::mysql under NaviServer

2019-05-03 Thread David Osborne
cuting "load {} Tdbcmysql" invoked from within "ns_ictl update" (procedure "ns_cleanup" line 8) invoked from within "ns_cleanup" while executing callback ns:tcltrace ns_cleanup (context: trace proc) (context: trace deallocate) Regards, --

Re: [naviserver-devel] Using tcllib ::try

2019-01-29 Thread David Osborne
Thanks for looking at that Gustaf. I can definitely relate to Zoran's comment: > "In the moment all is pretty "spooky" and it is difficult to grasp..." Your change doesn't seem to help the problems I'm looking at here unfortunately. The output you have pasted is the same as mine up to that

[naviserver-devel] Using tcllib ::try

2019-01-25 Thread David Osborne
Hi, I was wondering of you would be able to help us get past this issue. When attempting to use the tip version of NaviServer, whenever we use the tcllib's ::try command (which we need because we're running Tcl8.5) we get an error: "unknown namespace in import pattern "::tcl::control::try" I've

Re: [naviserver-devel] NaviServer 4.99.17 available

2018-11-06 Thread David Osborne
Thanks for all the work on this Gustaf. Looks like a great release! Regards, Dave On Sun, 4 Nov 2018 at 17:54, Gustaf Neumann wrote: > > Dear all, > > on sourceforge is now the release of NaviServer 4.99.17 [1]. > Below is a summary of the changes. This release was tested > under Windows 8.1

[naviserver-devel] Naviserver SSL connections to Postgresql DB

2018-10-09 Thread David Osborne
Hi, I wanted to ask how NaviServer users typically go about encrypting their connection to a remote PostgreSQL backend under Linux? In the past we've used the nsdbpg drivers (which don't support SSL connections) and encrypted using stunnel SSL wrapper, but in later versions of libpq5 this seems

Re: [naviserver-devel] Long Accept Times

2018-03-14 Thread David Osborne
eum...@wu.ac.at> wrote: > On 3/7/18 5:28 PM, David Osborne wrote: > > Thank you very much for that info Gustaf. > We'll have a look at whether we can adapt the ideas in boomerang to our > environment. > > Yes we suspected the Accept timings might be outwith the control of

Re: [naviserver-devel] Long Accept Times

2018-03-07 Thread David Osborne
Thank you very much for that info Gustaf. We'll have a look at whether we can adapt the ideas in boomerang to our environment. Yes we suspected the Accept timings might be outwith the control of the server. Am I right in thinking that Runtimes (if not using writer threads) are also affected by

[naviserver-devel] Long Accept Times

2018-03-05 Thread David Osborne
Hi, One of our installations is appearing to randomly experience long accept times. While all other requests are flowing nicely, suddenly one request will have an accept time of between 2-20 seconds (as measured in the access log with logpartialtimes). Our understanding is that this is the time

[naviserver-devel] Profiling

2018-02-13 Thread David Osborne
Hi, We've been looking at profiling the execution times of various application-level procs running NaviServer. We want the ability to turn on profiling for selected commands, so we started off using Tcl traces in a manner similar to: http://nikit.tcl.tk/page/Profiling+with+execution+traces So

Re: [naviserver-devel] "ns_server active" request IP - proxy aware?

2017-10-06 Thread David Osborne
ratic operation on the number of running connections. I would > rather recommend to use > either nsvs to track connections/peers, or to use a separate monitoring > thread keeping this information > up to date, receiving information about starting and ending threads. > > please che

Re: [naviserver-devel] "ns_server active" request IP - proxy aware?

2017-10-06 Thread David Osborne
Thanks Gustaf, I can only really reproduce it by hammering my development instance using apache bench (and only with a concurrency level of >100 after a few hundred requests) - not very scientific. The purpose of the filter_dos_check filter is to examine the server load (via ns_server active)

Re: [naviserver-devel] "ns_server active" request IP - proxy aware?

2017-10-05 Thread David Osborne
Hi, After an initial rollout we made a change to our code which massive increased the number of times ns_server is called with the -checkforproxy switch - we than started to get intermittent seg faults. I can reproduce this is test with gdb. It does seem to be related to the new section of code

Re: [naviserver-devel] "ns_server active" request IP - proxy aware?

2017-10-04 Thread David Osborne
alid commands. > By moving it to the end, it can be made specific to the three sub-commands > where it makes sense ("active", "queued", and "all"). > > all the best > -gn > > Am 29.09.17 um 5:50 PM schrieb David Osborne: > > Hi, > &

[naviserver-devel] "ns_server active" request IP - proxy aware?

2017-09-29 Thread David Osborne
Hi, We have a need for ns_server active & queued commands to be proxy-aware in the IP addresses that they report. Similar to how the access log IP address can be configured to report the X-Forwarded-For value (if it exists)

Re: [naviserver-devel] ns_proxy hang

2017-05-29 Thread David Osborne
Thanks very much Gustaf! We'll schedule a build and let you know how it goes. On 25 May 2017 at 11:32, Gustaf Neumann wrote: > > but i will be looking into this when times allows. > David, There are now some updates in the repository, removing falso > positives about zombies

Re: [naviserver-devel] ns_proxy hang

2017-05-15 Thread David Osborne
In some cases (but not all), Ns_WaitProcess is logging an error immediately after the reaper sends a signal 15. It's under these circumstances ReleaseProxy seems to find the proxy Busy without a slave. [15/May/2017:16:38:39][23802.7f2f0d7fa700][-nsproxy:reap-] Warning: [testpool]: pid 5174

Re: [naviserver-devel] ns_proxy hang

2017-05-15 Thread David Osborne
/2017:15:03:52][7028.7fffc67fc700][-nsproxy:reap-] Error: process 7640 killed with signal 15 (Terminated) [15/May/2017:15:03:53][7028.7fffc67fc700][-nsproxy:reap-] Warning: nsproxy: zombie: 7622 On 15 May 2017 at 14:45, Gustaf Neumann <neum...@wu.ac.at> wrote: > Am 15.05.17 um 15:15 schr

Re: [naviserver-devel] ns_proxy hang

2017-05-15 Thread David Osborne
Thanks. This does indeed seem to fix the test case I sent - however I'm now getting a segmentation fault during the "make test". It's during the second ns_proxy 5.10 test (there are two 5.10s) - "test ns_proxy-5.10 {check killing active proxy}". ns_close is being called on an invalid SlavePtr

Re: [naviserver-devel] ns_proxy hang

2017-05-12 Thread David Osborne
libc-start.c:287 #4 0x00400765 in _start () On 10 May 2017 at 15:27, Gustaf Neumann <neum...@wu.ac.at> wrote: > I'll look at it at the weekend - unless someone else can fix this before > this. > > -g > Am 10.05.17 um 16:00 schrieb David Osborne: > > Increasing waittimeout

Re: [naviserver-devel] ns_urlencode on Windows Naviserver

2017-05-12 Thread David Osborne
ng, distribution, or reliance upon the contents of > this e-mail is strictly prohibited. Quest Computing Ltd., Ushers Court, > 31-33 Ushers Quay, Dublin 8, Ireland. Quest Computing Ltd. is registered in > Ireland No. 146435. > > > > > > -----

Re: [naviserver-devel] ns_proxy hang

2017-05-10 Thread David Osborne
/linux/x86_64/clone.S:111 On 10 May 2017 at 13:04, Gustaf Neumann <neum...@wu.ac.at> wrote: > We don't see such hangs either. Does increasing the waittimeout solve this > issue? > ... not as a fix, but to narrow the problem down. > -gn > > Am 10.05.17 um 13:

Re: [naviserver-devel] ns_proxy hang

2017-05-10 Thread David Osborne
The manifestation for us in production is quite insidious and difficult to spot the root cause of. Certainly not easy to see via the logs. One place we are experiencing it is in code which generates outgoing emails. Sometimes, when the outgoing email is particularly large, nsproxy times out while

Re: [naviserver-devel] ns_proxy hang

2017-05-10 Thread David Osborne
Has anyone got any ideas on debugging this issue? Turns out it's causing us many more operational issues that we first thought... Regards, -- David On 18 April 2017 at 15:44, David Osborne <da...@qcode.co.uk> wrote: > Hi, > > We're encountering an occasional problem wh

[naviserver-devel] ns_proxy hang

2017-04-18 Thread David Osborne
Hi, We're encountering an occasional problem which means calls to ns_proxy hang forever. It seems to be when an ns_proxy eval command timeouts during evaluation & it produces a large amount of output. Under those circumstances ns_proxy becomes inoperable. Following is a contrived test case which

Re: [naviserver-devel] revproxy callback setup

2017-04-10 Thread David Osborne
. On 10 April 2017 at 14:29, David Osborne <da...@qcode.co.uk> wrote: > Hi, > > While testing the revproxy code I coming across the occasional error like > the following: > > connchan "conn237" does not exist > while executing > "ns_connchan callback -

[naviserver-devel] revproxy callback setup

2017-04-10 Thread David Osborne
Hi, While testing the revproxy code I coming across the occasional error like the following: connchan "conn237" does not exist while executing "ns_connchan callback -timeout $timeout $frontendChan [list spool $frontendChan $backendChan client $timeout 0 ] rex" Seems like

Re: [naviserver-devel] revproxy timeouts

2017-04-07 Thread David Osborne
29235e156a781ca5599284c3e9d > > The bug was a problem in nsssl triggered by "ns_connchan close". > The same verison of revproxy should work. > > all the best > -g > > > Am 06.04.17 um 12:21 schrieb David Osborne: > > Hi Gustaf, > > I can confirm that if I swi

Re: [naviserver-devel] ns_chan and ns_connchan

2017-04-07 Thread David Osborne
icht: Landesgericht Linz > > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > __

Re: [naviserver-devel] revproxy timeouts

2017-04-06 Thread David Osborne
Hi Gustaf, I can confirm that if I switch to using http (rather than https) then your timeout modification works perfectly. So looking like it's (at least) ssl specific, On 5 April 2017 at 18:03, Gustaf Neumann <neum...@wu.ac.at> wrote: > Am 05.04.17 um 18:45 schrieb David Osborn

Re: [naviserver-devel] revproxy timeouts

2017-04-05 Thread David Osborne
Thanks very much. That's similar to what I was trying (although I was setting the timeout for the backendReply callback only since it should be the first thing to be called on reply if I understand rightly). I get the same behaviour with your code. Say I request a backend URL that takes 10

[naviserver-devel] Build Errors in Debian

2017-04-04 Thread David Osborne
Hi, This recent change means I no longer build naviserver tip on Debian 8.6 due to the compiler complaining the following sockfile.c switch statement has the same case twice. https://bitbucket.org/naviserver/naviserver/commits/1fadec70fa7612f9a541116a1a2e72b69a9dacf7#chg-nsd/sockfile.c The

Re: [naviserver-devel] revproxy timeouts

2017-04-03 Thread David Osborne
On 2 April 2017 at 12:00, Gustaf Neumann wrote: > Just to be sure: timeouts during connection creation work fine, but read > timeouts from the back-end lead to the problem. ... and this happened with > an https backend. And you are experiencing this under Linux. Did i >

[naviserver-devel] revproxy timeouts

2017-03-31 Thread David Osborne
Hi, I've been trying to get revproxy to work with a timeout. So if the backend doesn't reply within $timeout secs, then the connection is closed. What I thought we could do was set a timeout when creating the callback to call backendReply. Then when the callback is triggered with $when eq "t" we

Re: [naviserver-devel] revproxy

2017-03-01 Thread David Osborne
On 28 February 2017 at 17:55, Gustaf Neumann <neum...@wu.ac.at> wrote: > Am 28.02.17 um 17:02 schrieb David Osborne: > > In that case, we get a ~5 second delay, > do you see the same behavior for static content (fastpath) and dynamic > content? > -g > Yes, identical re

Re: [naviserver-devel] revproxy

2017-02-28 Thread David Osborne
af Neumann <neum...@wu.ac.at> wrote: > >> Am 24.02.17 um 18:21 schrieb David Osborne: >> >> Thanks, that's working great for me. >> Have tested it with large POST data, binary data, UTF-8 data and all gets >> handled perfectly. >> >> Only

Re: [naviserver-devel] revproxy

2017-02-24 Thread David Osborne
(In asnwer to the previous question, yes this is debian 8.7 with openssl 1.0.1t) On 24 February 2017 at 06:24, Gustaf Neumann wrote: > There are reports that SSL writes with large content cause problems. > I've uploaded a version of revproxy that avoids memory bloats on huge >

Re: [naviserver-devel] revproxy

2017-02-23 Thread David Osborne
After this, I've been trying to get POST requests to work. I needed to change the revproxy code a little since, for me, the backend would wait forever for the POST body unless I explicitly sent it after opening the backend channel. This will now work fine for POSTs with a body of ~32,768 bytes

Re: [naviserver-devel] revproxy

2017-02-22 Thread David Osborne
On 22 February 2017 at 13:49, Gustaf Neumann <neum...@wu.ac.at> wrote: > Am 22.02.17 um 13:48 schrieb David Osborne: > > Short question: > Is there a way to influence which driver thread "ns_connchan open" will > use? > > ... no, since ns_connchan does

[naviserver-devel] revproxy

2017-02-22 Thread David Osborne
Hi, We've been looking at using the new revproxy module in our environment. I'm not sure we can use the module as is, but we're interested in using the spool & backendReply procs in out our code. I think we should be able to replace ns_http queue/ns_http wait with the same ns_connchan callbacks

Re: [naviserver-devel] NsWriterQueue Assertion fails

2017-02-17 Thread David Osborne
to be enabled. I'll try your suggestions - thanks again. On 16 February 2017 at 02:26, Gustaf Neumann <neum...@wu.ac.at> wrote: > Am 15.02.17 um 16:22 schrieb David Osborne: > > We spotted that, if the following conditions are met, then a malformed > HTTP request will make a Navi

[naviserver-devel] NsWriterQueue Assertion fails

2017-02-15 Thread David Osborne
Hi, We spotted that, if the following conditions are met, then a malformed HTTP request will make a Naviserver assert fail (4.99.15). - Naviserver is compiled with --enable-symbols - At least 1 writerthread is running - A malformed HTTP request is made to a fastpath file which is bigger

Re: [naviserver-devel] Access logs logging 2 IPs for single request

2017-01-24 Thread David Osborne
24.01.17 um 11:02 schrieb David Osborne: > > Can anyone come up with a scenario where a request could have 2 peer IP > addresses? > > We occasionally see entries in the naviserver access logs like this one > and I'm really not sure what to make of them: > > 10.x.x.x, 37.x.x.

Re: [naviserver-devel] gzipping static content - how to do?

2017-01-16 Thread David Osborne
On 16 January 2017 at 09:17, John from Decent Espresso < j...@decentespresso.com> wrote: > Thanks Gustaf (Hahaha) that is the cause of the mime-type problem. I’ve > upgraded to ns 4.99.15 and the MIME type bug is now gone. > > However, I’m still not having gzipped files automatically made. I’ve

Re: [naviserver-devel] Logpartialtimes seemingly giving runtime of *previous* request

2016-11-03 Thread David Osborne
Thanks Gustaf. I've patched 4.99.12 with your changes and it seems to work fine now. Thanks for the fast response! Regards, David On 3 November 2016 at 10:12, Gustaf Neumann <neum...@wu.ac.at> wrote: > > Am 02.11.16 um 16:49 schrieb David Osborne: > > I've spotted an issue

[naviserver-devel] Logpartialtimes seemingly giving runtime of *previous* request

2016-11-02 Thread David Osborne
Hi, I've spotted an issue that seems to have been introduced at some point between 4.99.10 & 4.99.11 (and still present in tip as of today). When logpartialtimes are enabled, the access log seems to be logging the runtime of the previous entry. As a testcase on Debian 8: hg clone

[naviserver-devel] Permission Denied for Ns_SockBind

2016-07-29 Thread David Osborne
Hi, I'm having a few problems with a new build of Naviserver on Debian (wheezy & Jessie). Using Tip, when I try to start naviserver listening on a privileged port, with a non-root (but system) user, I'm getting a permission denied error from within Ns_SockBind: eg.

Re: [naviserver-devel] Contention on mu12

2016-07-08 Thread David Osborne
Thanks for the info Gustaf. You gave us enough info to set us down the right path to working out what was the likely issue. If mu12 is the same or this server, and I think it is, this server seemed to be spending far too much time creating interpreters. Looking at potential causes of a high

[naviserver-devel] Contention on mu12

2016-07-07 Thread David Osborne
I was wondering if someone could help us understand this Mutex Lock output from nsstats. It's ordered by Contention and it would seem the figures for mu12, and to a lesser extent, nsd:pool:, are quite high. I don't know what mu12 is used for. Could someone give an example of the sort of workload

Re: [naviserver-devel] Building naviserver with merged nsssl

2016-06-27 Thread David Osborne
Thanks very much for looking at this Gustaf. We'll have another go at doing a build in due course. Regards, -- David On 25 June 2016 at 14:10, Gustaf Neumann wrote: > Dear David, > > i have addressed the problem with libnssock.dylib + rpath in the > repository. > Background:

[naviserver-devel] Building naviserver with merged nsssl

2016-06-22 Thread David Osborne
Hi, I've hit a problem trying to build naviserver (on Debian wheezy) since nsssl was merged into the main repo (which is a great idea btw). In the nsssl Makefile it references: include $(NAVISERVER)/include/Makefile.module but on a clean box that file will not exist yet. If I change it to

[naviserver-devel] Form vars + 302 redirect with trailing slash

2016-03-19 Thread David Osborne
HI, Quick question. We've noticed that when fastpath does an internal 302 redirect for URLs without a trailing slash, it seems to drop the query string. Is this intended behaviour? eg. http://site.com/news?a=1 302 redirects to http://site.com/news/ I think it maybe this redirect that's in

Re: [naviserver-devel] NaviServer 4.99.9 available

2016-01-12 Thread David Osborne
Thanks Gustaf - that looks great. On-the-fly minification in particular will be very helpful. I noticed there is a recent intent-to-package ticket with Debian. Are there moves afoot to have Naviserver officially included in Debian?

[naviserver-devel] ns_sendmail's use of ns_sockopen

2015-10-15 Thread David Osborne
Hi, Been trying to use Naviserver's ns_sendmail for the first time. By default, speaking to localhost:25 (under Debian) I'm getting a "Operation now in progress" error at the point where ns_sockopen is called against localhost:25. I modified the code to use the -nonblock switch (just as an

Re: [naviserver-devel] ns_ssl and binary data

2015-07-30 Thread David Osborne
Thanks Gustaf - I have just tried these changes and it appear to work just fine. I just have to be careful when handling the binary content to pass to the -body option, if I inadvertently force an internal string representation to be generated then if will be corrupted by ns_ssl in transit since

Re: [naviserver-devel] ns_ssl and binary data

2015-07-29 Thread David Osborne
*body = (char *) Tcl_GetByteArrayFromObj(bodyPtr, len); Ns_DStringPrintf(httpPtr-ds, Content-Length: %d\r\n\r\n, len); Tcl_DStringAppend(httpPtr-ds, body, len); } else { On 23 July 2015 at 16:52, David Osborne da...@qcode.co.uk wrote: Question - do ns_http / ns_ssl support raw

[naviserver-devel] Disabling directorylisting

2015-07-23 Thread David Osborne
, and leave the default value of directorylisting to be set in fastpath.tcl as in the following commit? https://bitbucket.org/davidqc/naviserver/commits/afd8de3e5bdc6813ab49c6bc5713faeaaf68b854 Regards, -- David Osborne Qcode Software Limited

Re: [naviserver-devel] Escaping characters in access log

2015-07-15 Thread David Osborne
potentially on external input, such as the user agent field or the referrer field. all the best -g Am 14.07.15 um 09:05 schrieb David Osborne: Hi, We're coming up against a problem where we attempt to parse data in a naviserver access log to analyse server use. We were relying

  1   2   >