FOSDEM 2016 - take action by 4th of December 2015
As most of you probably know FOSDEM 2016 (the biggest, 100% free open source developer conference) is right around the corner: https://fosdem.org/2016/ We hope to have an ASF booth and we would love to see as many ASF projects as possible present at various tracks (AKA Developer rooms): https://fosdem.org/2016/schedule/#devrooms This year, for the first time, we are running a dedicated Big Data and HPC Developer Room and given how much of that open source development is done at ASF it would be great to have folks submit talks to: https://hpc-bigdata-fosdem16.github.io While the CFPs for different Developer Rooms follow slightly different schedules, but if you submit by the end of this week you should be fine. Finally if you don't want to fish for CFP submission URL, here it is: https://fosdem.org/submit If you have any questions -- please email me *directly* and hope to see as many of you as possible in two months! Thanks, Roman.
Re: apr_token_* conclusions (was: Better casecmpstr[n]?)
I've hijacked Yann's thoughts and replied on a dev@apr thread. There is merit in httpd's deliberations but the issue is sufficiently larger than 'just us ourselves'.
Re: No H2 Window updates!
1.0.8 is known problematic, 1.0.5 mod_h2 is known good. I wouldn't expect any change there On Nov 30, 2015 17:41, "Jan Ehrhardt" wrote: > Jim Jagielski in gmane.comp.apache.devel (Mon, 30 Nov 2015 07:24:07 -0500): > >I'm assuming that the brokenness also shows up on trunk, > >right? > > > >Bert, Jan, can you check if trunk shows the same behavior? > >I would prefer not hacking away on 2.4 directly and independently. > > FYI: the only differences between trunk and mod_http2 1.0.8 were > ap_strcasecmp > vs. strcasecmp and 1.0.9-dev vs 1.0.8. > -- > Jan > >
Re: No H2 Window updates!
Jim Jagielski in gmane.comp.apache.devel (Mon, 30 Nov 2015 07:24:07 -0500): >I'm assuming that the brokenness also shows up on trunk, >right? > >Bert, Jan, can you check if trunk shows the same behavior? >I would prefer not hacking away on 2.4 directly and independently. FYI: the only differences between trunk and mod_http2 1.0.8 were ap_strcasecmp vs. strcasecmp and 1.0.9-dev vs 1.0.8. -- Jan
Re: No H2 Window updates!
Jim Jagielski in gmane.comp.apache.devel (Mon, 30 Nov 2015 07:24:07 -0500): >I'm assuming that the brokenness also shows up on trunk, >right? > >Bert, Jan, can you check if trunk shows the same behavior? >I would prefer not hacking away on 2.4 directly and independently. I checked out revision 1717334 from trunk and tried to update mod_http2 without updating the other sources. Surprise: Creating library .\Release/mod_http2.lib and object .\Release/mod_http2.exp 1>h2_from_h1.obj : error LNK2019: unresolved external symbol _ap_casecmpstr referenced in function _create_response 1>.\Release\mod_http2.so : fatal error LNK1120: 1 unresolved externals Somebody followed your advice and actually wrote some code! I cannot easily test mod_http2 from trunk, because it involves updating APR as well. -- Jan
Re: apr_token_* conclusions (was: Better casecmpstr[n]?)
Sorry for the late, was afk this times... Regarding the name, I'm fine with ap[r]_cstr[n]casecmp(), ap[r]_casecmpcstr[n]() or ap[r]_cstr_*() (if we need a set of functions in this area).. I think we all agree that the new function(s) would help protocol "validation" being agnostic wrt the locale, though httpd (as any *nix program) runs in the "C" locale by default (hence str[n]casecmp() behave as expected), and this can't be changed unless some (third-party-)module plays with setlocale(), as Bill said). So the new function(s) would address two concerns: 1. doing the right thing at the protocol level if/when modules need custom locales, 2. have an effecient "C"-string caseless comparison function on all platforms (see tests results below). For 1. I agree we should not hurry and take the time to review the kind of changes I proposed in [1]. For 2. I think we can start using the new function(s) whenever we are dealing with "C"-strings and this is a fast path (eg. Jean-Frederic's report about ap_proxy_port_of_scheme(), which should be addressed both in httpd and APR IMHO). Regarding performances, attached are the tests (and results) I ran on different systems (linuxes+glibc+gcc only!, i.e. Debian6+glibc-2.11+gcc-4.4, Debian8+glibc-2.19+gcc-4.9 and CentOS7+glibc-2.17+gcc-4.8) for the different implementations that were discussed so far (including standard strncasecmp, svn_cstring_casecmp, and Mikhail's mi_strcasecmp). a. Our implementation(s) are faster than str[n]casecmp() for strings lengths < 4 or 8 (depending on sizeof(long), ie. 32bit vs 64bit system), which matters not only for such short strings but also when the compared strings differ in these first bytes (our implementation fails faster too here), b. Latests str[n]casecmp() (or/and gcc) are far faster (x3) than any of our proposal in the "C" (or "UTF-8") locale for longer strings, too bad there is no strcasecmp[_loc]() taking the locale as argument (à la stdc++)... *However*, whenever mappings are in place, eg. the famous mt_MT.ISO-8859, str[n]casecmp() takes the same time as our implementation (comparing the same number of caseless-equal characters), c. Our best implementation, which is performing well in all cases (ie. no "pathological" behaviour with some cases) is Jim's "ap_casestrcmp" (the current one). Actually the ones performing a bit better are those called "ap_casestrcmp_1" and "ap_casestrcmp_2" in the test, the former being the same as Jim's but with "++ps1; ++ps2;" done at the end of the loop, and the latter being my proposed version using an index instead of char pointers (no gain compared to "ap_casestrcmp_1", not worth the change...). So I'd be for using Jim's with the simple "++ps1; ++ps2;" change. The attached test results are the ones run on CentOS7 (because this is the system of a real/performant machine I can access, and running the tests on my Debian laptop make it hot enough to be unfair :) Since I'm not very used to CentOS, I could not make the "mt_MT.ISO-8859" locale work/being applied, either because I'm doing things wrong, or sowehow the locale has been updated to avoid this mapping (though I was able to make it work with latest debians, where strcasecmp() performs differently depending on the locale...). So for completeness, I'm pasting the results on a debian jessie for locales "mt_MT.ISO-8859" and "C" here, since it matters there: $ LC_ALL=mt_MT.iso88593 ./ap_casecmpstr-O2 'a' 1 'CyCyCyCyCyCyCyCyCyCoOoOoOoOoOaAaAaAaAaAa' 'cYcYcYcYcYcYcYcYcYcOoOoOoOoOoAaAaAaAaAaa' 0 ./ap_casecmpstr-O2 'a' 1 "CyCyCyCyCyCyCyCyCyCoOoOoOoOoOaAaAaAaAaAa" "cYcYcYcYcYcYcYcYcYcOoOoOoOoOoAaAaAaAaAaa" 0: locale "mt_MT.iso88593" - ap_casecmpstr : time=06.160937456, res=0 - ap_casecmpstr_1 : time=06.256894742, res=0 - ap_casecmpstr_2 : time=06.136213804, res=0 - ap_casecmpstr_4 : time=06.787756289, res=0 - ap_casecmpstr_3 : time=06.110559311, res=0 - ap_casecmpstr_7 : time=06.844624092, res=0 - ap_casecmpstr_5 : time=06.820174763, res=0 - ap_casecmpstr_6 : time=10.488436936, res=0 - svn_cstring_casecmp : time=07.329213881, res=0 - mi_strcasecmp : time=10.165367784, res=0 - strcasecmp_ext : time=06.274211596, res=0 - strcasecmp : time=06.126361486, res=0 - strcmp : time=00.590613344, res=-32 != str[n]casecmp()'s result! $ LC_ALL=mt_MT.iso88593 ./ap_casecmpstr-O2 'a' 1 $'\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9\xa9' '' 0 ./ap_casecmpstr-O2 'a' 1 "<...40 unprintable chars here...>" "" 0: locale "mt_MT.iso88593" - ap_casecmpstr : time=00.479203198, res=64 != str[n]casecmp()'s result! - ap_casecmpstr_1 : time=00.526867671, res=64 != str[n]casecmp()'s result! - ap_casecmpstr_2 : time=00.525341010, res=64 != str[n]casecmp()'s result! - a
Re: No H2 Window updates!
I'm assuming that the brokenness also shows up on trunk, right? Bert, Jan, can you check if trunk shows the same behavior? I would prefer not hacking away on 2.4 directly and independently. tia!! > On Nov 29, 2015, at 3:03 AM, Stefan Eissing > wrote: > > Ok, thanks. I think I have an idea of what's happening: > - on short request bodies, window updates get omitted and gives a shrinking > connection window > - the window size of the connection itself should be at max Value right from > the start > > I won't be able to do anything about it until later this week, though. > > //stefan > > Am 28.11.2015 um 22:29 schrieb Bert Huijben : > >>> -Original Message- >>> From: Bert Huijben [mailto:b...@qqmail.nl] >>> Sent: zaterdag 28 november 2015 14:32 >>> To: stefan.eiss...@greenbytes.de; dev@httpd.apache.org >>> Subject: RE: No H2 Window updates! >> >> >>> In case of Subversion's real usage, I want to commit potentially hundreds >> of >>> MByte, so a connection level window of more than a few bytes would be >>> very >>> welcome With HTTP/1 we send out the data as fast as we can and the TCP >>> windowing handles this from the httpd side... Now the http/2 level >>> windowing >>> should handle this. >>> >>> >>> Mod_dav and mod_dav_svn are usually fast readers; just limited by trivial >>> disk io or xml processing in the common operations, so I don't expect real >>> problems there. >> >> For my commits against the svn-master.apache.org repository my latency/ping >> time is +- 142 ms. >> >> With the current simple algorithm and a maximum window of 64 >> Kbyte/connection, I can send a theoretical maximum of 64 Kbyte/142 ms per >> connection to that server... which would be about 450 Kbyte/s. >> (=1/0.142*65536/1024) >> >> That is < 1/30th of the bandwidth I have at my disposal (+- 150 Mbit). >> >> But currently I don't even get anywhere near that as my window continues to >> shrink because some data is missed in accounting even after my simple patch. >> >> >> >> For httpd we have to think carefully which algorithm we want to implement >> here. Preferably the algorithm should be better than TCP could, as we know >> the specifics of the http algorithm better than plain TCP could for >> HTTP/1,1. TCP does send a lot of window updates though... Almost every >> packet does. >> >> >> Solving the really hard problems, like this one, is the reason I didn't >> think the nghttp2 library would really be the solution for serf. >> It is nice for such a library to provide a head start on the protocol level, >> but there is no way a standard library can really solve this scheduling >> problem in a generic way. (If it could do it, we had used that solution for >> TCP... and never had to resort to designing a HTTP/2 in the first place). >> >> See https://en.wikipedia.org/wiki/TCP_window_scale_option for some >> introduction on how TCP was extended to work above the limit that currently >> applies to H2. >> >> >> Dynamic window sizing/scaling will have to be implemented at some point, at >> both the stream and the connection level... This will involve timing, >> measuring, etc. etc. Things nghttp2 can't do for us right now. >> And if I'm right this might take continuous optimizing for years to come. >> >> >> >> When I connect to sites like google I immediately receive a large connection >> level window, to allow me to post huge blobs without a delay. (Haven't >> tested their stream level windows behavior yet) >> In serf I do something similar... and then apply a bit of throttling at the >> stream level. >> >> >> I would guess some clients have already implemented some of this, so we >> might be able to learn from their implementations... Clients will see much >> more incoming data than servers of course :). >> >> >> Bert >> >>
