> On 23 Apr 2017, at 15:19, Aleksandar Lazic <al-hapr...@none.at> wrote: > > Hi Thierry > > Am 20-04-2017 15:05, schrieb Thierry Fournier: >> Hi, >> After a quick private brainstorm, Willy propose to me a new binary encoding >> for the headers. It is useless to give the numbers of headers contained in >> the block, so the end of headers is marked by an empty header (header name >> and value have a length of 0). >> The new patches in attachment (first patch is 0002). > > I have now created a docker image > > https://hub.docker.com/r/me2digital/haproxy-waf/ > > The build was successfully and haproxy starts. > > I have also try to check if the modsecurity starts and yeah it starts ;-) > > ### > > 1492953204.290643 [00] ModSecurity for nginx (STABLE)/2.9.1 > (http://www.modsecurity.org/) configured. > 1492953204.290705 [00] ModSecurity: APR compiled version="1.4.8"; loaded > version="1.4.8" > 1492953204.290730 [00] ModSecurity: PCRE compiled version="8.32 "; loaded > version="8.32 2012-11-30" > 1492953204.290739 [00] ModSecurity: YAJL compiled version="2.0.4" > 1492953204.290743 [00] ModSecurity: LIBXML compiled version="2.9.1" > 1492953204.290747 [00] ModSecurity: Status engine is currently disabled, > enable it by set SecStatusEngine to On. > 1492953209.298799 [03] 0 clients connected > 1492953209.298937 [04] 0 clients connected > ... > ### > > Now we can go the next step and create a complete solution ;-) > > It would be interesting to go the same way as mod_sec & nginx is going. > > https://www.trustwave.com/Resources/SpiderLabs-Blog/Announcing-the-availability-of-ModSecurity-extension-for-Nginx/ >
Hi aleksander, Thanks for the news. I integrated ModSec using the same way than NGinx, so it is absolutely a good idea to follow the same way than ModSec/NGinx. I will write an howto and publish benchmark when I found a lot of time. A feedback on your test is welcome. Thierry >> Thierry > > Regards > Aleks > >>> On 19 Apr 2017, at 15:34, thierry.fourn...@arpalert.org wrote: >>> Hi, >>> There is a new lot of patches for the spoa/modescurity contrib. >>> Thierry >>> On Wed, 19 Apr 2017 11:24:36 +0200 >>> Thierry Fournier <thierry.fourn...@ozon.io> wrote: >>>>> On 19 Apr 2017, at 09:16, Aleksandar Lazic <al-hapr...@none.at> wrote: >>>>> Am 19-04-2017 05:51, schrieb Willy Tarreau: >>>>>> On Tue, Apr 18, 2017 at 11:55:46PM +0200, Aleksandar Lazic wrote: >>>>>>> Why not reuse the upcoming http/2 format. >>>>>>> HTTP/2 is *easy* to parse and the implementations of servers are >>>>>>> growing? >>>>>> Are you kidding ? I mean you want everyone to have to implement HPACK >>>>>> etc ? >>>>> Well there are a currently lot of http/2 servers out there, but I >>>>> understand your concerns. >>>>>>> I know this is still on the todo list but I think this is worth to go >>>>>>> instead to add another protocol. >>>>>>> Or we can take a look into grpc, which is on top of http/2 with >>>>>>> protobuffer. >>>>>>> http://www.grpc.io/ >>>>>>> As I take more time to think in this direction why not add a grpc filter >>>>>>> like the spoa filter? >>>>>>> Opinions? >>>>>> Code generators are always a pain to deal with. It reminds me the old >>>>>> days >>>>>> when I was using rpcgen. And grpc doesn't provide a C implementation so >>>>>> that >>>>>> kills it :-) >>>>> Yeah I have also seen that they prefer c++, c#, go and other languages >>>>> but not offer a c lib. >>>>> This is strange just because the core is written in c, but I don't need >>>>> to understand everything ;-) >>>>>> Thierry told me he's going to implement a simple TLV-like list which will >>>>>> be much easier to implement and easy to code on both sides. >>>>> @Thierry: >>>>> Ah something like netstrings ? >>>>> https://cr.yp.to/proto/netstrings.txt >>>> Hi thanks for the link. I do not like this encoding because it is not easy >>>> to >>>> parse. I must read ascii representation of numbers and convert-it. I prefer >>>> simple binary format, with length (encoding in the same way than the SPOE >>>> protocol) and value. >>>> Hi, the main goal is providing: >>>> - a format easy to decode >>>> - not adding lot of dependencies to haproxy >>>> So, after brainstorm, I propose: >>>> - remove all data already accessible via existing sample-fetches from the >>>> new >>>> sample-fetch introduced by my previous patch. These data are: >>>> method, path, query string, http-version and the body. All of these data >>>> are >>>> available in some sample fetches and it will be great to reuse it. Note >>>> that >>>> the only data which keep are the headers. >>>> - For the headers I propose two format: The first format is for general >>>> use. It >>>> is simply the header block in the same format that we received (with the >>>> final >>>> \r\n for detecting truncated header bloc) >>>> - For header I propose also a binary format like this: >>>> <number of headers> >>>> [ >>>> <header name length><header name content> >>>> <header value length><header value content> >>>> ]… >>>> This format is easy to decode because each length is encoded with the SPOE >>>> numeric values encoding method, so it is always embedded in SPOE client. >>>> The SPOE message description will become >>>> spoe-message check-request >>>> uniqueid args method path query req.ver req.hdrs_bin req.body_size >>>> req.body >>>> uniqueid is a string which allow to do matching between the mod security >>>> logs and >>>> the haproxy logs. I propose the uniqueid, but this string can be >>>> another >>>> think like an header containing a uniqueid previously generated. >>>> req.hrds_bin is the header bloc in binary format. >>>> req.body_size is the advertised size of the body and it is used to >>>> determine if the >>>> body is full or truncated. >>>> Thierry >>>>>> I personally >>>>>> think that adding dependencies on tons of libs to implement simple stuff >>>>>> is more of a problem than an help even for implementors, because when you >>>>>> start to enter some dependency hell or version incompatibilities, it's a >>>>>> real pain, especially when it was done only to find how to implement >>>>>> just a small list. These things are useful if you want to implement heavy >>>>>> inter-application communications but it's not really the case here. >>>>> I got you. >>>>>> Cheers, >>>>>> Willy >>>>> Regards >>>>> Aleks >>> <0001-BUG-MINOR-change-header-declared-function-to-static-.patch><0002-REORG-spoe-move-spoe_encode_varint-spoe_decode_varin.patch><0003-MINOR-Add-binary-encoding-request-header-sample-fetc.patch><0004-MINOR-proto-http-Add-sample-fetch-wich-returns-all-H.patch><0005-MINOR-Add-ModSecurity-wrapper-as-contrib.patch>
