Re: ModSecurity testing
Hi Joao, On Sat, Dec 14, 2019 at 11:30 PM Joao Morais wrote: > > > > Em 13 de dez de 2019, à(s) 10:09, Christopher Faulet < > cfau...@haproxy.com> escreveu: > > > > Le 10/12/2019 à 05:24, Igor Cicimov a écrit : > >> > >> Testing with Haproxy 2.0.10 but same result with 1.8.23. The versions > of ModSecurity is 2.9.2 and the OWASP rules v3.0.2 > >> What am I doing wrong? Can anyone provide a request that should confirm > if the module is working or not from or share the experience from their own > setup? > > > > Hi Igor, > > > > First of all, I don't know how the modsecurity agent really work. But > I'm surprised to see it returns -101. In the code, -1, 0 or an HTTP status > code is expected. And only 0 or the HTTP status code is returned to > HAProxy. I don't know if -101 is a valid return value from modsecurity > point of view. But it is not from the agent one. > > > > Then, You don't have an error 403 because the variable txn.modsec.code > is negative, so the deny http-request rule is never triggered. So, I guess > your error 400 comes from your webserver. You can enabled HTTP log to have > more information. > > > > Finally, I notice some requests to the SPOA agent seems to have failed. > The variable is not set (- in the logs). You can try to enable SPOE logs in > your SPOE engine configuration. Take a look at the SPOE documentation > (doc/SPOE.txt) for more information. > > > Hi, perhaps this thread helps: > > https://www.mail-archive.com/haproxy@formilux.org/msg30061.html > > And perhaps this building of ModSecurity SPOA will also help: > > > https://github.com/jcmoraisjr/modsecurity-spoa/blob/v0.5/rootfs/Dockerfile > > ~jm > First thanks for your reply, I've been following your work on haproxy-ingress for Kubernetes (where I can see you have incorporated ModSecurity) and your input is certainly appreciated on this matter. I had some time today to quickly run the test again after enabling the log for SPOE: [modsecurity] spoe-agent modsecurity-agent log global messagescheck-request option var-prefix modsec option continue-on-error timeout hello 100ms timeout idle30s timeout processing 1s use-backend spoe-modsecurity spoe-message check-request args unique-id method path query req.ver req.hdrs_bin req.body_size req.body event on-frontend-http-request and can see that the empty values coming from SPOE are legit and are due to SEARCH method that is not allowed by Haproxy: Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.accept(000a)=0016 from [127.0.0.1:53206] ALPN= Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.clireq[0016:]: SEARCH / HTTP/1.1 Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.clihdr[0016:]: user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36 Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.clihdr[0016:]: content-type: application/x-www-form-urlencoded Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.clihdr[0016:]: content-length: 1 Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.clihdr[0016:]: host: localhost:9080 Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1343:my-front.srvcls[0017:0013] Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1343:my-front.clicls[0017:0013] Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1343:my-front.closed[0017:0013] Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.srvrep[0016:0013]: HTTP/1.1 405 Method Not Allowed Dec 17 00:40:01 ip-172-31-17-121 haproxy[21508]: SPOE: [modsecurity-agent] sid=4933 st=0 0/0/0/0/0 4/4 0/0 0/2768 Dec 17 00:40:01 ip-172-31-17-121 haproxy[21508]: The txn.modsec.code is: - Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.srvhdr[0016:0013]: server: ecstatic-3.3.2 Dec 17 00:40:01 ip-172-31-17-121 haproxy[21508]: The txn.modsec.code is: - Dec 17 00:40:01 ip-172-31-17-121 haproxy[21508]: SPOE: [modsecurity-agent] sid=4935 st=0 0/0/0/0/0 4/4 0/0 0/2769 Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.srvhdr[0016:0013]: date: Tue, 17 Dec 2019 00:40:01 GMT Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.srvhdr[0016:0013]: content-length: 0 Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.srvcls[0016:0013] Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.clicls[0016:0013] Dec 17 00:40:01 ip-172-31-17-121 haproxy[17468]: 1345:my-front.closed[0016:0013] and some other methods like TRACK, PROPFIND, DEBUG that NIkto tries out. Apart from that, I'm still equally stumped as I was before about the rules not working as expected. For example this request: $ curl -I -H "User-Agent: nikto" http://localhost:9080/ HTTP/1.1 200 OK server: ecstatic-3.3.2 cache-control: max-age=3600 last-modified:
Re: ModSecurity testing
> Em 13 de dez de 2019, à(s) 10:09, Christopher Faulet > escreveu: > > Le 10/12/2019 à 05:24, Igor Cicimov a écrit : >> >> Testing with Haproxy 2.0.10 but same result with 1.8.23. The versions of >> ModSecurity is 2.9.2 and the OWASP rules v3.0.2 >> What am I doing wrong? Can anyone provide a request that should confirm if >> the module is working or not from or share the experience from their own >> setup? > > Hi Igor, > > First of all, I don't know how the modsecurity agent really work. But I'm > surprised to see it returns -101. In the code, -1, 0 or an HTTP status code > is expected. And only 0 or the HTTP status code is returned to HAProxy. I > don't know if -101 is a valid return value from modsecurity point of view. > But it is not from the agent one. > > Then, You don't have an error 403 because the variable txn.modsec.code is > negative, so the deny http-request rule is never triggered. So, I guess your > error 400 comes from your webserver. You can enabled HTTP log to have more > information. > > Finally, I notice some requests to the SPOA agent seems to have failed. The > variable is not set (- in the logs). You can try to enable SPOE logs in your > SPOE engine configuration. Take a look at the SPOE documentation > (doc/SPOE.txt) for more information. Hi, perhaps this thread helps: https://www.mail-archive.com/haproxy@formilux.org/msg30061.html And perhaps this building of ModSecurity SPOA will also help: https://github.com/jcmoraisjr/modsecurity-spoa/blob/v0.5/rootfs/Dockerfile ~jm
Re: ModSecurity testing
Le 10/12/2019 à 05:24, Igor Cicimov a écrit :
Hi all,
I have a quick question about running ModSecurity in Haproxy. I followed the
guide https://github.com/haproxy/haproxy/tree/master/contrib/modsecurity, have
compiled the modsecurity binary and have setup all required configuration for
Haproxy as per the guide.
I have ModSecurity running locally on port 12345:
$ modsecurity -d -n 1 -p 12345 -f /etc/modsecurity/modsecurity.conf -f
/etc/modsecurity/owasp-modsecurity-crs.conf575948204.684882 [00] ModSecurity for
nginx (STABLE)/2.9.2 (http://www.modsecurity.org/) configured.
1575948204.684938 [00] ModSecurity: APR compiled version="1.7.0"; loaded
version="1.7.0"
1575948204.684949 [00] ModSecurity: PCRE compiled version="8.38 "; loaded
version="8.38 2015-11-23"
1575948204.685084 [00] ModSecurity: YAJL compiled version="2.1.0"
1575948204.685096 [00] ModSecurity: LIBXML compiled version="2.9.3"
1575948204.685103 [00] ModSecurity: Status engine is currently disabled, enable
it by set SecStatusEngine to On.
1575948204.701154 [00] Worker 01 initialized
and can see Haproxy connecting to the service in its own logs and ModSecurity
output:
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
Using epoll() as the polling mechanism.
localhost haproxy[518]: Proxy my-front started.
localhost haproxy[518]: Proxy my-front started.
localhost haproxy[518]: Proxy spoe-modsecurity started.
The Haproxy config is quite simple as per the guide:
listen my-front
timeout client 5s
timeout connect 5s
timeout server 5s
mode http
bind :9080
log-format "The txn.modsec.code is: %[var(txn.modsec.code)]"
filter spoe engine modsecurity config /etc/haproxy/spoe-modsecurity.conf
http-request deny if { var(txn.modsec.code) -m int gt 0 }
server local 127.0.0.1:8080
backend spoe-modsecurity
mode tcp
timeout connect 5s
timeout server 3m
server iprep1 127.0.0.1:12345
As you can see I have the OWASP rules setup under /etc/modsecurity/ and the
SecRuleEngine is enabled:
$ grep SecRuleEngine /etc/modsecurity/modsecurity.conf
SecRuleEngine On
and the rules loaded (I guess):
$ cat /etc/modsecurity/owasp-modsecurity-crs.conf
Include /etc/modsecurity/owasp-modsecurity-crs/crs-setup.conf
Include
/etc/modsecurity/owasp-modsecurity-crs/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf
[...]
and SecDefaultAction set to 403:
$ grep DefaultAction /etc/modsecurity/owasp-modsecurity-crs/crs-setup.conf |
grep -v "^#" | grep .
SecDefaultAction "phase:1,log,auditlog,deny,status:403"
SecDefaultAction "phase:2,log,auditlog,deny,status:403"
However, for the life of me I can not make any successful test and get 403 error
from Haproxy when sending test load (as per the guide). For the example query
mentioned there "?param=">alert(1);" Haproxy replies with 400
instead of 403. I have also tried running Nikto2 scanner that should for sure be
detected by the scanner rules but all I get is negative value or not value at
all for the txn.modsec.code variable return by ModSecurity:
haproxy[32752]: The txn.modsec.code is: -101
haproxy[32752]: The txn.modsec.code is: -
haproxy[32752]: The txn.modsec.code is: -101
haproxy[32752]: message repeated 1408 times: [ The txn.modsec.code is: -101]
haproxy[32752]: The txn.modsec.code is: -
haproxy[32752]: The txn.modsec.code is: -101
The ModSecurity output during the test:
1575948214.855512 [00] <1> New Client connection accepted and assigned to
worker 01
1575948214.855689 [01] <1> read_frame_cb
1575948214.855767 [01] <1> New Frame of 129 bytes received
1575948214.855787 [01] <1> Decode HAProxy HELLO frame
1575948214.855804 [01] <1> Supported versions : 2.0
1575948214.855819 [01] <1> HAProxy maximum frame size : 16380
1575948214.855836 [01] <1> HAProxy capabilities : pipelining,async
1575948214.855855 [01] <1> HAProxy supports frame pipelining
1575948214.855872 [01] <1> HAProxy supports asynchronous frame
1575948214.855888 [01] <1> HAProxy engine id :
c2accfac-1da0-4593-81c5-1ad2749be68b
1575948214.855908 [01] <1> Encode Agent HELLO frame
1575948214.855926 [01] <1> Agent version : 2.0
1575948214.855943 [01] <1> Agent maximum frame size : 16380
1575948214.855958 [01] <1> Agent capabilities :
1575948214.855994 [01] <1> write_frame_cb
1575948214.856472 [01] <1> Frame of 54 bytes send
1575948214.856521 [01] <1> read_frame_cb
1575948214.856546 [01] <1> New Frame of 196 bytes received
1575948214.856562 [01] <1> Decode HAProxy NOTIFY frame
1575948214.856578 [01] <1> STREAM-ID=2232 - FRAME-ID=1 - unfragmented frame
received - frag_len=0 - len=196 - offset=8
1575948214.856606 [01] Process frame messages : STREAM-ID=2232 - FRAME-ID=1 -
length=188 bytes
1575948214.856623 [01] Process SPOE Message 'check-request'
1575948214.857123 [01] Encode Agent ACK frame
1575948214.857154 [01] STREAM-ID=2232 - FRAME-ID=1
1575948214.857169 [01] Add action : set variable code=4294967195
ModSecurity testing
Hi all,
I have a quick question about running ModSecurity in Haproxy. I followed
the guide https://github.com/haproxy/haproxy/tree/master/contrib/modsecurity,
have compiled the modsecurity binary and have setup all required
configuration for Haproxy as per the guide.
I have ModSecurity running locally on port 12345:
$ modsecurity -d -n 1 -p 12345 -f /etc/modsecurity/modsecurity.conf -f
/etc/modsecurity/owasp-modsecurity-crs.conf575948204.684882 [00]
ModSecurity for nginx (STABLE)/2.9.2 (http://www.modsecurity.org/)
configured.
1575948204.684938 [00] ModSecurity: APR compiled version="1.7.0"; loaded
version="1.7.0"
1575948204.684949 [00] ModSecurity: PCRE compiled version="8.38 "; loaded
version="8.38 2015-11-23"
1575948204.685084 [00] ModSecurity: YAJL compiled version="2.1.0"
1575948204.685096 [00] ModSecurity: LIBXML compiled version="2.9.3"
1575948204.685103 [00] ModSecurity: Status engine is currently disabled,
enable it by set SecStatusEngine to On.
1575948204.701154 [00] Worker 01 initialized
and can see Haproxy connecting to the service in its own logs and
ModSecurity output:
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
Using epoll() as the polling mechanism.
localhost haproxy[518]: Proxy my-front started.
localhost haproxy[518]: Proxy my-front started.
localhost haproxy[518]: Proxy spoe-modsecurity started.
The Haproxy config is quite simple as per the guide:
listen my-front
timeout client 5s
timeout connect 5s
timeout server 5s
mode http
bind :9080
log-format "The txn.modsec.code is: %[var(txn.modsec.code)]"
filter spoe engine modsecurity config /etc/haproxy/spoe-modsecurity.conf
http-request deny if { var(txn.modsec.code) -m int gt 0 }
server local 127.0.0.1:8080
backend spoe-modsecurity
mode tcp
timeout connect 5s
timeout server 3m
server iprep1 127.0.0.1:12345
As you can see I have the OWASP rules setup under /etc/modsecurity/ and the
SecRuleEngine is enabled:
$ grep SecRuleEngine /etc/modsecurity/modsecurity.conf
SecRuleEngine On
and the rules loaded (I guess):
$ cat /etc/modsecurity/owasp-modsecurity-crs.conf
Include /etc/modsecurity/owasp-modsecurity-crs/crs-setup.conf
Include
/etc/modsecurity/owasp-modsecurity-crs/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf
[...]
and SecDefaultAction set to 403:
$ grep DefaultAction /etc/modsecurity/owasp-modsecurity-crs/crs-setup.conf
| grep -v "^#" | grep .
SecDefaultAction "phase:1,log,auditlog,deny,status:403"
SecDefaultAction "phase:2,log,auditlog,deny,status:403"
However, for the life of me I can not make any successful test and get 403
error from Haproxy when sending test load (as per the guide). For the
example query mentioned there "?param=">alert(1);" Haproxy
replies with 400 instead of 403. I have also tried running Nikto2 scanner
that should for sure be detected by the scanner rules but all I get is
negative value or not value at all for the txn.modsec.code variable return
by ModSecurity:
haproxy[32752]: The txn.modsec.code is: -101
haproxy[32752]: The txn.modsec.code is: -
haproxy[32752]: The txn.modsec.code is: -101
haproxy[32752]: message repeated 1408 times: [ The txn.modsec.code is: -101]
haproxy[32752]: The txn.modsec.code is: -
haproxy[32752]: The txn.modsec.code is: -101
The ModSecurity output during the test:
1575948214.855512 [00] <1> New Client connection accepted and assigned to
worker 01
1575948214.855689 [01] <1> read_frame_cb
1575948214.855767 [01] <1> New Frame of 129 bytes received
1575948214.855787 [01] <1> Decode HAProxy HELLO frame
1575948214.855804 [01] <1> Supported versions : 2.0
1575948214.855819 [01] <1> HAProxy maximum frame size : 16380
1575948214.855836 [01] <1> HAProxy capabilities : pipelining,async
1575948214.855855 [01] <1> HAProxy supports frame pipelining
1575948214.855872 [01] <1> HAProxy supports asynchronous frame
1575948214.855888 [01] <1> HAProxy engine id :
c2accfac-1da0-4593-81c5-1ad2749be68b
1575948214.855908 [01] <1> Encode Agent HELLO frame
1575948214.855926 [01] <1> Agent version : 2.0
1575948214.855943 [01] <1> Agent maximum frame size : 16380
1575948214.855958 [01] <1> Agent capabilities :
1575948214.855994 [01] <1> write_frame_cb
1575948214.856472 [01] <1> Frame of 54 bytes send
1575948214.856521 [01] <1> read_frame_cb
1575948214.856546 [01] <1> New Frame of 196 bytes received
1575948214.856562 [01] <1> Decode HAProxy NOTIFY frame
1575948214.856578 [01] <1> STREAM-ID=2232 - FRAME-ID=1 - unfragmented frame
received - frag_len=0 - len=196 - offset=8
1575948214.856606 [01] Process frame messages : STREAM-ID=2232 - FRAME-ID=1
- length=188 bytes
1575948214.856623 [01] Process SPOE Message 'check-request'
1575948214.857123 [01] Encode Agent ACK frame
1575948214.857154 [01] STREAM-ID=2232 - FRAME-ID=1
1575948214.857169 [01] Add action : set variable code=4294967195
1575948214.857219 [01] <1> write_frame_cb
1575948214.857648 [01] <1> Frame of 31 bytes send
Testing with Haproxy 2.0.10 but same result with
