Re: Syntax to capture tcp-request content? How to access the captured data later?

[Christian Rohmann]

Hello again and very sorry for the late response,


1)

On 26.04.2017 15:38, Swapnil Dubey wrote:
For logging please if you can try this to see if it works, though I 
have not tested it.


log-format %{+Q}[req.payload(3,8)]


That does not work, but I figured how the capturing and logging does work

 # capture 20 bytes starting at byte 16
 tcp-request content capture req.payload(16,20) len 20

 # log capture slot 0#
 log-format "capture0: %[capture.req.hdr(0)]"


In the HAProxy log this is then logged:

 haproxy[$pid]: capture0: A


I am now able to capture some bytes, or rather "string", of the payload 
and also log it. I was even able apply sample filters like base64 or hex 
to it ;-)






2) Unfortunately my intended capture would actually be longer, But the 
it contains terminating null-bytes separating multiple strings and 
HAProxy seems to stop capturing at the first \0 byte there or at least 
only let's me log the first string. According to the documentation 
(https://cbonte.github.io/haproxy-dconv/1.6/configuration.html#7.3.5-req.payload) 
is captured as binary, so this should not be the case, but it apparently is.


  a) I tried to log / dump it using hex or base64 to preserve the whole 
capture ... but no luck, yet.
  b) As for logging the raw data in nothing like capture-req seems 
available. Maybe I missed some other custom log variable to use here?





Maybe you have some other ideas or hints on how to log that captured data?



Regards

Christian

Re: Syntax to capture tcp-request content? How to access the captured data later?

[Swapnil Dubey]

For logging please if you can try this to see if it works, though I have
not tested it.

log-format %{+Q}[req.payload(3,8)]



On Wed, Apr 26, 2017 at 2:07 PM, Christian Rohmann <
christian.rohm...@inovex.de> wrote:

> On 25.04.2017 19:29, Swapnil Dubey wrote:
>
>> I had the similar requirement and I had to add the following statement in
>> order to make it work. I have put comment on the top of each statement.
>>
>
> Awesome, thanks very much for your granular and well commented config
> sample!
>
>
> The ACL part I believe I can get to work this way.
> But from the documentation I read I can also capture a few bytes and,
> instead of just using them in an ACL or to switch to different backend,
> also use them to log. Quite a few non-HTTP based protocols contain
> interesting fields which one might want to log.
>
>
>
> Thanks again,
> Regards
>
> Christian
>

Re: Syntax to capture tcp-request content? How to access the captured data later?

[Christian Rohmann]

On 25.04.2017 19:29, Swapnil Dubey wrote:
I had the similar requirement and I had to add the following statement 
in order to make it work. I have put comment on the top of each 
statement. 


Awesome, thanks very much for your granular and well commented config 
sample!



The ACL part I believe I can get to work this way.
But from the documentation I read I can also capture a few bytes and, 
instead of just using them in an ACL or to switch to different backend, 
also use them to log. Quite a few non-HTTP based protocols contain 
interesting fields which one might want to log.




Thanks again,
Regards

Christian

Re: Syntax to capture tcp-request content? How to access the captured data later?

[Swapnil Dubey]

Just to make it clear that my answer is to solve the problem similar to
what you mentioned in the link w.r.t mqtt. If you problem is different from
that then it might not work.

On Tue, Apr 25, 2017 at 10:59 PM, Swapnil Dubey 
wrote:

> Hi Christian,
>
> I had the similar requirement and I had to add the following statement in
> order to make it work. I have put comment on the top of each statement.
>
>  *# Acl to make haproxy wait till enough information is received from
> client to intercept and route*
>
>   *  acl client_wait req_len gt 28  *
>
> *# Acl to match the data in the payload i.e. match the 28th  byte
> in the payload *
>
> *# and if it matches hex 32 (ascii char 2) then return success.*
>
> *acl last_msisdn req.payload(28,1) -m bin 32 *
>
>   * # Delay the connection for max 10 sec to  receive  and inspect
> the packet and match one of the ACL*
>
>*tcp-request inspect-delay 3s*
>
> *# When payload is received match for acl and accept the
> connection if it matches*
>
>* tcp-request content accept if client_wait*
>
> option tcplog
>
>  *   # Use the  tcp-gcp if acl rule names **last_msisdn** matches*
>
>* use_backend tcp-gcp if **last_msisdn*
>
>
> *Regards,*
>
> *Swapnil*
>
>
>
> On Tue, Apr 25, 2017 at 10:28 PM, Christian Rohmann <
> christian.rohm...@inovex.de> wrote:
>
>> Hello haproxy users!
>>
>> I am trying to get my head around the feature to capture arbitrary
>> tcp-request content.
>> (I've already found a previous thread - http://haproxy.formilux.narkiv
>> e.com/dwcoXP7y/haproxy-with-mqtt where someone asked a something similar
>> to extract data from MQTT requests)
>>
>> In the documentation https://cbonte.github.io/hapro
>> xy-dconv/1.5/configuration.html#4.2-tcp-request%20content
>> Is says that the action "capture" will allow for samples to be captured.
>> In my case I want to capture a few bytes of the payload
>> like it's described for acl samples at https://cbonte.github.io/hapro
>> xy-dconv/1.5/configuration.html#7.3.5-req.payload .
>>
>>
>> After some trial an error I found that config syntax:
>>  tcp-request content capture req.payload_lv(3,8) len 8
>>
>> will at least parse correctly. But how can I access the captured bytes
>> later to i.e. log them to the access log or to use them for load balancing
>> rules?
>>
>>
>>
>>
>> Thanks for any hints
>> Regards
>>
>>
>> Christian
>>
>>
>>
>

Re: Syntax to capture tcp-request content? How to access the captured data later?

[Swapnil Dubey]

Hi Christian,

I had the similar requirement and I had to add the following statement in
order to make it work. I have put comment on the top of each statement.

 *# Acl to make haproxy wait till enough information is received from
client to intercept and route*

  *  acl client_wait req_len gt 28  *

*# Acl to match the data in the payload i.e. match the 28th  byte
in the payload *

*# and if it matches hex 32 (ascii char 2) then return success.*

*acl last_msisdn req.payload(28,1) -m bin 32 *

  * # Delay the connection for max 10 sec to  receive  and inspect the
packet and match one of the ACL*

   *tcp-request inspect-delay 3s*

*# When payload is received match for acl and accept the connection
if it matches*

   * tcp-request content accept if client_wait*

option tcplog

 *   # Use the  tcp-gcp if acl rule names **last_msisdn** matches*

   * use_backend tcp-gcp if **last_msisdn*


*Regards,*

*Swapnil*



On Tue, Apr 25, 2017 at 10:28 PM, Christian Rohmann <
christian.rohm...@inovex.de> wrote:

> Hello haproxy users!
>
> I am trying to get my head around the feature to capture arbitrary
> tcp-request content.
> (I've already found a previous thread - http://haproxy.formilux.narkiv
> e.com/dwcoXP7y/haproxy-with-mqtt where someone asked a something similar
> to extract data from MQTT requests)
>
> In the documentation https://cbonte.github.io/hapro
> xy-dconv/1.5/configuration.html#4.2-tcp-request%20content
> Is says that the action "capture" will allow for samples to be captured.
> In my case I want to capture a few bytes of the payload
> like it's described for acl samples at https://cbonte.github.io/hapro
> xy-dconv/1.5/configuration.html#7.3.5-req.payload .
>
>
> After some trial an error I found that config syntax:
>  tcp-request content capture req.payload_lv(3,8) len 8
>
> will at least parse correctly. But how can I access the captured bytes
> later to i.e. log them to the access log or to use them for load balancing
> rules?
>
>
>
>
> Thanks for any hints
> Regards
>
>
> Christian
>
>
>

Syntax to capture tcp-request content? How to access the captured data later?

[Christian Rohmann]

Hello haproxy users!

I am trying to get my head around the feature to capture arbitrary 
tcp-request content.
(I've already found a previous thread - 
http://haproxy.formilux.narkive.com/dwcoXP7y/haproxy-with-mqtt where 
someone asked a something similar to extract data from MQTT requests)


In the documentation 
https://cbonte.github.io/haproxy-dconv/1.5/configuration.html#4.2-tcp-request%20content
Is says that the action "capture" will allow for samples to be captured. 
In my case I want to capture a few bytes of the payload
like it's described for acl samples at 
https://cbonte.github.io/haproxy-dconv/1.5/configuration.html#7.3.5-req.payload 
.



After some trial an error I found that config syntax:
 tcp-request content capture req.payload_lv(3,8) len 8

will at least parse correctly. But how can I access the captured bytes 
later to i.e. log them to the access log or to use them for load 
balancing rules?





Thanks for any hints
Regards


Christian