Re: [Zeek-Dev] Fwd: Re: Fwd: binpac crash triggered by exportsourcedata

[Song]

Hi Ronka,


The protocol I'm trying to analyze supports multiple authentication methods, 
including SASL Kerberos GSSAPI.
After authentication, according to the authentication method chosen and 
security layer negotiated, the RPC
requests/responses followed could be in plain text, signed or encrypted.

In the plain text form, the PDU is like:
   <4 bytes length field>
   



While in signed or encrypted form, the outmost layer of PDU is like:
   <4 bytes length field>
   


In the later case, the RPC requests/responses PDU (including the 4 bytes length 
field indicating the length of the
request/response data) is encapsulated in the Wrap Tokens. It is possible that 
a big RPC request/response will
be carried by multiple Wrap Token PDUs.



So I have two analyzers:



- controlling analyzer: deal with authentication and decryption, forward 
decrypted RPC PDU data to RPC analyzer
- RPC analyzer: decode RPC request/response


I need the  for the plain text case in which the whole 
controlling analyzer PDU should be forwarded

to the RPC analyzer.


Today I will try to change the type of controlling analyzer to datagram.


Best regards,
Song

-- Original --
From:  "ronka_mata";
Date:  Fri, Mar 8, 2019 10:08 PM
To:  "Song";
Cc:  "zeek-dev@zeek.org"; 
Subject:  Re: Fwd: Re: Fwd: [Zeek-Dev] binpac crash triggered by 
exportsourcedata



Hi Song,
Could you explain a bit more what you are trying to achieve? Do you want to 
deliver the same data into two analyzers? Or is it just part of it? Or deliver 
to one if condition is met and to second one otherwise? 

Do you have to wait with second analyzer after it was passed by the first 
interpreter or can you call second in the deliverstream function? Make the 
second one into child analyzer and then call ForwardStream function or some 
similar approach?

I understand where your problem is with your current code. I was not able to 
get around the len problem yet, but I will give it a go a bit later today, 
unless someone else knows solution first hand.

For delivery of parts of stream data, defined as a bytestream, you can take 
example from smb-pipe.pac forward_dce_rpc funct.
Hope this helps a bit.

Ronka

__
> Od: "Song" 
> Komu: "ronka_mata" 
> Datum: 08.03.2019 05:01
> Předmět: Re: Fwd: Re: Fwd: [Zeek-Dev] binpac crash triggered by 
> exportsourcedata
>
> CC: 
Thank you Ronka.It is a flowunit analyzer. I checked zeek source tree and found 
that there is only 1 flowunitanalyzer (tls-handshake) uses exportsourcedata 
directive. I guess that exportsourcedata onlyapply to non-incremental types. 
Maybe these are true:  - all types in a datagram analyzer can use 
exportsourcedata directive  - only non-incremental types in a flowunit analyzer 
can use exportsourcedataBut I'm not sure about what is non-incremental type, I 
have to check the generated code.The reason that I want sourcedata field is 
that I want to feed the whole test_pdu to anotheranalyzer. Now as a workaround, 
I have to do something like this: test_rpc->DeliverStream(${data}.length() + 4, 
${data}.begin() - 4, is_orig);to bring back the first 4 bytes to form the 
original whole PDU.Maybe I should try datagram analyzer.Song-- 
Original --From:  "ronka_mata";Date:  
Thu, Mar 7, 2019 10:05 PMTo:  "Song";Cc:  "zee
 k-dev"; Subject:  Fwd: Re: Fwd: [Zeek-Dev] binpac crash 
triggered by exportsourcedataHi,What might help is checking how you defined the 
the PDU in .pac file. If it is datagram, mostly used for DNS type traffic or if 
it is flowunit. You can read more on it here 
https://github.com/zeek/binpac/blob/master/README.rst#flowYou do not need to 
define length for datagrams. Look at other protocols for example of 
differences. Eg radius for datagrams and smb for flows.Ronka-- 
Forwarded message -From: Song Date: Thu, Mar 7, 
2019, 13:40Subject: [Zeek-Dev] binpac crash triggered by exportsourcedataTo: 
zeek-dev Hi,I define a PDU like below:type test_pdu = record 
{lenAB   : uint32;pduAB  : test_pdu_ab(lenAB);} =(lenAB 
+ 4),   # fail to compile without ,  
will cause binpac crashtype test_pdu_ab(len: uint32) = record {lenA
: uint
 16;dataA  : bytestring  = lenA;dataB  : bytestring 
 = (len - 2 - lenA);}   #  here is 
OKThe error message is:binpac: 
/home/grid/git/zeek/aux/binpac/src/pac_type.cc:857: std::__cxx11::string 
Type::EvalLengthExpr(Output*, Env*): Assertion `!incremental_input()!` 
failed.Aborted (core dumped)___
zeek-dev mailing list
zeek-dev@zeek.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek-dev

Re: [Zeek-Dev] Zeek using Network attached storage

[ODell, John]

Thank you for the feedback! I am not familiar with Zeek and my customer wanted 
a guarantee they can use a NAS device (nfs) to store collected data for long 
term analysis. Sounds like they can use it!!! Great news!

Thank You,
Dell/EMC -Federal - Isilon
John W. ODell
614-309-3085

On Mar 8, 2019, at 11:43, Michał Purzyński 
mailto:michalpurzyns...@gmail.com>> wrote:


[EXTERNAL EMAIL]

Just make sure you do not sniff the interface that you use for packet storage. 
Nothing like a positive feedback loop that can happen because of some crazy 
network configurations ;]

On Fri, Mar 8, 2019 at 11:31 AM Vern Paxson 
mailto:v...@corelight.com>> wrote:
> They would like to use a NAS (network attached storage) and have asked
> me to validate that it will work.

Zeek will happily read pcaps from Unix files.  Assuming that's the interface
that the NAS provides, sure this will work.  Maybe though I'm not understanding
the question, as it seems quite straightforward.

Vern
___
zeek-dev mailing list
zeek-dev@zeek.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek-dev
___
zeek-dev mailing list
zeek-dev@zeek.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek-dev

Re: [Zeek-Dev] Zeek using Network attached storage

[Michał Purzyński]

Just make sure you do not sniff the interface that you use for packet
storage. Nothing like a positive feedback loop that can happen because of
some crazy network configurations ;]

On Fri, Mar 8, 2019 at 11:31 AM Vern Paxson  wrote:

> > They would like to use a NAS (network attached storage) and have asked
> > me to validate that it will work.
>
> Zeek will happily read pcaps from Unix files.  Assuming that's the
> interface
> that the NAS provides, sure this will work.  Maybe though I'm not
> understanding
> the question, as it seems quite straightforward.
>
> Vern
> ___
> zeek-dev mailing list
> zeek-dev@zeek.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek-dev
>
___
zeek-dev mailing list
zeek-dev@zeek.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek-dev

Re: [Zeek-Dev] Zeek using Network attached storage

[Vern Paxson]

> They would like to use a NAS (network attached storage) and have asked
> me to validate that it will work.

Zeek will happily read pcaps from Unix files.  Assuming that's the interface
that the NAS provides, sure this will work.  Maybe though I'm not understanding
the question, as it seems quite straightforward.

Vern
___
zeek-dev mailing list
zeek-dev@zeek.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek-dev

Re: [Zeek-Dev] Fwd: Re: Fwd: binpac crash triggered byexportsourcedata

[Song]

It turns out that my workaround is wrong. Every field is a copy, so 
field.begin() - 4 will not give you the address of the data 4 bytes before the 
field. It may contains random data even be an illegal address.

-- Original --
From:  "Song";
Date:  Fri, Mar 8, 2019 12:00 PM
To:  "ronka_mata";
Cc:  "zeek-dev@zeek.org"; 
Subject:  Re: [Zeek-Dev] Fwd: Re: Fwd: binpac crash triggered byexportsourcedata



Thank you Ronka.


It is a flowunit analyzer. I checked zeek source tree and found that there is 
only 1 flowunit
analyzer (tls-handshake) uses exportsourcedata directive. I guess that 
exportsourcedata only
apply to non-incremental types. Maybe these are true:


  - all types in a datagram analyzer can use exportsourcedata directive


  - only non-incremental types in a flowunit analyzer can use exportsourcedata


But I'm not sure about what is non-incremental type, I have to check the 
generated code.


The reason that I want sourcedata field is that I want to feed the whole 
test_pdu to another
analyzer. Now as a workaround, I have to do something like this:


 test_rpc->DeliverStream(${data}.length() + 4, ${data}.begin() - 4, is_orig);


to bring back the first 4 bytes to form the original whole PDU.


Maybe I should try datagram analyzer.


Song
-- Original --
From:  "ronka_mata";
Date:  Thu, Mar 7, 2019 10:05 PM
To:  "Song";
Cc:  "zeek-dev"; 
Subject:  Fwd: Re: Fwd: [Zeek-Dev] binpac crash triggered by exportsourcedata



Hi,
What might help is checking how you defined the the PDU in .pac file. If it is 
datagram, mostly used for DNS type traffic or if it is flowunit. You can read 
more on it here 
https://github.com/zeek/binpac/blob/master/README.rst#flow

You do not need to define length for datagrams. Look at other protocols for 
example of differences. Eg radius for datagrams and smb for flows.

Ronka


-- Forwarded message -
From: Song 
Date: Thu, Mar 7, 2019, 13:40
Subject: [Zeek-Dev] binpac crash triggered by exportsourcedata
To: zeek-dev 


Hi,I define a PDU like below:


type test_pdu = record {
lenAB   : uint32;
pduAB  : test_pdu_ab(lenAB);
} =(lenAB + 4),   # fail to compile without , 
 will cause binpac crash


type test_pdu_ab(len: uint32) = record {
lenA: uint16;
dataA  : bytestring  = lenA;
dataB  : bytestring  = (len - 2 - lenA);}   # 
 here is OK


The error message is:
binpac: /home/grid/git/zeek/aux/binpac/src/pac_type.cc:857: 
std::__cxx11::string Type::EvalLengthExpr(Output*, Env*):
 Assertion `!incremental_input()!` failed.
Aborted (core dumped)___
zeek-dev mailing list
zeek-dev@zeek.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek-dev