Re: [tcpdump-workers] New DLT_ type request

2022-01-06 Thread Timotej Ecimovic via tcpdump-workers
--- Begin Message ---



On 1/6/22 1:22 PM, Guy Harris wrote:

CAUTION: This email originated from outside of the organization. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.


On Jan 5, 2022, at 6:53 PM, Timotej Ecimovic  
wrote:


No. Like the document describes: tooling that deals with deframing is expected 
to remove the starting `[`, the ending `]` and the 2 byte length right after 
the `[`.
In case of creating a PCAPNG file out of this stream, the payload of the packet blocks 
will NOT contain the framing. So the "packet" starts with the debug message.

I.e., in LINKTYPE_SILABS_DEBUG_CHANNEL files, the packet doesn't include the 
'[', the length value, or the ']'?
Correct. The "packet" inside the PCAP files will only include the 
payload that is wedged between `[`, the two length bytes, and the `]`.


As far as PCAP files are concerned, this whole debate about framing is 
just "informative", and might be helpful for people who will attempt to 
understand how the TCP/IP stream of this incoming data turns into 
individual "packets" inside PCAP files.


We also have tooling that are NOT performing deframing, but simply 
record raw streams of data. If someone encounters those files, then this 
framing information will help them convert those raw binary files into 
PCAP files.


I will make this more explicit in the document. It was clearly a bit 
confusing.



What do the bits in the "Flags" field of the 3.0 debug message mean?  Does "few 
bytes of future-proofing flags" mean that there are currently no flag bits defined, so that 
the field should always be zero, but there might be flag bits defined in the future?

They mean. "Reserved for future use". The value currently can be arbitrary and 
until someone defines values for them, they have no meaning. I'll make this more specific 
in the doc.

So is there something in the debug message to indicate whether the field has no 
meaning and should be ignored, or has a meaning and should be interpreted?
Yes. The version. If the version is 3, then these are "Undefined, 
reserved for future use". If the version becomes greater than 3, then 
these may develop meaning.
--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


Re: [tcpdump-workers] New DLT_ type request

2022-01-05 Thread Timotej Ecimovic via tcpdump-workers
--- Begin Message ---



I'm requesting an addition of the new DLT type. I'd call it: 
DLT_SILABS_DEBUG_CHANNEL.
The description of the protocol is here:
https://github.com/SiliconLabs/java_packet_trace_library/blob/master/doc/debug-channel.md

 ...


In case of errors (such as the ] not being present after the length bytes) the 
recovery is typically accomplished by the deframing state engine reading 
forward until a next [ is found, and then attempting to resume the deframing. 
This case can be detected, because the payload of individual message contains 
the sequence number.

So, presumably:

 1) all packets in a LINKTYPE_SILABS_DEBUG_CHANNEL capture begin with a 
'[';
No. Like the document describes: tooling that deals with deframing is 
expected to remove the starting `[`, the ending `]` and the 2 byte 
length right after the `[`.
In case of creating a PCAPNG file out of this stream, the payload of the 
packet blocks will NOT contain the framing. So the "packet" starts with 
the debug message.



 2) all bytes after the '[' and the payload bytes specified by the 
length should be ignored as being from a framing error, unless there's just one 
byte equal to ']'?
Deframing engine is looking for a `]` after the end of the  
bytes. If it doesn't find it, it's considered "in error". It will stay 
"in error" and not emit any deframed messages, until it:

   a.) Finds a next `[`.
   b.) Succesfully reads the  bytes.
   c.) Succesfully reads the ']' after the bytes.
If this happens, the deframing is back out of error, and continues 
properly emitting the individual messages.


This design is intended to favor simplicity over recoverability. You can 
be unlucky, and encounter a sequence `[`, 0xFF, 0xFF inside the payload 
when you're in-error, and this would immediately cause you to skip  
bytes. Which may be way too much. And if you have fatefully many `[` 
characters in the payload, you may never recover.
That's ok though. The use of this protocol is such, that the user 
typically simply breaks off all the communication and gives up, when the 
deframer ends up in-error. Deframing errors are rare, because all this 
data in majority of cases travels over TCP/IP. What gets lost are 
typically whole messages because of various buffer exhaustion, not 
individual bytes that would mess up with the deframing.



I.e., code reading the capture file does *not* have to do any deframing?

Correct. Capture file is already deframed into individual debug messages.

What do the bits in the "Flags" field of the 3.0 debug message mean?  Does "few 
bytes of future-proofing flags" mean that there are currently no flag bits defined, so that 
the field should always be zero, but there might be flag bits defined in the future?
They mean. "Reserved for future use". The value currently can be 
arbitrary and until someone defines values for them, they have no 
meaning. I'll make this more specific in the doc.



The types supported are listed in this file.

The file in question:

 
https://github.com/SiliconLabs/java_packet_trace_library/blob/master/silabs-pti/src/main/java/com/silabs/pti/debugchannel/DebugMessageType.java

lists a bunch of message types; is there a document that describes the format 
of messages with each of those types?


Good eyes :)

I am currently in the process of gathering all this documentation, but I 
moved forward with starting with the DLT_ process before I finish all 
that, since it's a decent digital archaeology project and meanwhile I 
can wait out the pull-requests for the new DLT_ numbers to be approved.


The outcome of documentation gathering effort will be:
  a.) The message types that are actually of immediate concern (the 
TX/RX packet formats of various radios, mostly envelopes that 
essentially wrap the payload of a well-known protocol, such as Bluetooth 
or Zigbee) are well documented, and I'll gather all that and document it 
in a same location.
  b.) Several message types that are helpful, and would be "nice to 
show" in the utilities that visualize or analyze these messages are 
simple, and will be documented as well.
  c.) Many message types will be declared "Obsolete." as the format and 
purpose was lost over time, but the message types will remain reserved 
forever.
  d.) Some message types will be declared "Reserved, vendor specific.", 
as the format of those messages is somewhat proprietary. (There may be 
just one or two of those, if any).


--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers


[tcpdump-workers] New DLT_ type request

2022-01-05 Thread Timotej Ecimovic via tcpdump-workers
--- Begin Message ---

Hello!

I'm requesting an addition of the new DLT type. I'd call it: 
DLT_SILABS_DEBUG_CHANNEL.

The description of the protocol is here:
https://github.com/SiliconLabs/java_packet_trace_library/blob/master/doc/debug-channel.md

I'll open a corresponding pull requests shortly

Cheers,
   Timotej Ecimovic



--- End Message ---
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers