[
https://issues.apache.org/jira/browse/DRILL-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967041#comment-15967041
]
Ted Dunning commented on DRILL-5432:
------------------------------------
Charles,
I don't understand your comment. Tug reported the following output from a
sample file:
{code}
select *
from dfs.`data`.`airtunes.pcap`
limit 10
+-------+----------+--------------------------+-----------------+-----------------+-----------+-----------+----------------+-------+
| Type | Network | Timestamp | dst_ip | src_ip
| src_port | dst_port | packet_length | data |
+-------+----------+--------------------------+-----------------+-----------------+-----------+-----------+----------------+-------+
| TCP | 1 | 2012-03-29 22:05:41.808 | /192.168.3.123 |
/192.168.3.107 | 51594 | 5000 | 78 | [] |
| TCP | 1 | 2012-03-29 22:05:41.808 | /192.168.3.107 |
/192.168.3.123 | 5000 | 51594 | 78 | [] |
| TCP | 1 | 2012-03-29 22:05:41.808 | /192.168.3.123 |
/192.168.3.107 | 51594 | 5000 | 66 | [] |
+-------+----------+--------------------------+-----------------+-----------------+-----------+-----------+----------------+-------+
{code}
What is your change going to do?
> Want a memory format for PCAP files
> -----------------------------------
>
> Key: DRILL-5432
> URL: https://issues.apache.org/jira/browse/DRILL-5432
> Project: Apache Drill
> Issue Type: New Feature
> Reporter: Ted Dunning
>
> PCAP files [1] are the de facto standard for storing network capture data. In
> security and protocol applications, it is very common to want to extract
> particular packets from a capture for further analysis.
> At a first level, it is desirable to query and filter by source and
> destination IP and port or by protocol. Beyond that, however, it would be
> very useful to be able to group packets by TCP session and eventually to look
> at packet contents. For now, however, the most critical requirement is that
> we should be able to scan captures at very high speed.
> I previously wrote a (kind of working) proof of concept for a PCAP decoder
> that did lazy deserialization and could traverse hundreds of MB of PCAP data
> per second per core. This compares to roughly 2-3 MB/s for widely available
> Apache-compatible open source PCAP decoders.
> This JIRA covers the integration and extension of that proof of concept as a
> Drill file format.
> Initial work is available at https://github.com/mapr-demos/pcap-query
> [1] https://en.wikipedia.org/wiki/Pcap
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
