Take a look at SiLK. Very robust, very powerful, command line:
https://tools.netsa.cert.org/silk/silk.html
FlowViewer sits on top and provides a full GUI interface: flowviewer.net
Joe
On 1/15/2026 12:57 PM, Jonas Muecke via NANOG wrote:
Hi,
I'm looking for recommendations on conversion and long-term storage of
raw IPFIX flow data. Specifically, I need to convert IPFIX flows
stored in pcap files into a columnar format like Parquet to enable
easy and detailed historic analysis that isn't possible with
aggregated data.
Requirements:
- Parse IPFIX from pcap files (including templates)
- Preserve ALL information elements, including custom IEs with
enterprise PENs
- Output enterprise number + IE ID + data (detailed interpretation of
the data not needed)
- Handle IP fragment reassembly or large IP packets
nfdump [1] gets close, but it skips custom IEs. Other tools require
replaying the pcaps which risks overflowing buffers, so reading
directly from pcap files would be preferred.
Has anyone had similar requirements and found a solution? I'm open to
multi-step conversions (e.g., via JSON). Long-term I'll capture
directly to a better format, but need to process existing pcap
archives first.
Thanks,
Jonas
[1] https://github.com/phaag/nfdump
--
Jonas Muecke
Phd Student, TU Dresden
_______________________________________________
NANOG mailing list
https://lists.nanog.org/archives/list/[email protected]/message/U7ZR5BJFNCBWI4EBLRDUVPVEV45GHID6/
_______________________________________________
NANOG mailing list
https://lists.nanog.org/archives/list/[email protected]/message/4YRD6RUYRNOYWYA2KGRP45PHLDYRBCZN/
