Hey Louis

For 10GbE, you *should* be able to capture everything just by tuning your
linux install. Jonathon Kocz et. al. spent quite a lot of time fine tuning
parameters, some of which are detailed in http://arxiv.org/abs/1411.3751

The two main components to the problem are 1) high-speed packet capture is
really hard using only sockets, and 2) capturing and arranging packets into
a ring buffer that can then be the start of a pipeline (or just write it to
disk).

## Capture to a ring buffer

For capture to a ring buffer, there are a few pipelines that might be of
help. For LEDA, we used PSRDADA (http://psrdada.sourceforge.net). VEGAS
uses HASHPIPE (https://github.com/david-macmahon/hashpipe/tree/master/src).
CHIME called their pipeline kotekan (http://arxiv.org/pdf/1503.06189v1.pdf
).

We are actually currently working on a friendlier pipeline and are making
some good progress, and I believe Andrew Jameson has made some good
progress on a similar path. Hopefully these come to fruition and can help
form a "CASPER toolflow for GPUs". This is probably still a bit far off for
your immediate use though :(


## High-speed packet capture

Returning to the packet capture problem, there are now a few competing
"accelerator frameworks" out there for high-speed packet capture. Most of
it comes down to doing a zero-copy from the NIC to the CPU by bypassing the
kernel.

* If you're using a Mellanox NIC, you can use libvma.
http://www.mellanox.com/page/software_vma?mtag=vma

This can be used in two ways:
1) Not changing your code: use a LD_LIBRARY_PRELOAD to load libvma to
intercept socket calls in your C code. We haven't had much luck making this
improve our code.
2) Changing your code: using the "VMA Extra API" and using VMA offload
instead of sockets (see Chapter 7 of
http://www.mellanox.com/related-docs/prod_acceleration_software/VMA_8_0_4_User_Manual_DOC-00393.pdf).
I haven't personally used the API yet, but have heard some success stories.

* CHIME used NTOP's PF_RING ZC and got very impressive results
http://www.ntop.org/products/packet-capture/pf_ring/pf_ring-zc-zero-copy/

* And as Dave mentioned, there is DPDK as well (http://dpdk.org).

Of all of those, I would suggest trying DPDK first, should fine tuning not
get you 100% of the way there.

Good luck!
- Danny

On Thu, Jun 30, 2016 at 3:18 AM, David George <[email protected]> wrote:

> Use a NIC that can do dpdk:
> http://dpdk.org/doc/nics
> then look on the web for dpdk packet capture utils.
>
> David
>
> On 30 June 2016 at 10:30, Andrew Siemion <[email protected]> wrote:
>
>> Hi Louis,
>>
>> We found this paper quite useful:
>>
>> http://www.ieee-hpec.org/2014/CD/index_htm_files/FinalPapers/95.pdf
>>
>> Cheers,
>>
>> Andrew
>>
>> On Jun 30, 2016, at 6:58 AM, Louis P. Dartez <[email protected]>
>> wrote:
>>
>> Hi all,
>> Can anyone guide me on what tools are being used for packet capture
>> purposes right now? I’ve been working on a baseband recorder using a ROACH1
>> using the 10GbE lines but I can’t seem to avoid losing packets when writing
>> data to memory or disk. I wrote a threaded packet sniffer in C with a
>> pretty tight loop that write directly to memory in Ubuntu Linux. I’m
>> wondering if there are any optimizations that I could be missing out.
>>
>>
>> Any advice would be greatly appreciated, Thanks in advance!
>> LD
>>
>> Louis P. Dartez
>>
>> Ph.D. Candidate
>>
>> STARGATE
>>
>> Center for Advanced Radio Astronomy
>>
>> University of Texas Rio Grande Valley
>>
>> (956) 372-5812
>>
>>
>>
>>
>


-- 
Danny Price

[email protected] | +1 617-386-3700

Reply via email to