Yeah, I only thought of perl cause I'm used to running through 'while true' loops and someone showed me Perl was about 400x faster....good thing I'm not running through 10gb/s worth of data :-D
Figured getting closer to hardware was the way to go.....I'll have to check out PF_RING. On Thu, Jun 13, 2013 at 4:49 PM, Jonathan Lassoff <j...@thejof.com> wrote: > On Thu, Jun 13, 2013 at 3:38 PM, Phil Fagan <philfa...@gmail.com> wrote: > > I would assume something FreeBSD based might be best.... > > Meh... personal choice. I prefer Linux, mostly because I know it best > and most network application development is taking place there. > > > On Thu, Jun 13, 2013 at 4:37 PM, Phil Fagan <philfa...@gmail.com> wrote: > >> > >> I really like the idea of a stripe of linux boxes doing the heavy > lifting. > >> Any suggestions on platforms, card types, and chip types that might be > >> better purposed at processing this type of data? > > Personally, I'd use modern-ish Intel Ethernet NICs. They seem to have > the best support in the kernel. > > >> I assume you could write some fast Perl to ingest and manage the tables? > >> What would the package of choice be for something like this? > > Heh... "fast" Perl. > As for programming the processing, I would do as much as possible in > the kernel, as passing packets off to userland really slows everything > down. > If you really need to, I'd do something with Go and/or C these days. > > Using iptables and the "string" module to match patterns, you can chew > through packets pretty efficiently. This comes with the caveat that > this can only match against strings contained within a single packet; > this doesn't do L4 stream reconstruction. > > You can do some incredibly-parallel stuff with ntop's PF_RING code, if > you blow more traffic through a single core than it can chew through. > > It all depends on what you're trying to do. > > --j > >> > >> > >> On Thu, Jun 13, 2013 at 3:11 PM, Jonathan Lassoff <j...@thejof.com> > wrote: > >>> > >>> Are you trying to block flows from becoming established, knowing what > >>> you're looking for ahead of time, or are you looking to examine a > >>> stream of flow establishments, and will snipe off some flows once > >>> you've determined that they should be blocked? > >>> > >>> If you know a 5-tuple (src/dst IP, IP protocol, src/dst L4 ports) you > >>> want to block ahead of time, just place an ACL. It depends on the > >>> platform, but those that implement them in hardware can filter a lot > >>> of traffic very quickly. > >>> However, they're not a great tool when you want to dynamically > >>> reconfigure the rules. > >>> > >>> For high-touch inspection, I'd recommend a stripe of Linux boxes, with > >>> traffic being ECMP-balanced across all of them, sitting in-line on the > >>> traffic path. It adds a tiny bit of latency, but can scale up to > >>> process large traffic paths and apply complex inspections on the > >>> traffic. > >>> > >>> Cheers, > >>> jof > >>> > >>> On Thu, Jun 13, 2013 at 12:32 PM, Eric Wustrow <ew...@umich.edu> > wrote: > >>> > Hi all, > >>> > > >>> > I'm looking for a way to block individual TCP flows (5-tuple) on a > 1-10 > >>> > gbps > >>> > link, with new blocked flows being dropped within a millisecond or so > >>> > of > >>> > being > >>> > added. I've been looking into using OpenFlow on an HP Procurve, but I > >>> > don't > >>> > know much in this area, so I'm looking for better alternatives. > >>> > > >>> > Ideally, such a device would add minimal latency (many/expandable CAM > >>> > entries?), can handle many programatically added flows (hundreds per > >>> > second), > >>> > and would be deployable in a production network (fails in bypass > mode). > >>> > Are > >>> > there any > >>> > COTS devices I should be looking at? Or is the market for this all > >>> > under > >>> > the table to > >>> > pro-censorship governments? > >>> > > >>> > Thanks, > >>> > > >>> > -Eric > >>> > >> > >> > >> > >> -- > >> Phil Fagan > >> Denver, CO > >> 970-480-7618 > > > > > > > > > > -- > > Phil Fagan > > Denver, CO > > 970-480-7618 > -- Phil Fagan Denver, CO 970-480-7618
