Re: Hardware capture platforms
Jay R. Ashworth wrote: And, note carefully: some "dual-speed hubs" are actually a 10BT hub and a 100BT hub *with a switch between them*. I forget which brand I caught this on, but it bit me a couple of years back. 3COM Dual-Speed 10/100 hubs were this way. Got bit by that too back in the day. Technically I think all hubs supporting both 10 and 100 would have to do this. I can't think of any technical way of getting around the problem without doing this. Justin
Re: Hardware capture platforms
All, On the subject of turning off mac learning on a switch, I've just discovered this - an unusual way of using RSPAN to force the MAC learning off on Cisco switches: http://blog.internetworkexpert.com/2008/02/05/turning-switch-into-hub/ # Turn MAC learning on ports Fa0/1 - 3 vtp mode transparent ! vlan 555 remote-span ! interface range Fa 0/1 - 3 switchport trunk encapsulation dot1q switchport mode trunk switchport trunk allowed vlan 555 switchport trunk native vlan 555 Sam Sam Stickland wrote: Lynda wrote: Warren Kumari wrote: What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps I don't believe that such a thing ever existed. Hubs that did 10/100, certainly, but I've never ever seen a hub that did gig speeds. Depends what you mean by 'hub' I guess. I thought the term referred to a device that was half-duplex only, and had no address learning. GE has never supported half-duplex. Sam
Re: Hardware capture platforms
On Wed, 30 Jul 2008, Jon Kibler wrote: However, there is a problem with your specification: No hub (that I am aware of) can do 1Gbps. All hubs are 10/100 AFAIK. GigE is PtP at the physical-layer by the IEEE 802.3ad specification. It's just not possible to have a dumb, GigE hub. You have to have a switch that can be told to L2-forward everything to one or more ports (e.g. through a port-mirroring feature, or by disabling MAC learning). Also, though probably not terribly relevant, various switches have various bugs/malfeatures that cause them to consume certain kinds of frames rather than forward them (e.g. consuming all or certain kinds of ISO frames). regards, -- Paul Jakma [EMAIL PROTECTED] [EMAIL PROTECTED] Key ID: 64A2FF6A Fortune: lisp, v.: To call a spade a thpade.
Re: Hardware capture platforms
On Fri, 1 Aug 2008, Paul Jakma wrote: GigE is PtP at the physical-layer by the IEEE 802.3ad specification. It's Gah, I meant 802.3ab, of course. just not possible to have a dumb, GigE hub. You have to have a switch that can be told to L2-forward everything to one or more ports (e.g. through a port-mirroring feature, or by disabling MAC learning). Also, though probably not terribly relevant, various switches have various bugs/malfeatures that cause them to consume certain kinds of frames rather than forward them (e.g. consuming all or certain kinds of ISO frames). regards, -- Paul Jakma [EMAIL PROTECTED] [EMAIL PROTECTED] Key ID: 64A2FF6A Fortune: Anything is possible, unless it's not.
Re: Hardware capture platforms
Hey, On Thu, 31 Jul 2008 16:00:36 +0100 Leon Ward <[EMAIL PROTECTED]> wrote: > > On 31 Jul 2008, at 14:16, Juuso Lehtinen wrote: > > > Second that. > > > > Using hub to tap into a single link is also risky. I used to monitor > > single FE link with 100M hub. After link had moderate utilization > > >20%, collision led was lit all the time. > > > > I've had good experience with VSS Monitoring Ethernet Aggregator > > taps. Also Catalyst 2960 SPAN seems to work OK. > > > > As for capture PC, we've been using regular PC with Wireshark. > > That's good for single FE link, but has problem with GE and multiple > > links. > > If you need to increase the speed of your capture tool, maybe this [1] > link may be of use. > It is an implementation of a libpcap that implements a shared memory > ring buffer which can result in some capture performance gains. > > [1] http://public.lanl.gov/cpw/ Better off - http://www.ntop.org/PF_RING.html I've seen tenfold decrease in CPU usage using PF_RING. > > -Leon [ cut ] -- Best regards, Nickola Kolev
Re: Hardware capture platforms
I have had the same problem and solved it with a rare (even then) 100BT Only hub. I still have at least one stashed away. For years though, I have been using bonding on Linux to combine multiple tap streams. We also use hardware aggregators for the higher volume applications. Jon On Thu, Jul 31, 2008 at 12:31 PM, Jay R. Ashworth <[EMAIL PROTECTED]> wrote: > > And, note carefully: some "dual-speed hubs" are actually a 10BT hub and > a 100BT hub *with a switch between them*. I forget which brand I > caught this on, but it bit me a couple of years back. > > Which speed cable you plug in determines which hub you're talking to. > > Yes, it's weird. > > Cheers, > -- jra > -- > Jay R. Ashworth Baylink [EMAIL > PROTECTED] > Designer The Things I Think RFC 2100 > Ashworth & Associates http://baylink.pitas.com '87 e24 > St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274 > > Those who cast the vote decide nothing. > Those who count the vote decide everything. > -- (Josef Stalin) > >
Re: Hardware capture platforms
On Jul 31, 2008, at 12:31 PM, Jay R. Ashworth wrote: On Wed, Jul 30, 2008 at 02:47:11PM -0400, Jon Kibler wrote: Hubs are still available that are REAL hubs. I got 4 netgears about a year ago and they are still available. However, there is a problem with your specification: No hub (that I am aware of) can do 1Gbps. All hubs are 10/100 AFAIK. Ok, so I guess what I am speaking is not strictly a hub, it is a non- learning bridge (single collision domain per port, full duplex, etc). There used to be a bunch of devices sold like this -- there were a few really cheap chipsets (AFAIR, Vitesse SparX VSCsomething was one of them -- basically a standard switch chipset that they shaved a few cents off because there was no learning logic / memory) that many people used in cheap "hubs"... I still have some of these somewhere and will rip the lid off to figure out exactly what it was so I can get some more... And, note carefully: some "dual-speed hubs" are actually a 10BT hub and a 100BT hub *with a switch between them*. I forget which brand I caught this on, but it bit me a couple of years back. Which speed cable you plug in determines which hub you're talking to. I see your weird hub story and raise you one: I went along to one of my wife's clients to help lug a printer up the stairs... We get it on the desk and I go to plug in the Ethernet port -- I follow some cables and find this small white switch jammed behind a photocopier -- I pull it out and it has, emblazoned in large red letters on the front, "10/100 Hub with Switch" -- this was back in the day when switches were still cool... I turn it around, and on the back there is... a switch, one side is marked "10M" and the other is marked "100M"... After I stopped laughing I tested it, and sure enough, its a standard hub, and you can make the ports either run at 10Mbps or 100Mbps by flipping the switch... I *really* wish I had replaced and kept it... W Yes, it's weird. Cheers, -- jra -- Jay R. Ashworth Baylink [EMAIL PROTECTED] Designer The Things I Think RFC 2100 Ashworth & Associates http:// baylink.pitas.com '87 e24 St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274 Those who cast the vote decide nothing. Those who count the vote decide everything. -- (Josef Stalin) -- Do not meddle in the affairs of wizards, for they are subtle and quick to anger. -- J.R.R. Tolkien
Re: Hardware capture platforms
On Wed, Jul 30, 2008 at 02:47:11PM -0400, Jon Kibler wrote: > Hubs are still available that are REAL hubs. I got 4 netgears about a > year ago and they are still available. > > However, there is a problem with your specification: No hub (that I am > aware of) can do 1Gbps. All hubs are 10/100 AFAIK. And, note carefully: some "dual-speed hubs" are actually a 10BT hub and a 100BT hub *with a switch between them*. I forget which brand I caught this on, but it bit me a couple of years back. Which speed cable you plug in determines which hub you're talking to. Yes, it's weird. Cheers, -- jra -- Jay R. Ashworth Baylink [EMAIL PROTECTED] Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com '87 e24 St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274 Those who cast the vote decide nothing. Those who count the vote decide everything. -- (Josef Stalin)
Re: Hardware capture platforms
On 31 Jul 2008, at 14:16, Juuso Lehtinen wrote: Second that. Using hub to tap into a single link is also risky. I used to monitor single FE link with 100M hub. After link had moderate utilization >20%, collision led was lit all the time. I've had good experience with VSS Monitoring Ethernet Aggregator taps. Also Catalyst 2960 SPAN seems to work OK. As for capture PC, we've been using regular PC with Wireshark. That's good for single FE link, but has problem with GE and multiple links. If you need to increase the speed of your capture tool, maybe this [1] link may be of use. It is an implementation of a libpcap that implements a shared memory ring buffer which can result in some capture performance gains. [1] http://public.lanl.gov/cpw/ -Leon BR, Juuso On Wed, Jul 30, 2008 at 4:26 PM, Leon Ward <[EMAIL PROTECTED]> wrote: On 30 Jul 2008, at 03:26, James Pleger wrote: Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea. Never try to aggregate multiple TAPs with a hub. You will just create a bucket load of collisions and end up with a useless data feed presented to your monitoring tool. If you want to aggregate multiple TAP feeds into a smaller number of devices(s), most of the TAP vendors make some form of link aggregation device. Or, depending on the OS and sniffer you use, you may be able to bond the interfaces on the capture device. -Leon You can use regular old tcpdump with the -C option to rotate logs tcpdump -i blah -s0 -C , etc. or you can use Daemonlogger which does pretty much the same thing... http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html
Re: Hardware capture platforms
Second that. Using hub to tap into a single link is also risky. I used to monitor single FE link with 100M hub. After link had moderate utilization >20%, collision led was lit all the time. I've had good experience with VSS Monitoring Ethernet Aggregator taps. Also Catalyst 2960 SPAN seems to work OK. As for capture PC, we've been using regular PC with Wireshark. That's good for single FE link, but has problem with GE and multiple links. BR, Juuso On Wed, Jul 30, 2008 at 4:26 PM, Leon Ward <[EMAIL PROTECTED]> wrote: > > On 30 Jul 2008, at 03:26, James Pleger wrote: > >> >> Something you might want to look into is traffic aggregation with a >> switch or hub. You can buy an Allied Telesyn switch and basically turn >> it into a hub by disabling switchport learning. Just an idea. >> > > Never try to aggregate multiple TAPs with a hub. > You will just create a bucket load of collisions and end up with a useless > data feed presented to your monitoring tool. If you want to aggregate > multiple TAP feeds into a smaller number of devices(s), most of the TAP > vendors make some form of link aggregation device. > > Or, depending on the OS and sniffer you use, you may be able to bond the > interfaces on the capture device. > > -Leon > > > > >> You can use regular old tcpdump with the -C option to rotate logs >> >> tcpdump -i blah -s0 -C , etc. >> >> or you can use Daemonlogger which does pretty much the same thing... >> >> http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html >> > > >
Re: Hardware capture platforms
Warren Kumari wrote: On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote: Hubs sure are fun... This might be a stupid question, but where can one get small hubs these days? All of the common commodity (eg: 4 port Netgear) "hubs" these days are actually switches. What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps You won't find the gig-e hub out there for sale despite some ieee 802.3 participants staunch defense of 1/2 duplex gig-e support and the resulting complications that caused/s... Perversely when traveling I actually use the Ethernet ports on my soekris configured as a bridge for this application. A device with 4 Ethernet ports plus a wifi radio which can be configured as bridges, routed, nated etc if that's what's desired. the soekris is not gig-e capable and it's forwarding capacity is a bit closer to the low hundreds of megs, but it travels in my bag, has disk, wifi etc. MSI industrial makes a mini-itx mainboard that will take an intel core2 has 3 embedded gig-e ports and a 16x pci-e slot that you can put a multiport gig or 2 x 10Gbe interface in... I have a utility 10" deep rackmount that I drag around with that in it when I need more power than the soekris can deliver... http://www.logicsupply.com/products/ms_9642 While a tap would work, I'd prefer a hub because I can then use it to connect machines together in a pinch. W --- In the past I have bought some cheap 4 port commodity switches (form Circuit City or somewhere similar), found the datasheet for the chipset (it was a Broadcom something or other) and tied the pin to ground that disables the learning mode (actually, I think that the pin just set the size of the learning table to be 0 entries). While this works, doing it once was more than enough :-) I would trunk the ports you are monitoring, and run the port monitor on the trunk port instead (one trunk port, one port per VLAN, plus one span) which will help with your density. This is assuming the analysis software you have can read the dot1q tags, but means you do not need to burn two ports per monitor. -Original Message- From: James Pleger [mailto:[EMAIL PROTECTED] Sent: Tuesday, July 29, 2008 19:26 To: [EMAIL PROTECTED] Subject: Re: Hardware capture platforms There are several things that you can do with open source solutions, however looking at the data may be a bit more difficult than something like Network Generals or Solera Networks capture appliances. It is still doable and is definitely much much cheaper... Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea. You can use regular old tcpdump with the -C option to rotate logs tcpdump -i blah -s0 -C , etc. or you can use Daemonlogger which does pretty much the same thing... http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html On Tue, Jul 29, 2008 at 6:45 PM, Network Fortius <[EMAIL PROTECTED]> wrote: Richard's blog @ http://taosecurity.blogspot.com/search?q=taps and especially his books (Tao of Network Security Monitoring and Extrusion Detection) are the best sources I have ever found, concerning [not only] taps and[/but] so much more on the subject - proper usage and best methodologies and practices for network monitoring (and not only for security!!!) Stefan On Tue, Jul 29, 2008 at 7:12 PM, Christopher Morrow <[EMAIL PROTECTED] wrote: On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch <[EMAIL PROTECTED]> wrote: Check out packet forensics depending on what your ultimate requirements are. I would also add a 'see packet forensics'... On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <[EMAIL PROTECTED]> wrote: We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware? -- John A. Kilpatrick [EMAIL PROTECTED]Email| http://www.hypergeek.net/ [EMAIL PROTECTED] Text pages| ICQ: 19147504 remember: no obstacles/only challenges -- "Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life." -- Terry Pratchett
Re: Hardware capture platforms
Lynda wrote: Warren Kumari wrote: What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps I don't believe that such a thing ever existed. Hubs that did 10/100, certainly, but I've never ever seen a hub that did gig speeds. Depends what you mean by 'hub' I guess. I thought the term referred to a device that was half-duplex only, and had no address learning. GE has never supported half-duplex. Sam
Re: Hardware capture platforms
Warren Kumari wrote: On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote: Hubs sure are fun... This might be a stupid question, but where can one get small hubs these days? All of the common commodity (eg: 4 port Netgear) "hubs" these days are actually switches. What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps While a tap would work, I'd prefer a hub because I can then use it to connect machines together in a pinch. D-Link sells a smallish 8-port managed Gigabit switch that allows you to disable learning on the ports -- DGS-3200-10 -- http://www.dlink.com/products/?sec=0&pid=674 I don't know where they hide the manuals on the D-Link US site, but Google turned them up on their Russian ftp server ?? While not incredibly cheap, it seems reasonable at about $300. As a bonus, it seems to have pretty complete IPv6 support. We wanted to do something similar with a 10G switch (SMC8708L2). It let's you set the size of the MAC table, but not to zero. However, we found that setting the size of the table to 1 entry effectively disabled learning. W --- In the past I have bought some cheap 4 port commodity switches (form Circuit City or somewhere similar), found the datasheet for the chipset (it was a Broadcom something or other) and tied the pin to ground that disables the learning mode (actually, I think that the pin just set the size of the learning table to be 0 entries). While this works, doing it once was more than enough :-) Nice hack!
Re: Hardware capture platforms
On Jul 31, 2008, at 5:44 AM, [EMAIL PROTECTED] wrote: Check out Endace cards, that will let you do line rate gig e or better and has native libpcap interface. I believe Endace also have a productized box containing their capture cards (NinjaProbe); it can be used to capture packets, and can also export NetFlow telemetry based upon the captured traffic. Arbor, Narus, and Lancope have similar NetFlow-via-packet-capture capabilities. --- Roland Dobbins <[EMAIL PROTECTED]> // +66.83.266.6344 mobile History is a great teacher, but it also lies with impunity. -- John Robb
Re: Hardware capture platforms
Jon Kibler wrote: Hubs are still available that are REAL hubs. I got 4 netgears about a year ago and they are still available. However, there is a problem with your specification: No hub (that I am aware of) can do 1Gbps. All hubs are 10/100 AFAIK. Grand Junction made a gigabit Ethernet repeater around 1996. It was based on the "carrier extension" part of the gigabit Ethernet spec that allows for half-duplex operation. Carrier extension pads any frame shorter than 512 bytes to be 512 bytes long. For that reason (in case frame size distribution matters), as well as the tons of collisions that others have mentioned, I'd also stay away from hubs for the OP's needs. Also, many 10/100 hubs have a 2-port switch to move frames between speeds, so it's conceivable that even a "hub" may have multiple collision domains. dn
Re: Hardware capture platforms
On Tue, 29 Jul 2008, John A. Kilpatrick wrote: We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware? A hardware based capture card is the only way to get to any real throughput. Check out Endace cards, that will let you do line rate gig e or better and has native libpcap interface. You also may want to check out WildPackets cards. <> Nathan StrattonCTO, BlinkMind, Inc. nathan at robotics.net nathan at blinkmind.com http://www.robotics.nethttp://www.blinkmind.com
RE: Hardware capture platforms
The Cisco 8 port 10/100/1000 switch (WS-C2960G-8TC-L) supports RSPAN which would allow you to tap all the ports even though it's a switch. It's about $750, so it's not a cheap option, but it's not outrageous either. It's the right size also. Matthew Huff | One Manhattanville Rd OTA Management LLC | Purchase, NY 10577 www.otaotr.com | Phone: 914-460-4039 aim: matthewbhuff | Fax: 914-460-4139 -Original Message- From: Lynda [mailto:[EMAIL PROTECTED] Sent: Wednesday, July 30, 2008 2:52 PM To: Nanog Subject: Re: Hardware capture platforms Warren Kumari wrote: > > On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote: > >> Hubs sure are fun... > This might be a stupid question, but where can one get small hubs > these days? All of the common commodity (eg: 4 port Netgear) "hubs" > these days are actually switches. True enough. For those of us who need and want something non-switched, eBay and other used hardware places are the only real option. > What I am looking for is: Small enough to live in my notebook bag > (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps I don't believe that such a thing ever existed. Hubs that did 10/100, certainly, but I've never ever seen a hub that did gig speeds. When I realized hubs were about to be an endangered species, I started purchasing new and used. I have at least two that (other than testing) have never been used. > While a tap would work, I'd prefer a hub because I can then use it to > connect machines together in a pinch. The original poster needed to deploy a tap, and a hub (for him) would defeat the purpose entirely. If you really really need a hub (or two), your best bet is to start looking at various resellers. Pity you're not closer; I'm retired, and no longer really need the six or eight that I still have. -- In April 1951, Galaxy published C.M. Kornbluth's "The Marching Morons". The intervening years have proven Kornbluth right. --Valdis Kletnieks
Re: Hardware capture platforms
Warren Kumari wrote: On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote: Hubs sure are fun... This might be a stupid question, but where can one get small hubs these days? All of the common commodity (eg: 4 port Netgear) "hubs" these days are actually switches. True enough. For those of us who need and want something non-switched, eBay and other used hardware places are the only real option. What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps I don't believe that such a thing ever existed. Hubs that did 10/100, certainly, but I've never ever seen a hub that did gig speeds. When I realized hubs were about to be an endangered species, I started purchasing new and used. I have at least two that (other than testing) have never been used. While a tap would work, I'd prefer a hub because I can then use it to connect machines together in a pinch. The original poster needed to deploy a tap, and a hub (for him) would defeat the purpose entirely. If you really really need a hub (or two), your best bet is to start looking at various resellers. Pity you're not closer; I'm retired, and no longer really need the six or eight that I still have. -- In April 1951, Galaxy published C.M. Kornbluth's "The Marching Morons". The intervening years have proven Kornbluth right. --Valdis Kletnieks
Re: Hardware capture platforms
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Warren Kumari wrote: > > On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote: > >> Hubs sure are fun... >> > > This might be a stupid question, but where can one get small hubs these > days? All of the common commodity (eg: 4 port Netgear) "hubs" these > days are actually switches. > > What I am looking for is: > Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) > Cheap > Simple > 10/100/1000Mbps > > While a tap would work, I'd prefer a hub because I can then use it to > connect machines together in a pinch. > Hubs are still available that are REAL hubs. I got 4 netgears about a year ago and they are still available. However, there is a problem with your specification: No hub (that I am aware of) can do 1Gbps. All hubs are 10/100 AFAIK. Jon Kibler - -- Jon R. Kibler Chief Technical Officer Advanced Systems Engineering Technology, Inc. Charleston, SC USA o: 843-849-8214 c: 843-224-2494 s: 843-564-4224 My PGP Fingerprint is: BAA2 1F2C 5543 5D25 4636 A392 515C 5045 CF39 4253 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkiQty8ACgkQUVxQRc85QlOA1ACfWWGa6FcwzcKT1PN+0pBRky46 bUQAnAxgqV4hfGEZBSgPoMXP8+3/PS+k =ynxx -END PGP SIGNATURE- == Filtered by: TRUSTEM.COM's Email Filtering Service http://www.trustem.com/ No Spam. No Viruses. Just Good Clean Email.
Re: Hardware capture platforms
On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote: Hubs sure are fun... This might be a stupid question, but where can one get small hubs these days? All of the common commodity (eg: 4 port Netgear) "hubs" these days are actually switches. What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps While a tap would work, I'd prefer a hub because I can then use it to connect machines together in a pinch. W --- In the past I have bought some cheap 4 port commodity switches (form Circuit City or somewhere similar), found the datasheet for the chipset (it was a Broadcom something or other) and tied the pin to ground that disables the learning mode (actually, I think that the pin just set the size of the learning table to be 0 entries). While this works, doing it once was more than enough :-) I would trunk the ports you are monitoring, and run the port monitor on the trunk port instead (one trunk port, one port per VLAN, plus one span) which will help with your density. This is assuming the analysis software you have can read the dot1q tags, but means you do not need to burn two ports per monitor. -Original Message- From: James Pleger [mailto:[EMAIL PROTECTED] Sent: Tuesday, July 29, 2008 19:26 To: [EMAIL PROTECTED] Subject: Re: Hardware capture platforms There are several things that you can do with open source solutions, however looking at the data may be a bit more difficult than something like Network Generals or Solera Networks capture appliances. It is still doable and is definitely much much cheaper... Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea. You can use regular old tcpdump with the -C option to rotate logs tcpdump -i blah -s0 -C , etc. or you can use Daemonlogger which does pretty much the same thing... http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html On Tue, Jul 29, 2008 at 6:45 PM, Network Fortius <[EMAIL PROTECTED]> wrote: Richard's blog @ http://taosecurity.blogspot.com/search?q=taps and especially his books (Tao of Network Security Monitoring and Extrusion Detection) are the best sources I have ever found, concerning [not only] taps and[/but] so much more on the subject - proper usage and best methodologies and practices for network monitoring (and not only for security!!!) Stefan On Tue, Jul 29, 2008 at 7:12 PM, Christopher Morrow <[EMAIL PROTECTED] wrote: On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch <[EMAIL PROTECTED]> wrote: Check out packet forensics depending on what your ultimate requirements are. I would also add a 'see packet forensics'... On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <[EMAIL PROTECTED]> wrote: We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware? -- John A. Kilpatrick [EMAIL PROTECTED]Email| http://www.hypergeek.net/ [EMAIL PROTECTED] Text pages| ICQ: 19147504 remember: no obstacles/only challenges -- "Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life." -- Terry Pratchett
Re: Hardware capture platforms
On 30 Jul 2008, at 03:26, James Pleger wrote: Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea. Never try to aggregate multiple TAPs with a hub. You will just create a bucket load of collisions and end up with a useless data feed presented to your monitoring tool. If you want to aggregate multiple TAP feeds into a smaller number of devices(s), most of the TAP vendors make some form of link aggregation device. Or, depending on the OS and sniffer you use, you may be able to bond the interfaces on the capture device. -Leon You can use regular old tcpdump with the -C option to rotate logs tcpdump -i blah -s0 -C , etc. or you can use Daemonlogger which does pretty much the same thing... http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html
Re: Hardware capture platforms
Hi John, You might want to check out www.opencalea.org. We have just released opencalea-lite which is a complete re-write of the original opencalea software. OpenCalea-lite is a much better and cleaner re-write(we learnt from our mistakes in the previous releases). One of the problems of the original version was that we were getting bogged down in details over the precise standard format instead of making the core more stable. OpenCalea-lite takes a step back form this and aims at doing well the essense of what packet taps should be able to. It has a nice clean tap/controller/collector architecture which is much more robust. Taps will register with the controller irrespective of which is started first. Process control has also been improved. Starting and stopping taps is handled in a much cleaner way. In addtion TCP streams are used to transfer data. We were about to send out an announcement regarding opencalea-lite on the [EMAIL PROTECTED] mailing list. Aside from calea requirements opencalea-lite is actually a fairly good platform for running remote-taps in your network. -manish Message: 4 Date: Tue, 29 Jul 2008 16:10:09 -0700 (PDT) From: "John A. Kilpatrick" <[EMAIL PROTECTED]> Subject: Hardware capture platforms To: [EMAIL PROTECTED] Message-ID: <[EMAIL PROTECTED]> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware? -- John A. Kilpatrick [EMAIL PROTECTED]Email| http://www.hypergeek.net/ [EMAIL PROTECTED] Text pages| ICQ: 19147504 remember: no obstacles/only challenges
RE: Hardware capture platforms
Hubs sure are fun... I would trunk the ports you are monitoring, and run the port monitor on the trunk port instead (one trunk port, one port per VLAN, plus one span) which will help with your density. This is assuming the analysis software you have can read the dot1q tags, but means you do not need to burn two ports per monitor. -Original Message- From: James Pleger [mailto:[EMAIL PROTECTED] Sent: Tuesday, July 29, 2008 19:26 To: [EMAIL PROTECTED] Subject: Re: Hardware capture platforms There are several things that you can do with open source solutions, however looking at the data may be a bit more difficult than something like Network Generals or Solera Networks capture appliances. It is still doable and is definitely much much cheaper... Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea. You can use regular old tcpdump with the -C option to rotate logs tcpdump -i blah -s0 -C , etc. or you can use Daemonlogger which does pretty much the same thing... http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html On Tue, Jul 29, 2008 at 6:45 PM, Network Fortius <[EMAIL PROTECTED]> wrote: > Richard's blog @ http://taosecurity.blogspot.com/search?q=taps and > especially his books (Tao of Network Security Monitoring and Extrusion > Detection) are the best sources I have ever found, concerning [not only] > taps and[/but] so much more on the subject - proper usage and best > methodologies and practices for network monitoring (and not only for > security!!!) > > > Stefan > > On Tue, Jul 29, 2008 at 7:12 PM, Christopher Morrow <[EMAIL PROTECTED] >> wrote: > >> On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch <[EMAIL PROTECTED]> >> wrote: >> > Check out packet forensics depending on what your ultimate requirements >> are. >> > >> >> I would also add a 'see packet forensics'... >> >> > On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <[EMAIL PROTECTED]> >> > wrote: >> > >> >> >> >> We've deployed a bunch taps in our network and now we need a platform on >> >> which to capture the data. Our bandwidth is currently pretty low but >> I've >> >> got 8 links to tap, which means I need 16 ports. Has anyone done any >> >> research on doing accurate packet capture with commodity hardware? >> >> >> >> >> >> -- >> >> John A. Kilpatrick >> >> [EMAIL PROTECTED]Email| http://www.hypergeek.net/ >> >> [EMAIL PROTECTED] Text pages| ICQ: 19147504 >> >>remember: no obstacles/only challenges >> >> >> >> >> > >> > >> >> >
Re: Hardware capture platforms
There are several things that you can do with open source solutions, however looking at the data may be a bit more difficult than something like Network Generals or Solera Networks capture appliances. It is still doable and is definitely much much cheaper... Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea. You can use regular old tcpdump with the -C option to rotate logs tcpdump -i blah -s0 -C , etc. or you can use Daemonlogger which does pretty much the same thing... http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html On Tue, Jul 29, 2008 at 6:45 PM, Network Fortius <[EMAIL PROTECTED]> wrote: > Richard's blog @ http://taosecurity.blogspot.com/search?q=taps and > especially his books (Tao of Network Security Monitoring and Extrusion > Detection) are the best sources I have ever found, concerning [not only] > taps and[/but] so much more on the subject - proper usage and best > methodologies and practices for network monitoring (and not only for > security!!!) > > > Stefan > > On Tue, Jul 29, 2008 at 7:12 PM, Christopher Morrow <[EMAIL PROTECTED] >> wrote: > >> On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch <[EMAIL PROTECTED]> >> wrote: >> > Check out packet forensics depending on what your ultimate requirements >> are. >> > >> >> I would also add a 'see packet forensics'... >> >> > On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <[EMAIL PROTECTED]> >> > wrote: >> > >> >> >> >> We've deployed a bunch taps in our network and now we need a platform on >> >> which to capture the data. Our bandwidth is currently pretty low but >> I've >> >> got 8 links to tap, which means I need 16 ports. Has anyone done any >> >> research on doing accurate packet capture with commodity hardware? >> >> >> >> >> >> -- >> >> John A. Kilpatrick >> >> [EMAIL PROTECTED]Email| http://www.hypergeek.net/ >> >> [EMAIL PROTECTED] Text pages| ICQ: 19147504 >> >>remember: no obstacles/only challenges >> >> >> >> >> > >> > >> >> >
Re: Hardware capture platforms
Richard's blog @ http://taosecurity.blogspot.com/search?q=taps and especially his books (Tao of Network Security Monitoring and Extrusion Detection) are the best sources I have ever found, concerning [not only] taps and[/but] so much more on the subject - proper usage and best methodologies and practices for network monitoring (and not only for security!!!) Stefan On Tue, Jul 29, 2008 at 7:12 PM, Christopher Morrow <[EMAIL PROTECTED] > wrote: > On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch <[EMAIL PROTECTED]> > wrote: > > Check out packet forensics depending on what your ultimate requirements > are. > > > > I would also add a 'see packet forensics'... > > > On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <[EMAIL PROTECTED]> > > wrote: > > > >> > >> We've deployed a bunch taps in our network and now we need a platform on > >> which to capture the data. Our bandwidth is currently pretty low but > I've > >> got 8 links to tap, which means I need 16 ports. Has anyone done any > >> research on doing accurate packet capture with commodity hardware? > >> > >> > >> -- > >> John A. Kilpatrick > >> [EMAIL PROTECTED]Email| http://www.hypergeek.net/ > >> [EMAIL PROTECTED] Text pages| ICQ: 19147504 > >>remember: no obstacles/only challenges > >> > >> > > > > > >
Re: Hardware capture platforms
On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch <[EMAIL PROTECTED]> wrote: > Check out packet forensics depending on what your ultimate requirements are. > I would also add a 'see packet forensics'... > On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <[EMAIL PROTECTED]> > wrote: > >> >> We've deployed a bunch taps in our network and now we need a platform on >> which to capture the data. Our bandwidth is currently pretty low but I've >> got 8 links to tap, which means I need 16 ports. Has anyone done any >> research on doing accurate packet capture with commodity hardware? >> >> >> -- >> John A. Kilpatrick >> [EMAIL PROTECTED]Email| http://www.hypergeek.net/ >> [EMAIL PROTECTED] Text pages| ICQ: 19147504 >>remember: no obstacles/only challenges >> >> > >
Re: Hardware capture platforms
solera makes some nice boxes also On Tue, Jul 29, 2008 at 7:35 PM, Jared Mauch <[EMAIL PROTECTED]> wrote: > Check out packet forensics depending on what your ultimate requirements > are. > > Jared Mauch > > > On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <[EMAIL PROTECTED]> > wrote: > > >> We've deployed a bunch taps in our network and now we need a platform on >> which to capture the data. Our bandwidth is currently pretty low but I've >> got 8 links to tap, which means I need 16 ports. Has anyone done any >> research on doing accurate packet capture with commodity hardware? >> >> >> -- >> John A. Kilpatrick >> [EMAIL PROTECTED]Email| http://www.hypergeek.net/ >> [EMAIL PROTECTED] Text pages| ICQ: 19147504 >>remember: no obstacles/only challenges >> >> >> >
Re: Hardware capture platforms
Check out packet forensics depending on what your ultimate requirements are. Jared Mauch On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <[EMAIL PROTECTED]> wrote: We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware? -- John A. Kilpatrick [EMAIL PROTECTED]Email| http://www.hypergeek.net/ [EMAIL PROTECTED] Text pages| ICQ: 19147504 remember: no obstacles/only challenges
Hardware capture platforms
We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware? -- John A. Kilpatrick [EMAIL PROTECTED]Email| http://www.hypergeek.net/ [EMAIL PROTECTED] Text pages| ICQ: 19147504 remember: no obstacles/only challenges
