Cheers Glenn.
Symlinks and -R worked a trick.
On Thu, Oct 15, 2009 at 8:48 PM, Glenn Forbes Fleming Larratt <
g...@cornell.edu> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> The -R option is flexible enough, I think, but comes with certain
> constraints (:
>
> - the directories you want all need to be at the same hierarchical
> level ( as they are in your example below);
>
> - the directories you want need to be sequential (as I suspect they
> *aren't* in your example below - I presume each of BNE-BRD-1 and
> SYD-BRD-1 have an structure of YYYY/MM/DD below them, yes?);
>
> - the files you want need to go from the lexigraphically last in the
> first directory to the lexigraphically first in the last directory,
> i.e. you could go from BNE-BRD-1/.../nfcapd.200910151420 through
> all the remaining files BNE-BRD-1 that were later in the day than
> 1420, then all the files in BNE-SYD-1/ that were earlier in the day
> than 1420, and finally BNE-SYD-1/...nfcapd.200910151420 itself. I'm
> quite sure your datafiles and what you're trying to do don't line
> up this way.
>
> If you don't mind a heavy dose of cruft, you could wrapper nfdump in your
> scripting language of choice, and write a script that would:
>
> - take a work direcory and a list of files as arguments;
> - create soft links in the work directory to each of the list of files;
> - use "nfdump -R" over the work directory to perform the processing;
> - clean the soft links out of the work directory and exit.
>
> -g
> - --
> Glenn Forbes Fleming Larratt
> Cornell University IT Security Office
>
> On Wed, 14 Oct 2009, Jason Luxton wrote:
>
> > Hi All,
> >
> > This seem like a simple request and I'm sure the answer is staring me in
> the face.
> >
> > How do I supply a list of data files collected by nfcapd to processed by
> nfdump?
> >
> > I have tried to cat all the neccessary files together and pipe them into
> nfdump as follows but also get a 'corrupt data file' message. The
> individual files are fine.
> >
> > <snip>
> > jas...@syd-netflow-01$ cat BNE-BRD-1/2009/10/15/nfcapd.200910151420
> SYD-BRD-1/2009/10/15/nfcapd.200910151420 | nfdump -s dstip:p
> > Can't process block type 20. Skip block.
> > Skip corrupt data file '': 'Corrupt data file: Requested buffer size
> 759452226 exceeds max. buffer size.
> > '
> > Top 10 Dst IP Addr ordered by flows:
> > 2009-10-15 14:13:43.910 667.061 UDP xxx.xxx.xxx.xxx 24957( 4.6)
> 32953( 0.3) 4.1 M( 0.1) 49 49273 124
> > 2009-10-15 14:12:45.521 720.938 TCP xxx.xxx.xxx.xxx 8571( 1.6)
> 153038( 1.6) 145.2 M( 2.3) 212 1.6 M 948
> > 2009-10-15 14:18:50.765 339.602 UDP xxx.xxx.xxx.xxx 6666( 1.2)
> 6978( 0.1) 782377( 0.0) 20 18430 112
> > ...
> > </snip>
> >
> > I am using a snapshot of nfdump as below but have found the same problem
> on version 1.5.7.
> >
> > <snip>
> > nfdump: Version: snapshot-1.6b-20090930 $LastChangedDate: 2009-09-30
> 10:04:28 +0200 (Wed, 30 Sep 2009) $
> > $Id: nfdump.c 31 2009-09-30 08:04:28Z haag $
> > </snip>
> >
> > I can't use multiple '-r' options and -R requires the files to be in the
> same directory. Using the -M option to specify multiple directories doesn't
> help me either. Maybe because the files have the same name but in different
> directories?
> >
> > I'm sure this is a user error but yet to find out how.
> >
> > Cheers
> > Jason
> >
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
>
> iEYEARECAAYFAkrW/hUACgkQLyw7nZwiKgQK6gCglX5SHgklXqxGDmrlSmCEXLYC
> 3gsAoKErleycV9OUIwsh0pWF+YCz/k9/
> =3FK4
> -----END PGP SIGNATURE-----
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Nfdump-discuss mailing list
> Nfdump-discuss@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
>
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Nfdump-discuss mailing list
Nfdump-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
