What I need to do is nfdump from both the last 5 minute rolled file (currently nfcapd.201411141005) and nfcapd.current.18967. Currently my scripts simply use the latest rolled nfcapd file which introduces up to 5 minutes of latency. For my purposes I need to analyze netflow data as real time as possible with nfdump.
In theory this should be as simple as loading both nfcapd.201411141005 and nfcapd.current.18967 into nfdump at the same time as one would with a range of similarly named files using -R or a number of other options available in nfdump. Unfortunately in practice I have not found a way to load both of these files simultaneously for analysis in the same nfdump. Here are some methods I have tried. Specifying multiple -r opts on the command line like: nfdump -r nfcapd.201411141005 -r nfcapd.current.18967 -n 1 -s record/bps -A proto,dstip -o extended result: only reads 1 file, the last -r option, in this case nfcapd.current.18967 Using stdin to read multiple files: cat nfcapd.201411141000 nfcapd.201411141005 | nfdump -n 1 -s record/bps -A proto,dstip -o extended Results in: ReadBlock() error decompression failed in nffile.c line 779: LZO error: -4 Skip corrupt data file '(null)' So the main problem I am having here is piping multiple files to nfdump via stdin. I receive the above error any time I attempt to pipe multiple files to nfdump. Since the error was LZO related, I tried decompressing both files first.. # nfdump -j nfcapd.201411140930 Uncompress file nfcapd.201411140930 .. # nfdump -j nfcapd.201411140935 Uncompress file nfcapd.201411140935 .. # cat nfcapd.20141114093* | nfdump -n 1 -s record/bps -A proto,dstip -o extended Can't process block type 670. Skip block. Can't process block type 0. Skip block. Corrupt data file: Requested buffer size 3489660928 exceeds max. buffer size. Skip corrupt data file '(null)' Reading the files separately works fine and yields no errors. Any ideas on how I can get nfdump to read nfcapd.201411141005 and nfcapd.current.18967 in one run would be appreciated. ------------------------------------------------------------------------------ Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk _______________________________________________ Nfdump-discuss mailing list Nfdump-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
