I am using -x to rename the files to nfcapd.hhmm. Nfsen looks for files named
nfcapd.YYYYMMDDhhmm. I think that's why Nfsen is no longer populating the RRDs,
cannot process netflow via the webinterface, and nfexpire isn't purging the old
flows..
If I remove the optional argument to move the flows, everything starts working
again.
--Chad
On Apr 6, 2010, at 4:23 AM, Peter Haag wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>
> On 4/2/10 22:06, ckotil wrote:
>> I was able to rename the nfcapd files by using a perl script.
>>
>> 'optarg' => '-x "perl /usr/local/bin/nfcapd-rewrite.pl %d/%f %d %f'"},
>>
>> However, This makes Nfsen sad. Nfsen loses track of the files. The rrd
>> graphs have holes in them. Nfexpire does not purge flows, and the Nfsen
>> frontend is unable to process stats on the flows.
>
> There is no reason, why NfSen shouldn't like this additional -x. It's
> executed in the additional nfcapd/launcher
> process, which NfSen has no knowledge about it's existance anyway.
> So there must be another reason for that. What do the logfiles say??
>
> - Peter
>
>>
>> It looks like we need a better way to rename nfcapd files. Feature request?
>>
>> Thanks,
>>
>> --Chad
>>
>>
>>
>> On Apr 1, 2010, at 2:02 PM, ckotil wrote:
>>
>>> As long as NfSen can find the renamed nfcapd files, then that will be OK.
>>> However, So far I am unable to get optarg -x to move the nfcapd file from
>>> nfcapd.YYYYmmddhhmm to nfcapd.hhmm. It seems to be a problem with the -x
>>> variables; %d %f. Whenever you try to combine any -x variable such as %d or
>>> %f with any string, they stop working.
>>>
>>> None of the examples below work.
>>> 'cr-ul' => { 'port' => '10151', 'col' => '#00ff00', 'type'
>>> => 'netflow', 'optarg' => '-t 5 -x "mv -f %d/%f %d/nfcapd.new"' },
>>> The next two examples , are how i envision renaming the nfcapd files.
>>> Stripping out YYYYmmdd from the filename and replacing it with hhmm:
>>> 'cr-ul' => { 'port' => '10151', 'col' => '#00ff00', 'type'
>>> => 'netflow', 'optarg' => '-t 5 -x "perl -e \"my ($suf) = $ARGV[0] =~
>>> m/(....)$/; `mv %d/$ARGV[0] nfcapd.$suf`;\" %f"' },
>>> 'cr-ul' => { 'port' => '10151', 'col' => '#00ff00', 'type'
>>> => 'netflow', 'optarg' => '-t 5 -x "suf=`expr substr %f 16 4`;mv -f %d/%f
>>> %d/nfcapd.$suf"' },
>>>
>>>
>>>
>>> The examples below actually work. But as soon as I combine the use of a -x
>>> variable, such as %d. They no longer work, as seen above.
>>> 'cr-ul' => { 'port' => '10151', 'col' => '#00ff00', 'type'
>>> => 'netflow', 'optarg' => '-t 5 -x "mv -f %d/%f /tmp/testflow"'
>>> 'cr-ul' => { 'port' => '10151', 'col' => '#00ff00', 'type'
>>> => 'netflow', 'optarg' => '-t 5 -x "mv -f %d/%f
>>> /var/data/nfsen/profiles-data/live/cr-ul/2010/04/01/nfcapd.new"' },
>>>
>>> Any ideas?
>>>
>>> Thanks,
>>>
>>> --Chad
>>>
>>>
>>>
>>> On Apr 1, 2010, at 1:41 AM, Peter Haag wrote:
>>>
>>
>>
>> On 3/30/10 15:11, ckotil wrote:
>>>>>> That's exactly what I am trying to do.
>>>>>> I did consider using the -x parameter after reading through the man page
>>>>>> for nfdump, but I wasn't exactly sure how to use it.
>>>>>> One problem I had with hacking up the source is that the nfsen frontend
>>>>>> then needed to be modified to look for filenames named `nfcapd.hhmm`;
>>>>>> the filenames with hour and minute.
>>>>>>
>>>>>> If -x is used with nfcapd, will nfsen still need to be modified or is
>>>>>> there a config bit we can set , instructing nfsen what filenames to look
>>>>>> for?
>>
>> No - you can use the 'optarg' argument in the %sources definition. 'optarg'
>> => '-x whatever ...'
>>
>> - Peter
>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> --Chad
>>>>>>
>>>>>> On Mar 30, 2010, at 12:53 AM, Manish Kumar wrote:
>>>>>>
>>>>>>> Hi ckotil,
>>>>>>> If I get your problem, there is one way out. At the time of
>>>>>>> capturing itself you can rename your file like this.
>>>>>>>
>>>>>>> ./nfcapd -p port_no -t rotating_time -l location_of_files -I
>>>>>>> Binary_file_name -x 'mv file_location_dir/%f file_location_dir/%i'
>>>>>>>
>>>>>>> By this You will always have a single file in ur directory with the
>>>>>>> name of Binary_file_name, so that you don't have to use wild card while
>>>>>>> reading with nfdump -r, you can run the collector at the specified time
>>>>>>> only and stop it by controlling with a script.
>>>>>>>
>>>>>>> May be it work for you.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Mar 29, 2010 at 9:08 PM, ckotil <[email protected]> wrote:
>>>>>>> Hi,
>>>>>>> I would like to collect statistics on my netflow from multiple hosts ,
>>>>>>> spanning multiple days and a specific time. For example from host1 ,
>>>>>>> host2, and host3, on 03/26/2010, 03/27/2010, and 03/28/2010 at 0800.
>>>>>>> The problem I am having is that nfdump seems unable to use a wildcard.
>>>>>>> Here is the command:
>>>>>>>
>>>>>>> [u...@netflow]$ nfdump -M
>>>>>>> /var/data/nfsen/profiles-data/live/cr-ul/2010/03/26:27:28 -R
>>>>>>> nfcapd.*0800 'inet6 and not dst ip fec0:0:0:ffff::1' -S
>>>>>>> WARNING: -S depricated! use -s record/packets/bytes instead. Option
>>>>>>> will get removed.
>>>>>>> stat() error
>>>>>>> '/var/data/nfsen/profiles-data/live/cr-ul/2010/03/26/nfcapd.*0800':
>>>>>>> File not found!
>>>>>>>
>>>>>>> I was able to wrap this command in a script, and by using the -R
>>>>>>> command I could make this work.
>>>>>>> Another solution I found was to hack the source code so that filenames
>>>>>>> were written to disk without year, month, and day; nfcapd.0800 for
>>>>>>> example. Then I could use the command above without a wildcard.
>>>>>>>
>>>>>>> Is there another way to do this without additional scripting or hacking
>>>>>>> up the source?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> --Chad
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Download Intel® Parallel Studio Eval
>>>>>>> Try the new software tools for yourself. Speed compiling, find bugs
>>>>>>> proactively, and fine-tune applications for parallel performance.
>>>>>>> See why Intel Parallel Studio got high marks during beta.
>>>>>>> http://p.sf.net/sfu/intel-sw-dev
>>>>>>> _______________________________________________
>>>>>>> Nfsen-discuss mailing list
>>>>>>> [email protected]
>>>>>>> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Thanks & Regards,
>>>>>>> Manish Kumar,
>>>>>>> Project Associate,
>>>>>>> Computer Networks & Internet Engineering Division,
>>>>>>> Centre for Development of Advanced Computing R&D,
>>>>>>> #68,Electronics City,
>>>>>>> Bangalore 560100,
>>>>>>> Karnataka, India
>>>>>>> Mobile:9886739073
>>>>>>> Ph: 080 28523300 Extn: 2511
>>>>>>> Email: [email protected]
>>>>>>> http://cens.cdac.in/
>>>>>>
>>>>>> Chad E. Kotil
>>>>>> GRNOC Systems Engineer
>>>>>> 812-855-5288
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Download Intel® Parallel Studio Eval
>>>>>> Try the new software tools for yourself. Speed compiling, find bugs
>>>>>> proactively, and fine-tune applications for parallel performance.
>>>>>> See why Intel Parallel Studio got high marks during beta.
>>>>>> http://p.sf.net/sfu/intel-sw-dev
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Nfsen-discuss mailing list
>>>>>> [email protected]
>>>>>> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Download Intel® Parallel Studio Eval
>>> Try the new software tools for yourself. Speed compiling, find bugs
>>> proactively, and fine-tune applications for parallel performance.
>>> See why Intel Parallel Studio got high marks during beta.
>>> http://p.sf.net/sfu/intel-sw-dev
>>> _______________________________________________
>>> Nfsen-discuss mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>
> - --
> _______ SWITCH - The Swiss Education and Research Network ______
> Peter Haag, Security Engineer, Member of SWITCH CERT
> PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7
> SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland
> E-mail: [email protected] Web: http://www.switch.ch/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (Darwin)
>
> iQCVAwUBS7rvk/5AbZRALNr/AQKC/gP/fYF5XpGKIzS0L3o3m8hMHNdX88Vg0ccF
> rjQDs/UEFBUD1At48evZwG4ODmIMhgcFgNATI+8b4P7Yin9Kth4RWCPf+RIr011Q
> 8SqRrJ5uijIZZuy+h0AerKdUFeLKFLRUrQoF/LIVN59/PyQVHfWHX5BjlcI55iou
> MlCPymsM1/k=
> =XuyW
> -----END PGP SIGNATURE-----
------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Nfsen-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
