-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Patch for nfdump-1.6b profiler:
For all those running nfdump-1.6b and nfsen: Please apply the patch appended,
recompile and reinstall nfprofile.
Thanks Ivan for pointing out this issue!
- Peter
Ivan Gasparik wrote:
> Hi
>
> After upgrade to last snapshot of nfdump I've run into similar
> problem - no profiles are beeing updated except the live profile.
> These are the symptoms that I've found:
>
> - after a new profile creation there is .nfstat file missing inside
> the profiles-data structure in the new profile directory
>
> - web interface shows the profile is created too quickly and the
> reason is there are no nfcapd.200906xxxx files created, I can see
> just nfprofile.22xxx files of size 276 bytes for every timeslot since
> the profile start date
>
> - all statistics in the nfsen web interface show x's
>
> I tried to replace nfprofile binary with the older one from nfdump
> 1.5.7 and created a new profile. Everything was fine, nfprofile
> created new profile containing all data except those collected by the
> 1.6b version of nfdump of course.
> Rebuilding profiles using nfsen -r did not help.
>
> Thanks
> Ivan
>
>
>
> On Tuesday 23 June 2009, Donnelly, Michael (OFT) wrote:
>> I'm sorry if I'm not being clear..
>>
>> For shadow profiles. I have a empty graphs and statistics grids
>> full of zeros .. generating a report by moving the slider around
>> on the blank graph pulls data from that time period.
>>
>> for continuous profile The graphs are empty, and the statistics
>> grid beneath the graph holds all xxx's. Generating a report
>> by moving the slider around on the blank graph produces
>> file-not-found errors..
>>
>> Also note botnet plugin produces the following every five minutes
>> Error reading statinfo of 'botnets': No flow file for requested
>> time slot
>>
>> Attached screenshot of shadow profile.. wierd
>>
>>
>>
>> -----Original Message-----
>> From: Peter Haag [mailto:[email protected]]
>> Sent: Tuesday, June 23, 2009 3:02 PM
>> To: Donnelly, Michael (OFT)
>> Cc: [email protected]
>> Subject: Re: [Nfsen-discuss] Recreate / Reinitialize RRD files..
>> Channel Info File Missing?
>>
>> Donnelly, Michael (OFT) wrote:
>>> Peter, Thank you for your guidance..
>>>
>>> 'nfprofile -h' returns help info as expected..
>>> 'ldd nfprofile' show all libs, including librrd.so.2
>>> nfdump -Z against all filters returns fast and quiet
>>> 'nfsen -r live' behaves normal / as expected..
>>> 'nfsen -r ServerNets' returns one of these for each/every
>>> channel. ERR Error reading channel stat information. Missing key
>>> 'first'
>>>
>>> I can create a new profile now, and the new profile mis-behaves
>>> the same way ..
>> As I said - this is not a 'mis-behaving' set an expire time and
>> this messages is gone! The imprtant issue is, that your profiles do
>> work. Ignore channel messages like these.
>>
>> - Peter
>>
>>> Thanks !
>>>
>>> -----Original Message-----
>>> From: Peter Haag [mailto:[email protected]]
>>> Sent: Tuesday, June 23, 2009 1:29 PM
>>> To: Donnelly, Michael (OFT)
>>> Cc: [email protected]
>>> Subject: Re: [Nfsen-discuss] Recreate / Reinitialize RRD files..
>>> Channel Info File Missing?
>>>
>>> Hi Michael,
>>> First - there is no reason to panic! I've upgraded more than one
>>> system to 64bit - and it works. :)
>>>
>>> Donnelly, Michael (OFT) wrote:
>>>> What does this tell me about my troubles?
>>>>
>>>> " nfsen[13856]: Channel info file missing for channel .... "
>>> This is no reason for problems. It simply tells you, that you
>>> have not set any expire limits on that profile. "Missing" does
>>> not mean necessarily "bad".
>>>
>>> 1. Make sure nfprofile is correctly compiled. Run nfprofile on
>>> the command line to see, if it's properly linked with lib rrd:
>>> ./nfprofile -h
>>> 2. If so verify all your channels:
>>>
>>> cd $PROFILESTATDIR ( whatever you have set in nfsen.conf for
>>> $PROFILESTATDIR ) Run this command:
>>> find . -name \*filter.txt -exec nfdump -Z -f {} \;
>>>
>>> This verifies all your filters - just to make sure everything is
>>> clean. Very old installations had an inconsistency in filter and
>>> accepted undocumented filter syntax such as 'srcport' which is
>>> syntactically wrong. Make sure this filer check completes with
>>> any error output. Fix filters, if required.
>>>
>>> If the command above completes you may rebuild the profiles with
>>> 'channel missing' errors although not strictly required:
>>>
>>> ./nfsen -r ServerNets
>>>
>>> Again - if you have a very old installation, you may want to do
>>> that for every profile - it does not harm, but needs some time to
>>> do:
>>> in bourne shell and friends:
>>> sh-3.2$ for profile in `bin/nfsen -A`; do echo Rebuilding
>>> $profile; bin/nfsen -r $profile; done
>>>
>>> for shadow profiles, nfsen spews an error - it can be ignored of
>>> course.
>>>
>>> Afterall nfsen should be in sync again.
>>>
>>> Hope this helps, otherwise come back.
>>>
>>> - Peter
>>>
>>>> My "live" profile is working fine and tracking all data , but
>>>> shadow profiles and graphs never track any data just empty
>>>> repositories and blank graphs .. The above message is one of a
>>>> set.. I get one for each router .. "RouteReflect1" is a valid
>>>> router in , and a member of my live profile..
>>>>
>>>> I'm very much regretting upgrading to 64bit, but needed to
>>>> exceed the per user memory limits posed in 32bit. I'm running
>>>> RHEL 5.3 x86_64
>>>>
>>>> I've recompiled all binaries involved and am running the latest
>>>> nfdump/nfsen code.
>>>>
>>>> Since my move to 64bit i've got a live profile working normally,
>>>> but no new profiles or alerts or plugins behave.
>>>>
>>>> local3 10:50:23 nfsen[9253]: End expire at Tue
>>>> Jun 23 10:50:00 2009 local3 10:50:23 nfsen[9253]: Expire
>>>> profile live group . low water mark: 75%% local3 10:50:22
>>>> nfsen[9253]: Expire profile Cookie group . low water mark: 75%%
>>>> local3 10:50:22 nfsen[9253]: Run expire at Tue Jun 23
>>>> 10:50:00 2009 local3 10:50:22 nfsen[9253]: Error reading
>>>> statinfo of 'botnets': No flow file for requested time slot
>>>> local3 10:50:22 nfsen[9253]: Process alert 'botnets'
>>>> local3 10:50:22 nfsen[9622]: Plugin Cycle: Time:
>>>> 200906231045, Profile: live, Group: ., Module: Events, local3
>>>> 10:50:21 nfsen[9253]: Update profile live in group .
>>>> local3 10:50:20 nfsen[9253]: Update profile ServerNets
>>>> in group . local3 10:50:19 nfsen[9253]: Update profile
>>>> Pookie in group . local3 10:50:19 nfsen[9253]: Channel
>>>> info file missing for channel 'Routerwing' in './Mookie' local3
>>>> 10:50:19 nfsen[9253]: Channel info file missing for
>>>> channel 'Router100sen' in './Mookie' local3 10:50:19
>>>> nfsen[9253]: Channel info file missing for channel
>>>> 'RouteReflect1' in './Mookie' local3 10:50:19
>>>> nfsen[9253]: Channel info file missing for channel
>>>> 'RouterWarner' in './Mookie' local3 10:50:19
>>>> nfsen[9253]: Channel info file missing for channel 'Router2laf'
>>>> in './Mookie' local3 10:50:19 nfsen[9253]: Channel info
>>>> file missing for channel 'Routerwpl' in './Mookie' local3
>>>> 10:50:19 nfsen[9253]: Channel info file missing for
>>>> channel 'Routerc22' in './Mookie' local3 10:50:19
>>>> nfsen[9253]: Channel info file missing for channel 'Routercap'
>>>> in './Mookie' local3 10:50:18 nfsen[9253]: Channel info
>>>> file missing for channel 'Router15met' in './Mookie' local3
>>>> 10:50:18 nfsen[9253]: Channel info file missing for
>>>> channel 'Routeratt' in './Mookie' local3 10:50:18
>>>> nfsen[9253]: Channel info file missing for channel
>>>> 'Routercogent' in './Mookie' local3 10:50:18
>>>> nfsen[9253]: Channel info file missing for channel
>>>> 'RouterG_Tech' in './Mookie' local3 10:50:18
>>>> nfsen[9253]: Update profile Mookie in group . local3 10:50:17
>>>> nfsen[9253]: Update profile Dookie in group . local3
>>>> 10:50:17 last message repeated 11 times local3 10:50:17
>>>> nfsen[9253]: Error reading channel stat information.
>>>> Missing key 'first' local3 10:50:16 nfsen[9253]: Update
>>>> profile Cookie in group . local3 10:50:15 nfsen[9253]:
>>>> 50 channels/alerts to profile daemon 10:50:10
>>>> /usr/local/bin/nfcapd[9218]: Total ignored packets: 0 daemon
>>>> 10:50:10 /usr/local/bin/nfcapd[9218]: Ident:
>>>> 'RouterG_Tech' Flows: 24, Packets: 183, Bytes: 10974, Sequence
>>>> Errors: 12, Bad Packets: 0 daemon 10:50:05
>>>> /usr/local/bin/nfcapd[9239]: Total ignored packets: 0 daemon
>>>> 10:50:05 /usr/local/bin/nfcapd[9239]: Ident: 'Router2laf'
>>>> Flows: 769, Packets: 5087637, Bytes: 2256297523, Sequence
>>>> Errors: 0, Bad Packets: 0 daemon 10:50:02
>>>> /usr/local/bin/nfcapd[9230]: Total ignored packets: 0 daemon
>>>> 10:50:02 /usr/local/bin/nfcapd[9230]: Ident: 'Routercap'
>>>> Flows: 12644, Packets: 8721169, Bytes: 3308263407, Sequence
>>>> Errors: 0, Bad Packets: 0 daemon 10:50:01
>>>> /usr/local/bin/nfcapd[9227]: Total ignored packets: 0 daemon
>>>> 10:50:01 /usr/local/bin/nfcapd[9227]: Ident: 'Routerc22'
>>>> Flows: 11937, Packets: 11231279, Bytes: 5472391616, Sequence
>>>> Errors: 0, Bad Packets: 0 daemon 10:50:01
>>>> /usr/local/bin/nfcapd[9251]: Total ignored packets: 0 daemon
>>>> 10:50:01 /usr/local/bin/nfcapd[9251]: Ident:
>>>> 'Router100sen' Flows: 1048, Packets: 1674423, Bytes: 936777615,
>>>> Sequence Errors: 0, Bad Packets: 0 daemon 10:50:00
>>>> /usr/local/bin/nfcapd[9224]: Total ignored packets: 0 daemon
>>>> 10:50:00 /usr/local/bin/nfcapd[9224]: Ident:
>>>> 'Router15met' Flows: 6644, Packets: 4546867, Bytes: 2135841733,
>>>> Sequence Errors: 0, Bad Packets: 0 daemon 10:50:00
>>>> /usr/local/bin/nfcapd[9245]: Total ignored packets: 0 daemon
>>>> 10:50:00 /usr/local/bin/nfcapd[9245]: Ident:
>>>> 'RouteReflect1' Flows: 0, Packets: 0, Bytes: 0, Sequence Errors:
>>>> 0, Bad Packets: 0 daemon 10:50:00
>>>> /usr/local/bin/nfcapd[9242]: Total ignored packets: 0 daemon
>>>> 10:50:00 /usr/local/bin/nfcapd[9242]: Ident: 'TimeWarner'
>>>> Flows: 295980, Packets: 7668673, Bytes: 5035803183, Sequence
>>>> Errors: 0, Bad Packets: 0 daemon 10:50:00
>>>> /usr/local/bin/nfcapd[9248]: Total ignored packets: 0 daemon
>>>> 10:50:00 /usr/local/bin/nfcapd[9248]: Ident:
>>>> 'Routerbwing' Flows: 278310, Packets: 5924814, Bytes:
>>>> 5642547873, Sequence Errors: 0, Bad Packets: 0 daemon 10:50:00
>>>> /usr/local/bin/nfcapd[9236]: Total ignored packets: 0
>>>> daemon 10:50:00 /usr/local/bin/nfcapd[9236]: Ident:
>>>> 'Routerwpl' Flows: 4582, Packets: 1665665, Bytes: 689980205,
>>>> Sequence Errors: 0, Bad Packets: 0 daemon 10:50:00
>>>> /usr/local/bin/nfcapd[9233]: Total ignored packets: 0 daemon
>>>> 10:50:00 /usr/local/bin/nfcapd[9233]: Ident: 'Routeratt'
>>>> Flows: 238200, Packets: 3813227, Bytes: 1356438479, Sequence
>>>> Errors: 0, Bad Packets: 0 daemon 10:50:00
>>>> /usr/local/bin/nfcapd[9221]: Total ignored packets: 0 daemon
>>>> 10:50:00 /usr/local/bin/nfcapd[9221]: Ident:
>>>> 'Routercogent' Flows: 732270, Packets: 16143537, Bytes:
>>>> 7453599742, Sequence Errors: 0, Bad Packets: 0
>>>>
>>>> Thanks
>>>>
>>>> This e-mail, including any attachments, may be confidential,
>>>> privileged or otherwise legally protected. It is intended only
>>>> for the addressee. If you received this e-mail in error or from
>>>> someone who was not authorized to send it to you, do not
>>>> disseminate, copy or otherwise use this e-mail or its
>>>> attachments. Please notify the sender immediately by reply
>>>> e-mail and delete the e-mail from your system.
>>>>
>>>>
>>>> ----------------------------------------------------------------
>>>> -------------- Are you an open source citizen? Join us for the
>>>> Open Source Bridge conference! Portland, OR, June 17-19. Two
>>>> days of sessions, one day of unconference: $250. Need another
>>>> reason to go? 24-hour hacker lounge. Register today!
>>>> http://ad.doubleclick.net/clk;215844324;13503038;v?http://openso
>>>> urcebridge.org _______________________________________________
>>>> Nfsen-discuss mailing list
>>>> [email protected]
>>>> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>> -------------------------------------------------------------------
>> ----------- _______________________________________________
>> Nfsen-discuss mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Nfsen-discuss mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
- --
_______ SWITCH - The Swiss Education and Research Network ______
Peter Haag, Security Engineer, Member of SWITCH CERT
PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7
SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland
E-mail: [email protected] Web: http://www.switch.ch/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
iQCVAwUBSkIdvf5AbZRALNr/AQIj7QP7BeRqVLwG3yR5lc77Agpetz2aZgB8dXtH
LbEM6HFS2WiNi7pCaHU8dGsMb04dkVZa8Xl6DVP/nkyksYZtRgZ6puke7xnmd4mJ
A5w10a4VS2dtbL9ui3A3jPkVt6dOqCjZIPLbFJ1kHi51NBqvUg02H/+3C/tcJ3WE
QdVq3+KtDUQ=
=nuw6
-----END PGP SIGNATURE-----
--- profile.c.orig Wed Jun 24 14:00:11 2009
+++ profile.c Wed Jun 24 14:05:21 2009
@@ -335,10 +335,6 @@
profile_channels[num_channels].nffile.wfd
= wfd;
profile_channels[num_channels].nffile.compress
= compress;
- profile_channels[num_channels].nffile.block_header->size
= 0;
- profile_channels[num_channels].nffile.block_header->NumRecords = 0;
- profile_channels[num_channels].nffile.block_header->pad
= 0;
- profile_channels[num_channels].nffile.block_header->id
= DATA_BLOCK_TYPE_2;
if ( wfd ) {
profile_channels[num_channels].nffile.block_header =
(data_block_header_t *)malloc(BUFFSIZE);
@@ -346,6 +342,10 @@
fprintf(stderr, "Buffer allocation error: %s",
strerror(errno));
exit(255);
}
+ profile_channels[num_channels].nffile.block_header->size
= 0;
+ profile_channels[num_channels].nffile.block_header->NumRecords
= 0;
+ profile_channels[num_channels].nffile.block_header->pad
= 0;
+ profile_channels[num_channels].nffile.block_header->id
= DATA_BLOCK_TYPE_2;
profile_channels[num_channels].nffile.writeto = (void
*)((pointer_addr_t)profile_channels[num_channels].nffile.block_header +
sizeof(data_block_header_t));
} else {
profile_channels[num_channels].nffile.block_header = NULL;
patch-nfdump-1.6b-001.txt.sig
Description: video/flv
------------------------------------------------------------------------------
_______________________________________________ Nfsen-discuss mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
