[
https://bro-tracker.atlassian.net/browse/BIT-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21500#comment-21500
]
Justin Azoff commented on BIT-1441:
-----------------------------------
files on try.bro.org eventually expire, so I uploaded it here so it does not
get lost.
> Logrotation cannot be set when using path_func
> ----------------------------------------------
>
> Key: BIT-1441
> URL: https://bro-tracker.atlassian.net/browse/BIT-1441
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
> Affects Versions: 2.4
> Environment: SLC6, PF_RING, broctl
> Reporter: Jan Grashoefer
> Attachments: path_func_bug.bro
>
>
> I had a problem using Bro's filtering on my Bro cluster (using broctl). I
> wanted to create separate logfiles in JSON format for some streams. As the
> file name should include the current date, I specified a path_func. So far
> everything worked as expected. Then I tried to disable the logrotation for
> these files by setting interv = 0. Unfortunately this did not work. Setting a
> fixed path, disabling logrotation worked as intended (see
> [http://try.bro.org/#/trybro/saved/14143] an example of the code I used).
> I investigated this issue and think, I have discovered a problem. The
> rotation interval for a writer is determined in CreateWriter in manager.cc
> (see
> [https://github.com/bro/bro/blob/2b1cd66f17194a30b90490965cbdffdd71c18c09/src/logging/Manager.cc#L1064])
> based on the filter. The filter again is determined by writer and path (I
> don't understand why the name of the filter is not used but there may be
> reasons). To see whether the interval is set correctly I added some debug
> output here. Then I did a test specifying a filter for HTTP using path_func
> and a filter for CONN using a fixed path.
> On my worker I get the expected output (except the interval seems wrong):
> {quote} 0.000000/1437813255.656896 [logging] Set interval for
> 'packet_filter' (filter 'default') to '86400.000000'
> 0.000000/1437813255.658523 [logging] Set interval for
> 'loaded_scripts' (filter 'default') to '86400.000000'
> 0.000000/1437813255.685123 [logging] Set interval for
> 'communication' (filter 'default') to '86400.000000'
> 1437813255.644956/1437813255.709181 [logging] Set interval for 'stats'
> (filter 'default') to '86400.000000'
> 1437813255.644965/1437813255.710468 [logging] Set interval for 'weird'
> (filter 'default') to '86400.000000'
> 1437813255.822196/1437813255.834760 [logging] Set interval for 'reporter'
> (filter 'default') to '86400.000000'
> 1437813256.015793/1437813256.027556 [logging] Set interval for 'software'
> (filter 'default') to '86400.000000'
> 1437813256.015793/1437813256.039455 [logging] Set interval for 'files'
> (filter 'default') to '86400.000000'
> 1437813256.015793/1437813256.040269 [logging] Set interval for 'http' (filter
> 'default') to '86400.000000'
> 1437813256.015793/1437813256.040504 [logging] Set interval for
> '/var/opt/bro/logs-json/http-2015-07-25' (filter 'http_json') to '0.000000'
> 1437813257.512453/1437813257.523782 [logging] Set interval for 'x509' (filter
> 'default') to '86400.000000'
> 1437813260.645607/1437813260.656385 [logging] Set interval for 'conn' (filter
> 'default') to '86400.000000'
> 1437813260.645607/1437813260.656526 [logging] Set interval for
> '/var/opt/bro/logs-json/conn' (filter 'conn_json') to '0.000000'
> 1437813262.827012/1437813262.839179 [logging] Set interval for 'dns' (filter
> 'default') to '86400.000000'
> 1437813263.401981/1437813263.411552 [logging] Set interval for 'ssl' (filter
> 'default') to '86400.000000'
> 1437813293.565530/1437813293.575182 [logging] Set interval for 'kerberos'
> (filter 'default') to '86400.000000'{quote}
> But on the manager I get the following:
> {quote}1437813085.377826/1437813085.387819 [logging] Set interval for
> 'loaded_scripts' (filter 'default') to '3600.000000'
> 1437813085.377826/1437813085.400927 [logging] Set interval for
> 'communication' (filter 'default') to '3600.000000'
> 1437813089.408731/1437813089.409921 [logging] Set interval for 'reporter'
> (filter '') to '3600.000000'
> 1437813089.410046/1437813089.411141 [logging] Set interval for 'weird'
> (filter '') to '3600.000000'
> 1437813089.410046/1437813089.411314 [logging] Set interval for
> 'packet_filter' (filter '') to '3600.000000'
> 1437813089.411802/1437813089.412948 [logging] Set interval for 'stats'
> (filter '') to '3600.000000'
> 1437813089.444066/1437813089.445155 [logging] Set interval for 'files'
> (filter '') to '3600.000000'
> 1437813089.453163/1437813089.454249 [logging] Set interval for 'software'
> (filter '') to '3600.000000'
> 1437813089.472973/1437813089.474123 [logging] Set interval for 'dns' (filter
> '') to '3600.000000'
> 1437813089.507522/1437813089.508617 [logging] Set default interval for
> '/var/opt/bro/logs-json/http-2015-07-25' (filter '')
> 1437813089.508759/1437813089.509852 [logging] Set interval for 'http' (filter
> '') to '3600.000000'
> 1437813089.523751/1437813089.524868 [logging] Set interval for 'x509' (filter
> '') to '3600.000000',
> 1437813089.983185/1437813089.984342 [logging] Set interval for 'ssl' (filter
> '') to '3600.000000'
> 1437813093.316215/1437813093.317350 [logging] Set interval for 'ftp' (filter
> '') to '3600.000000'
> 1437813094.076354/1437813094.077442 [logging] Set interval for 'conn' (filter
> '') to '3600.000000'
> 1437813094.077580/1437813094.078657 [logging] Set interval for
> '/var/opt/bro/logs-json/conn' (filter '') to '0.000000'
> 1437813100.949465/1437813100.950567 [logging] Set interval for 'syslog'
> (filter '') to '3600.000000'{quote}
> On the manager you can see, that for all worker-generated logs the filter is
> not known and that the interval for my HTTP-JSON log is set to the default
> value (Note: The instantiating filter is not known because it is not set in
> the call in SendAllWritersTo - see
> [https://github.com/bro/bro/blob/2b1cd66f17194a30b90490965cbdffdd71c18c09/src/logging/Manager.cc#L1174]).
> So why does it work on the worker? Its because the path of the filter is
> determined and set during the write: The first write triggers determining the
> path by the filter. Then the writer is created and path of writer and filter
> match. The writers on the manager seem to be created without a write and
> therefore the filter cannot be determined.
> At first I tried to fix the issue by using the name of the filter but as seen
> in the debug output, the name is not set. I also thought about setting the
> interval using the WriterBackend::WriterInfo, which is passed to CreateWriter
> and has a field for the interval, but there is also the postprocessor set in
> the CreateWriter method. Unfortunately I don't understand how logging is
> distributed between manager and worker in detail, so I do not know how I can
> fix this issue.
--
This message was sent by Atlassian JIRA
(v6.5-OD-08-001#65007)
_______________________________________________
bro-dev mailing list
[email protected]
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
