[Bro-Dev] [JIRA] (BIT-1042) Caret in regex pattern doesn't match start of line
[
https://bro-tracker.atlassian.net/browse/BIT-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13312#comment-13312
]
Seth Hall commented on BIT-1042:
The caret matches beginning of string. If you want to match newlines, use /\n/.
> Caret in regex pattern doesn't match start of line
> --
>
> Key: BIT-1042
> URL: https://bro-tracker.atlassian.net/browse/BIT-1042
> Project: Bro Issue Tracker
> Issue Type: Bug
> Components: Bro
>Affects Versions: 2.1
> Environment: Ubuntu 12.04 and 13.04. Dependencies from apt-get ubuntu
> repos only.
>Reporter: Brian Little
>Priority: Normal
> Labels: pattern, regex
> Attachments: caret.bro
>
>
> print split_all("some string", /^t/);
> I would expect it to not match ^t, but it matches any t in the string.
> output:
> {
> [1] = some s,
> [3] = ring,
> [2] = t
> }
> expected:
> {
> [1] = some string
> }
> tested on bro 2.1 and github master 2.1-824
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1042) Caret in regex pattern doesn't match start of line
[
https://bro-tracker.atlassian.net/browse/BIT-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1042:
---
Resolution: Won't Fix
Status: Closed (was: Open)
> Caret in regex pattern doesn't match start of line
> --
>
> Key: BIT-1042
> URL: https://bro-tracker.atlassian.net/browse/BIT-1042
> Project: Bro Issue Tracker
> Issue Type: Bug
> Components: Bro
>Affects Versions: 2.1
> Environment: Ubuntu 12.04 and 13.04. Dependencies from apt-get ubuntu
> repos only.
>Reporter: Brian Little
>Priority: Normal
> Labels: pattern, regex
> Attachments: caret.bro
>
>
> print split_all("some string", /^t/);
> I would expect it to not match ^t, but it matches any t in the string.
> output:
> {
> [1] = some s,
> [3] = ring,
> [2] = t
> }
> expected:
> {
> [1] = some string
> }
> tested on bro 2.1 and github master 2.1-824
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1043) LRU Table implementation
[
https://bro-tracker.atlassian.net/browse/BIT-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1043:
---
Affects Version/s: (was: 2.1)
2.3
> LRU Table implementation
>
>
> Key: BIT-1043
> URL: https://bro-tracker.atlassian.net/browse/BIT-1043
> Project: Bro Issue Tracker
> Issue Type: Improvement
> Components: Bro
>Affects Versions: 2.3
>Reporter: Jordi Ros-Giralt
>Priority: Medium
>
> Attaching below the email description i exchanged with Seth and Robin
> describing this work.
> --
> Hi Seth and Robin,
> We got the repo up, you can get to our branch as follows:
> git clone --recursive https://github.com/giralt/bro.git
> cd bro/
> git checkout lru-table
> We would be happy to contribute this code to the Bro community. This is what
> it does:
> - It implements LRU tables for Bro
> - A Bro table can be enhanced with the LRU functionality with the following
> new table attributes:
> &lru_table: enhance the table with LRU functionality
> &size_limit=n: if adding an element to the table makes the size of the table
> larger than n, then drop the LRU element from that table before inserting the
> new element. n=0 means table size can be infinite (so don't drop elements
> from it)
> &drop_func=callback_func: defines a programmable callback function that gets
> called automatically every time an element from the LRU table is dropped due
> to hitting the size_limit. The prototype of this callback must be as follows:
> function callback_func(t: table[keytype] of valuetype, key: keytype, val:
> valuetype): count
> - It adds the following bif functions:
> function get_lru%(v: any%): any
> function get_mru%(v: any%): any
> function get_lru_key%(v: any%): any
> function get_mru_key%(v: any%): any
> - Example:
> function freed(t: table[port] of string, key: port, val: string): count {
> print "Dropped"; }
> local port_names: table[port] of string &lru &size_limit=2 &drop_func=freed;
> In terms of applications, we are currently using this feature for the
> chimera-to-bro compiler we are working on:
> http://www.chimera-query.org/index.html
> We thought that we could also use this feature to provide a sort of memory
> management facility for Bro. I had a talk with Seth some weeks ago about
> this. Something like the LRU implementation allows programmers to put bounds
> on the size of tables and prioritize which elements can be dropped first upon
> memory exhaustion scenarios. Perhaps an idea here would be to develop a
> garbage collector (could be done using Bro language itself, perhaps as a
> framework) which would be run upon hitting a certain memory usage watermark
> and which would mainly run through the set of tables marked as "garbage
> collectable" dropping LRU elements from them to help reduce/eliminate the
> risk of running out of memory.
> Should this be something interesting, what are the steps we would need to do
> to open source the LRU code into Bro?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1044) topic/seth/faf-updates ready for merge
[ https://bro-tracker.atlassian.net/browse/BIT-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1044: --- Status: Merge Request (was: Open) > topic/seth/faf-updates ready for merge > -- > > Key: BIT-1044 > URL: https://bro-tracker.atlassian.net/browse/BIT-1044 > Project: Bro Issue Tracker > Issue Type: Task >Reporter: Seth Hall >Assignee: Robin Sommer >Priority: Normal > > Big updates to logs and functionality of the files framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1044) topic/seth/faf-updates ready for merge
Seth Hall created BIT-1044: -- Summary: topic/seth/faf-updates ready for merge Key: BIT-1044 URL: https://bro-tracker.atlassian.net/browse/BIT-1044 Project: Bro Issue Tracker Issue Type: Task Reporter: Seth Hall Assignee: Robin Sommer Priority: Normal Big updates to logs and functionality of the files framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-927) topic/seth/metrics-merge: Metrics framework updates
[ https://bro-tracker.atlassian.net/browse/BIT-927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-927: -- Resolution: Fixed Status: Closed (was: Open) Closing. This ticket is wildly outdated. > topic/seth/metrics-merge: Metrics framework updates > --- > > Key: BIT-927 > URL: https://bro-tracker.atlassian.net/browse/BIT-927 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro >Affects Versions: git/master >Reporter: Seth Hall >Assignee: Seth Hall > Fix For: 2.2 > > > This branch is in a workable state and basically ready to be merged, but I'd > appreciate a more detailed API/sanity review from anyone willing to take a > look before it gets merged. This code is starting to get more and more > important and I don't think we can afford to get it wrong for a release. New > scripts include policy/misc/scan.bro, policy/misc/detect-traceroute, and > various metrics test scripts that you can find by searching for > "base/frameworks/metrics". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-951) Error message from SSL delay logging
[
https://bro-tracker.atlassian.net/browse/BIT-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-951:
--
Resolution: Fixed
Status: Closed (was: Open)
> Error message from SSL delay logging
>
>
> Key: BIT-951
> URL: https://bro-tracker.atlassian.net/browse/BIT-951
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: Seth Hall
>Assignee: Seth Hall
>Priority: Low
> Fix For: 2.2
>
>
> This is mostly a reminder for me..
> {noformat}
> 1362104865.826919 ./scripts/base/protocols/ssl/./main.bro, lines 193-194: SSL
> delay tokens not released in time ({
> notary
> })
> {noformat}
> I got that message in my console when I ran Bro with the local.bro script. I
> think that we should just go ahead and log the line without waiting if Bro is
> terminating. I'd rather get the log line even without any extended
> information and just skip the error message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-951) Error message from SSL delay logging
[
https://bro-tracker.atlassian.net/browse/BIT-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416#comment-13416
]
Seth Hall commented on BIT-951:
---
I fixed this with a commit directly to master this morning.
> Error message from SSL delay logging
>
>
> Key: BIT-951
> URL: https://bro-tracker.atlassian.net/browse/BIT-951
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: Seth Hall
>Assignee: Seth Hall
>Priority: Low
> Fix For: 2.2
>
>
> This is mostly a reminder for me..
> {noformat}
> 1362104865.826919 ./scripts/base/protocols/ssl/./main.bro, lines 193-194: SSL
> delay tokens not released in time ({
> notary
> })
> {noformat}
> I got that message in my console when I ran Bro with the local.bro script. I
> think that we should just go ahead and log the line without waiting if Bro is
> terminating. I'd rather get the log line even without any extended
> information and just skip the error message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-176) Adding names to mails from tracker
[ https://bro-tracker.atlassian.net/browse/BIT-176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-176: -- Resolution: Fixed Status: Closed (was: Open) This one definitely doesn't apply anymore. > Adding names to mails from tracker > -- > > Key: BIT-176 > URL: https://bro-tracker.atlassian.net/browse/BIT-176 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: TicketTracker >Reporter: Robin Sommer >Assignee: Seth Hall > > Would be cool if the mails generated by the tracker came with the real-name > of the person triggering the change in the From line (rather than "Bro > Tracker"). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-333) Add workflow transitions to Trac's email interface.
[ https://bro-tracker.atlassian.net/browse/BIT-333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-333: -- Resolution: Fixed Status: Closed (was: Open) Again, doesn't apply anymore. > Add workflow transitions to Trac's email interface. > --- > > Key: BIT-333 > URL: https://bro-tracker.atlassian.net/browse/BIT-333 > Project: Bro Issue Tracker > Issue Type: Task > Components: TicketTracker >Reporter: Robin Sommer >Assignee: Seth Hall > > Currently one can't simply close a ticket via mail while setting the > resolution correctly. > See > https://subtrac.sara.nl/oss/email2trac/wiki/Email2tracConfiguration#Workflow > Note that this should probably wait until we have updated the workflow we use > in Trac (like with getting rid of the distinction between new/seen). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer
Seth Hall created BIT-1054: -- Summary: Merge unified2 file analyzer Key: BIT-1054 URL: https://bro-tracker.atlassian.net/browse/BIT-1054 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.2 Reporter: Seth Hall The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer
[ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1054: --- Status: Merge Request (was: Open) > Merge unified2 file analyzer > > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.2 >Reporter: Seth Hall > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer
[ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503#comment-13503 ] Seth Hall commented on BIT-1054: Oops, that definitely wasn't supposed to happen. Will fix. > Merge unified2 file analyzer > > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.2 >Reporter: Seth Hall >Assignee: Seth Hall > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer
[ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1054: --- Assignee: Robin Sommer (was: Seth Hall) > Merge unified2 file analyzer > > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.2 >Reporter: Seth Hall >Assignee: Robin Sommer > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer
[ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504#comment-13504 ] Seth Hall commented on BIT-1054: Fixed it. > Merge unified2 file analyzer > > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.2 >Reporter: Seth Hall >Assignee: Seth Hall > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-779) missing values cause bro to crash when used inside of a 'when' statement.
[
https://bro-tracker.atlassian.net/browse/BIT-779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506#comment-13506
]
Seth Hall commented on BIT-779:
---
This is on git master? Keep in mind that this is *not* fixed in 2.1.
> missing values cause bro to crash when used inside of a 'when' statement.
> -
>
> Key: BIT-779
> URL: https://bro-tracker.atlassian.net/browse/BIT-779
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: justin
>Priority: High
> Labels: InterpreterException, when
> Fix For: 2.2
>
>
> Simplest test case:
> {noformat}
> event bro_init()
> {
> local loc: geo_location;
> when (local hostname = lookup_addr(127.0.0.1)){
> print "Location", loc$country_code;
> print "ok";
> terminate();
> }
> }
> {noformat}
> gives:
> {noformat}
> terminate called after throwing an instance of 'InterpreterException'
> {noformat}
> outside of the when block, reporter.log would get:
> {noformat}
> Reporter::ERROR field value missing [loc$country_code]
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1056) incorrect values in num_pkts and num_bytes_ip
[
https://bro-tracker.atlassian.net/browse/BIT-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508#comment-13508
]
Seth Hall commented on BIT-1056:
What do you mean by a "network probe"?
In the logs that you sent that show 0 outbound packets and 0 outbound bytes was
that all from hosts sniffing their own local interface? If that's the case
then you have invalid checksums (due to on-NIC checksum offloading) which would
cause exactly what you were showing.
> incorrect values in num_pkts and num_bytes_ip
> -
>
> Key: BIT-1056
> URL: https://bro-tracker.atlassian.net/browse/BIT-1056
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master, 2.1
> Environment: Two separate testing environments:
> 1. Intel i3, OpenSuse x64, public IP, Bro is on the computer monitoring its
> own traffic
> 2. Intel i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual
> machine has private IP, Bro is on virtual machine monitoring its own traffic
>Reporter: Jakub Svoboda
> Labels: issue
>
> Problem: Bro does not see traffic correctly.
> * conn.log contains invalid information, with values differing from a
> network monitoring probe. (The probe runs on a different computer, so there's
> no interference.)
> * Our own scripts logging connections using the connection_state_remove
> event report the same bad values as conn.log.
> * Specifically, all following values are off in the connection_state_remove
> event handler: c$orig$num_pkts c$resp$num_pkts c$conn$orig_pkts
> c$conn$resp_pkts c$orig$num_bytes_ip c$resp$num_bytes_ip c$conn$orig_ip_bytes
> c$conn$resp_ip_bytes. (Yes, we know some of the values are copied one from
> another in conn/main.bro.)
> * Problem is the same on both tested machines.
> * Problem is the same both when capturing an interface and when analyzing a
> pcap.
> * Problem is the same on both Bro 2.1 and the current git version (as of
> 14.08.2013)
> The problem is exactly the same on two different computers.
> 1. i3, OpenSuse, public IP
> 2. i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine
> has private IP, Bro is on virtual machine monitoring its own traffic
> Both computers use 64bit OS and Bro 2.1 compiled from source.
> Notes: The chance of the network monitoring probe reporting bad values is
> very low as this probe is used by our department of computer science and
> several people checked it works correctly.
> Also, the values reported by Bro are obviously wrong even without no
> comparison with the probe---e.g. an outgoing flow with 0 outgoing packets, 0
> outgoing bytes and nonzero incoming packets/bytes.
> Test 1:
> Bro log files were read after bro shutdown so that there should be nothing
> missing.
> Bro was installed on a computer with IP 147.251.13.65, monitoring its own
> traffic. A web browser was used to generate connections. One of the contacted
> addresses was 77.75.77.170.
> Data from conn.log:
> id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts
> orig_ip_bytes resp_pkts resp_ip_bytes
> 147.251.13.65 44690 77.75.77.170993 tcp 0 151 0
> 0 3 307
> (this was the only line with 147.251.13.65 and 77.75.77.170)
> Data from the network monitoring probe:
> nfdump filter:
> ip 147.251.13.65 and ip 77.75.77.170
> Date first seen Duration Proto Src IP Addr:Port Dst IP
> Addr:Port Flags Tos PacketsBytes pps bpsBpp Flows
> 2013-08-14 09:54:30.355 0.054 TCP 77.75.77.170:993 ->
> 147.251.13.65:44690 .AP... 03 307 5545481102
> 1
> 2013-08-14 09:54:30.347 0.062 TCP 147.251.13.65:44690 ->
> 77.75.77.170:993 .AP... 05 348 8044903 69 1
> Summary: total flows: 2, total bytes: 655, total packets: 8, avg bps: 84516,
> avg pps: 129, avg bpp: 81
> Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00
> Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460
> Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5795248.1
> Test 2:
> Bro log files were read after bro shutdown so that there should be nothing
> missing.
> Bro was installed on a computer with IP 147.251.13.65, monitoring its own
> traffic. A web browser was used to generate connections. One of the contacted
> addresses was 195.113.232.73.
> Debug output of our own Bro script watching connections using
> connection_state_remove event handler.
> The script's code for the event handler is:
> event connection_state_remove(c: connection) {
> NOTICE([$note=profile_creation::Test3, $msg=cat("DBG: ", c$id$orig_h, "->",
> c$id$resp_h, " c$orig$num_pkts=", c$orig$n
[Bro-Dev] [JIRA] (BIT-1056) incorrect values in num_pkts and num_bytes_ip
[
https://bro-tracker.atlassian.net/browse/BIT-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509#comment-13509
]
Seth Hall commented on BIT-1056:
Nevermind, is re-read and I see that that is the case. You are seeing effects
of TCP checksum offloading.
Refer to this blog post for more information about the problem:
http://securityonion.blogspot.com/2011/10/when-is-full-packet-capture-not-full.html
For testing purposes (*not in production!*) you can put the following line into
local.bro:
bq. redef ignore_checksums = T;
> incorrect values in num_pkts and num_bytes_ip
> -
>
> Key: BIT-1056
> URL: https://bro-tracker.atlassian.net/browse/BIT-1056
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master, 2.1
> Environment: Two separate testing environments:
> 1. Intel i3, OpenSuse x64, public IP, Bro is on the computer monitoring its
> own traffic
> 2. Intel i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual
> machine has private IP, Bro is on virtual machine monitoring its own traffic
>Reporter: Jakub Svoboda
> Labels: issue
>
> Problem: Bro does not see traffic correctly.
> * conn.log contains invalid information, with values differing from a
> network monitoring probe. (The probe runs on a different computer, so there's
> no interference.)
> * Our own scripts logging connections using the connection_state_remove
> event report the same bad values as conn.log.
> * Specifically, all following values are off in the connection_state_remove
> event handler: c$orig$num_pkts c$resp$num_pkts c$conn$orig_pkts
> c$conn$resp_pkts c$orig$num_bytes_ip c$resp$num_bytes_ip c$conn$orig_ip_bytes
> c$conn$resp_ip_bytes. (Yes, we know some of the values are copied one from
> another in conn/main.bro.)
> * Problem is the same on both tested machines.
> * Problem is the same both when capturing an interface and when analyzing a
> pcap.
> * Problem is the same on both Bro 2.1 and the current git version (as of
> 14.08.2013)
> The problem is exactly the same on two different computers.
> 1. i3, OpenSuse, public IP
> 2. i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine
> has private IP, Bro is on virtual machine monitoring its own traffic
> Both computers use 64bit OS and Bro 2.1 compiled from source.
> Notes: The chance of the network monitoring probe reporting bad values is
> very low as this probe is used by our department of computer science and
> several people checked it works correctly.
> Also, the values reported by Bro are obviously wrong even without no
> comparison with the probe---e.g. an outgoing flow with 0 outgoing packets, 0
> outgoing bytes and nonzero incoming packets/bytes.
> Test 1:
> Bro log files were read after bro shutdown so that there should be nothing
> missing.
> Bro was installed on a computer with IP 147.251.13.65, monitoring its own
> traffic. A web browser was used to generate connections. One of the contacted
> addresses was 77.75.77.170.
> Data from conn.log:
> id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts
> orig_ip_bytes resp_pkts resp_ip_bytes
> 147.251.13.65 44690 77.75.77.170993 tcp 0 151 0
> 0 3 307
> (this was the only line with 147.251.13.65 and 77.75.77.170)
> Data from the network monitoring probe:
> nfdump filter:
> ip 147.251.13.65 and ip 77.75.77.170
> Date first seen Duration Proto Src IP Addr:Port Dst IP
> Addr:Port Flags Tos PacketsBytes pps bpsBpp Flows
> 2013-08-14 09:54:30.355 0.054 TCP 77.75.77.170:993 ->
> 147.251.13.65:44690 .AP... 03 307 5545481102
> 1
> 2013-08-14 09:54:30.347 0.062 TCP 147.251.13.65:44690 ->
> 77.75.77.170:993 .AP... 05 348 8044903 69 1
> Summary: total flows: 2, total bytes: 655, total packets: 8, avg bps: 84516,
> avg pps: 129, avg bpp: 81
> Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00
> Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460
> Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5795248.1
> Test 2:
> Bro log files were read after bro shutdown so that there should be nothing
> missing.
> Bro was installed on a computer with IP 147.251.13.65, monitoring its own
> traffic. A web browser was used to generate connections. One of the contacted
> addresses was 195.113.232.73.
> Debug output of our own Bro script watching connections using
> connection_state_remove event handler.
> The script's code for the event handler is:
> event connection_state_remove(c: connection) {
> NOTICE([$note=profile_creation::Test3, $msg=cat("DBG:
[Bro-Dev] [JIRA] (BIT-1056) incorrect values in num_pkts and num_bytes_ip
[
https://bro-tracker.atlassian.net/browse/BIT-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1056:
---
Resolution: Rejected
Status: Closed (was: Open)
> incorrect values in num_pkts and num_bytes_ip
> -
>
> Key: BIT-1056
> URL: https://bro-tracker.atlassian.net/browse/BIT-1056
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master, 2.1
> Environment: Two separate testing environments:
> 1. Intel i3, OpenSuse x64, public IP, Bro is on the computer monitoring its
> own traffic
> 2. Intel i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual
> machine has private IP, Bro is on virtual machine monitoring its own traffic
>Reporter: Jakub Svoboda
> Labels: issue
>
> Problem: Bro does not see traffic correctly.
> * conn.log contains invalid information, with values differing from a
> network monitoring probe. (The probe runs on a different computer, so there's
> no interference.)
> * Our own scripts logging connections using the connection_state_remove
> event report the same bad values as conn.log.
> * Specifically, all following values are off in the connection_state_remove
> event handler: c$orig$num_pkts c$resp$num_pkts c$conn$orig_pkts
> c$conn$resp_pkts c$orig$num_bytes_ip c$resp$num_bytes_ip c$conn$orig_ip_bytes
> c$conn$resp_ip_bytes. (Yes, we know some of the values are copied one from
> another in conn/main.bro.)
> * Problem is the same on both tested machines.
> * Problem is the same both when capturing an interface and when analyzing a
> pcap.
> * Problem is the same on both Bro 2.1 and the current git version (as of
> 14.08.2013)
> The problem is exactly the same on two different computers.
> 1. i3, OpenSuse, public IP
> 2. i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine
> has private IP, Bro is on virtual machine monitoring its own traffic
> Both computers use 64bit OS and Bro 2.1 compiled from source.
> Notes: The chance of the network monitoring probe reporting bad values is
> very low as this probe is used by our department of computer science and
> several people checked it works correctly.
> Also, the values reported by Bro are obviously wrong even without no
> comparison with the probe---e.g. an outgoing flow with 0 outgoing packets, 0
> outgoing bytes and nonzero incoming packets/bytes.
> Test 1:
> Bro log files were read after bro shutdown so that there should be nothing
> missing.
> Bro was installed on a computer with IP 147.251.13.65, monitoring its own
> traffic. A web browser was used to generate connections. One of the contacted
> addresses was 77.75.77.170.
> Data from conn.log:
> id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts
> orig_ip_bytes resp_pkts resp_ip_bytes
> 147.251.13.65 44690 77.75.77.170993 tcp 0 151 0
> 0 3 307
> (this was the only line with 147.251.13.65 and 77.75.77.170)
> Data from the network monitoring probe:
> nfdump filter:
> ip 147.251.13.65 and ip 77.75.77.170
> Date first seen Duration Proto Src IP Addr:Port Dst IP
> Addr:Port Flags Tos PacketsBytes pps bpsBpp Flows
> 2013-08-14 09:54:30.355 0.054 TCP 77.75.77.170:993 ->
> 147.251.13.65:44690 .AP... 03 307 5545481102
> 1
> 2013-08-14 09:54:30.347 0.062 TCP 147.251.13.65:44690 ->
> 77.75.77.170:993 .AP... 05 348 8044903 69 1
> Summary: total flows: 2, total bytes: 655, total packets: 8, avg bps: 84516,
> avg pps: 129, avg bpp: 81
> Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00
> Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460
> Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5795248.1
> Test 2:
> Bro log files were read after bro shutdown so that there should be nothing
> missing.
> Bro was installed on a computer with IP 147.251.13.65, monitoring its own
> traffic. A web browser was used to generate connections. One of the contacted
> addresses was 195.113.232.73.
> Debug output of our own Bro script watching connections using
> connection_state_remove event handler.
> The script's code for the event handler is:
> event connection_state_remove(c: connection) {
> NOTICE([$note=profile_creation::Test3, $msg=cat("DBG: ", c$id$orig_h, "->",
> c$id$resp_h, " c$orig$num_pkts=", c$orig$num_pkts, " c$resp$num_pkts=",
> c$resp$num_pkts, " c$conn$orig_pkts=", c$conn$orig_pkts, "
> c$conn$resp_pkts=", c$conn$resp_pkts, " c$orig$num_bytes_ip=",
> c$orig$num_bytes_ip, " c$resp$num_bytes_ip=", c$resp$num_bytes_ip, "
> c$conn$orig_ip_bytes=", c$conn$orig_ip_bytes, " c$conn$resp_ip_bytes=",
> c$conn$res
[Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers
[
https://bro-tracker.atlassian.net/browse/BIT-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13703#comment-13703
]
Seth Hall commented on BIT-1064:
It seems to be working correctly for me. Could you send a packet capture that
exhibits the problem?
> DNS Analyzer does not correctly log NXDOMAIN answers
>
>
> Key: BIT-1064
> URL: https://bro-tracker.atlassian.net/browse/BIT-1064
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.1
> Environment: Bro 2.1 running on SecurityOnion 12.04-2
>Reporter: Pietro Delsante
> Labels: dns, nxdomain
> Attachments: nxdomain_pcap.png
>
>
> Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest
> available packages.
> It looks like Bro's DNS analyzer is not assigning the correct rcode and
> rcode_name in the output log when the query is of type A and the server
> answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both
> fields, like this:
> {noformat}
> 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|-
> {noformat}
> that is, exploded:
> {noformat}
> ts: 1377179281.104465
> uid:prGZzGRr1M4
> id: 192.168.X.Y|45406|8.8.8.8|53
> proto: udp
> trans_id: 64928
> query: www.this-domain-does-not-exist.it
> qclass: 1
> qclass_name:C_INTERNET
> qtype: 1
> qtype_name: A
> rcode: -
> rcode_name: -
> AA: F
> TC: F
> RD: T
> RA: F
> Z: 0
> answers:-
> TTLs: -
> {noformat}
> The only case in which I see those values set correctly (rcode: 3,
> rcode_name: NXDOMAIN) is when Bro is logging a PTR query:
> {noformat}
> 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|-
> {noformat}
> The attachment is a screenshot from a wireshark capture of the DNS query
> showing that the server is actually answering with NXDOMAIN.
> The only change I made to the default configuration was to enable the
> extraction of executable files from HTTP and SMTP fluxes, so this should have
> nothing to do with this issue.
> Should you need any more info about my setup, please let me know.
> Thanks,
> Pietro
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1065) packet_filter.log is broken
Seth Hall created BIT-1065: -- Summary: packet_filter.log is broken Key: BIT-1065 URL: https://bro-tracker.atlassian.net/browse/BIT-1065 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.2 Reporter: Seth Hall Apparently I broke the packet_filter.log when I overhauled the packet filter framework. I need to make sure and fix that prior to the 2.2 release (I think it's an ok thing to be broken during the beta). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers
[
https://bro-tracker.atlassian.net/browse/BIT-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1064:
---
Resolution: Fixed
Fix Version/s: 2.2
Status: Closed (was: Open)
> DNS Analyzer does not correctly log NXDOMAIN answers
>
>
> Key: BIT-1064
> URL: https://bro-tracker.atlassian.net/browse/BIT-1064
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.1
> Environment: Bro 2.1 running on SecurityOnion 12.04-2
>Reporter: Pietro Delsante
> Labels: dns, nxdomain
> Fix For: 2.2
>
> Attachments: nxdomain.pcap, nxdomain_pcap.png
>
>
> Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest
> available packages.
> It looks like Bro's DNS analyzer is not assigning the correct rcode and
> rcode_name in the output log when the query is of type A and the server
> answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both
> fields, like this:
> {noformat}
> 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|-
> {noformat}
> that is, exploded:
> {noformat}
> ts: 1377179281.104465
> uid:prGZzGRr1M4
> id: 192.168.X.Y|45406|8.8.8.8|53
> proto: udp
> trans_id: 64928
> query: www.this-domain-does-not-exist.it
> qclass: 1
> qclass_name:C_INTERNET
> qtype: 1
> qtype_name: A
> rcode: -
> rcode_name: -
> AA: F
> TC: F
> RD: T
> RA: F
> Z: 0
> answers:-
> TTLs: -
> {noformat}
> The only case in which I see those values set correctly (rcode: 3,
> rcode_name: NXDOMAIN) is when Bro is logging a PTR query:
> {noformat}
> 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|-
> {noformat}
> The attachment is a screenshot from a wireshark capture of the DNS query
> showing that the server is actually answering with NXDOMAIN.
> The only change I made to the default configuration was to enable the
> extraction of executable files from HTTP and SMTP fluxes, so this should have
> nothing to do with this issue.
> Should you need any more info about my setup, please let me know.
> Thanks,
> Pietro
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers
[
https://bro-tracker.atlassian.net/browse/BIT-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13709#comment-13709
]
Seth Hall commented on BIT-1064:
It's fixed in 2.2 (git master). I think this was related to some bugs I fixed
a while ago in the DNS base scripts. I'm closing the ticket because we aren't
going to back port the fix to prior releases.
> DNS Analyzer does not correctly log NXDOMAIN answers
>
>
> Key: BIT-1064
> URL: https://bro-tracker.atlassian.net/browse/BIT-1064
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.1
> Environment: Bro 2.1 running on SecurityOnion 12.04-2
>Reporter: Pietro Delsante
> Labels: dns, nxdomain
> Fix For: 2.2
>
> Attachments: nxdomain.pcap, nxdomain_pcap.png
>
>
> Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest
> available packages.
> It looks like Bro's DNS analyzer is not assigning the correct rcode and
> rcode_name in the output log when the query is of type A and the server
> answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both
> fields, like this:
> {noformat}
> 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|-
> {noformat}
> that is, exploded:
> {noformat}
> ts: 1377179281.104465
> uid:prGZzGRr1M4
> id: 192.168.X.Y|45406|8.8.8.8|53
> proto: udp
> trans_id: 64928
> query: www.this-domain-does-not-exist.it
> qclass: 1
> qclass_name:C_INTERNET
> qtype: 1
> qtype_name: A
> rcode: -
> rcode_name: -
> AA: F
> TC: F
> RD: T
> RA: F
> Z: 0
> answers:-
> TTLs: -
> {noformat}
> The only case in which I see those values set correctly (rcode: 3,
> rcode_name: NXDOMAIN) is when Bro is logging a PTR query:
> {noformat}
> 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|-
> {noformat}
> The attachment is a screenshot from a wireshark capture of the DNS query
> showing that the server is actually answering with NXDOMAIN.
> The only change I made to the default configuration was to enable the
> extraction of executable files from HTTP and SMTP fluxes, so this should have
> nothing to do with this issue.
> Should you need any more info about my setup, please let me know.
> Thanks,
> Pietro
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1065) packet_filter.log is broken
[ https://bro-tracker.atlassian.net/browse/BIT-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13711#comment-13711 ] Seth Hall commented on BIT-1065: I doubt it would be hard, but I wanted to add a feature too where each worker reports it's packet filter too (which mostly just involves adding a "node" field to the log). > packet_filter.log is broken > --- > > Key: BIT-1065 > URL: https://bro-tracker.atlassian.net/browse/BIT-1065 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.2 >Reporter: Seth Hall > > Apparently I broke the packet_filter.log when I overhauled the packet filter > framework. I need to make sure and fix that prior to the 2.2 release (I > think it's an ok thing to be broken during the beta). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1068) cpu_pin problem
Seth Hall created BIT-1068: -- Summary: cpu_pin problem Key: BIT-1068 URL: https://bro-tracker.atlassian.net/browse/BIT-1068 Project: Bro Issue Tracker Issue Type: Problem Components: BroControl Affects Versions: 2.2 Reporter: Seth Hall Assignee: Daniel Thayer I seem to be having a problem with the cpu_pin feature of broctl. I'm getting the following output... [rootsh@xx worker-1-6]# cat stderr.log sched_setaffinity: Invalid argument failed to set pid 0's affinity. Daniel, any clue what I should be looking into or information I can provide? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1068) cpu_pin problem
[ https://bro-tracker.atlassian.net/browse/BIT-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716#comment-13716 ] Seth Hall commented on BIT-1068: I feel so ashamed. :) I entered '8' as a cpu because it was an 8 core cpu. Unfortunately I was off by one since the last cpu core is addressed at '7'. Sigh. Do you think that's something you could catch and detect? A better error message would be super helpful. :) > cpu_pin problem > --- > > Key: BIT-1068 > URL: https://bro-tracker.atlassian.net/browse/BIT-1068 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl >Affects Versions: 2.2 >Reporter: Seth Hall >Assignee: Daniel Thayer > > I seem to be having a problem with the cpu_pin feature of broctl. I'm > getting the following output... > [rootsh@xx worker-1-6]# cat stderr.log > sched_setaffinity: Invalid argument > failed to set pid 0's affinity. > Daniel, any clue what I should be looking into or information I can provide? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1090) fatal error Val::CONVERTER
[
https://bro-tracker.atlassian.net/browse/BIT-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14504#comment-14504
]
Seth Hall commented on BIT-1090:
That syntax is wrong, use...
redef SSH::ignore_guessers += {
[172.0.0.0/16] = 10.0.0.1/32,
[192.168.1.0/16] = 192.168.2.0/32
};
Also, the yield value for that table is just a subnet. Not a set of subnets.
> fatal error Val::CONVERTER
> --
>
> Key: BIT-1090
> URL: https://bro-tracker.atlassian.net/browse/BIT-1090
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.1
> Environment: Ubuntu 10.04.03 LTS, bro 2.1-179
>Reporter: tyler.schoenke
> Attachments: my-detect-bruteforcing.bro, sigsup-ssh-pass2.bro
>
>
> Hi guys,
> I get the following message when I modified a data structure in
> detect-bruteforcing.bro. I didn't get a chance to test against the current
> version, but did a quick check against the mailing lists and tracker and
> didn't see this issue mentioned.
> $ bro my-detect-bruteforcing.bro sigsup-ssh-pass2.bro
> fatal error in ./sigsup-ssh-pass2.bro, line 2: Val::CONVERTER (types/table)
> (10.0.0.1/32)
> Here is the modification to detect-bruteforcing.bro:
> const ignore_guessers: table[subnet] of set[subnet] = {} &redef;
> I found the need to whitelist from a single host to multiple subnets instead
> of a single subnet. The following minimal script will produce the error.
> cat sigsup-ssh-pass2.bro
> redef SSH::ignore_guessers = {
> [172.0.0.0/16] = set( 10.0.0.1/32 )
> };
> Any help would be appreciated.
> Thanks,
> Tyler
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1065) packet_filter.log is broken
[ https://bro-tracker.atlassian.net/browse/BIT-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1065: --- Resolution: Fixed Fix Version/s: git/master Status: Closed (was: Open) Fixed > packet_filter.log is broken > --- > > Key: BIT-1065 > URL: https://bro-tracker.atlassian.net/browse/BIT-1065 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.2 >Reporter: Seth Hall > Fix For: git/master > > > Apparently I broke the packet_filter.log when I overhauled the packet filter > framework. I need to make sure and fix that prior to the 2.2 release (I > think it's an ok thing to be broken during the beta). -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1082) Trash after SNI information
[ https://bro-tracker.atlassian.net/browse/BIT-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1082: --- Resolution: Cannot Reproduce Status: Closed (was: Open) I'm going to close this ticket since there is really nothing to do about it at this time. > Trash after SNI information > --- > > Key: BIT-1082 > URL: https://bro-tracker.atlassian.net/browse/BIT-1082 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: git/master >Reporter: Bernhard Amann > Fix For: 2.3 > > > When using Bro to extract the server name indication of TLS connections, Bro > sometimes does not seem to be able to determine the correct end of the server > name field -- instead it also returns additional binary data at the end of > the server name. > I do not know if this is reproducible, so far I have not managed to get a > trace of a case where it happens. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1038) Input framework memory usage issue with huge files
[
https://bro-tracker.atlassian.net/browse/BIT-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1038:
---
Resolution: Fixed
Status: Closed (was: Open)
I believe this ticket is fixed.
> Input framework memory usage issue with huge files
> --
>
> Key: BIT-1038
> URL: https://bro-tracker.atlassian.net/browse/BIT-1038
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: scampbell
>Assignee: Bernhard Amann
> Fix For: 2.2
>
>
> When using the input framework to read a data file and convert into an event
> stream, it seems that none of the objects instantiated for event generation
> are freed up. For a file of ~ 5.25 M lines, I am seeing > 4 GB memory used.
> Version is 2.1-798
> A quick demo script looks like:
> event bro_init()
> \{
> # input stream setup
> Input::add_event([$reader=Input::READER_RAW, $mode=Input::STREAM,
> $name="issh", $fields=lineVals, $ev=sshLine|$source=data_file,]);
> }
> event sshLine(description: Input::EventDescription, tpe: Input::Event, LV:
> lineVals)
> \{
> return;
> }
> thanks\!
> scott
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1073) Make the MIME analyzer a FAF analyzer
[ https://bro-tracker.atlassian.net/browse/BIT-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1073: --- Fix Version/s: 2.3 > Make the MIME analyzer a FAF analyzer > - > > Key: BIT-1073 > URL: https://bro-tracker.atlassian.net/browse/BIT-1073 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro >Affects Versions: git/master >Reporter: Matthias Vallentin > Labels: analyzer, file-analysis, mime > Fix For: 2.3 > > > We should convert the MIME analyzer to use FAF, allowing other components to > reuse it. Specifically, I noted this in the process of bringing back the POP3 > analyzer. Ideally, we can just feed the contents of the download emails via > the RETR command into a FAF-based MIME analyzer. Then we wouldn't have to > rebuild functionality that's close to the SMTP analyzer. > In summary, we should factor the MIME analysis into a separate analysis unit. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1051) smtp-url-extraction.bro misses/truncates urls between data chunks
[
https://bro-tracker.atlassian.net/browse/BIT-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1051:
---
Resolution: Won't Fix
Status: Closed (was: Open)
We aren't going to "fix" the current functionality, but we will be replacing it
with a better mechanism.
> smtp-url-extraction.bro misses/truncates urls between data chunks
> -
>
> Key: BIT-1051
> URL: https://bro-tracker.atlassian.net/browse/BIT-1051
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: Brian Little
>Priority: Low
>
> Files::add_analyzer(f, Files::ANALYZER_DATA_EVENT,
> [$stream_event=intel_mime_data]);
> event intel_mime_data(f: fa_file, data: string) {}
> I think the file analysis framework sends the data through to the
> intel_mime_data event in sections (appears that way from adding print
> debugging). The cutting point between the data sections can fall in the
> middle of an url, causing the regex to miss the url, or truncate it.
> What would be the recommended way around for this? (and other usage of file
> analysis framework)
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-896) SCP and SFTP rotators printing
[ https://bro-tracker.atlassian.net/browse/BIT-896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-896: -- Resolution: Fixed Status: Closed (was: Open) > SCP and SFTP rotators printing > -- > > Key: BIT-896 > URL: https://bro-tracker.atlassian.net/browse/BIT-896 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: git/master >Reporter: Seth Hall > Fix For: 2.2 > > > The SCP and SFTP log rotators are causing output on stdout and shouldn't be. > I think the output is coming from the scp and sftp clients, maybe they just > need a quiet flag? -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1025) internal error: over-ran key in CompositeHash::RecoverVals
[
https://bro-tracker.atlassian.net/browse/BIT-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1025:
---
Fix Version/s: (was: 2.2)
2.3
> internal error: over-ran key in CompositeHash::RecoverVals
> --
>
> Key: BIT-1025
> URL: https://bro-tracker.atlassian.net/browse/BIT-1025
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: dmandelb
> Fix For: 2.3
>
> Attachments: bbn-correlator.tgz
>
>
> {noformat}
> $ bro bbn-correlator
> internal error: over-ran key in CompositeHash::RecoverVals
> Aborted
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1051) smtp-url-extraction.bro misses/truncates urls between data chunks
[
https://bro-tracker.atlassian.net/browse/BIT-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603#comment-14603
]
Seth Hall commented on BIT-1051:
That's a known limitation. The plan at the moment is to create a file analyzer
that let's you extract with a regular expression. Internally it would be
provided as a stream so the chunking issue will go away. What in place now is
a hack unfortunately.
> smtp-url-extraction.bro misses/truncates urls between data chunks
> -
>
> Key: BIT-1051
> URL: https://bro-tracker.atlassian.net/browse/BIT-1051
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: Brian Little
>Priority: Low
>
> Files::add_analyzer(f, Files::ANALYZER_DATA_EVENT,
> [$stream_event=intel_mime_data]);
> event intel_mime_data(f: fa_file, data: string) {}
> I think the file analysis framework sends the data through to the
> intel_mime_data event in sections (appears that way from adding print
> debugging). The cutting point between the data sections can fall in the
> middle of an url, causing the regex to miss the url, or truncate it.
> What would be the recommended way around for this? (and other usage of file
> analysis framework)
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1050) Merge request for DHCP analyzer
[ https://bro-tracker.atlassian.net/browse/BIT-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1050: --- Fix Version/s: 2.2 Status: Closed (was: Reopened) Merged. > Merge request for DHCP analyzer > --- > > Key: BIT-1050 > URL: https://bro-tracker.atlassian.net/browse/BIT-1050 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro >Affects Versions: 2.2 >Reporter: Vlad Grigorescu >Assignee: Seth Hall > Labels: analyzer > Fix For: 2.2 > > > topic/vladg/dhcp is ready to go. I've been running it in prod with no > problems. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-988) Bug in HTTP body extraction
[
https://bro-tracker.atlassian.net/browse/BIT-988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-988:
--
Resolution: Fixed
Status: Closed (was: Open)
Functionality removed in favor of FAF
> Bug in HTTP body extraction
> ---
>
> Key: BIT-988
> URL: https://bro-tracker.atlassian.net/browse/BIT-988
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: Matthias Vallentin
>Assignee: Seth Hall
> Labels: file-analysis
> Fix For: 2.2
>
>
> There exists a bug in HTTP body extraction that prevents certain bodies from
> being dumped, even though having set
> {noformat}
> redef extract_file_types = /.*/;
> {noformat}
> This happens presumably because Bro does not figure out the correct MIME type
> and does not set {{c$http$mime_type}}. It results in this check failing:
> {noformat}
> if ( c$http?$mime_type && extract_file_types in c$http$mime_type )
> {
> c$http$extract_file = T;
> }
> {noformat}
> On a related note, I also find missing responses to HTTP POST requests which
> I assume come from the same issues.
> I have a trace that I could attach, but wanted to make sure it's worth the
> effort in face of the upcoming file analysis framework, or if we plan on
> pushing a 2.1 hotfix for this.
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-990) Sumstats documentation
[ https://bro-tracker.atlassian.net/browse/BIT-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-990: -- Resolution: Fixed Status: Closed (was: Open) Done > Sumstats documentation > -- > > Key: BIT-990 > URL: https://bro-tracker.atlassian.net/browse/BIT-990 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro >Affects Versions: git/master >Reporter: Robin Sommer > Fix For: 2.2 > > > We should prepare an overview how-to for using the sumstats framework for 2.2 -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-942) Generic log delaying mechanism for logging framework
[ https://bro-tracker.atlassian.net/browse/BIT-942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-942: -- Fix Version/s: (was: 2.2) 2.3 > Generic log delaying mechanism for logging framework > > > Key: BIT-942 > URL: https://bro-tracker.atlassian.net/browse/BIT-942 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro >Affects Versions: git/master >Reporter: Seth Hall >Assignee: Seth Hall >Priority: Low > Fix For: 2.3 > > > We need to add a mechanism for delaying log writes within the logging > framework for the case where some asynchronous lookup needs to happen in a > non-base script. There are a few requirements: > \\- The mechanism needs to copy the log record so that future modifications > of the record aren't impacted unless deliberately modifying the delayed > record. > \\- Three functions in Log:: namespace to register and unregister delays for > logs and one to get access to the delayed log by it's delay token. > \\- Additional configuration option in logging framework to configure a > default logging delay. It's possible that we should set the default stream > delay in the stream configuration record. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-948) add bif for URI -> binary decoding
[ https://bro-tracker.atlassian.net/browse/BIT-948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-948: -- Fix Version/s: (was: 2.2) 2.3 > add bif for URI -> binary decoding > -- > > Key: BIT-948 > URL: https://bro-tracker.atlassian.net/browse/BIT-948 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro >Affects Versions: git/master >Reporter: scampbell >Priority: Low > Fix For: 2.3 > > > The current URI_decode() bif returns non-ascii data in a x\nn format which is > safe, but not useful in all situations (such as when you need the literal > binary data). > thanks\! > scott -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-977) retransmit in connection history
[ https://bro-tracker.atlassian.net/browse/BIT-977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-977: -- Fix Version/s: (was: 2.2) 2.3 > retransmit in connection history > > > Key: BIT-977 > URL: https://bro-tracker.atlassian.net/browse/BIT-977 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro >Affects Versions: git/master >Reporter: Seth Hall >Priority: Low > Fix For: 2.3 > > > In the connection record's $history field, it would be useful to include > another character to indicate retransmits sent during the connection. I > think that T and t for first orig and resp retransmit respectively would be > appropriate. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-816) Reworked PacketFilter framework
[ https://bro-tracker.atlassian.net/browse/BIT-816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-816: -- Resolution: Fixed Status: Closed (was: Open) Finished > Reworked PacketFilter framework > --- > > Key: BIT-816 > URL: https://bro-tracker.atlassian.net/browse/BIT-816 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro >Affects Versions: git/master >Reporter: Seth Hall >Assignee: Seth Hall > Fix For: 2.2 > > > This is in the topic/seth/scripts-for-2.1 branch, apologies for the poor > naming. One test is failing for me (coverage.test-all-policy) but I'm not > sure what to do to fix it. > This branch reworks the packet filter framework to make it easier to > accomplish common actions. > \\- Removes the PacketFilter::all_packets variable and instead makes "ip or > not ip" the default capture filter. > \\- Adds some convenience methods for restricting the traffic that is > monitored and shunting traffic away with BPF. > \\- Adds the beginning of load balancing support that is necessary to tie in > with some load balancing methods through broctl. > \\- Change the queue manager to flush the event queue before initializing > analyzers through DPD. > \\- New protocols framework that adds some convenience support for defining > the analyzer->DPD linkage. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-903) -b turns off -f
[
https://bro-tracker.atlassian.net/browse/BIT-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-903:
--
Fix Version/s: (was: 2.2)
2.3
> -b turns off -f
> ---
>
> Key: BIT-903
> URL: https://bro-tracker.atlassian.net/browse/BIT-903
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: Vern Paxson
> Fix For: 2.3
>
> Attachments: single-tcp-conn-est.trace
>
>
> Running with \-b (bare bones) disables processing by \-f. Boy did this take
> me a long time to figure out :-(.
> Reproduce using the appended trace. Invoking with *-e 'event
> connection_established(c:connection) \{ print "yep"; }*' will print "yep".
> Invoking with that plus *-f 'not tcp*' won't print anything. But invoking
> with *-f 'not tcp' \-b* _does_ print "yep".
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-521) Bro's secondary path does not handle IPv6) (only v4
[ https://bro-tracker.atlassian.net/browse/BIT-521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14614#comment-14614 ] Seth Hall commented on BIT-521: --- Let's say we kill off secondary path in 2.3? > Bro's secondary path does not handle IPv6) (only v4 > --- > > Key: BIT-521 > URL: https://bro-tracker.atlassian.net/browse/BIT-521 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: git/master >Reporter: gregor > Labels: IPv6 > Fix For: 2.3 > > > Bro's secondary path only handles IPv4 packets > See NetSessions::NextPacketSecondary -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-903) -b turns off -f
[
https://bro-tracker.atlassian.net/browse/BIT-903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608#comment-14608
]
Seth Hall commented on BIT-903:
---
This issue begs a couple of questions.
Should Bro scripts have access to command line arguments? If they did, we
could have a script that monitors for the flag being given and complaining if
the PacketFilter framework isn't loaded.
Should we always load the PacketFilter framework? (I don't think we should).
> -b turns off -f
> ---
>
> Key: BIT-903
> URL: https://bro-tracker.atlassian.net/browse/BIT-903
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: Vern Paxson
> Fix For: 2.3
>
> Attachments: single-tcp-conn-est.trace
>
>
> Running with \-b (bare bones) disables processing by \-f. Boy did this take
> me a long time to figure out :-(.
> Reproduce using the appended trace. Invoking with *-e 'event
> connection_established(c:connection) \{ print "yep"; }*' will print "yep".
> Invoking with that plus *-f 'not tcp*' won't print anything. But invoking
> with *-f 'not tcp' \-b* _does_ print "yep".
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-560) Child analyzer Init() problem
[
https://bro-tracker.atlassian.net/browse/BIT-560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611#comment-14611
]
Seth Hall commented on BIT-560:
---
Need to bump this again. No one looked into it.
> Child analyzer Init() problem
> -
>
> Key: BIT-560
> URL: https://bro-tracker.atlassian.net/browse/BIT-560
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: gregor
>Assignee: Robin Sommer
> Fix For: 2.3
>
>
> {noformat}
> #!rst
> I think there is an inherent problem in the way analyzers and child analyzers
> are initialized. If analyzers are added by BuildInitialAnalyzerTree() they
> are not initialized at first but in a batch by calling::
>
> root->Init();
> root->InitChildren();
> If an analyzer wants to add a child in its Init(), the parent doesn't know
> whether it needs to init this child or not. If the parent was added by
> ``BuildInitialAnalyzerTree()``, it *must not* ``Init()`` the child, because
> ``BuildInitialAnalyzerTree()`` will do it. OTOH, if the parent was added
> dynamically, e.g., by DPD signatures, then it *must* ``Init()`` the child.
> What was the reason for ``BuildInitialAnalyzerTree()`` to defer
> initialization of the tree until the end of the function? Initializing when
> they are added would solve the problem but I guess there was a good reason to
> do it this way.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-924) String BIFs Return 1-indexed string_arrays
[ https://bro-tracker.atlassian.net/browse/BIT-924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-924: -- Fix Version/s: (was: 2.2) 2.3 > String BIFs Return 1-indexed string_arrays > -- > > Key: BIT-924 > URL: https://bro-tracker.atlassian.net/browse/BIT-924 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: git/master >Reporter: grigorescu > Fix For: 2.3 > > > The following BIFs return 1-indexed string_arrays: > * sort_string_array > * split > * split1 > * split_all > * split_n -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-671) Test Bro core and script layer simultaneously
[
https://bro-tracker.atlassian.net/browse/BIT-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-671:
--
Fix Version/s: (was: 2.2)
2.3
> Test Bro core and script layer simultaneously
> -
>
> Key: BIT-671
> URL: https://bro-tracker.atlassian.net/browse/BIT-671
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: BTest
>Affects Versions: git/master
>Reporter: Matthias Vallentin
> Fix For: 2.3
>
>
> If we record all events during testing, say by adding {{events.bst}} to each
> Bro run, we can simultaneously test the core. Moreover, we instantly know
> whether a bug manifests at script land or at the core.
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-867) GRE support
[ https://bro-tracker.atlassian.net/browse/BIT-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-867: -- Fix Version/s: (was: 2.2) 2.3 > GRE support > --- > > Key: BIT-867 > URL: https://bro-tracker.atlassian.net/browse/BIT-867 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro >Affects Versions: git/master >Reporter: Robin Sommer > Fix For: 2.3 > > > Should be rather easy to add support for GRE tunnels now. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-579) "Raw" logging writer
[
https://bro-tracker.atlassian.net/browse/BIT-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-579:
--
Resolution: Fixed
Status: Closed (was: Open)
I can now recognize that this wasn't a great idea. :)
> "Raw" logging writer
>
>
> Key: BIT-579
> URL: https://bro-tracker.atlassian.net/browse/BIT-579
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: Seth Hall
>Priority: High
> Fix For: 2.2
>
>
> This was formerly a ticket about creating syslog logging writer, but I think
> we found a better and more general approach in a "raw" writer. The raw
> writer would abandon the normal tab separated output from the Ascii writer
> and instead would be based on a templating format passed through the config
> filter field. There should also be options for sending the formatted data to
> files, sockets, and syslog.
> This writer would open several doors for us:
> * Direct integration from script-land with ELSA.
> * Functional replacement for PRADS in script-land with integration into
> Sguil.
> * Direct script-land integration with the metrics framework and Graphite.
> Here is a made up example of creating a metrics filter for sending data to
> Graphite:
> {noformat}
> Log::add_filter(Metrics::LOG, [$name="graphite",
>$writer=Log::WRITER_RAW,
>$path="tcp://1.2.3.4:2003/",
>$config = table(["fmt"] = "{{metric}}
> {{value}} {{ts}}")]);
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-896) SCP and SFTP rotators printing
[ https://bro-tracker.atlassian.net/browse/BIT-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609#comment-14609 ] Seth Hall commented on BIT-896: --- I believe this was fixed. > SCP and SFTP rotators printing > -- > > Key: BIT-896 > URL: https://bro-tracker.atlassian.net/browse/BIT-896 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: git/master >Reporter: Seth Hall > Fix For: 2.2 > > > The SCP and SFTP log rotators are causing output on stdout and shouldn't be. > I think the output is coming from the scp and sftp clients, maybe they just > need a quiet flag? -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-310) Higher time resolution?
[ https://bro-tracker.atlassian.net/browse/BIT-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-310: -- Resolution: Fixed Status: Closed (was: Open) I'm just going to close this ticket actually. It will come up again if it's important enough. > Higher time resolution? > --- > > Key: BIT-310 > URL: https://bro-tracker.atlassian.net/browse/BIT-310 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro >Affects Versions: git/master >Reporter: Robin Sommer > Fix For: 2.3 > > > Something to discuss: do we need timestamps with higher resolution > (if the NIC provides them, like the DAGs for example). -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-560) Child analyzer Init() problem
[
https://bro-tracker.atlassian.net/browse/BIT-560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-560:
--
Fix Version/s: (was: 2.2)
2.3
> Child analyzer Init() problem
> -
>
> Key: BIT-560
> URL: https://bro-tracker.atlassian.net/browse/BIT-560
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: gregor
>Assignee: Robin Sommer
> Fix For: 2.3
>
>
> {noformat}
> #!rst
> I think there is an inherent problem in the way analyzers and child analyzers
> are initialized. If analyzers are added by BuildInitialAnalyzerTree() they
> are not initialized at first but in a batch by calling::
>
> root->Init();
> root->InitChildren();
> If an analyzer wants to add a child in its Init(), the parent doesn't know
> whether it needs to init this child or not. If the parent was added by
> ``BuildInitialAnalyzerTree()``, it *must not* ``Init()`` the child, because
> ``BuildInitialAnalyzerTree()`` will do it. OTOH, if the parent was added
> dynamically, e.g., by DPD signatures, then it *must* ``Init()`` the child.
> What was the reason for ``BuildInitialAnalyzerTree()`` to defer
> initialization of the tree until the end of the function? Initializing when
> they are added would solve the problem but I guess there was a good reason to
> do it this way.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-266) broctl cron hangs with dead hosts
[ https://bro-tracker.atlassian.net/browse/BIT-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-266: -- Resolution: Fixed Status: Closed (was: Open) Without more debugging information or activity on this ticket, I'm just going to close it. > broctl cron hangs with dead hosts > - > > Key: BIT-266 > URL: https://bro-tracker.atlassian.net/browse/BIT-266 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl >Affects Versions: git/master >Reporter: solomon >Assignee: Daniel Thayer > Fix For: 2.2 > > > If a bro node has become completely inaccessible (dead or otherwise offline), > the broctl 'cron' task will hang for far too long while attempting to restart > bro on that host. When 'broctl cron' is run from the system crontab, this > negatively impacts any other use of broctl and prevents almost all > interactive use of broctl, which repeatedly times out with 'cannot get lock' > error message. > One is forced to either kill all running python processes on the master > (thereby killing the blocking 'broctl cron' task), or to run broctl > non-interactively (such as running "broctl top" from the command line) > over-and-over again until the process successfully gets the lock. > broctl should timeout more quickly with hosts that are completely offline. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-341) Verify MACs seen
[ https://bro-tracker.atlassian.net/browse/BIT-341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-341: -- Resolution: Fixed Status: Closed (was: Open) I think this is too deployment specific and ultimately something that Bro itself should be doing. I'm going to close this ticket since it doesn't have a terribly concrete task item. > Verify MACs seen > > > Key: BIT-341 > URL: https://bro-tracker.atlassian.net/browse/BIT-341 > Project: Bro Issue Tracker > Issue Type: Task > Components: BroControl >Affects Versions: git/master >Reporter: Robin Sommer >Assignee: Robin Sommer > > BroControl's "cron" should check the number of MAC addresses seen by > the worker nodes. If it changes, that may be an indicator of a > problem (like the switch flooding packets to all nodes). -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-310) Higher time resolution?
[ https://bro-tracker.atlassian.net/browse/BIT-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-310: -- Fix Version/s: (was: 2.2) 2.3 > Higher time resolution? > --- > > Key: BIT-310 > URL: https://bro-tracker.atlassian.net/browse/BIT-310 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro >Affects Versions: git/master >Reporter: Robin Sommer > Fix For: 2.3 > > > Something to discuss: do we need timestamps with higher resolution > (if the NIC provides them, like the DAGs for example). -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-697) Equivalent of capture-events.bro in 2.x
[
https://bro-tracker.atlassian.net/browse/BIT-697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-697:
--
Fix Version/s: 2.3
> Equivalent of capture-events.bro in 2.x
> ---
>
> Key: BIT-697
> URL: https://bro-tracker.atlassian.net/browse/BIT-697
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: Matthias Vallentin
> Fix For: 2.3
>
>
> How should we handle the functionality provided by the 1.5 script
> {{capture-events.bro}} in 2.x? It currently does not exist. Since it's
> implementation only consists of this one-liner
> {noformat}
> event bro_init()
> {
> capture_events("events.bst");
> }
> {noformat}
> I think we make that a redefinable script variable rather than shipping a
> separate script.
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-758) New function split_esc
[
https://bro-tracker.atlassian.net/browse/BIT-758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622#comment-14622
]
Seth Hall commented on BIT-758:
---
Oh no, I'm ridiculous. You can't because that would grab that character.
> New function split_esc
> --
>
> Key: BIT-758
> URL: https://bro-tracker.atlassian.net/browse/BIT-758
> Project: Bro Issue Tracker
> Issue Type: New Feature
> Components: Bro
>Affects Versions: git/master
>Reporter: Matthias Vallentin
>
> In the topic branch {{topic/matthias/split-escaped}}, I added new
> functionality to split strings that may contain an escaped split expression.
> E.g., now it would be possible to split strings of the form
> {noformat}
> a#b\#c#d
> {noformat}
> into {{[b\#c, d|a,]}}. This would otherwise not be possible, because one
> cannot perform a lookahead with Bro's regular expressions.
> I implemented this function as an utility function at the scripting layer,
> rather than adding a new BiF. Ideally, we'd enhance {{split_n}} with this
> ability and then propagate the changes through the respective {{split*}}
> functions. But before this happens, it makes probably more sense to address
> BIT-757 first.
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-553) Script variable to set pcap's buffer size
[ https://bro-tracker.atlassian.net/browse/BIT-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-553: -- Resolution: Feedback Missing Status: Closed (was: Open) I don't think we're going to bother with this especially since packet capture might get overhauled a bit soon. > Script variable to set pcap's buffer size > - > > Key: BIT-553 > URL: https://bro-tracker.atlassian.net/browse/BIT-553 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro >Affects Versions: git/master >Reporter: gregor > Fix For: 2.2 > > > Newer pcap versions have a pcap_set_buffer_size(). This call can be used to > adjust the kernel level buffer size for captures. (E.g., Linux PF_RING's > buffer size, which will be quite important if we start using 64KB snaplen) > It's supported for all platforms we care about (Linux, BPF-based OSes), but > we probably want to make sure that it obsoletes the manual tuning through: > http://www.net.t-labs.tu-berlin.de/research/bpcs/ > (It might well be that one still has to manually increase the max buffer > sizes) -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-505) Invalid Unref crash
[
https://bro-tracker.atlassian.net/browse/BIT-505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-505:
--
Resolution: Fixed
Status: Closed (was: Open)
I'm not seeing crashes.
> Invalid Unref crash
> ---
>
> Key: BIT-505
> URL: https://bro-tracker.atlassian.net/browse/BIT-505
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: gclark
> Attachments: bro.trace.bz2
>
>
> Using latest from bro-master:
> Last few lines of Bro execution trace:
> {noformat}
> 1311220077.263882
> /home/gilbert/Code/bro/policy/frameworks/notice/weird.bro:384
> Builtin Function called: network_time()
> 1311220077.263882
> /home/gilbert/Code/bro/policy/frameworks/notice/weird.bro:384
> Function return: 1311220077.26388
> 1311220077.263882 /home/gilbert/Code/bro/policy/utils/conn-ids.bro:23
> function called: id_string(id = '[orig_h=204.152.191.37, orig_p=80/tcp,
> resp_h=212.110.251.3, resp_p=33595/tcp]')
> 1311220077.263882 /home/gilbert/Code/bro/policy/utils/conn-ids.bro:23
> Builtin Function called: fmt(va_args = '%s:%d > %s:%d', vararg0 =
> '204.152.191.37', vararg1 = '80/tcp', vararg2 = '212.110.251.3', vararg3 =
> '33595/tcp')
> 1311220077.263882 /home/gilbert/Code/bro/policy/utils/conn-ids.bro:23
> Function return: 204.152.191.37:80 > 212.110.251.3:33595
> 1311220077.263882 /home/gilbert/Code/bro/policy/utils/conn-ids.bro:23
> Function return: 204.152.191.37:80 > 212.110.251.3:33595
> 1311220077.263882
> /home/gilbert/Code/bro/policy/frameworks/notice/weird.bro:359
> function called: Weird::report_weird_conn(t = '1311220077.26388', name =
> 'above_hole_data_without_any_acks', id = '204.152.191.37:80 >
> 212.110.251.3:33595', addl = '', c = '[id=[orig_h=204.152.191.37,
> orig_p=80/tcp, resp_h=212.110.251.3, resp_p=33595/tcp], orig=[size=7240,
> state=3, num_pkts=, num_bytes_ip=],
> resp=[size=0, state=0, num_pkts=,
> num_bytes_ip=], start_time=1311220077.2637,
> duration=0.000185012817382812, service={
> }, addl=, hot=0, history=D, uid=vwfIafnipTj, conn=,
> extract_orig=F, extract_resp=F, dns=,
> dns_state=, ftp=, http=,
> http_state=, irc=, mime=,
> mime_state=, smtp=, smtp_state=,
> ssh=, ssl=, syslog=]')
> 1311220077.263882
> /home/gilbert/Code/bro/policy/frameworks/notice/weird.bro:310
> function called: Weird::report_weird(t = '1311220077.26388', name =
> 'above_hole_data_without_any_acks', id = '204.152.191.37:80 >
> 212.110.251.3:33595', have_conn = 'T', addl = '', action = 'WEIRD_FILE',
> no_log = 'F')
> 1311220077.263882
> /home/gilbert/Code/bro/policy/frameworks/logging/base.bro:188
> function called: Log::write(id = 'WEIRD', columns =
> '[ts=1311220077.26388, uid=vwfIafnipTj, id=[orig_h=204.152.191.37,
> orig_p=80/tcp, resp_h=212.110.251.3, resp_p=33595/tcp],
> msg=above_hole_data_without_any_acks, addl=, notice=F]')
> 1311220077.263882
> /home/gilbert/Code/bro/policy/frameworks/logging/base.bro:188
> Builtin Function called: Log::__write(id = 'WEIRD',
> columns = '[ts=1311220077.26388, uid=vwfIafnipTj, id=[orig_h=204.152.191.37,
> orig_p=80/tcp, resp_h=212.110.251.3, resp_p=33595/tcp],
> msg=above_hole_data_without_any_acks, addl=, notice=F]')
> 1311220077.263882
> /home/gilbert/Code/bro/policy/frameworks/logging/base.bro:188
> Function return: T
> 1311220077.263882
> /home/gilbert/Code/bro/policy/frameworks/logging/base.bro:188
> Function return: T
> 1311220077.461318 /home/gilbert/Code/bro/build/src/event.bif.bro:104 event
> called: conn_weird(name = 'spontaneous_RST', c = '[id=[orig_h=212.110.251.3,
> orig_p=113/tcp, resp_h=161.53.178.240, resp_p=50349/tcp], orig=[size=0,
> state=6, num_pkts=, num_bytes_ip=],
> resp=[size=0, state=0, num_pkts=,
> num_bytes_ip=], start_time=1311220077.46132, duration=0.0,
> service={
> }, addl=, hot=0, history=R, uid=0Wn1J3f4jD4, conn=,
> extract_orig=F, extract_resp=F, dns=,
> dns_state=, ftp=, http=,
> http_state=, irc=, mime=,
> mime_state=, smtp=, smtp_state=,
> ssh=, ssl=, syslog=]', addl = '')
> 1311220077.461318
> /home/gilbert/Code/bro/policy/frameworks/notice/weird.bro:384
> Builtin Function called: network_time()
> 1311220077.461318
> /home/gilbert/Code/bro/policy/frameworks/notice/weird.bro:384
> Function return: 1311220077.46132
> 1311220077.461318 /home/gilbert/Code/bro/policy/utils/conn-ids.bro:23
> function called: id_string(id = '[orig_h=212.110.251.3, orig_p=113/tcp,
> resp_h=161.53.178.240, resp_p=50349/tcp]')
> 1311220077.461318 /home/gilbert/Code/bro/policy/utils/conn-ids.bro
[Bro-Dev] [JIRA] (BIT-348) Reassembler integer overflow issues. Data not delivered after 2GB
[
https://bro-tracker.atlassian.net/browse/BIT-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14624#comment-14624
]
Seth Hall commented on BIT-348:
---
Sigh, now I guess it's high priority for 2.3.
> Reassembler integer overflow issues. Data not delivered after 2GB
> -
>
> Key: BIT-348
> URL: https://bro-tracker.atlassian.net/browse/BIT-348
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: gregor
>Priority: High
> Labels: inttypes
> Fix For: 2.3
>
>
> {noformat}
> #!rst
> The TCP Reassembler does not deliver any data to analyzers after the first
> 2GB due to signed integer overflow (Actually it will deliver again between
> 4--6GB, etc.) This happens silently, i.e., without content_gap events or
> Undelivered calls.
> This report superseded BIT-315, BIT-137
> The TCP Reassembler (and Reassem) base class use ``int`` to keep track of
> sequence numbers and ``seq_delta`` to check for differences. If a connection
> exceeds 2GB, the relative sequence numbers (int) used by the Reassembler
> become negative. While many parts of the Reassembler still work (because
> seq_delta still reports the correct difference) some parts do not. In
> particular ``seq_to_skip`` is broken (and fails silently). There might well
> be other parts of the Reassembler that fail
> silently as well, that I haven't found yet.
> See Comments in TCP_Reassembler.cc for more details.
> The Reassembler should use int64. However this will require deep changes to
> the Reassembler and the TCP Analyzer and TCP_Endpoint classes (since we also
> store sequence numbers there). Also, the analyzer framework will need tweaks
> as well (e.g., Undelivered uses ``int`` for sequence numbers, also has to go
> to 64 bit)
> As a hotfix that seems to work I disabled the ``seq_to_skip`` features. It
> wasn't used by any analyzer or policy script (Note, that seq_to_skip is
> different from skip_deliveries). Hotfix is in
> topic/gregor/reassembler-hotfix
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-688) [Fwd] Re: content_gap vs. ack_above_hole
[ https://bro-tracker.atlassian.net/browse/BIT-688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-688: -- Fix Version/s: 2.3 > [Fwd] Re: [Bro-Dev] content_gap vs. ack_above_hole > -- > > Key: BIT-688 > URL: https://bro-tracker.atlassian.net/browse/BIT-688 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro >Affects Versions: git/master >Reporter: Robin Sommer > Labels: cleanup > Fix For: 2.3 > > > From: Vern Paxson > Subject: Re: [Bro-Dev] content_gap vs. ack_above_hole > > Can somebody remind me what exactly the difference between these two > > is (and/or why we have both?). > Yeah, my fault :-P. As best as I can tell (from revisiting the code), > content-gap is a superset of ack-above-hole. Content gaps can also occur > in situations where we're not expecting to see ACKs (for example, due to > split routing, or because we're not processing traffic from the receiver). > I think merging the two into a single content_gap event would make sense. > Vern -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-688) [Fwd] Re: content_gap vs. ack_above_hole
[ https://bro-tracker.atlassian.net/browse/BIT-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14625#comment-14625 ] Seth Hall commented on BIT-688: --- This should be really easy to take care of for 2.3. > [Fwd] Re: [Bro-Dev] content_gap vs. ack_above_hole > -- > > Key: BIT-688 > URL: https://bro-tracker.atlassian.net/browse/BIT-688 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro >Affects Versions: git/master >Reporter: Robin Sommer > Labels: cleanup > Fix For: 2.3 > > > From: Vern Paxson > Subject: Re: [Bro-Dev] content_gap vs. ack_above_hole > > Can somebody remind me what exactly the difference between these two > > is (and/or why we have both?). > Yeah, my fault :-P. As best as I can tell (from revisiting the code), > content-gap is a superset of ack-above-hole. Content gaps can also occur > in situations where we're not expecting to see ACKs (for example, due to > split routing, or because we're not processing traffic from the receiver). > I think merging the two into a single content_gap event would make sense. > Vern -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-521) Bro's secondary path does not handle IPv6) (only v4
[ https://bro-tracker.atlassian.net/browse/BIT-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-521: -- Fix Version/s: 2.3 > Bro's secondary path does not handle IPv6) (only v4 > --- > > Key: BIT-521 > URL: https://bro-tracker.atlassian.net/browse/BIT-521 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: git/master >Reporter: gregor > Labels: IPv6 > Fix For: 2.3 > > > Bro's secondary path only handles IPv4 packets > See NetSessions::NextPacketSecondary -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-758) New function split_esc
[
https://bro-tracker.atlassian.net/browse/BIT-758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621#comment-14621
]
Seth Hall commented on BIT-758:
---
Couldn't you get the same functionality with one of the split functions using
this pattern?
/[^\/]#/
> New function split_esc
> --
>
> Key: BIT-758
> URL: https://bro-tracker.atlassian.net/browse/BIT-758
> Project: Bro Issue Tracker
> Issue Type: New Feature
> Components: Bro
>Affects Versions: git/master
>Reporter: Matthias Vallentin
>
> In the topic branch {{topic/matthias/split-escaped}}, I added new
> functionality to split strings that may contain an escaped split expression.
> E.g., now it would be possible to split strings of the form
> {noformat}
> a#b\#c#d
> {noformat}
> into {{[b\#c, d|a,]}}. This would otherwise not be possible, because one
> cannot perform a lookahead with Bro's regular expressions.
> I implemented this function as an utility function at the scripting layer,
> rather than adding a new BiF. Ideally, we'd enhance {{split_n}} with this
> ability and then propagate the changes through the respective {{split*}}
> functions. But before this happens, it makes probably more sense to address
> BIT-757 first.
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-699) Reorganizing layout of protocol analyzers
[
https://bro-tracker.atlassian.net/browse/BIT-699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-699:
--
Resolution: Fixed
Status: Closed (was: Open)
Done.
> Reorganizing layout of protocol analyzers
> -
>
> Key: BIT-699
> URL: https://bro-tracker.atlassian.net/browse/BIT-699
> Project: Bro Issue Tracker
> Issue Type: Task
> Components: Bro
>Affects Versions: git/master
>Reporter: Robin Sommer
> Fix For: 2.2
>
>
> We should restructure how protocol analyzers are organized inside the
> C++ core. Ideally, I'd like to have a single directory per analyzer
> with everything in there that relates to that analyzer, with no need
> to touch other places (currently, one needs to touch quite many for
> adding an analyzer). This includes the analyzers' `{{*.cc}}{{, }}{{*.h}}{{,
> }}{{*.bif}}{{, and }}{{*.pac}}{{ files. For the }}{{bif}}{{s, we should also
> move
> the corresponding parts from }}{{event.bif}}{{ and }}{{init-bare.bif}}{{ into
> that directory.
> This restructuring is also a good opportuntity to reorganize the
> Broxygen generation for analyzers: I think it would be nice to have
> one page per analyzer that has (1) a general description, (2) all
> tuning parameters, (3) all core events, and (4) links to all relevant
> }}{{base}}{{ and }}{{policy}}` scripts.
> I'm setting this to 2.1 for now. It's not high-priority for that
> release and we can bump it further out if necessary. But I think it's
> good to keep in mind, and perhaps work on parts of this as time
> permits.
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-348) Reassembler integer overflow issues. Data not delivered after 2GB
[
https://bro-tracker.atlassian.net/browse/BIT-348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-348:
--
Fix Version/s: (was: 2.2)
2.3
> Reassembler integer overflow issues. Data not delivered after 2GB
> -
>
> Key: BIT-348
> URL: https://bro-tracker.atlassian.net/browse/BIT-348
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: gregor
>Priority: High
> Labels: inttypes
> Fix For: 2.3
>
>
> {noformat}
> #!rst
> The TCP Reassembler does not deliver any data to analyzers after the first
> 2GB due to signed integer overflow (Actually it will deliver again between
> 4--6GB, etc.) This happens silently, i.e., without content_gap events or
> Undelivered calls.
> This report superseded BIT-315, BIT-137
> The TCP Reassembler (and Reassem) base class use ``int`` to keep track of
> sequence numbers and ``seq_delta`` to check for differences. If a connection
> exceeds 2GB, the relative sequence numbers (int) used by the Reassembler
> become negative. While many parts of the Reassembler still work (because
> seq_delta still reports the correct difference) some parts do not. In
> particular ``seq_to_skip`` is broken (and fails silently). There might well
> be other parts of the Reassembler that fail
> silently as well, that I haven't found yet.
> See Comments in TCP_Reassembler.cc for more details.
> The Reassembler should use int64. However this will require deep changes to
> the Reassembler and the TCP Analyzer and TCP_Endpoint classes (since we also
> store sequence numbers there). Also, the analyzer framework will need tweaks
> as well (e.g., Undelivered uses ``int`` for sequence numbers, also has to go
> to 64 bit)
> As a hotfix that seems to work I disabled the ``seq_to_skip`` features. It
> wasn't used by any analyzer or policy script (Note, that seq_to_skip is
> different from skip_deliveries). Hotfix is in
> topic/gregor/reassembler-hotfix
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-781) Case sensitive (non-normalized) HTTP header names
[ https://bro-tracker.atlassian.net/browse/BIT-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-781: -- Fix Version/s: (was: 2.2) 2.3 > Case sensitive (non-normalized) HTTP header names > - > > Key: BIT-781 > URL: https://bro-tracker.atlassian.net/browse/BIT-781 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro >Affects Versions: git/master >Reporter: sconzo > Labels: analyzer > Fix For: 2.3 > > > There are a lot of usecases to have access to the normalized HTTP header > names, I'm asking for an extension that will allow access to the > non-normalized version. This can be used to find non-standard/suspicious > HTTP connections. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-768) Inline monitoring of modified scripts.
[ https://bro-tracker.atlassian.net/browse/BIT-768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-768: -- Fix Version/s: (was: 2.2) 2.3 > Inline monitoring of modified scripts. > -- > > Key: BIT-768 > URL: https://bro-tracker.atlassian.net/browse/BIT-768 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl >Affects Versions: git/master >Reporter: Seth Hall >Assignee: Daniel Thayer > Fix For: 2.3 > > > We need to train users to do check, install, restart through broctl better. > I'd like to reduce the barrier to entry a bit more and if broctl can coach > new users through the process better and remind existing users of the process > it would be great. > Here are my suggestions for what I think needs to be done: > \\- Track hashes for all copied scripts (maybe in broctl.dat?) and watch for > changes to notify the user. I think it would be ok to only notify the user > when they are in broctl but I can see that people may want that to also check > and occasionally email from broctl cron (let's save emailing for later > though, inline notification in broctl may be enough). > \\- Track hashes for scripts that have been "checked" because then we can > coach people about what step in the process they are at. If someone has > already run "check" on the current scripts we can recommend that they need to > \\- Create variables to turn off various suggestions. I think the various > suggestions would be "need to check scripts", "need to install scripts", and > "ready to restart" or something along those lines. I'm not even sure I like > this idea though. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-781) Case sensitive (non-normalized) HTTP header names
[ https://bro-tracker.atlassian.net/browse/BIT-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630#comment-14630 ] Seth Hall commented on BIT-781: --- Well, looks like this didn't happen yet again. We may end up just waiting on this until the HTTP binpac++ analyzer shows up in Bro. > Case sensitive (non-normalized) HTTP header names > - > > Key: BIT-781 > URL: https://bro-tracker.atlassian.net/browse/BIT-781 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro >Affects Versions: git/master >Reporter: sconzo > Labels: analyzer > Fix For: 2.3 > > > There are a lot of usecases to have access to the normalized HTTP header > names, I'm asking for an extension that will allow access to the > non-normalized version. This can be used to find non-standard/suspicious > HTTP connections. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-854) problem with VLAN/MPLS packet dumping
[
https://bro-tracker.atlassian.net/browse/BIT-854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633#comment-14633
]
Seth Hall commented on BIT-854:
---
I think the real question with this is what level of support we provide to
"dumping" packets in Bro? Right now it's not something we consider much or put
much effort into validating that it works correctly. I'm going to remove the
milestone from this because it's possible that we address the issue later
either by having timemachine actually dump the packets or from further work on
protocol analysis through the upcoming binpac++ integration.
> problem with VLAN/MPLS packet dumping
> -
>
> Key: BIT-854
> URL: https://bro-tracker.atlassian.net/browse/BIT-854
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: Jon Siwek
>
> report from Carsten Langer:
> {noformat}
> By the way: you have in my opinion a problem with packet dumping. If the
> trace contains VLAN or MPLS, you strip off VLAN/MPLS and if then you
> dump the packet, then the dumped trace is missing the Ethernet header
> for these packets, while the Ethernet header is still there for packets
> which did not have VLAN/MPLS. My previous GTP-detunneling did the same
> mistake, now I have introduced a fake Ethernet header so that if the
> packet is dumped, is still has its Ethernet header.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-699) Reorganizing layout of protocol analyzers
[
https://bro-tracker.atlassian.net/browse/BIT-699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14627#comment-14627
]
Seth Hall commented on BIT-699:
---
I think this ticket was completed by the work Robin did on reorganizing the
analyzers for 2.2.
> Reorganizing layout of protocol analyzers
> -
>
> Key: BIT-699
> URL: https://bro-tracker.atlassian.net/browse/BIT-699
> Project: Bro Issue Tracker
> Issue Type: Task
> Components: Bro
>Affects Versions: git/master
>Reporter: Robin Sommer
> Fix For: 2.2
>
>
> We should restructure how protocol analyzers are organized inside the
> C++ core. Ideally, I'd like to have a single directory per analyzer
> with everything in there that relates to that analyzer, with no need
> to touch other places (currently, one needs to touch quite many for
> adding an analyzer). This includes the analyzers' `{{*.cc}}{{, }}{{*.h}}{{,
> }}{{*.bif}}{{, and }}{{*.pac}}{{ files. For the }}{{bif}}{{s, we should also
> move
> the corresponding parts from }}{{event.bif}}{{ and }}{{init-bare.bif}}{{ into
> that directory.
> This restructuring is also a good opportuntity to reorganize the
> Broxygen generation for analyzers: I think it would be nice to have
> one page per analyzer that has (1) a general description, (2) all
> tuning parameters, (3) all core events, and (4) links to all relevant
> }}{{base}}{{ and }}{{policy}}` scripts.
> I'm setting this to 2.1 for now. It's not high-priority for that
> release and we can bump it further out if necessary. But I think it's
> good to keep in mind, and perhaps work on parts of this as time
> permits.
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-854) problem with VLAN/MPLS packet dumping
[
https://bro-tracker.atlassian.net/browse/BIT-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-854:
--
Fix Version/s: (was: 2.2)
> problem with VLAN/MPLS packet dumping
> -
>
> Key: BIT-854
> URL: https://bro-tracker.atlassian.net/browse/BIT-854
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: Jon Siwek
>
> report from Carsten Langer:
> {noformat}
> By the way: you have in my opinion a problem with packet dumping. If the
> trace contains VLAN or MPLS, you strip off VLAN/MPLS and if then you
> dump the packet, then the dumped trace is missing the Ethernet header
> for these packets, while the Ethernet header is still there for packets
> which did not have VLAN/MPLS. My previous GTP-detunneling did the same
> mistake, now I have introduced a fake Ethernet header so that if the
> packet is dumped, is still has its Ethernet header.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-856) more documentation for utilities would be cool
[ https://bro-tracker.atlassian.net/browse/BIT-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-856: -- Fix Version/s: 2.3 > more documentation for utilities would be cool > -- > > Key: BIT-856 > URL: https://bro-tracker.atlassian.net/browse/BIT-856 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: bro-aux >Affects Versions: git/master >Reporter: Vern Paxson > Fix For: 2.3 > > > Utilities like bro-cut only supply \--help documentation, as far as I can > tell. Man pages would be handy. (In particular, I was looking for some sort > of statement of exactly to what degree bro-cut can munch on the concatenation > of multiple log files that have different column layouts.) -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-903) -b turns off -f
[
https://bro-tracker.atlassian.net/browse/BIT-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-903:
--
Attachment: signature.asc
If we could check command line arguments in Bro scripts we could implement that
functionality. ;)
> -b turns off -f
> ---
>
> Key: BIT-903
> URL: https://bro-tracker.atlassian.net/browse/BIT-903
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: git/master
>Reporter: Vern Paxson
> Fix For: 2.3
>
> Attachments: signature.asc, single-tcp-conn-est.trace
>
>
> Running with \-b (bare bones) disables processing by \-f. Boy did this take
> me a long time to figure out :-(.
> Reproduce using the appended trace. Invoking with *-e 'event
> connection_established(c:connection) \{ print "yep"; }*' will print "yep".
> Invoking with that plus *-f 'not tcp*' won't print anything. But invoking
> with *-f 'not tcp' \-b* _does_ print "yep".
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-959) Issue with HTTP POST file extraction
[ https://bro-tracker.atlassian.net/browse/BIT-959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-959: -- Resolution: Fixed Status: Closed (was: Open) This feature has been replace by the file analysis framework. > Issue with HTTP POST file extraction > > > Key: BIT-959 > URL: https://bro-tracker.atlassian.net/browse/BIT-959 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.1 >Reporter: gregoire.moreau >Priority: Low > Fix For: 2.2 > > > I've had a problem with the extraction of HTTP POST file content with bro2.1 > stable, there's no problem with incoming content. I use a modified > http/file-extract.bro script. > My tests were mainly done with PDF content. > The problem is whenever a 0x0d is found in the content, it is replaced with > 0x0d0a. > I've found a little workaround, but I'm not sure about all the borders > effects it could have. Also, it may not be the good way to correct the > problem... > The workaround is as follow in HTTP.cc : > *** HTTP_Analyzer::HTTP_Analyzer(Connection* > *** 808,813 > \--\- 808,814 \---\- > reply_reason_phrase = 0; > content_line_orig = new ContentLine_Analyzer(conn, true); > + content_line_orig->SetCRLFAsEOL(CR_as_EOL & LF_as_EOL); > AddSupportAnalyzer(content_line_orig); > With the workaround it still add one CRLF at the end of some PDF files. > As I wish to keep the hashes of the files it does matter :) -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1042) Caret in regex pattern doesn't match start of line
[
https://bro-tracker.atlassian.net/browse/BIT-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1042:
---
Status: Closed (was: Reopened)
Closing this ticket as there is not really any action resulting from it.
> Caret in regex pattern doesn't match start of line
> --
>
> Key: BIT-1042
> URL: https://bro-tracker.atlassian.net/browse/BIT-1042
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.1
> Environment: Ubuntu 12.04 and 13.04. Dependencies from apt-get ubuntu
> repos only.
>Reporter: Brian Little
> Labels: pattern, regex
> Attachments: caret.bro
>
>
> print split_all("some string", /^t/);
> I would expect it to not match ^t, but it matches any t in the string.
> output:
> {
> [1] = some s,
> [3] = ring,
> [2] = t
> }
> expected:
> {
> [1] = some string
> }
> tested on bro 2.1 and github master 2.1-824
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1042) Caret in regex pattern doesn't match start of line
[
https://bro-tracker.atlassian.net/browse/BIT-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638#comment-14638
]
Seth Hall commented on BIT-1042:
Brian, you can refer to the scripts here:
https://github.com/sethhall/bro-junk-drawer/tree/master/domain-tld
There are some examples of dealing with domain names (and I'm avoiding string
creation as much as possible which is good for performance).
> Caret in regex pattern doesn't match start of line
> --
>
> Key: BIT-1042
> URL: https://bro-tracker.atlassian.net/browse/BIT-1042
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.1
> Environment: Ubuntu 12.04 and 13.04. Dependencies from apt-get ubuntu
> repos only.
>Reporter: Brian Little
> Labels: pattern, regex
> Attachments: caret.bro
>
>
> print split_all("some string", /^t/);
> I would expect it to not match ^t, but it matches any t in the string.
> output:
> {
> [1] = some s,
> [3] = ring,
> [2] = t
> }
> expected:
> {
> [1] = some string
> }
> tested on bro 2.1 and github master 2.1-824
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1092) Handling of variable size 802.11 link-headers
[ https://bro-tracker.atlassian.net/browse/BIT-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1092: --- Resolution: Rejected Status: Closed (was: Open) No plans to support variable length headers at this time and this ticket doesn't have any concrete tasks in it so I'm closing. > Handling of variable size 802.11 link-headers > -- > > Key: BIT-1092 > URL: https://bro-tracker.atlassian.net/browse/BIT-1092 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.0, 2.1 >Reporter: Fahad Arshad > > This issue has been discussed before at this location: > http://comments.gmane.org/gmane.comp.security.detection.bro/3932 > Running Bro on version 2.0 or 2.1 results in the following error > $ bro -r ~/Documents/airportSniff-5000-packets.cap > fatal error: bro: problem with trace file > /home/bro/Documents/airportSniff-5000-packets.cap - unknown data link type > 0x7f > From my understanding (by looking at get_link_header_size() method in > src/PktSrc.cc), currently Bro cannot handle variable size header information. > Is there any future plan to handle variable size header information? -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1022) HTTP bogus events
[
https://bro-tracker.atlassian.net/browse/BIT-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1022:
---
Resolution: Fixed
Status: Closed (was: Open)
Closing since no traffic was ever provided.
> HTTP bogus events
> -
>
> Key: BIT-1022
> URL: https://bro-tracker.atlassian.net/browse/BIT-1022
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.1
>Reporter: thorkill
>Priority: High
> Labels: http
> Fix For: 2.2
>
> Attachments: local-http.bro
>
>
> I am using attached script to watch for suspected activity in
> http-connections. This happens a lot in our network:
> > 2013-06-10-16:32:00 HTTP::HTTP_strange_event 87.139.xxx.2xx:3916/tcp \->
> > xx.xx.xx.xx:80/tcp (uid ngRQOFjBgsg)
> bq. unknown_HTTP_method=\{Accept: text/*} (0 missed bytes)
> bq. # 87.139.xxx.2xx = p57xxx4xx.dip0.t-ipconnect.de xx.xx.xx.xx =
> I can not find out what the problem is. httpd logs tell me that everything
> was just fine.
> In most cases it happens after some POST request but not all the time.
> I will provide a pcap if I catch it somehow.
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1090) fatal error Val::CONVERTER
[
https://bro-tracker.atlassian.net/browse/BIT-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1090:
---
Resolution: Works for Me
Status: Closed (was: Open)
> fatal error Val::CONVERTER
> --
>
> Key: BIT-1090
> URL: https://bro-tracker.atlassian.net/browse/BIT-1090
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.1
> Environment: Ubuntu 10.04.03 LTS, bro 2.1-179
>Reporter: tyler.schoenke
> Attachments: my-detect-bruteforcing.bro, sigsup-ssh-pass2.bro
>
>
> Hi guys,
> I get the following message when I modified a data structure in
> detect-bruteforcing.bro. I didn't get a chance to test against the current
> version, but did a quick check against the mailing lists and tracker and
> didn't see this issue mentioned.
> $ bro my-detect-bruteforcing.bro sigsup-ssh-pass2.bro
> fatal error in ./sigsup-ssh-pass2.bro, line 2: Val::CONVERTER (types/table)
> (10.0.0.1/32)
> Here is the modification to detect-bruteforcing.bro:
> const ignore_guessers: table[subnet] of set[subnet] = {} &redef;
> I found the need to whitelist from a single host to multiple subnets instead
> of a single subnet. The following minimal script will produce the error.
> cat sigsup-ssh-pass2.bro
> redef SSH::ignore_guessers = {
> [172.0.0.0/16] = set( 10.0.0.1/32 )
> };
> Any help would be appreciated.
> Thanks,
> Tyler
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1043) LRU Table implementation
[
https://bro-tracker.atlassian.net/browse/BIT-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1043:
---
Affects Version/s: (was: 2.1)
git/master
> LRU Table implementation
>
>
> Key: BIT-1043
> URL: https://bro-tracker.atlassian.net/browse/BIT-1043
> Project: Bro Issue Tracker
> Issue Type: Improvement
> Components: Bro
>Affects Versions: git/master
>Reporter: Jordi Ros-Giralt
> Fix For: 2.3
>
>
> Attaching below the email description i exchanged with Seth and Robin
> describing this work.
> --
> Hi Seth and Robin,
> We got the repo up, you can get to our branch as follows:
> git clone --recursive https://github.com/giralt/bro.git
> cd bro/
> git checkout lru-table
> We would be happy to contribute this code to the Bro community. This is what
> it does:
> - It implements LRU tables for Bro
> - A Bro table can be enhanced with the LRU functionality with the following
> new table attributes:
> &lru_table: enhance the table with LRU functionality
> &size_limit=n: if adding an element to the table makes the size of the table
> larger than n, then drop the LRU element from that table before inserting the
> new element. n=0 means table size can be infinite (so don't drop elements
> from it)
> &drop_func=callback_func: defines a programmable callback function that gets
> called automatically every time an element from the LRU table is dropped due
> to hitting the size_limit. The prototype of this callback must be as follows:
> function callback_func(t: table[keytype] of valuetype, key: keytype, val:
> valuetype): count
> - It adds the following bif functions:
> function get_lru%(v: any%): any
> function get_mru%(v: any%): any
> function get_lru_key%(v: any%): any
> function get_mru_key%(v: any%): any
> - Example:
> function freed(t: table[port] of string, key: port, val: string): count {
> print "Dropped"; }
> local port_names: table[port] of string &lru &size_limit=2 &drop_func=freed;
> In terms of applications, we are currently using this feature for the
> chimera-to-bro compiler we are working on:
> http://www.chimera-query.org/index.html
> We thought that we could also use this feature to provide a sort of memory
> management facility for Bro. I had a talk with Seth some weeks ago about
> this. Something like the LRU implementation allows programmers to put bounds
> on the size of tables and prioritize which elements can be dropped first upon
> memory exhaustion scenarios. Perhaps an idea here would be to develop a
> garbage collector (could be done using Bro language itself, perhaps as a
> framework) which would be run upon hitting a certain memory usage watermark
> and which would mainly run through the set of tables marked as "garbage
> collectable" dropping LRU elements from them to help reduce/eliminate the
> risk of running out of memory.
> Should this be something interesting, what are the steps we would need to do
> to open source the LRU code into Bro?
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1062) Issues fragmented packets and BRO
[
https://bro-tracker.atlassian.net/browse/BIT-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1062:
---
Resolution: Won't Fix
Status: Closed (was: Open)
We aren't sure these packets are legitimate.
> Issues fragmented packets and BRO
> -
>
> Key: BIT-1062
> URL: https://bro-tracker.atlassian.net/browse/BIT-1062
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.1
> Environment: Ubuntu/Debian
>Reporter: john blaze
> Attachments: fraggy_out_EVILSTUFF, more_frag.pcap
>
>
> I was doing some testing with fragmented attacks trying to bypass IDS sensors
> and noticed that BRO does not identify/populate the SRC & DST IP's in the
> weird log and other fields such as the URI in the http.log when doing stuff
> like:
> >>> f=fragment(IP(dst="80.69.77.211")/ICMP()/("X"*50), fragsize=10)
> >>> for frag in f:
> ... send(frag)
> 1377062338.222065 - - - - -
> excessively_small_fragment - F bro
> Also,. I fragmented a GET /EVILSTUFF HTTP request,. and noticed:
> 1377056289.770819 - - - - -
> excessively_small_fragment - F bro
> 1377056289.787032 - - - - -
> fragment_inconsistency - F bro
> 1377056290.141267 iL6Ki3ncjV1 192.168.1.5 17384 192.168.1.16
> 80 unmatched_HTTP_reply- F bro
> PCAPS are attached.
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-478) Move BinPAC docs over to new server
[ https://bro-tracker.atlassian.net/browse/BIT-478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-478: -- Resolution: Fixed Status: Closed (was: Open) > Move BinPAC docs over to new server > --- > > Key: BIT-478 > URL: https://bro-tracker.atlassian.net/browse/BIT-478 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Website >Reporter: Robin Sommer >Assignee: Daniel Thayer > Fix For: 2.2 > > > There's some BinPAC documentation in the old Wiki that we should move over. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1090) fatal error Val::CONVERTER
[
https://bro-tracker.atlassian.net/browse/BIT-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14641#comment-14641
]
Seth Hall commented on BIT-1090:
This short test script works for me with 2.2...
const foobar: table[subnet] of set[subnet] = table() &redef;
redef foobar += { [172.0.0.0/16] = set(10.0.0.1/32) };
> fatal error Val::CONVERTER
> --
>
> Key: BIT-1090
> URL: https://bro-tracker.atlassian.net/browse/BIT-1090
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.1
> Environment: Ubuntu 10.04.03 LTS, bro 2.1-179
>Reporter: tyler.schoenke
> Attachments: my-detect-bruteforcing.bro, sigsup-ssh-pass2.bro
>
>
> Hi guys,
> I get the following message when I modified a data structure in
> detect-bruteforcing.bro. I didn't get a chance to test against the current
> version, but did a quick check against the mailing lists and tracker and
> didn't see this issue mentioned.
> $ bro my-detect-bruteforcing.bro sigsup-ssh-pass2.bro
> fatal error in ./sigsup-ssh-pass2.bro, line 2: Val::CONVERTER (types/table)
> (10.0.0.1/32)
> Here is the modification to detect-bruteforcing.bro:
> const ignore_guessers: table[subnet] of set[subnet] = {} &redef;
> I found the need to whitelist from a single host to multiple subnets instead
> of a single subnet. The following minimal script will produce the error.
> cat sigsup-ssh-pass2.bro
> redef SSH::ignore_guessers = {
> [172.0.0.0/16] = set( 10.0.0.1/32 )
> };
> Any help would be appreciated.
> Thanks,
> Tyler
--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-594) Control via git not reliable
[ https://bro-tracker.atlassian.net/browse/BIT-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-594: -- Resolution: Fixed Status: Closed (was: Open) I believe this is about the old tracker. > Control via git not reliable > > > Key: BIT-594 > URL: https://bro-tracker.atlassian.net/browse/BIT-594 > Project: Bro Issue Tracker > Issue Type: Problem > Components: TicketTracker >Reporter: Robin Sommer > > Adding "Closes #xxx" to commit messages works sometimes, but not > always. See ccad24b68584aabecaf0af69f2202914221be9e4 for a commit > where it didn't. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-595) Batch edit plugin missing
[ https://bro-tracker.atlassian.net/browse/BIT-595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-595: -- Resolution: Fixed Status: Closed (was: Open) About the old tracker. > Batch edit plugin missing > - > > Key: BIT-595 > URL: https://bro-tracker.atlassian.net/browse/BIT-595 > Project: Bro Issue Tracker > Issue Type: Problem > Components: TicketTracker >Reporter: Robin Sommer > > The batch edit plugin for Trac has disappaered. -- This message was sent by Atlassian JIRA (v6.2-OD-01#6204) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1096) Should check version of libmagic not version of file
[ https://bro-tracker.atlassian.net/browse/BIT-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14802#comment-14802 ] Seth Hall commented on BIT-1096: Unfortunately there probably isn't much we will do about this. We're already had some early discussions about forking and modifying libmagic and building it directly into Bro which would make it problem go away. I'm going to close this ticket because it's unlikely we'll address it before taking some other course of action anyway. > Should check version of libmagic not version of file > > > Key: BIT-1096 > URL: https://bro-tracker.atlassian.net/browse/BIT-1096 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.2 > Environment: Mac OS X 10.6.8; libmagic and other dependencies > installed using MacPorts >Reporter: Ryan Schmidt > > As far as I can tell, bro requires the libmagic library, but not the file > program. However bro's configuration script appears not to be checking the > version of the libmagic library, but the version of the file program. This is > a problem in distributions like MacPorts where the libmagic library and the > file program are in separate packages; installing the libmagic package does > not mean you will automatically get the corresponding version of the file > program. > This causes a build failure on Mac OS X 10.6 Snow Leopard for example which > ships with /usr/bin/file version 5.03. Even though libmagic 5.15 is installed > from MacPorts, bro fails to configure, thinking it's too old. The MacPorts > project's bug report for that is https://trac.macports.org/ticket/41457 > Could you change bro's configuration script to check the version of libmagic > instead? You can check MAGIC_VERSION in magic.h. -- This message was sent by Atlassian JIRA (v6.2-OD-03#6206) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1096) Should check version of libmagic not version of file
[ https://bro-tracker.atlassian.net/browse/BIT-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1096: --- Resolution: Won't Fix Status: Closed (was: Open) Unfortunately there isn't a better way to check the version at this time. > Should check version of libmagic not version of file > > > Key: BIT-1096 > URL: https://bro-tracker.atlassian.net/browse/BIT-1096 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.2 > Environment: Mac OS X 10.6.8; libmagic and other dependencies > installed using MacPorts >Reporter: Ryan Schmidt > > As far as I can tell, bro requires the libmagic library, but not the file > program. However bro's configuration script appears not to be checking the > version of the libmagic library, but the version of the file program. This is > a problem in distributions like MacPorts where the libmagic library and the > file program are in separate packages; installing the libmagic package does > not mean you will automatically get the corresponding version of the file > program. > This causes a build failure on Mac OS X 10.6 Snow Leopard for example which > ships with /usr/bin/file version 5.03. Even though libmagic 5.15 is installed > from MacPorts, bro fails to configure, thinking it's too old. The MacPorts > project's bug report for that is https://trac.macports.org/ticket/41457 > Could you change bro's configuration script to check the version of libmagic > instead? You can check MAGIC_VERSION in magic.h. -- This message was sent by Atlassian JIRA (v6.2-OD-03#6206) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1103) Memory leak in Bro Intel framework
[
https://bro-tracker.atlassian.net/browse/BIT-1103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1103:
---
Status: In Progress (was: Open)
> Memory leak in Bro Intel framework
> --
>
> Key: BIT-1103
> URL: https://bro-tracker.atlassian.net/browse/BIT-1103
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.2
> Environment: Red Hat Enterprise Linux Server release 6.5
>Reporter: Andrew Hoying
>Priority: High
> Labels: intel, leak
>
> The policy/frameworks/intel/seen bro scripts have a memory leak. On my
> moderately busy Bro installation I am leaking about a gig of memory a day per
> worker process with the Intel framework enabled. I can replicate by adding
> the following to the local.bro default script and then running through a
> small PCAP with primarily dns, dhcp and syslog traffic.
> {{
> @load policy/frameworks/intel/seen
> redef Intel::read_files += {
> "/usr/local/bro/spool/domain_suspicious.txt",
> };
> }}
> The intel file is in the following format, here's a few sample lines. It is
> generated automatically by CIF:
> {{
> #fields indicator indicator_type meta.source meta.desc
> meta.urlmeta.cif_impact meta.cif_severity meta.cif_confidence
> mete-tools.biz Intel::DOMAIN CIF - need-to-know spammed domain
> http://www.spamhaus.org/query/dbl?domain=mete-tools.biz (public)-
> - 95
> rttvxygkmwlqmq.net Intel::DOMAIN CIF - need-to-know spammed
> domain http://www.spamhaus.org/query/dbl?domain=rttvxygkmwlqmq.net (public)
> - - 95
> podserveruho.comIntel::DOMAIN CIF - need-to-know spammed
> domain http://www.spamhaus.org/query/dbl?domain=podserveruho.com (public)
> - - 95
> wwfcogdgntlxw.biz Intel::DOMAIN CIF - need-to-know spammed
> domain http://www.spamhaus.org/query/dbl?domain=wwfcogdgntlxw.biz (public)
> - - 95
> }}
> I compiled bro with gperftool debug support and followed the instructions
> here: http://www.bro.org/development/howtos/leaks.html. (Note, the
> instructions are wrong on the flags for ./configure, you need to add
> --enable-perftools-debug to get the -m option for bro)
> Here's the output from pprof top after running a PCAP trace with 10,000
> packets. Running traces with more packets show a greater number of lost
> objects in the same code locations.
> {{
> # pprof bin/bro "/tmp/bro.24541.net_run-end.heap" --inuse_objects --lines
> --heapcheck --edgefraction=1e-10 --nodefraction=1e-10
> Using local file bin/bro.
> Using local file /tmp/bro.24541.net_run-end.heap.
> Welcome to pprof! For help, type 'help'.
> (pprof) top
> Total: 4295 objects
> 2150 50.1% 50.1% 2150 50.1% AsciiFormatter::ParseValue
> /usr/src/bro-2.2/src/threading/AsciiFormatter.cc:186
> 2141 49.8% 99.9% 2141 49.8% copy_string
> /usr/src/bro-2.2/src/util.cc:155
>2 0.0% 100.0%2 0.0% re_alloc
> /usr/src/bro-2.2/build/src/re-scan.cc:2287
>1 0.0% 100.0%1 0.0% RE_parse
> /usr/src/bro-2.2/build/src/re-parse.y:110
>1 0.0% 100.0%1 0.0% RE_parse
> /usr/src/bro-2.2/build/src/re-parse.y:133
>0 0.0% 100.0% 2141 49.8% AsciiFormatter::ParseValue
> /usr/src/bro-2.2/src/threading/AsciiFormatter.cc:195
>0 0.0% 100.0%4 0.1% Connection::NextPacket
> /usr/src/bro-2.2/src/Conn.cc:259
>0 0.0% 100.0%4 0.1% NetSessions::DispatchPacket
> /usr/src/bro-2.2/src/Sessions.cc:189
>0 0.0% 100.0%4 0.1% NetSessions::DoNextPacket
> /usr/src/bro-2.2/src/Sessions.cc:709
>0 0.0% 100.0%4 0.1% NetSessions::NextPacket
> /usr/src/bro-2.2/src/Sessions.cc:247
> }}
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1103) Memory leak in Bro Intel framework
[
https://bro-tracker.atlassian.net/browse/BIT-1103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908#comment-14908
]
Seth Hall commented on BIT-1103:
I have confirmed this leak, it seems to be an issue with the input framework.
We'll be fixing it ASAP. Thanks for tracking it down.
> Memory leak in Bro Intel framework
> --
>
> Key: BIT-1103
> URL: https://bro-tracker.atlassian.net/browse/BIT-1103
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.2
> Environment: Red Hat Enterprise Linux Server release 6.5
>Reporter: Andrew Hoying
>Priority: High
> Labels: intel, leak
>
> The policy/frameworks/intel/seen bro scripts have a memory leak. On my
> moderately busy Bro installation I am leaking about a gig of memory a day per
> worker process with the Intel framework enabled. I can replicate by adding
> the following to the local.bro default script and then running through a
> small PCAP with primarily dns, dhcp and syslog traffic.
> {{
> @load policy/frameworks/intel/seen
> redef Intel::read_files += {
> "/usr/local/bro/spool/domain_suspicious.txt",
> };
> }}
> The intel file is in the following format, here's a few sample lines. It is
> generated automatically by CIF:
> {{
> #fields indicator indicator_type meta.source meta.desc
> meta.urlmeta.cif_impact meta.cif_severity meta.cif_confidence
> mete-tools.biz Intel::DOMAIN CIF - need-to-know spammed domain
> http://www.spamhaus.org/query/dbl?domain=mete-tools.biz (public)-
> - 95
> rttvxygkmwlqmq.net Intel::DOMAIN CIF - need-to-know spammed
> domain http://www.spamhaus.org/query/dbl?domain=rttvxygkmwlqmq.net (public)
> - - 95
> podserveruho.comIntel::DOMAIN CIF - need-to-know spammed
> domain http://www.spamhaus.org/query/dbl?domain=podserveruho.com (public)
> - - 95
> wwfcogdgntlxw.biz Intel::DOMAIN CIF - need-to-know spammed
> domain http://www.spamhaus.org/query/dbl?domain=wwfcogdgntlxw.biz (public)
> - - 95
> }}
> I compiled bro with gperftool debug support and followed the instructions
> here: http://www.bro.org/development/howtos/leaks.html. (Note, the
> instructions are wrong on the flags for ./configure, you need to add
> --enable-perftools-debug to get the -m option for bro)
> Here's the output from pprof top after running a PCAP trace with 10,000
> packets. Running traces with more packets show a greater number of lost
> objects in the same code locations.
> {{
> # pprof bin/bro "/tmp/bro.24541.net_run-end.heap" --inuse_objects --lines
> --heapcheck --edgefraction=1e-10 --nodefraction=1e-10
> Using local file bin/bro.
> Using local file /tmp/bro.24541.net_run-end.heap.
> Welcome to pprof! For help, type 'help'.
> (pprof) top
> Total: 4295 objects
> 2150 50.1% 50.1% 2150 50.1% AsciiFormatter::ParseValue
> /usr/src/bro-2.2/src/threading/AsciiFormatter.cc:186
> 2141 49.8% 99.9% 2141 49.8% copy_string
> /usr/src/bro-2.2/src/util.cc:155
>2 0.0% 100.0%2 0.0% re_alloc
> /usr/src/bro-2.2/build/src/re-scan.cc:2287
>1 0.0% 100.0%1 0.0% RE_parse
> /usr/src/bro-2.2/build/src/re-parse.y:110
>1 0.0% 100.0%1 0.0% RE_parse
> /usr/src/bro-2.2/build/src/re-parse.y:133
>0 0.0% 100.0% 2141 49.8% AsciiFormatter::ParseValue
> /usr/src/bro-2.2/src/threading/AsciiFormatter.cc:195
>0 0.0% 100.0%4 0.1% Connection::NextPacket
> /usr/src/bro-2.2/src/Conn.cc:259
>0 0.0% 100.0%4 0.1% NetSessions::DispatchPacket
> /usr/src/bro-2.2/src/Sessions.cc:189
>0 0.0% 100.0%4 0.1% NetSessions::DoNextPacket
> /usr/src/bro-2.2/src/Sessions.cc:709
>0 0.0% 100.0%4 0.1% NetSessions::NextPacket
> /usr/src/bro-2.2/src/Sessions.cc:247
> }}
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1103) Memory leak in Bro Intel framework
[
https://bro-tracker.atlassian.net/browse/BIT-1103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall reassigned BIT-1103:
--
Assignee: Bernhard Amann
> Memory leak in Bro Intel framework
> --
>
> Key: BIT-1103
> URL: https://bro-tracker.atlassian.net/browse/BIT-1103
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
>Affects Versions: 2.2
> Environment: Red Hat Enterprise Linux Server release 6.5
>Reporter: Andrew Hoying
>Assignee: Bernhard Amann
>Priority: High
> Labels: intel, leak
>
> The policy/frameworks/intel/seen bro scripts have a memory leak. On my
> moderately busy Bro installation I am leaking about a gig of memory a day per
> worker process with the Intel framework enabled. I can replicate by adding
> the following to the local.bro default script and then running through a
> small PCAP with primarily dns, dhcp and syslog traffic.
> {{
> @load policy/frameworks/intel/seen
> redef Intel::read_files += {
> "/usr/local/bro/spool/domain_suspicious.txt",
> };
> }}
> The intel file is in the following format, here's a few sample lines. It is
> generated automatically by CIF:
> {{
> #fields indicator indicator_type meta.source meta.desc
> meta.urlmeta.cif_impact meta.cif_severity meta.cif_confidence
> mete-tools.biz Intel::DOMAIN CIF - need-to-know spammed domain
> http://www.spamhaus.org/query/dbl?domain=mete-tools.biz (public)-
> - 95
> rttvxygkmwlqmq.net Intel::DOMAIN CIF - need-to-know spammed
> domain http://www.spamhaus.org/query/dbl?domain=rttvxygkmwlqmq.net (public)
> - - 95
> podserveruho.comIntel::DOMAIN CIF - need-to-know spammed
> domain http://www.spamhaus.org/query/dbl?domain=podserveruho.com (public)
> - - 95
> wwfcogdgntlxw.biz Intel::DOMAIN CIF - need-to-know spammed
> domain http://www.spamhaus.org/query/dbl?domain=wwfcogdgntlxw.biz (public)
> - - 95
> }}
> I compiled bro with gperftool debug support and followed the instructions
> here: http://www.bro.org/development/howtos/leaks.html. (Note, the
> instructions are wrong on the flags for ./configure, you need to add
> --enable-perftools-debug to get the -m option for bro)
> Here's the output from pprof top after running a PCAP trace with 10,000
> packets. Running traces with more packets show a greater number of lost
> objects in the same code locations.
> {{
> # pprof bin/bro "/tmp/bro.24541.net_run-end.heap" --inuse_objects --lines
> --heapcheck --edgefraction=1e-10 --nodefraction=1e-10
> Using local file bin/bro.
> Using local file /tmp/bro.24541.net_run-end.heap.
> Welcome to pprof! For help, type 'help'.
> (pprof) top
> Total: 4295 objects
> 2150 50.1% 50.1% 2150 50.1% AsciiFormatter::ParseValue
> /usr/src/bro-2.2/src/threading/AsciiFormatter.cc:186
> 2141 49.8% 99.9% 2141 49.8% copy_string
> /usr/src/bro-2.2/src/util.cc:155
>2 0.0% 100.0%2 0.0% re_alloc
> /usr/src/bro-2.2/build/src/re-scan.cc:2287
>1 0.0% 100.0%1 0.0% RE_parse
> /usr/src/bro-2.2/build/src/re-parse.y:110
>1 0.0% 100.0%1 0.0% RE_parse
> /usr/src/bro-2.2/build/src/re-parse.y:133
>0 0.0% 100.0% 2141 49.8% AsciiFormatter::ParseValue
> /usr/src/bro-2.2/src/threading/AsciiFormatter.cc:195
>0 0.0% 100.0%4 0.1% Connection::NextPacket
> /usr/src/bro-2.2/src/Conn.cc:259
>0 0.0% 100.0%4 0.1% NetSessions::DispatchPacket
> /usr/src/bro-2.2/src/Sessions.cc:189
>0 0.0% 100.0%4 0.1% NetSessions::DoNextPacket
> /usr/src/bro-2.2/src/Sessions.cc:709
>0 0.0% 100.0%4 0.1% NetSessions::NextPacket
> /usr/src/bro-2.2/src/Sessions.cc:247
> }}
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1102) Logs use tab delimiter yet do not remove tabs from logged data
[ https://bro-tracker.atlassian.net/browse/BIT-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall reassigned BIT-1102: -- Assignee: Seth Hall > Logs use tab delimiter yet do not remove tabs from logged data > -- > > Key: BIT-1102 > URL: https://bro-tracker.atlassian.net/browse/BIT-1102 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.2 > Environment: ubuntu 12.04LTS server >Reporter: April Lorenzen >Assignee: Seth Hall > Labels: http.log > Attachments: 4872c1c6d6983c4b6cf4e0f83baf5ec5.pcap > > > The attached pcap processed with bro -r pcapname will demonstrate the issue. > View the http.log after processing and note that the field delimiter appears > within a field. > I tried searching the official github mirror but don't know bro well enough > to find the spot to look. I'd be happy to work on fixing this if you point me > to the problem area (that's the hard part, right?) -- This message was sent by Atlassian JIRA (v6.2-OD-03#6206) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1102) Logs use tab delimiter yet do not remove tabs from logged data
[ https://bro-tracker.atlassian.net/browse/BIT-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1102: --- Status: In Progress (was: Open) > Logs use tab delimiter yet do not remove tabs from logged data > -- > > Key: BIT-1102 > URL: https://bro-tracker.atlassian.net/browse/BIT-1102 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.2 > Environment: ubuntu 12.04LTS server >Reporter: April Lorenzen > Labels: http.log > Attachments: 4872c1c6d6983c4b6cf4e0f83baf5ec5.pcap > > > The attached pcap processed with bro -r pcapname will demonstrate the issue. > View the http.log after processing and note that the field delimiter appears > within a field. > I tried searching the official github mirror but don't know bro well enough > to find the spot to look. I'd be happy to work on fixing this if you point me > to the problem area (that's the hard part, right?) -- This message was sent by Atlassian JIRA (v6.2-OD-03#6206) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1102) Logs use tab delimiter yet do not remove tabs from logged data
[ https://bro-tracker.atlassian.net/browse/BIT-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14910#comment-14910 ] Seth Hall commented on BIT-1102: Which line in http.log are you specifically referring to? I don't see any problems with this log. We specifically handle the case of field separators within fields by hex escaping. > Logs use tab delimiter yet do not remove tabs from logged data > -- > > Key: BIT-1102 > URL: https://bro-tracker.atlassian.net/browse/BIT-1102 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro >Affects Versions: 2.2 > Environment: ubuntu 12.04LTS server >Reporter: April Lorenzen > Labels: http.log > Attachments: 4872c1c6d6983c4b6cf4e0f83baf5ec5.pcap > > > The attached pcap processed with bro -r pcapname will demonstrate the issue. > View the http.log after processing and note that the field delimiter appears > within a field. > I tried searching the official github mirror but don't know bro well enough > to find the spot to look. I'd be happy to work on fixing this if you point me > to the problem area (that's the hard part, right?) -- This message was sent by Atlassian JIRA (v6.2-OD-03#6206) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1104) Add tracking for MSIE 11
[
https://bro-tracker.atlassian.net/browse/BIT-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1104:
---
Status: Merge Request (was: Open)
> Add tracking for MSIE 11
>
>
> Key: BIT-1104
> URL: https://bro-tracker.atlassian.net/browse/BIT-1104
> Project: Bro Issue Tracker
> Issue Type: Patch
> Components: Bro
>Affects Versions: 2.1
> Environment: Ubuntu
>Reporter: Michael Stone
> Labels: analyzer
>
> MSIE 11.0 currently shows up as . It looks like MS might
> have changed it's user agent string and doesn't include "MSIE". I added the
> following to /usr/local/bro/share/bro/base/frameworks/software/main.bro
> just below the "MSIE" block and above the "Safari" block.
> else if ( /Trident\/7.0/ in uparsed_version )
>{
>if ( /rv:11\.0/ in unparsed_version ) {
>software_name = "MSIE";
>v = [$major=11,$minor=0];
> }
>}
> Disclaimer: I'm fairly new to working with Bro so this might not be the best
> way, but it seems to be working for me.
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1104) Add tracking for MSIE 11
[
https://bro-tracker.atlassian.net/browse/BIT-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14913#comment-14913
]
Seth Hall commented on BIT-1104:
I have made some changes to your edit and it's in the
topic/seth/ie11-software-parsing branch. Thanks.
> Add tracking for MSIE 11
>
>
> Key: BIT-1104
> URL: https://bro-tracker.atlassian.net/browse/BIT-1104
> Project: Bro Issue Tracker
> Issue Type: Patch
> Components: Bro
>Affects Versions: 2.1
> Environment: Ubuntu
>Reporter: Michael Stone
> Labels: analyzer
>
> MSIE 11.0 currently shows up as . It looks like MS might
> have changed it's user agent string and doesn't include "MSIE". I added the
> following to /usr/local/bro/share/bro/base/frameworks/software/main.bro
> just below the "MSIE" block and above the "Safari" block.
> else if ( /Trident\/7.0/ in uparsed_version )
>{
>if ( /rv:11\.0/ in unparsed_version ) {
>software_name = "MSIE";
>v = [$major=11,$minor=0];
> }
>}
> Disclaimer: I'm fairly new to working with Bro so this might not be the best
> way, but it seems to be working for me.
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1104) Add tracking for MSIE 11
[
https://bro-tracker.atlassian.net/browse/BIT-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall reassigned BIT-1104:
--
Assignee: Seth Hall
> Add tracking for MSIE 11
>
>
> Key: BIT-1104
> URL: https://bro-tracker.atlassian.net/browse/BIT-1104
> Project: Bro Issue Tracker
> Issue Type: Patch
> Components: Bro
>Affects Versions: 2.1
> Environment: Ubuntu
>Reporter: Michael Stone
>Assignee: Seth Hall
> Labels: analyzer
>
> MSIE 11.0 currently shows up as . It looks like MS might
> have changed it's user agent string and doesn't include "MSIE". I added the
> following to /usr/local/bro/share/bro/base/frameworks/software/main.bro
> just below the "MSIE" block and above the "Safari" block.
> else if ( /Trident\/7.0/ in uparsed_version )
>{
>if ( /rv:11\.0/ in unparsed_version ) {
>software_name = "MSIE";
>v = [$major=11,$minor=0];
> }
>}
> Disclaimer: I'm fairly new to working with Bro so this might not be the best
> way, but it seems to be working for me.
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1108) Add broctl option to set PF_RING cluster type
[ https://bro-tracker.atlassian.net/browse/BIT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15001#comment-15001 ] Seth Hall commented on BIT-1108: Let's change the default to 4-tuple. > Add broctl option to set PF_RING cluster type > - > > Key: BIT-1108 > URL: https://bro-tracker.atlassian.net/browse/BIT-1108 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl >Reporter: Daniel Thayer > > Currently, when using PF_RING, broctl chooses the PF_RING > cluster type by setting the environment variable > PCAP_PF_RING_USE_CLUSTER_PER_FLOW. In order to use a > different cluster type, we would need to set a different > environment variable (the PF_RING-aware libpcap does not > look at the actual value of the environment variable, > just whether the variable is defined or not), but there is > no option in broctl to do this. > To address this issue, a new broctl option PFRINGClusterType > can be added, then a user could change the value of this > option to choose a different PF_RING cluster type (and the > broctl pf_ring plugin would set the appropriate env. variable). > The allowed values of this new broctl option would be: > "2-tuple", "4-tuple", "5-tuple", "tcp-5-tuple", "round-robin", > or "6-tuple" (this one corresponds to the current > cluster type used by broctl). By default, PFRINGClusterType > would be set to "6-tuple". -- This message was sent by Atlassian JIRA (v6.2-OD-03#6206) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1119) topic/jsiwek/tcp-improvements
[ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1119: --- Attachment: signature.asc We could probably do it similarly to how we're doing the detection of invalid checksums by sampling weirds for a little bit. I also like this approach a lot. I think that keeping the default settings of Bro working "correctly" in the normal case is good, but it's awesome to be able to notify people when things are failing and how they could fix it. > topic/jsiwek/tcp-improvements > - > > Key: BIT-1119 > URL: https://bro-tracker.atlassian.net/browse/BIT-1119 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro >Affects Versions: git/master >Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc > > > This branch is in the bro, bro-testing, and bro-testing-private repos and has > a few changes to improve reporting of TCP connection sizes and gaps (commit > messages explain in more detail). > The baseline changes in the external repos all seemed reasonable/explainable > (or actually fix a problem). There's too much changed to go through > case-by-case and actually check things, but I did do closer examinations of > unique differences as I came across them (e.g. try to corroborate Bro results > via wireshark). Then for those that seem to follow the same trend as > something I already inspected, I wouldn't manually check. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
