[Bro-Dev] [JIRA] (BIT-1554) broker (bro 2.4.1) fails to build against Python 3.{3, 4, 5}

[Matthias Vallentin (JIRA)]

 [ 
https://bro-tracker.atlassian.net/browse/BIT-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Vallentin reassigned BIT-1554:
---

Assignee: Matthias Vallentin

> broker (bro 2.4.1) fails to build against Python 3.{3,4,5}
> --
>
> Key: BIT-1554
> URL: https://bro-tracker.atlassian.net/browse/BIT-1554
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Broker
>Affects Versions: 2.4
> Environment: Trying to compile Bro 2.4.1 on Gentoo Linux (x86_64) 
> with broker enabled, against CAF 0.13.2, with python, using GCC support.
>Reporter: M.B.
>Assignee: Matthias Vallentin
>Priority: High
>  Labels: build
> Fix For: 2.5
>
> Attachments: bro-2.4.1.ebuild, build.log
>
>
> Bro fails to build. Details (in particular the options cmake gets called 
> with) can be seen from the build.log.
> For completeness I included the .ebuild.



--
This message was sent by Atlassian JIRA
(v7.2.0-OD-05-030#72002)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1522) Broker listener takes a long time to shut down on cluster stop/restart

[Matthias Vallentin (JIRA)]

 [ 
https://bro-tracker.atlassian.net/browse/BIT-1522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Vallentin reassigned BIT-1522:
---

Assignee: Matthias Vallentin

> Broker listener takes a long time to shut down on cluster stop/restart
> --
>
> Key: BIT-1522
> URL: https://bro-tracker.atlassian.net/browse/BIT-1522
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Broker
>Affects Versions: 2.4
> Environment: Ubuntu 14.04, Bro 2.4.1 with Broker
>Reporter: Stephen Hosom 
>Assignee: Matthias Vallentin
> Fix For: 2.5
>
> Attachments: broker-shutdown-test.bro
>
>
> It looks like when shutting down Broker, the listener sticks around for an 
> exceptionally long time (as much as a minute or more). Because of this, 
> Broker's listener actually fails to re-bind to the port on the next cluster 
> start silently. All Broker communication then fails to work silently. It can 
> take a while to notice this failure, since nothing really complains. 
> The listener should probably shut down faster than 1 minute... but it might 
> also make sense to add options to have the listener retry to start, or 
> generate a failure message when it doesn't start. Maybe listener starts in 
> bro_init should actually cause Bro to stop, so that the user sees the failure 
> immediately?



--
This message was sent by Atlassian JIRA
(v7.2.0-OD-05-030#72002)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1554) broker (bro 2.4.1) fails to build against Python 3.{3, 4, 5}

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=25709#comment-25709
 ] 

Matthias Vallentin commented on BIT-1554:
-

Fixing this "soon" would probably only be possible via a hotfix release. But 
since this is not a Bro issue, it doesn't seem to be the right path. We 
currently have major refactoring of Broker underway. In this scope, we can also 
convert the current bindings to Python 3.

What should we do with Python 2 support? From a maintaining point of view, I'd 
say we should go with one version only, but I know too little about the 
deployment of Python 2.

> broker (bro 2.4.1) fails to build against Python 3.{3,4,5}
> --
>
> Key: BIT-1554
> URL: https://bro-tracker.atlassian.net/browse/BIT-1554
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Broker
>Affects Versions: 2.4
> Environment: Trying to compile Bro 2.4.1 on Gentoo Linux (x86_64) 
> with broker enabled, against CAF 0.13.2, with python, using GCC support.
>Reporter: M.B.
>Priority: High
>  Labels: build
> Fix For: 2.5
>
> Attachments: bro-2.4.1.ebuild, build.log
>
>
> Bro fails to build. Details (in particular the options cmake gets called 
> with) can be seen from the build.log.
> For completeness I included the .ebuild.



--
This message was sent by Atlassian JIRA
(v7.2.0-OD-05-030#72002)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1556) bro-2.4.1 fails to compile broker with -march=i686

[Matthias Vallentin (JIRA)]

 [ 
https://bro-tracker.atlassian.net/browse/BIT-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Vallentin reassigned BIT-1556:
---

Assignee: Matthias Vallentin

> bro-2.4.1 fails to compile broker with -march=i686
> --
>
> Key: BIT-1556
> URL: https://bro-tracker.atlassian.net/browse/BIT-1556
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro, Broker
>Affects Versions: 2.4
> Environment: x86 chroot on Gentoo, with generic settings. E.g. 
> -march=i686.
>Reporter: M.B.
>Assignee: Matthias Vallentin
>  Labels: build
> Attachments: build.log
>
>
> Build fails due to missing SSE support.



--
This message was sent by Atlassian JIRA
(v7.2.0-OD-05-030#72002)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1450) Improve Python API

[Matthias Vallentin (JIRA)]

 [ 
https://bro-tracker.atlassian.net/browse/BIT-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Vallentin reassigned BIT-1450:
---

Assignee: Matthias Vallentin

> Improve Python API
> --
>
> Key: BIT-1450
> URL: https://bro-tracker.atlassian.net/browse/BIT-1450
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Broker
>Affects Versions: 2.4
>Reporter: Robin Sommer
>Assignee: Matthias Vallentin
> Fix For: 2.5
>
>
> The Python API is a bit cumbersome still as it requires (1) manually wrapping 
> values with {{data}} instances, and (2) also generally reflects C semantics a 
> bit too much, leading to some "unPythonic" idioms.



--
This message was sent by Atlassian JIRA
(v7.2.0-OD-05-030#72002)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1555) aux/broker/bindings/python/CMakeLists.txt doesn't respect -DINSTALL_LIB_DIR

[Matthias Vallentin (JIRA)]

 [ 
https://bro-tracker.atlassian.net/browse/BIT-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Vallentin reassigned BIT-1555:
---

Assignee: Matthias Vallentin

> aux/broker/bindings/python/CMakeLists.txt doesn't respect -DINSTALL_LIB_DIR
> ---
>
> Key: BIT-1555
> URL: https://bro-tracker.atlassian.net/browse/BIT-1555
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Broker
>Affects Versions: 2.4
> Environment: Building on Gentoo Linux (x86_64)
>Reporter: M.B.
>Assignee: Matthias Vallentin
>  Labels: build
> Fix For: 2.5
>
> Attachments: bro-2.4.1.ebuild, bro-2.4.1-fix-python-install-dir.patch
>
>
> During a normal build, this is a non-issue, as files get installed to 
> .../lib/...
> However, in a multilib environment this may become an issue. Hence it should 
> respect INSTALL_LIB_DIR, propagated from the top-level CMakeLists.txt.
> I wrote a simple patch that simply removes the logic for re-setting 
> PY_MOD_INSTALL_DIR, as I use this var to circumvent the issue.



--
This message was sent by Atlassian JIRA
(v7.2.0-OD-05-030#72002)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1447) Can't abort blocking Broker Python functions

[Matthias Vallentin (JIRA)]

 [ 
https://bro-tracker.atlassian.net/browse/BIT-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Vallentin reassigned BIT-1447:
---

Assignee: Matthias Vallentin

> Can't abort blocking Broker Python functions
> 
>
> Key: BIT-1447
> URL: https://bro-tracker.atlassian.net/browse/BIT-1447
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Broker
>Affects Versions: 2.4
>Reporter: Robin Sommer
>Assignee: Matthias Vallentin
> Fix For: 2.5
>
>
> When one of Broker's Python functions block, one can't abort with CTRL-C.



--
This message was sent by Atlassian JIRA
(v7.2.0-OD-05-030#72002)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1445) Broker crash when two stores go to the same SQLite DB

[Matthias Vallentin (JIRA)]

 [ 
https://bro-tracker.atlassian.net/browse/BIT-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Vallentin reassigned BIT-1445:
---

Assignee: Matthias Vallentin

> Broker crash when two stores go to the same SQLite DB
> -
>
> Key: BIT-1445
> URL: https://bro-tracker.atlassian.net/browse/BIT-1445
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Broker
>Affects Versions: 2.4
>Reporter: Robin Sommer
>Assignee: Matthias Vallentin
> Fix For: 2.5
>
>
> This crashes Bro:
> {code}
> [...]
> local s = BrokerStore::create_master("BroCon", BrokerStore::SQLITE);
> local t = BrokerStore::create_master("BroCon2", BrokerStore::SQLITE);
> [...]
> {code}
> Both stores go to the same file because the 3rd parameter with the file name 
> is optional and defaults to {{store.sqlite}}; and that is a problem.



--
This message was sent by Atlassian JIRA
(v7.2.0-OD-05-030#72002)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1388) Broker's integration in Bro's main/run loop

[Matthias Vallentin (JIRA)]

 [ 
https://bro-tracker.atlassian.net/browse/BIT-1388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Vallentin reassigned BIT-1388:
---

Assignee: Matthias Vallentin

> Broker's integration in Bro's main/run loop
> ---
>
> Key: BIT-1388
> URL: https://bro-tracker.atlassian.net/browse/BIT-1388
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro, Broker
>Reporter: Jon Siwek
>Assignee: Matthias Vallentin
> Fix For: 2.5
>
>
> * There's a cost to Broker queues being idle.  Whenever Broker gets a chance 
> to process messages, it looks for updates to all 
> connections/message-queues/data-stores.  That involves sending synchronous 
> messages between actors, and for empty queues, it just gets back an empty 
> deque object it needs to destroy.
> * Broker queues integrate into Bro's run loop by exposing a file descriptor 
> that's ready when the queue is non-empty.  Users have the capability of 
> adding arbitrary numbers of queues at run-time (e.g. they can freely add 
> subscriptions to any amount of logs, events, etc.).  Relying on select() may 
> become a bottleneck if someone has hundreds of Broker queues, or could 
> possibly break on some systems if FD_SETSIZE is limited to 1024.
> Ideas on how to fix:
> * Improve Bro's main run loop and dedicate an IOSource to each Broker queue 
> (instead of sharing a single IOSource like they do now).  There might be 
> several things that could be tweaked in the main run loop, but at a minimum, 
> epoll()/kqueue() could alternatively replace select().  Could also think 
> about using something like libev 
> (http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod) to abstract what 
> particular polling backend is used.  Might even be able to use libev's timers 
> to fix how Bro's timers are currently coupled w/ there being an active 
> IOSource consistently driving time forward.
> * Move the draining of Broker queues completely off to their own threads.  
> This maybe is adding a bit too much complexity (Broker/CAF uses threads for 
> queues, then Bro would use more threads just to talk to those other 
> threads...).  Since CAF becomes a requirement, it may be simpler to start 
> replacing/allowing some areas of Bro's threading to be done w/ CAF actors.  
> And then if Broker exposed an optional API to talk directly w/ CAF actors, 
> the integration w/ Bro may actually become more straightforward.
> And those ideas don't have to be mutually exclusive.



--
This message was sent by Atlassian JIRA
(v7.2.0-OD-05-030#72002)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1506) Bro fails to build on OS X 10.11 (El Capitan) due to OpenSSL header removal

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=25701#comment-25701
 ] 

Matthias Vallentin commented on BIT-1506:
-

Right, ideally we avoid a new release, because it's not really a Bro bug.

In my opinion, it suffices to update the 2.4.1 manual and mention how to use 
`--with-openssl` with Homebrew and El Capitan. Jon & Vlad have already fixed 
the issue for the next release.

> Bro fails to build on OS X 10.11 (El Capitan) due to OpenSSL header removal
> ---
>
> Key: BIT-1506
> URL: https://bro-tracker.atlassian.net/browse/BIT-1506
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro
>Affects Versions: 2.4
>Reporter: Vlad Grigorescu
> Fix For: 2.5
>
>
> It looks like Apple removed the OpenSSL headers with El Capitan[1] (OS X
> 10.11), and now Bro fails to build on OS X. Apple's recommendation is
> that we either include a copy of OpenSSL ourselves or we use their
> Secure Transport API.
> [1] - 



--
This message was sent by Atlassian JIRA
(v7.2.0-OD-05-030#72002)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1506) Bro fails to build on OS X 10.11 (El Capitan) due to OpenSSL header removal

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=25604#comment-25604
 ] 

Matthias Vallentin commented on BIT-1506:
-

This looks like the right way to search for OpenSSL under Homebrew.

> Bro fails to build on OS X 10.11 (El Capitan) due to OpenSSL header removal
> ---
>
> Key: BIT-1506
> URL: https://bro-tracker.atlassian.net/browse/BIT-1506
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro
>Affects Versions: 2.4
>Reporter: Vlad Grigorescu
> Fix For: 2.5
>
>
> It looks like Apple removed the OpenSSL headers with El Capitan[1] (OS X
> 10.11), and now Bro fails to build on OS X. Apple's recommendation is
> that we either include a copy of OpenSSL ourselves or we use their
> Secure Transport API.
> [1] - 



--
This message was sent by Atlassian JIRA
(v7.2.0-OD-05-030#72002)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1558) Bro's ascii formatter writing out scientific notation

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=25102#comment-25102
 ] 

Matthias Vallentin commented on BIT-1558:
-

Looks like a missing `std::fixed` in an I/O stream, or missing conversion from 
double to (un)signed integer.

> Bro's ascii formatter writing out scientific notation
> -
>
> Key: BIT-1558
> URL: https://bro-tracker.atlassian.net/browse/BIT-1558
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro
>Affects Versions: git/master
>Reporter: Seth Hall
>
> From the mailing list:
> ```
> Hello,
> in the x509.log normally the values regarding certificate.not_valid_before & 
> certificate.not_valid_after look like:
> 1444082400.00   1475791199.00
> I found some value like this:
> -3600.002.153226e+09
> Is it possible to modify something in order to have 2153226000 instead 
> 2.153226e+09 ?
> ```
> Bro's formatter's shouldn't use scientific notation because it complicates 
> parsing of the data.



--
This message was sent by Atlassian JIRA
(v7.2.0-OD-04-029#72002)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1474) Improve signaling of deprecated code

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=22009#comment-22009
 ] 

Matthias Vallentin commented on BIT-1474:
-

Ah, I missed that. That's definitely something we can work with. Let's keep 
this ticket open for now to track the enhancement with evaluated expressions.

> Improve signaling of deprecated code
> 
>
> Key: BIT-1474
> URL: https://bro-tracker.atlassian.net/browse/BIT-1474
> Project: Bro Issue Tracker
>  Issue Type: New Feature
>  Components: Bro
>Affects Versions: git/master
>Reporter: Matthias Vallentin
>  Labels: language
>
> Based on discussion in BIT-1411, we need a better mechanism for deprecating 
> code. In particular, Vern suggested the attribute {{&deprecate=expr}} syntax 
> where {{expr}} would be an expression that evaluates at the point of use of 
> the deprecated construct and would show the user a hint on how to migrate.



--
This message was sent by Atlassian JIRA
(v7.0.0-OD-04-018#70102)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1411) SQL_Injection_Victim is a misleading name

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21973#comment-21973
 ] 

Matthias Vallentin commented on BIT-1411:
-

I like this syntax and the proposed semantics. I've created a new ticket 
(BIT-1474) to track the addition of deprecation functionality explicitly.

> SQL_Injection_Victim is a misleading name
> -
>
> Key: BIT-1411
> URL: https://bro-tracker.atlassian.net/browse/BIT-1411
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro
>Reporter: Vern Paxson
>
> I suggest changing the name of this notice to {{SQL_Injection_Target}}.  
> Having "victim" in the name implies to me that the attack succeeded, which is 
> not what the associated logic is about.
> Indeed, I even wonder if this notice is useful.  The information should be 
> directly available from {{SQL_Injection_Attacker}} notices (though it doesn't 
> appear to be currently set up to provide this - why not?).



--
This message was sent by Atlassian JIRA
(v7.0.0-OD-02-259#70102)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1474) Improve signaling of deprecated code

[Matthias Vallentin (JIRA)]

Matthias Vallentin created BIT-1474:
---

 Summary: Improve signaling of deprecated code
 Key: BIT-1474
 URL: https://bro-tracker.atlassian.net/browse/BIT-1474
 Project: Bro Issue Tracker
  Issue Type: New Feature
  Components: Bro
Affects Versions: git/master
Reporter: Matthias Vallentin


Based on discussion in BIT-1411, we need a better mechanism for deprecating 
code. In particular, Vern suggested the attribute {{&deprecate=expr}} syntax 
where {{expr}} would be an expression that evaluates at the point of use of the 
deprecated construct and would show the user a hint on how to migrate.



--
This message was sent by Atlassian JIRA
(v7.0.0-OD-02-259#70102)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1411) SQL_Injection_Victim is a misleading name

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21971#comment-21971
 ] 

Matthias Vallentin commented on BIT-1411:
-

{quote}
There needs to be some model of deprecating/obsoleting deficient elements of 
the environment.
{quote}
How about adding a {{&deprecate}} attribute to such elements? Then we can 
inform users in one release cycle that the labeled functionality will cease to 
exist with the next release (or whatever deprecation policy we chose).

> SQL_Injection_Victim is a misleading name
> -
>
> Key: BIT-1411
> URL: https://bro-tracker.atlassian.net/browse/BIT-1411
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro
>Reporter: Vern Paxson
>
> I suggest changing the name of this notice to {{SQL_Injection_Target}}.  
> Having "victim" in the name implies to me that the attack succeeded, which is 
> not what the associated logic is about.
> Indeed, I even wonder if this notice is useful.  The information should be 
> directly available from {{SQL_Injection_Attacker}} notices (though it doesn't 
> appear to be currently set up to provide this - why not?).



--
This message was sent by Atlassian JIRA
(v7.0.0-OD-02-259#70102)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1454) Merge request for topic/mfischer/broker-bugfixes

[Matthias Vallentin (JIRA)]

 [ 
https://bro-tracker.atlassian.net/browse/BIT-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Vallentin updated BIT-1454:

Status: Closed  (was: Merge Request)

Good catch, thanks.

> Merge request for topic/mfischer/broker-bugfixes
> 
>
> Key: BIT-1454
> URL: https://bro-tracker.atlassian.net/browse/BIT-1454
> Project: Bro Issue Tracker
>  Issue Type: Patch
>  Components: Broker
>Affects Versions: git/master
>Reporter: Mathias Fischer
>Assignee: Matthias Vallentin
>
> Fixes the issue that Broker does not unpeer/disconnect from other endpoints. 
> Problem was a comparison in between two pointers instead of comparing their 
> dereferenced values in broker/src/peering.cc



--
This message was sent by Atlassian JIRA
(v7.0.0-OD-01-193#70101)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1454) Merge request for topic/mfischer/broker-bugfixes

[Matthias Vallentin (JIRA)]

 [ 
https://bro-tracker.atlassian.net/browse/BIT-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Vallentin reassigned BIT-1454:
---

Assignee: Matthias Vallentin

> Merge request for topic/mfischer/broker-bugfixes
> 
>
> Key: BIT-1454
> URL: https://bro-tracker.atlassian.net/browse/BIT-1454
> Project: Bro Issue Tracker
>  Issue Type: Patch
>  Components: Broker
>Affects Versions: git/master
>Reporter: Mathias Fischer
>Assignee: Matthias Vallentin
>
> Fixes the issue that Broker does not unpeer/disconnect from other endpoints. 
> Problem was a comparison in between two pointers instead of comparing their 
> dereferenced values in broker/src/peering.cc



--
This message was sent by Atlassian JIRA
(v7.0.0-OD-01-193#70101)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-672) Bring POP3 back into the distribution

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20014#comment-20014
 ] 

Matthias Vallentin commented on BIT-672:


We had student refactoring the code, but his changes never got merged: 
https://github.com/albert-magyar/bro/tree/topic/pop3. He refactored the scripts 
and I find their quality is good enough for us to ship them with the 
distribution, albeit disabled.

> Bring POP3 back into the distribution
> -
>
> Key: BIT-672
> URL: https://bro-tracker.atlassian.net/browse/BIT-672
> Project: Bro Issue Tracker
>  Issue Type: Task
>  Components: Bro
>Affects Versions: git/master
>Reporter: Matthias Vallentin
>Assignee: Seth Hall
> Fix For: 2.5
>
>
> The current master has no longer support for POP3. It lingers around but we 
> need to bring it back into the distribution.



--
This message was sent by Atlassian JIRA
(v6.4-OD-15-055#64014)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-757) Change split* to return a string_vec rather string_array

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19501#comment-19501
 ] 

Matthias Vallentin commented on BIT-757:


While I'd like to see this feature getting added, I think a new function 
{{vsplit}} would bloat the API, unless there is good use case for having 
{{string_set}} as well. I don't see that use case though. Anyone else?

> Change split* to return a string_vec rather string_array
> 
>
> Key: BIT-757
> URL: https://bro-tracker.atlassian.net/browse/BIT-757
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro
>Affects Versions: git/master
>Reporter: Matthias Vallentin
>  Labels: language
>
> Currently, `{{split}}{{ and friends return a }}{{string_array}}{{, which is a 
> }}{{table[count] of string}}{{. However, these BiFs should return a 
> }}{{string_vec}}{{ or }}{{vector of string}}{{ to allow for sequential 
> iteration over the result. The problem with the current approach is not only 
> that it is wrong modeled (the associative container does not make sense), but 
> also that iteration over the elements, which are obviously ordered, is 
> neither deterministic nor sequential. Presumably this mismatch exists because 
> vectors were not available when the }}{{split*}}` functions have been created.



--
This message was sent by Atlassian JIRA
(v6.4-OD-13-026#64011)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1140) Bloomfilter hashing problem

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16800#comment-16800
 ] 

Matthias Vallentin commented on BIT-1140:
-

I could reproduce the issue with your examples, thanks for providing them.

This is now fixed with 1d508742 in {{topic/matthias/bloomfilter-fix}}.

> Bloomfilter hashing problem
> ---
>
> Key: BIT-1140
> URL: https://bro-tracker.atlassian.net/browse/BIT-1140
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro
>Reporter: Robin Sommer
>Assignee: Matthias Vallentin
> Fix For: 2.3
>
> Attachments: bloom-test2.bro, bloom-test-short.bro
>
>
> It seems bloomfilter hashing isn't working correctly. Has that been 
> confirmed? Is there a fix?



--
This message was sent by Atlassian JIRA
(v6.3-OD-06-017#6327)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1140) Bloomfilter hashing problem

[Matthias Vallentin (JIRA)]

 [ 
https://bro-tracker.atlassian.net/browse/BIT-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Vallentin updated BIT-1140:

Status: Merge Request  (was: Open)

> Bloomfilter hashing problem
> ---
>
> Key: BIT-1140
> URL: https://bro-tracker.atlassian.net/browse/BIT-1140
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro
>Reporter: Robin Sommer
>Assignee: Matthias Vallentin
> Fix For: 2.3
>
> Attachments: bloom-test2.bro, bloom-test-short.bro
>
>
> It seems bloomfilter hashing isn't working correctly. Has that been 
> confirmed? Is there a fix?



--
This message was sent by Atlassian JIRA
(v6.3-OD-06-017#6327)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1166) installation does not take place in given prefix entirely

[Matthias Vallentin (JIRA)]

 [ 
https://bro-tracker.atlassian.net/browse/BIT-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Vallentin updated BIT-1166:


Resolution: Cannot Reproduce
Status: Closed  (was: Open)

> installation does not take place in given prefix entirely
> -
>
> Key: BIT-1166
> URL: https://bro-tracker.atlassian.net/browse/BIT-1166
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro, BroControl
>Affects Versions: git/master
>Reporter: Matthias Vallentin
>  Labels: build
>
> When configuring Bro to remain in a given prefix, say {{/opt/bro}}, the 
> installation of BroControl still attempts to create a spool directory outside 
> of the prefix:
> {code}
> ./configure --prefix=/opt/bro
> make
> make install
> [...]
> CMake Error at aux/broctl/cmake_install.cmake:200 (FILE):
>   file cannot create directory: /var/opt/bro/spool.  Maybe need
>   administrative privileges.
> {code}



--
This message was sent by Atlassian JIRA
(v6.2-OD-10-004-WN#6253)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1166) installation does not take place in given prefix entirely

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15824#comment-15824
 ] 

Matthias Vallentin commented on BIT-1166:
-

A student attempting to build Bro sent me this bug report. I asked for details 
on how to reproduce, but the problem apparently no longer occurs. Presumably 
this had to do with some submodule initialization problem. Closing it for now.

> installation does not take place in given prefix entirely
> -
>
> Key: BIT-1166
> URL: https://bro-tracker.atlassian.net/browse/BIT-1166
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro, BroControl
>Affects Versions: git/master
>Reporter: Matthias Vallentin
>  Labels: build
>
> When configuring Bro to remain in a given prefix, say {{/opt/bro}}, the 
> installation of BroControl still attempts to create a spool directory outside 
> of the prefix:
> {code}
> ./configure --prefix=/opt/bro
> make
> make install
> [...]
> CMake Error at aux/broctl/cmake_install.cmake:200 (FILE):
>   file cannot create directory: /var/opt/bro/spool.  Maybe need
>   administrative privileges.
> {code}



--
This message was sent by Atlassian JIRA
(v6.2-OD-10-004-WN#6253)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1140) Bloomfilter hashing problem

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822#comment-15822
 ] 

Matthias Vallentin commented on BIT-1140:
-

I have not yet had the chance to investigate the issue. [~aashish], since you 
reported the problem, maybe you could put together a small example, including 
data, that reproduces it? I sent along a patch which switches to 
double-hashing, but have lost track how it affected the problem.  

> Bloomfilter hashing problem
> ---
>
> Key: BIT-1140
> URL: https://bro-tracker.atlassian.net/browse/BIT-1140
> Project: Bro Issue Tracker
>  Issue Type: Problem
>  Components: Bro
>Reporter: Robin Sommer
>Assignee: Matthias Vallentin
> Fix For: 2.3
>
>
> It seems bloomfilter hashing isn't working correctly. Has that been 
> confirmed? Is there a fix?



--
This message was sent by Atlassian JIRA
(v6.2-OD-10-004-WN#6253)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1166) installation does not take place in given prefix entirely

[Matthias Vallentin (JIRA)]

Matthias Vallentin created BIT-1166:
---

 Summary: installation does not take place in given prefix entirely
 Key: BIT-1166
 URL: https://bro-tracker.atlassian.net/browse/BIT-1166
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro, BroControl
Affects Versions: git/master
Reporter: Matthias Vallentin


When configuring Bro to remain in a given prefix, say {{/opt/bro}}, the 
installation of BroControl still attempts to create a spool directory outside 
of the prefix:

{code}
./configure --prefix=/opt/bro
make
make install
[...]
CMake Error at aux/broctl/cmake_install.cmake:200 (FILE):
  file cannot create directory: /var/opt/bro/spool.  Maybe need
  administrative privileges.
{code}



--
This message was sent by Atlassian JIRA
(v6.2-OD-10-004-WN#6253)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (TM-15) Segfault in FifoDisk.cc

[Matthias Vallentin (JIRA)]

Matthias Vallentin created TM-15:


 Summary: Segfault in FifoDisk.cc
 Key: TM-15
 URL: https://bro-tracker.atlassian.net/browse/TM-15
 Project: Time Machine
  Issue Type: Patch
Affects Versions: git/master
 Environment: Confirmed on 32 and 64-bit Linux.
Reporter: Matthias Vallentin
 Fix For: git/master


Ahir consistently ran into segmentation faults with his workload. We debugged 
the issue and found the culprit: In FifoDisk.cc, a call to 
{{std::vector::front()}} violates its precondition

{{! std::vector::empty()}}

and thus causes UB. I fixed this here:

https://github.com/mavam/time-machine

It's a one-line fix and ready to merge.



--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1073) Make the MIME analyzer a FAF analyzer

[Matthias Vallentin (JIRA)]

Matthias Vallentin created BIT-1073:
---

 Summary: Make the MIME analyzer a FAF analyzer
 Key: BIT-1073
 URL: https://bro-tracker.atlassian.net/browse/BIT-1073
 Project: Bro Issue Tracker
  Issue Type: Task
  Components: Bro
Affects Versions: git/master
Reporter: Matthias Vallentin


We should convert the MIME analyzer to use FAF, allowing other components to 
reuse it. Specifically, I noted this in the process of bringing back the POP3 
analyzer. Ideally, we can just feed the contents of the download emails via the 
RETR command into a FAF-based MIME analyzer. Then we wouldn't have to rebuild 
functionality that's close to the SMTP analyzer.

In summary, we should factor the MIME analysis into a separate analysis unit.



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815#comment-13815
 ] 

Matthias Vallentin commented on BIT-1016:
-

{quote}
But also, I don't think we have any measurements yet that show how 
concerned/picky we need to be about how it's implemented?
{quote}

True that. A {{std::vector}} is not that much slower in terms of speed, it's 
just that it has some constant space overhead, roughly ~24 bytes. I cannot 
gauge how many UUID objects reside in memory at the same time, but this may be 
the ultimate measure.

> Option to extend uids to 128 bit
> 
>
> Key: BIT-1016
> URL: https://bro-tracker.atlassian.net/browse/BIT-1016
> Project: Bro Issue Tracker
>  Issue Type: New Feature
>  Components: Bro
>Affects Versions: git/master
>Reporter: rhave
>Assignee: Jon Siwek
>Priority: Low
> Fix For: 2.2
>
>
> Bro's uids are currently 64 bits, which makes them collide with a 50% chance 
> after 5.1 x 10^9^ different uids (see 
> http://en.wikipedia.org/wiki/Birthday_problem#Probability_table).
> I'm currently generating uuids of 128 bit to replace the native uids in bro, 
> as I'm using them as keys in a database, but this requires rewriting of the 
> bro-logs. I suspect that more people could benefit from an option to extend 
> the uids to 128 bit.
> I've made a quick and dirty patch to change most of the uids to 128 bit 
> (file_analysis uids are missing). The patch is ugly, and is only to show some 
> of the functionality I would like: http://pastebin.com/GkaGejNc



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13813#comment-13813
 ] 

Matthias Vallentin commented on BIT-1016:
-

{quote}
This is kind of what currently happens: the UID gets calculated in 64-bit 
chunks, then truncated if necessary. Except you can ask for more than 128-bits 
if you want.
{quote}

I wonder if we'd ever need more than 128-bit. I'm not aware of any UUID 
implementation that uses more than that. Using a {{uint64_t[2]}} wrapped in a 
small class with value semantics should suffice in my eyes.

> Option to extend uids to 128 bit
> 
>
> Key: BIT-1016
> URL: https://bro-tracker.atlassian.net/browse/BIT-1016
> Project: Bro Issue Tracker
>  Issue Type: New Feature
>  Components: Bro
>Affects Versions: git/master
>Reporter: rhave
>Assignee: Jon Siwek
>Priority: Low
> Fix For: 2.2
>
>
> Bro's uids are currently 64 bits, which makes them collide with a 50% chance 
> after 5.1 x 10^9^ different uids (see 
> http://en.wikipedia.org/wiki/Birthday_problem#Probability_table).
> I'm currently generating uuids of 128 bit to replace the native uids in bro, 
> as I'm using them as keys in a database, but this requires rewriting of the 
> bro-logs. I suspect that more people could benefit from an option to extend 
> the uids to 128 bit.
> I've made a quick and dirty patch to change most of the uids to 128 bit 
> (file_analysis uids are missing). The patch is ugly, and is only to show some 
> of the functionality I would like: http://pastebin.com/GkaGejNc



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13810#comment-13810
 ] 

Matthias Vallentin commented on BIT-1016:
-

Regarding performance: another option would be to use 128-bit UUIDs internally 
and just chop of 32 bytes if a 96-bit UUID is desired, assuming the bits in the 
UUID are distributed uniformly. Then we could use a fixed-size array and just 
change how the data is interpreted at script land.

> Option to extend uids to 128 bit
> 
>
> Key: BIT-1016
> URL: https://bro-tracker.atlassian.net/browse/BIT-1016
> Project: Bro Issue Tracker
>  Issue Type: New Feature
>  Components: Bro
>Affects Versions: git/master
>Reporter: rhave
>Assignee: Jon Siwek
>Priority: Low
> Fix For: 2.2
>
>
> Bro's uids are currently 64 bits, which makes them collide with a 50% chance 
> after 5.1 x 10^9^ different uids (see 
> http://en.wikipedia.org/wiki/Birthday_problem#Probability_table).
> I'm currently generating uuids of 128 bit to replace the native uids in bro, 
> as I'm using them as keys in a database, but this requires rewriting of the 
> bro-logs. I suspect that more people could benefit from an option to extend 
> the uids to 128 bit.
> I've made a quick and dirty patch to change most of the uids to 128 bit 
> (file_analysis uids are missing). The patch is ugly, and is only to show some 
> of the functionality I would like: http://pastebin.com/GkaGejNc



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1057) topic/jsiwek/bloomfilter-fix

[Matthias Vallentin (JIRA)]

[ 
https://bro-tracker.atlassian.net/browse/BIT-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510#comment-13510
 ] 

Matthias Vallentin commented on BIT-1057:
-

Yup, this looks good to me. 

For the record: the problem was indeed that the platform-specific block size of 
the bit vector. This could have resulted in different hash values, because the 
hashing algorithm took the underlying representations (i.e., the bit vector 
blocks) as opposed to the individual bits themselves. Moreover, the {{fmt}} 
function casted the 32-bit value into 64-bit one, potentially causing undefined 
behavior.

I will merge your changes into {{topic/matthias/bloom-filter}} and only do 
cosmetic polishing, such as using Bro's {{uint64}} instead of {{uint64_t}}.


> topic/jsiwek/bloomfilter-fix
> 
>
> Key: BIT-1057
> URL: https://bro-tracker.atlassian.net/browse/BIT-1057
> Project: Bro Issue Tracker
>  Issue Type: Patch
>  Components: Bro
>Affects Versions: git/master
>Reporter: Jon Siwek
>Assignee: Matthias Vallentin
> Fix For: 2.2
>
>
> The change (1) I did on this branch makes the bloomfilter-seed test start 
> passing on 32-bit systems, though I'm not confident how correct it is.  
> Matthias, can you review it and change this to a merge request if it looks 
> sane?
> (1) https://github.com/bro/bro/commit/774dadfe9aedc9fed98df69abcc83a3068bdf06b

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev