[Bro-Dev] Coding style enforcement
While porting Broker to the latest CAF version, I am realizing that the current pre C++11 coding style is not very conducive. Since the introduction of lambdas, and in particular with CAF's asynchronous and template-heavy programming model, the Whitesmiths style isn't very practical. Once can consider Broker a separate project, and perhaps a style change wouldn't be as complicated as in the main Bro code, I still wanted to check in with you whether anyone would object to changing the style. In particular, I'm planning to use CAF's coding style [1], which provides a unified style for meta programming as well as "regular" programming. There exists also a clang-format style file for this [2], which makes it really easy to enforce this style globally. Unfortunately, clang-format currently doesn't support the Whitesmiths style, so using this tool for Bro is not (yet) an option. (There exists an unmerged patch that needs some cleanup [3], if anyone wants to go for it.) On a related note: I'd also like to see stricter git commit message guidelines, at least putting strict rules on the first line [4]. Would you be in favor of such rules on commit messages? Matthias [1] https://github.com/actor-framework/actor-framework/blob/master/CONTRIBUTING.md [2] https://github.com/actor-framework/actor-framework/blob/master/.clang-format [3] http://reviews.llvm.org/D6833 [4] https://github.com/agis-/git-style-guide#messages ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1550) Please merge topic/johanna/netcontrol
[ https://bro-tracker.atlassian.net/browse/BIT-1550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1550: -- Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Please merge topic/johanna/netcontrol > - > > Key: BIT-1550 > URL: https://bro-tracker.atlassian.net/browse/BIT-1550 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro >Affects Versions: git/master >Reporter: Johanna Amann >Assignee: Robin Sommer > Fix For: 2.5 > > > Please merge topic/johanna/netcontrol, which contains the NetControl > framework and some small core changes necessary for it. > The core changes are: > - add support for the PrefixTable and patricia tree to dump lists of covered > IP addresses > - add a number of bifs > - add tracking of recursive types to prevent crash when a function contains > a record as an argument in which the function is a member of > The framework will get a few small updates in the future. However, these > mostly should be small missing features and either not affect the API at all, > or only contain minor changes. -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] [Bro-Commits] [git/bro] master: Files transferred over FTP were showing incorrect sizes. (08399da)
On Fri, Mar 11, 2016 at 12:56 -0500, Seth Hall wrote: > Files transferred over FTP were showing incorrect sizes. This seems to be causing a number of baseline mismatches in the external test suite. I can't tell if they are legitimate, did you run the tests? Robin -- Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1551) Broctl plugins in Bro plugins
[ https://bro-tracker.atlassian.net/browse/BIT-1551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=24807#comment-24807 ] Vlad Grigorescu commented on BIT-1551: -- Assigning to Daniel for the broctl piece. > Broctl plugins in Bro plugins > - > > Key: BIT-1551 > URL: https://bro-tracker.atlassian.net/browse/BIT-1551 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: bro-aux, BroControl, Documentation >Reporter: Vlad Grigorescu > > Right now, the Bro plugin skeleton creates: > /scripts > /src > /tests > I propose that a new directory, /broctl-plugins be created and that broctl > adds the following directories to the search path: > /lib/bro/plugins/*/broctl-plugins > $BRO_PLUGIN_PATH/*/broctl-plugins > The documentation here should also be updated: > https://www.bro.org/sphinx-git/devel/plugins.html -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1551) Broctl plugins in Bro plugins
Vlad Grigorescu created BIT-1551: Summary: Broctl plugins in Bro plugins Key: BIT-1551 URL: https://bro-tracker.atlassian.net/browse/BIT-1551 Project: Bro Issue Tracker Issue Type: New Feature Components: bro-aux, BroControl, Documentation Reporter: Vlad Grigorescu Right now, the Bro plugin skeleton creates: /scripts /src /tests I propose that a new directory, /broctl-plugins be created and that broctl adds the following directories to the search path: /lib/bro/plugins/*/broctl-plugins $BRO_PLUGIN_PATH/*/broctl-plugins The documentation here should also be updated: https://www.bro.org/sphinx-git/devel/plugins.html -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1551) Broctl plugins in Bro plugins
[ https://bro-tracker.atlassian.net/browse/BIT-1551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Grigorescu reassigned BIT-1551: Assignee: Daniel Thayer > Broctl plugins in Bro plugins > - > > Key: BIT-1551 > URL: https://bro-tracker.atlassian.net/browse/BIT-1551 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: bro-aux, BroControl, Documentation >Reporter: Vlad Grigorescu >Assignee: Daniel Thayer > > Right now, the Bro plugin skeleton creates: > /scripts > /src > /tests > I propose that a new directory, /broctl-plugins be created and that broctl > adds the following directories to the search path: > /lib/bro/plugins/*/broctl-plugins > $BRO_PLUGIN_PATH/*/broctl-plugins > The documentation here should also be updated: > https://www.bro.org/sphinx-git/devel/plugins.html -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1550) Please merge topic/johanna/netcontrol
[ https://bro-tracker.atlassian.net/browse/BIT-1550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1550: - Assignee: Robin Sommer > Please merge topic/johanna/netcontrol > - > > Key: BIT-1550 > URL: https://bro-tracker.atlassian.net/browse/BIT-1550 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro >Affects Versions: git/master >Reporter: Johanna Amann >Assignee: Robin Sommer > Fix For: 2.5 > > > Please merge topic/johanna/netcontrol, which contains the NetControl > framework and some small core changes necessary for it. > The core changes are: > - add support for the PrefixTable and patricia tree to dump lists of covered > IP addresses > - add a number of bifs > - add tracking of recursive types to prevent crash when a function contains > a record as an argument in which the function is a member of > The framework will get a few small updates in the future. However, these > mostly should be small missing features and either not affect the API at all, > or only contain minor changes. -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1540) Ifconfig is hardcoded in BroControl
[
https://bro-tracker.atlassian.net/browse/BIT-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Thayer reassigned BIT-1540:
--
Assignee: Daniel Thayer
> Ifconfig is hardcoded in BroControl
> ---
>
> Key: BIT-1540
> URL: https://bro-tracker.atlassian.net/browse/BIT-1540
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: BroControl
>Affects Versions: git/master
>Reporter: Johanna Amann
>Assignee: Daniel Thayer
> Fix For: 2.5
>
>
> From the mailing list:
> {quote}
> Hi Folks,
> On later versions of Linux distros iproute2 replaces ifconfig with ip
> Starting at line 601 at
> https://github.com/bro/broctl/blob/master/BroControl/config.py
> It looks like ifconfig is hard-written into the logic. Probably needs a
> patch to check for the ip command.
> Cheers,
> Harry
> {quote}
> We should probably check for the presence of the ip utility and use that, if
> present.
--
This message was sent by Atlassian JIRA
(v7.2.0-OD-03-014#72000)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Fw: Broker raw throughput
> To me, the performance numbers themselves don't matter as much as > managing expectations does: should I *expect* to be able to pass all > of my events through broker? This question depends on the event type and your concrete topology, and is hard to answer in general. We can say "in our point-to-point test scenario, our measurements show and upper bound of X events/sec for a workload consisting of message type Y." As Broker gets more traction, I assume we will get much more data points and a better understanding on the performance boundaries. > Trying to express things a slightly different way, I was concerned > that the different numbers from the different libraries were being > interpreted as an apples-to-apples comparison. Modifying CAF to > achieve the same results as e.g. 0mq would, at some point and in some > way, eventually require modifying CAF to be more like 0mq. I don't > think that would be good, because 0mq and CAF aren't (and shouldn't > be, in my humble opinion) the same thing. 0mq/nanomsg are only a thin wrapper around a blob of bytes, whereas CAF provides much more than that. However, I don't think the comparison we did was unrealistic: we looked at the overhead of sending a stream of simple (nearly empty) messages between two remote endpoints. This "dumbs down" CAF to a point where we're primarily stressing the messaging subsystem, without using much of the higher-level abstractions (CAF still has to go through its serialization layer). After Dominik's performance tweaks, the two libraries operate in the same order of magnitude, which strikes me as reasonable. 0mq still outperforms CAF in terms of maximum message rate for this specific workload, but this is also not surprising at this point, because it has received a lot of attention and optimizations over the past years specifically targeting high-throughput scenarios. Matthias ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1550) Please merge topic/johanna/netcontrol
Johanna Amann created BIT-1550: -- Summary: Please merge topic/johanna/netcontrol Key: BIT-1550 URL: https://bro-tracker.atlassian.net/browse/BIT-1550 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: git/master Reporter: Johanna Amann Fix For: 2.5 Please merge topic/johanna/netcontrol, which contains the NetControl framework and some small core changes necessary for it. The core changes are: - add support for the PrefixTable and patricia tree to dump lists of covered IP addresses - add a number of bifs - add tracking of recursive types to prevent crash when a function contains a record as an argument in which the function is a member of The framework will get a few small updates in the future. However, these mostly should be small missing features and either not affect the API at all, or only contain minor changes. -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1550) Please merge topic/johanna/netcontrol
[ https://bro-tracker.atlassian.net/browse/BIT-1550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1550: --- Status: Merge Request (was: Open) > Please merge topic/johanna/netcontrol > - > > Key: BIT-1550 > URL: https://bro-tracker.atlassian.net/browse/BIT-1550 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro >Affects Versions: git/master >Reporter: Johanna Amann > Fix For: 2.5 > > > Please merge topic/johanna/netcontrol, which contains the NetControl > framework and some small core changes necessary for it. > The core changes are: > - add support for the PrefixTable and patricia tree to dump lists of covered > IP addresses > - add a number of bifs > - add tracking of recursive types to prevent crash when a function contains > a record as an argument in which the function is a member of > The framework will get a few small updates in the future. However, these > mostly should be small missing features and either not affect the API at all, > or only contain minor changes. -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1498) add '-q' to ssh execution in ssh_runner.py
[ https://bro-tracker.atlassian.net/browse/BIT-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Schipp reassigned BIT-1498: --- Assignee: Jon Schipp > add '-q' to ssh execution in ssh_runner.py > -- > > Key: BIT-1498 > URL: https://bro-tracker.atlassian.net/browse/BIT-1498 > Project: Bro Issue Tracker > Issue Type: Patch > Components: BroControl >Affects Versions: 2.4 >Reporter: scampbell >Assignee: Jon Schipp >Priority: Trivial > Labels: broctl > Fix For: 2.5 > > > When using broctl in an environment with login banners, they will be > displayed in the broctl command. In the event that they can not be > configured away on the sshd end using '-q' avoids displaying the banner on > the client side. > The patch is trivial: > --- a/BroControl/ssh_runner.py > +++ b/BroControl/ssh_runner.py > @@ -108,6 +108,7 @@ class SSHMaster: > self.base_cmd = [ > "ssh", > "-o", "BatchMode=yes", > +"-q", > host, > ] > self.need_connect = True -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1498) add '-q' to ssh execution in ssh_runner.py
[ https://bro-tracker.atlassian.net/browse/BIT-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Schipp reassigned BIT-1498: --- Assignee: (was: Jon Schipp) > add '-q' to ssh execution in ssh_runner.py > -- > > Key: BIT-1498 > URL: https://bro-tracker.atlassian.net/browse/BIT-1498 > Project: Bro Issue Tracker > Issue Type: Patch > Components: BroControl >Affects Versions: 2.4 >Reporter: scampbell >Priority: Trivial > Labels: broctl > Fix For: 2.5 > > > When using broctl in an environment with login banners, they will be > displayed in the broctl command. In the event that they can not be > configured away on the sshd end using '-q' avoids displaying the banner on > the client side. > The patch is trivial: > --- a/BroControl/ssh_runner.py > +++ b/BroControl/ssh_runner.py > @@ -108,6 +108,7 @@ class SSHMaster: > self.base_cmd = [ > "ssh", > "-o", "BatchMode=yes", > +"-q", > host, > ] > self.need_connect = True -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1498) add '-q' to ssh execution in ssh_runner.py
[ https://bro-tracker.atlassian.net/browse/BIT-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Schipp updated BIT-1498: Status: Merge Request (was: Open) Assignee: (was: Jon Schipp) > add '-q' to ssh execution in ssh_runner.py > -- > > Key: BIT-1498 > URL: https://bro-tracker.atlassian.net/browse/BIT-1498 > Project: Bro Issue Tracker > Issue Type: Patch > Components: BroControl >Affects Versions: 2.4 >Reporter: scampbell >Priority: Trivial > Labels: broctl > Fix For: 2.5 > > > When using broctl in an environment with login banners, they will be > displayed in the broctl command. In the event that they can not be > configured away on the sshd end using '-q' avoids displaying the banner on > the client side. > The patch is trivial: > --- a/BroControl/ssh_runner.py > +++ b/BroControl/ssh_runner.py > @@ -108,6 +108,7 @@ class SSHMaster: > self.base_cmd = [ > "ssh", > "-o", "BatchMode=yes", > +"-q", > host, > ] > self.need_connect = True -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1540) Ifconfig is hardcoded in BroControl
[
https://bro-tracker.atlassian.net/browse/BIT-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Schipp reassigned BIT-1540:
---
Assignee: Jon Schipp
> Ifconfig is hardcoded in BroControl
> ---
>
> Key: BIT-1540
> URL: https://bro-tracker.atlassian.net/browse/BIT-1540
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: BroControl
>Affects Versions: git/master
>Reporter: Johanna Amann
>Assignee: Jon Schipp
> Fix For: 2.5
>
>
> From the mailing list:
> {quote}
> Hi Folks,
> On later versions of Linux distros iproute2 replaces ifconfig with ip
> Starting at line 601 at
> https://github.com/bro/broctl/blob/master/BroControl/config.py
> It looks like ifconfig is hard-written into the logic. Probably needs a
> patch to check for the ip command.
> Cheers,
> Harry
> {quote}
> We should probably check for the presence of the ip utility and use that, if
> present.
--
This message was sent by Atlassian JIRA
(v7.2.0-OD-03-014#72000)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1498) add '-q' to ssh execution in ssh_runner.py
[ https://bro-tracker.atlassian.net/browse/BIT-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=24806#comment-24806 ] Jon Schipp commented on BIT-1498: - [~dnthayer] I pushed topic/jschipp/broctl-quiet-ssh-banner > add '-q' to ssh execution in ssh_runner.py > -- > > Key: BIT-1498 > URL: https://bro-tracker.atlassian.net/browse/BIT-1498 > Project: Bro Issue Tracker > Issue Type: Patch > Components: BroControl >Affects Versions: 2.4 >Reporter: scampbell >Assignee: Jon Schipp >Priority: Trivial > Labels: broctl > Fix For: 2.5 > > > When using broctl in an environment with login banners, they will be > displayed in the broctl command. In the event that they can not be > configured away on the sshd end using '-q' avoids displaying the banner on > the client side. > The patch is trivial: > --- a/BroControl/ssh_runner.py > +++ b/BroControl/ssh_runner.py > @@ -108,6 +108,7 @@ class SSHMaster: > self.base_cmd = [ > "ssh", > "-o", "BatchMode=yes", > +"-q", > host, > ] > self.need_connect = True -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1498) add '-q' to ssh execution in ssh_runner.py
[ https://bro-tracker.atlassian.net/browse/BIT-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=24805#comment-24805 ] Jon Schipp commented on BIT-1498: - Using -o LogLevel=error will suppress the banner but still print the error messages. A happy medium I say. Going to push a new branch with it > add '-q' to ssh execution in ssh_runner.py > -- > > Key: BIT-1498 > URL: https://bro-tracker.atlassian.net/browse/BIT-1498 > Project: Bro Issue Tracker > Issue Type: Patch > Components: BroControl >Affects Versions: 2.4 >Reporter: scampbell >Assignee: Jon Schipp >Priority: Trivial > Labels: broctl > Fix For: 2.5 > > > When using broctl in an environment with login banners, they will be > displayed in the broctl command. In the event that they can not be > configured away on the sshd end using '-q' avoids displaying the banner on > the client side. > The patch is trivial: > --- a/BroControl/ssh_runner.py > +++ b/BroControl/ssh_runner.py > @@ -108,6 +108,7 @@ class SSHMaster: > self.base_cmd = [ > "ssh", > "-o", "BatchMode=yes", > +"-q", > host, > ] > self.need_connect = True -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1498) add '-q' to ssh execution in ssh_runner.py
[ https://bro-tracker.atlassian.net/browse/BIT-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=24804#comment-24804 ] Jon Schipp commented on BIT-1498: - I think the ssh messages are helpful in debugging problems but you're right we probably don't want to see the banner. Some notes: The banner prints when sshd_config is set to use the Banner option. Banner happens before authentication. The motd is not printed when the PrintMotd option is used. root@manager:~# broctl start starting manager ... starting proxy-1 ... starting worker-1 ... starting worker-2 ... This BANNER is displaying /etc/issue.net Ubuntu 14.04.1 LTS Stopping sshd on node1 will show banner and the informational ssh messages: root@manager:~# broctl start manager still running proxy-1 still running ssh: connect to host 10.1.1.20 port 22: Connection refused This BANNER is displaying /etc/issue.net Ubuntu 14.04.1 LTS Error: cannot connect to worker-1 worker-2 still running Stopping sshd on node1 while -q is set in ssh_runner.py yields a "Error: cannot connect", not the ssh errors. root@manager:~# broctl start ... Error: cannot connect to worker-1 worker-2 still running Shutting down the node will yield root@manager:~# broctl start ... ssh: connect to host 10.1.1.20 port 22: No route to host Error: cannot connect to worker-1 > add '-q' to ssh execution in ssh_runner.py > -- > > Key: BIT-1498 > URL: https://bro-tracker.atlassian.net/browse/BIT-1498 > Project: Bro Issue Tracker > Issue Type: Patch > Components: BroControl >Affects Versions: 2.4 >Reporter: scampbell >Assignee: Jon Schipp >Priority: Trivial > Labels: broctl > Fix For: 2.5 > > > When using broctl in an environment with login banners, they will be > displayed in the broctl command. In the event that they can not be > configured away on the sshd end using '-q' avoids displaying the banner on > the client side. > The patch is trivial: > --- a/BroControl/ssh_runner.py > +++ b/BroControl/ssh_runner.py > @@ -108,6 +108,7 @@ class SSHMaster: > self.base_cmd = [ > "ssh", > "-o", "BatchMode=yes", > +"-q", > host, > ] > self.need_connect = True -- This message was sent by Atlassian JIRA (v7.2.0-OD-03-014#72000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
