Re: [Bro-Dev] Dynamic plugin model (Re: [Bro-Commits] [git/bro] topic/robin/dynamic-plugins-2.3: Start of a plugin writing how-to. (87a1618))
Hi:
Still running through the plugin generation stuff ... seemed to build the
binary distribution okay for me after I set BRO (FC 19, x86_64), but haven't
tried to really load / run anything yet.
Personally, I'm a fan of this kind of development model. I like not having to
worry about synchronizing with master to work on a plugin. I also like that
the compilation / link times are much shorter for plugin development than they
are when trying to add code to bro directly.
+1 Seth's comment that the init-plugin script is very nice.
A few thoughts:
* Would a section on testing be appropriate? Both btest and unit testing might
be useful for plugins.
* A short section explaining how / when to modify CMakeLists.txt might be
useful. I had to look at one of the existing analyzer plugins to double-check
that all CC files needed to go into bro_plugin_cc(). Not hard, but still might
be nice to explicitly document.
* Should plugins be allowed to link to additional libraries? If so, how? I
believe this could become an issue if I write a plugin that links against
libXYZ, but libXYZ isn't available on the system that's trying to load the
plugin.
Cheers,
Gilbert
From: bro-dev-boun...@bro.org [bro-dev-boun...@bro.org] On Behalf Of Robin
Sommer [ro...@icir.org]
Sent: Monday, December 16, 2013 3:34 PM
To: bro-dev@bro.org
Subject: [Bro-Dev] Dynamic plugin model (Re: [Bro-Commits] [git/bro]
topic/robin/dynamic-plugins-2.3: Start of a plugin writing how-to.
(87a1618))
I'd appreciate feedback on the model for dynamic plugins described
below.
The document contains a short intro on writing a simple plugin, as
well as some background on how/when Bro integrates plugins. The code
is in topic/robin/dynamic-plugins-2.3. It hasn't seen much testing yet
but the basic infrastructure seems to work on Linux and Darwin at
least.
Robin
On Mon, Dec 16, 2013 at 12:09 -0800, I wrote:
+===
+Writing Bro Plugins
+===
+
+Bro is internally moving to a plugin structure that enables extending
+the system dynamically, without modifying the core code base. That way
+custom code remains self-contained and can be maintained, compiled,
+and installed independently. Currently, plugins can add the following
+functionality to Bro:
+
+- Bro scripts.
+
+- Builtin functions/events/types for the scripting language.
+
+- Protocol and file analyzers.
+
+- Packet sources and packet dumpers. TODO: Not yet.
+
+- Logging framework backends. TODO: Not yet.
+
+- Input framework readers. TODO: Not yet.
+
+A plugin's functionality is available to the user just as if Bro had
+the corresponding code built-in. Indeed, internally many of Bro's
+pieces are structured as plugins as well, they are just statically
+compiled into the binary rather than loaded dynamically at runtime, as
+external plugins are.
+
+Quick Start
+===
+
+Writing a basic plugin is quite straight-forward as long as one
+follows a few conventions. In the following we walk through adding a
+new built-in function (bif) to Bro; we'll add `a `rot13(s: string) :
+string``, a function that rotates every character in a string by 13
+places.
+
+A plugin comes in the form of a directory following a certain
+structure. To get started, Bro's distribution provides a helper script
+``aux/bro-aux/plugin-support/init-plugin`` that creates a skeleton
+plugin that can then be customized. Let's use that::
+
+# mkdir rot13-plugin
+# cd rot13-plugin
+# init-plugin Demo Rot13
+
+As you can see the script takes two arguments. The first is a
+namespace the plugin will live in, and the second a descriptive name
+for the plugin itself. Bro uses the combination of the two to identify
+a plugin. The namespace serves to avoid naming conflicts between
+plugins written by independent developers; pick, e.g., the name of
+your organisation (and note that the namespace ``Bro`` is reserved for
+functionality distributed by the Bro Project). In our example, the
+plugin will be called ``Demo::Rot13``.
+
+The ``init-plugin`` script puts a number of files in place. The full
+layout is described later. For now, all we need is
+``src/functions.bif``. It's initially empty, but we'll add our new bif
+there as follows::
+
+# cat scripts/functions.bif
+module CaesarCipher;
+
+function camel_case%(s: string%) : string
+%{
+char* rot13 = copy_string(s-CheckString());
+
+for ( char* p = rot13; *p; p++ )
+{
+char b = islower(*p) ? 'a' : 'A';
+*p = (*p - b + 13) % 26 + b;
+}
+
+return new StringVal(strlen(rot13), rot13);
+%}
+
+The syntax of this file is just like any other ``*.bif`` file; we
+won't go into it here.
+
+Now we can already compile our plugin, we just need to tell the
+Makefile put in place by
Re: [Bro-Dev] Dynamic plugin model (Re: [Bro-Commits] [git/bro] topic/robin/dynamic-plugins-2.3: Start of a plugin writing how-to. (87a1618))
On Wed, Dec 18, 2013 at 12:20 -0500, you wrote: I just build bro, cd into the build directory, source in the bro-path-dev.sh script and run Bro. Ah, I see. It's something else than I thought: a left-over from the earlier version that isn't needed anymore. Removed. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org ICSI/LBNL* Fax +1 (510) 666-2956 * www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Dynamic plugin model (Re: [Bro-Commits] [git/bro] topic/robin/dynamic-plugins-2.3: Start of a plugin writing how-to. (87a1618))
On Wed, Dec 18, 2013 at 21:07 -0500, you wrote: /tmp/bro/src/util.h:24:10: fatal error: 'magic.h' file not found I didn't consider Bro's CXX_FLAGS. I think I've fixed that, please try again. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org ICSI/LBNL* Fax +1 (510) 666-2956 * www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Dynamic plugin model (Re: [Bro-Commits] [git/bro] topic/robin/dynamic-plugins-2.3: Start of a plugin writing how-to. (87a1618))
On Thu, Dec 19, 2013 at 10:23 -0500, you wrote: * Would a section on testing be appropriate? Both btest and unit testing might be useful for plugins. Ack, that's a good point, the init-plugin script could put a basic setup in place for that, maybe even with a first test making sure things compile. * A short section explaining how / when to modify CMakeLists.txt might be useful. Yeah, likewise agreed. Indeed tthe documentation needs quite a bit more material to get people actually started without having to browse a ton of other code first. I'll leave that for later though once we've fleshed this all fully out. * Should plugins be allowed to link to additional libraries? Yes, definitly. My thinking is that the plugin author will extend the CMakeIndex.txt with the corresponding pieces, including compile-time logic to figure out if it's available. However, if the binary module aims to link against a lib that's not available at runtime where Bro executes, then I don't think there's much more we can do than fail loading the plugin: the dlopen will fail (iirc, Bro currently aborts in that case, I'm not sure if it should proceed without?) Thanks for the feedback. From chatting with Seth the other day, I took two more suggestions away: - I'm coming around that the BRO_PLUGIN_* macros aren't the best way of doing things. My main motivation for using them was hiding implementation details of the plugin API so that we can more easily change things without breaking existing code. However, it seems they are putting too much constraints on the plugin writer and/or, if one needs to get around them, require a lot of digging into the internals. So I'm mulling over creating a (simpler) C++ API to the Plugin class that can be used directly. - The static and dynamic plugins could be unified further. It's unclear what the right default is for shipping plugins that provide standard functionality, but it would be nice in any case if we could just flip a switch to change between static and dynamic builds for the in-tree stuff. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org ICSI/LBNL* Fax +1 (510) 666-2956 * www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1109) topic/dnthayer/doc-updates
Daniel Thayer created BIT-1109: -- Summary: topic/dnthayer/doc-updates Key: BIT-1109 URL: https://bro-tracker.atlassian.net/browse/BIT-1109 Project: Bro Issue Tracker Issue Type: Problem Components: Bro, BroControl Reporter: Daniel Thayer Fix For: 2.3 This branch (in bro and broctl repos) includes miscellaneous documentation fixes. -- This message was sent by Atlassian JIRA (v6.2-OD-05-4#6207) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1109) topic/dnthayer/doc-updates
[ https://bro-tracker.atlassian.net/browse/BIT-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1109: --- Status: Merge Request (was: Open) topic/dnthayer/doc-updates -- Key: BIT-1109 URL: https://bro-tracker.atlassian.net/browse/BIT-1109 Project: Bro Issue Tracker Issue Type: Problem Components: Bro, BroControl Reporter: Daniel Thayer Fix For: 2.3 This branch (in bro and broctl repos) includes miscellaneous documentation fixes. -- This message was sent by Atlassian JIRA (v6.2-OD-05-4#6207) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Broccoli question
On Dec 19, 2013, at 2:49 PM, Randy Caldejon ra...@packetchaser.org wrote: Can somebody tell me if BRO_TYPE_TABLE is supported at part of a record? Regardless of whether it’s part of a record, I don’t think broccoli can create and send tables (as well as sets/vectors). You may have to send individual key-value pairs and reconstruct the table on the Bro-side. - Jon ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Broccoli question
On Dec 19, 2013, at 6:09 PM, Siwek, Jonathan Luke jsi...@illinois.edu wrote: On Dec 19, 2013, at 2:49 PM, Randy Caldejon ra...@packetchaser.org wrote: Can somebody tell me if BRO_TYPE_TABLE is supported at part of a record? Regardless of whether it’s part of a record, I don’t think broccoli can create and send tables (as well as sets/vectors). You may have to send individual key-value pairs and reconstruct the table on the Bro-side. - Jon Ok, that's the path that I'll take. Thanks for confirming. -- Randy ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Dynamic plugin model (Re: [Bro-Commits] [git/bro] topic/robin/dynamic-plugins-2.3: Start of a plugin writing how-to. (87a1618))
The static and dynamic plugins could be unified further. It's unclear what the right default is for shipping plugins that provide standard functionality, but it would be nice in any case if we could just flip a switch to change between static and dynamic builds for the in-tree stuff. What's the reason for supporting both static and dynamic plugin types? Assuming that eliminating the static plugin type entirely would simplify the build / link process, and that the static plugin type exists largely due to performance concerns: if we could prove that dynamic linking didn't have a significant impact on performance, could we use that as rationale for eliminating static plugin linkage entirely? Cheers, Gilbert ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
