[jira] [Commented] (METRON-1865) Fix metron-bro-plugin-kafka tests
[ https://issues.apache.org/jira/browse/METRON-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682128#comment-16682128 ] ASF GitHub Bot commented on METRON-1865: Github user asfgit closed the pull request at: https://github.com/apache/metron-bro-plugin-kafka/pull/16 > Fix metron-bro-plugin-kafka tests > - > > Key: METRON-1865 > URL: https://issues.apache.org/jira/browse/METRON-1865 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1866) Improve metron-bro-plugin-kafka documentation
[ https://issues.apache.org/jira/browse/METRON-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682133#comment-16682133 ] ASF GitHub Bot commented on METRON-1866: Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/17#discussion_r232435113 --- Diff: README.md --- @@ -12,24 +12,72 @@ This software is a part of the [Apache Metron](http://metron.apache.org/) projec ## Installation +### `bro-pkg` Installation + +`bro-pkg` is the preferred mechanism for installing this plugin, as it will dynamically retrieve, build, test, and load the plugin. Note, that you will still need to [activate](#activation) and configure the plugin after your installation. + +1. Install [librdkafka](https://github.com/edenhill/librdkafka), a native client library for Kafka. This plugin has been tested against the latest release of librdkafka, which at the time of this writing is v0.9.4. + +In order to use this plugin within a kerberized Kafka environment, you will also need `libsasl2` installed and will need to pass `--enable-sasl` to the `configure` script. + +``` +$ curl -L https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz | tar xvz +$ cd librdkafka-0.9.4/ +$ ./configure --enable-sasl +$ make +$ sudo make install +``` + +1. Configure `bro-pkg` by following the quickstart guide [here](https://bro-package-manager.readthedocs.io/en/stable/quickstart.html). + +1. Install the plugin using `bro-pkg install`. + +``` +$ bro-pkg install apache/metron-bro-plugin-kafka --version master +The following packages will be INSTALLED: + bro/apache/metron-bro-plugin-kafka (master) + +Verify the following REQUIRED external dependencies: +(Ensure their installation on all relevant systems before proceeding): + from bro/apache/metron-bro-plugin-kafka (master): +librdkafka ~0.9.4 + +Proceed? [Y/n] +bro/apache/metron-bro-plugin-kafka asks for LIBRDKAFKA_ROOT (Path to librdkafka installation tree) ? [/usr/local/lib] +Saved answers to config file: /home/jonzeolla/.bro-pkg/config +Running unit tests for "bro/apache/metron-bro-plugin-kafka" +all 10 tests successful + + +Installing "bro/apache/metron-bro-plugin-kafka" +Installed "bro/apache/metron-bro-plugin-kafka" (master) +Loaded "bro/apache/metron-bro-plugin-kafka" +``` + --- End diff -- Sounds good, added a final step to validate manually. Initially, I left that out since `bro-pkg` does a very similar test, among others > Improve metron-bro-plugin-kafka documentation > - > > Key: METRON-1866 > URL: https://issues.apache.org/jira/browse/METRON-1866 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron-bro-plugin-kafka pull request #17: METRON-1866: Improve metron-bro-pl...
Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/17#discussion_r232435113 --- Diff: README.md --- @@ -12,24 +12,72 @@ This software is a part of the [Apache Metron](http://metron.apache.org/) projec ## Installation +### `bro-pkg` Installation + +`bro-pkg` is the preferred mechanism for installing this plugin, as it will dynamically retrieve, build, test, and load the plugin. Note, that you will still need to [activate](#activation) and configure the plugin after your installation. + +1. Install [librdkafka](https://github.com/edenhill/librdkafka), a native client library for Kafka. This plugin has been tested against the latest release of librdkafka, which at the time of this writing is v0.9.4. + +In order to use this plugin within a kerberized Kafka environment, you will also need `libsasl2` installed and will need to pass `--enable-sasl` to the `configure` script. + +``` +$ curl -L https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz | tar xvz +$ cd librdkafka-0.9.4/ +$ ./configure --enable-sasl +$ make +$ sudo make install +``` + +1. Configure `bro-pkg` by following the quickstart guide [here](https://bro-package-manager.readthedocs.io/en/stable/quickstart.html). + +1. Install the plugin using `bro-pkg install`. + +``` +$ bro-pkg install apache/metron-bro-plugin-kafka --version master +The following packages will be INSTALLED: + bro/apache/metron-bro-plugin-kafka (master) + +Verify the following REQUIRED external dependencies: +(Ensure their installation on all relevant systems before proceeding): + from bro/apache/metron-bro-plugin-kafka (master): +librdkafka ~0.9.4 + +Proceed? [Y/n] +bro/apache/metron-bro-plugin-kafka asks for LIBRDKAFKA_ROOT (Path to librdkafka installation tree) ? [/usr/local/lib] +Saved answers to config file: /home/jonzeolla/.bro-pkg/config +Running unit tests for "bro/apache/metron-bro-plugin-kafka" +all 10 tests successful + + +Installing "bro/apache/metron-bro-plugin-kafka" +Installed "bro/apache/metron-bro-plugin-kafka" (master) +Loaded "bro/apache/metron-bro-plugin-kafka" +``` + --- End diff -- Sounds good, added a final step to validate manually. Initially, I left that out since `bro-pkg` does a very similar test, among others ---
[jira] [Commented] (METRON-1304) Allow metron-bro-plugin-kafka to include or exclude logs
[ https://issues.apache.org/jira/browse/METRON-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682132#comment-16682132 ] ASF GitHub Bot commented on METRON-1304: Github user JonZeolla commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 Okay, pushed that fix and merged in master. > Allow metron-bro-plugin-kafka to include or exclude logs > > > Key: METRON-1304 > URL: https://issues.apache.org/jira/browse/METRON-1304 > Project: Metron > Issue Type: Improvement >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > > Right now, you must specify which logs you want to send to kafka via > metron-bro-plugin-kafka. This would allow the additional feature of > excluding certain logs, and sending everything else. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron-bro-plugin-kafka issue #2: METRON-1304: Allow metron-bro-plugin-kafka...
Github user JonZeolla commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 Okay, pushed that fix and merged in master. ---
[GitHub] metron-bro-plugin-kafka pull request #16: METRON-1865: Fix metron-bro-plugin...
Github user asfgit closed the pull request at: https://github.com/apache/metron-bro-plugin-kafka/pull/16 ---
[jira] [Commented] (METRON-1304) Allow metron-bro-plugin-kafka to include or exclude logs
[ https://issues.apache.org/jira/browse/METRON-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682120#comment-16682120 ] ASF GitHub Bot commented on METRON-1304: Github user JonZeolla commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 Yes that's why I said it depends on #16 and mentioned the order of review should be #16 -> #2 -> #17 -> #13 > Allow metron-bro-plugin-kafka to include or exclude logs > > > Key: METRON-1304 > URL: https://issues.apache.org/jira/browse/METRON-1304 > Project: Metron > Issue Type: Improvement >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > > Right now, you must specify which logs you want to send to kafka via > metron-bro-plugin-kafka. This would allow the additional feature of > excluding certain logs, and sending everything else. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1171: METRON-1740 make parser support CONFIG and SYSTEM log ty...
Github user JonZeolla commented on the issue: https://github.com/apache/metron/pull/1171 Yes, I outlined the specifics in my prior comments ---
[jira] [Commented] (METRON-1866) Improve metron-bro-plugin-kafka documentation
[ https://issues.apache.org/jira/browse/METRON-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682026#comment-16682026 ] ASF GitHub Bot commented on METRON-1866: Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/17#discussion_r232413911 --- Diff: README.md --- @@ -12,24 +12,72 @@ This software is a part of the [Apache Metron](http://metron.apache.org/) projec ## Installation +### `bro-pkg` Installation + +`bro-pkg` is the preferred mechanism for installing this plugin, as it will dynamically retrieve, build, test, and load the plugin. Note, that you will still need to [activate](#activation) and configure the plugin after your installation. + +1. Install [librdkafka](https://github.com/edenhill/librdkafka), a native client library for Kafka. This plugin has been tested against the latest release of librdkafka, which at the time of this writing is v0.9.4. + +In order to use this plugin within a kerberized Kafka environment, you will also need `libsasl2` installed and will need to pass `--enable-sasl` to the `configure` script. + +``` +$ curl -L https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz | tar xvz +$ cd librdkafka-0.9.4/ +$ ./configure --enable-sasl +$ make +$ sudo make install +``` + +1. Configure `bro-pkg` by following the quickstart guide [here](https://bro-package-manager.readthedocs.io/en/stable/quickstart.html). + +1. Install the plugin using `bro-pkg install`. + +``` +$ bro-pkg install apache/metron-bro-plugin-kafka --version master +The following packages will be INSTALLED: + bro/apache/metron-bro-plugin-kafka (master) + +Verify the following REQUIRED external dependencies: +(Ensure their installation on all relevant systems before proceeding): + from bro/apache/metron-bro-plugin-kafka (master): +librdkafka ~0.9.4 + +Proceed? [Y/n] +bro/apache/metron-bro-plugin-kafka asks for LIBRDKAFKA_ROOT (Path to librdkafka installation tree) ? [/usr/local/lib] +Saved answers to config file: /home/jonzeolla/.bro-pkg/config +Running unit tests for "bro/apache/metron-bro-plugin-kafka" +all 10 tests successful + + +Installing "bro/apache/metron-bro-plugin-kafka" +Installed "bro/apache/metron-bro-plugin-kafka" (master) +Loaded "bro/apache/metron-bro-plugin-kafka" +``` + --- End diff -- I think it would be good to add the same sanity check that we have at the end of the manual installation instructions, as the last step here. ``` [root@localhost ~]# bro -N Apache::Kafka Apache::Kafka - Writes logs to Kafka (dynamic, version 0.3) ``` > Improve metron-bro-plugin-kafka documentation > - > > Key: METRON-1866 > URL: https://issues.apache.org/jira/browse/METRON-1866 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1866) Improve metron-bro-plugin-kafka documentation
[ https://issues.apache.org/jira/browse/METRON-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682037#comment-16682037 ] ASF GitHub Bot commented on METRON-1866: Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/17#discussion_r232417353 --- Diff: README.md --- @@ -145,23 +217,35 @@ event bro_init() =-5 Notes * `logs_to_send` is mutually exclusive with `$pred`, thus for each log you want to set `$pred` on, you must individually setup a `Log::add_filter` and refrain from including that log in `logs_to_send`. + * In Bro 2.5.x the bro project introduced a [logger function](https://www.bro.org/sphinx/cluster/index.html#logger) which removes the logging functions from the manager thread, and taking advantage of that is highly recommended. If you are running this plugin on Bro 2.4.x, you may encounter issues where the manager thread is taking on too much responsibility and pinning a single CPU core without the ability to spread the load across additional cores. In this case, it may be in your best interest to prefer using a bro logging predicate over filtering in your Metron cluster [using Stellar](https://github.com/apache/metron/tree/master/metron-stellar/stellar-common) in order to lessen the load of that thread. * You can also filter IPv6 logs from within your Metron cluster [using Stellar](https://github.com/apache/metron/tree/master/metron-stellar/stellar-common#is_ip). In that case, you wouldn't apply a predicate in your bro configuration, and instead Stellar would filter the logs out before they were processed by the enrichment layer of Metron. * It is also possible to use the `is_v6_subnet()` bro function in your predicate, as of their [2.5 release](https://www.bro.org/sphinx-git/install/release-notes.html#bro-2-5), however the above example should work on [bro 2.4](https://www.bro.org/sphinx-git/install/release-notes.html#bro-2-4) and newer, which has been the focus of the kafka plugin. ## Settings -### `kafka_conf` +### `logs_to_send` --- End diff -- Nevermind. You're wanting to preceed this with #2 > Improve metron-bro-plugin-kafka documentation > - > > Key: METRON-1866 > URL: https://issues.apache.org/jira/browse/METRON-1866 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron-bro-plugin-kafka pull request #17: METRON-1866: Improve metron-bro-pl...
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/17#discussion_r232417353 --- Diff: README.md --- @@ -145,23 +217,35 @@ event bro_init() =-5 Notes * `logs_to_send` is mutually exclusive with `$pred`, thus for each log you want to set `$pred` on, you must individually setup a `Log::add_filter` and refrain from including that log in `logs_to_send`. + * In Bro 2.5.x the bro project introduced a [logger function](https://www.bro.org/sphinx/cluster/index.html#logger) which removes the logging functions from the manager thread, and taking advantage of that is highly recommended. If you are running this plugin on Bro 2.4.x, you may encounter issues where the manager thread is taking on too much responsibility and pinning a single CPU core without the ability to spread the load across additional cores. In this case, it may be in your best interest to prefer using a bro logging predicate over filtering in your Metron cluster [using Stellar](https://github.com/apache/metron/tree/master/metron-stellar/stellar-common) in order to lessen the load of that thread. * You can also filter IPv6 logs from within your Metron cluster [using Stellar](https://github.com/apache/metron/tree/master/metron-stellar/stellar-common#is_ip). In that case, you wouldn't apply a predicate in your bro configuration, and instead Stellar would filter the logs out before they were processed by the enrichment layer of Metron. * It is also possible to use the `is_v6_subnet()` bro function in your predicate, as of their [2.5 release](https://www.bro.org/sphinx-git/install/release-notes.html#bro-2-5), however the above example should work on [bro 2.4](https://www.bro.org/sphinx-git/install/release-notes.html#bro-2-4) and newer, which has been the focus of the kafka plugin. ## Settings -### `kafka_conf` +### `logs_to_send` --- End diff -- Nevermind. You're wanting to preceed this with #2 ---
[jira] [Commented] (METRON-1304) Allow metron-bro-plugin-kafka to include or exclude logs
[ https://issues.apache.org/jira/browse/METRON-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682033#comment-16682033 ] ASF GitHub Bot commented on METRON-1304: Github user nickwallen commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 I am getting a test failure. ``` [root@localhost ~]# bro-pkg install http://github.com/jonzeolla/metron-bro-plugin-kafka --version=METRON-1304 The following packages will be INSTALLED: http://github.com/jonzeolla/metron-bro-plugin-kafka (METRON-1304) Verify the following REQUIRED external dependencies: (Ensure their installation on all relevant systems before proceeding): from http://github.com/jonzeolla/metron-bro-plugin-kafka (METRON-1304): librdkafka ~0.9.4 Proceed? [Y/n] Y http://github.com/jonzeolla/metron-bro-plugin-kafka asks for LIBRDKAFKA_ROOT (Path to librdkafka installation tree) ? [/usr/local/lib] Saved answers to config file: /root/.bro-pkg/config Running unit tests for "http://github.com/jonzeolla/metron-bro-plugin-kafka; [ 90%] kafka.show-plugin ... failed % 'btest-diff output' failed unexpectedly (exit code 1) % cat .diag == File === Apache::Kafka - Writes logs to Kafka (dynamic, version 0.3) [Writer] KafkaWriter (Log::WRITER_KAFKAWRITER) [Constant] Kafka::kafka_conf [Constant] Kafka::topic_name [Constant] Kafka::max_wait_on_shutdown [Constant] Kafka::tag_json [Constant] Kafka::json_timestamps [Constant] Kafka::debug == Diff === --- /tmp/test-diff.22832.output.baseline.tmp 2018-11-09 22:51:27.714922560 + +++ /tmp/test-diff.22832.output.tmp 2018-11-09 22:51:27.719922348 + @@ -4,5 +4,6 @@ [Constant] Kafka::topic_name [Constant] Kafka::max_wait_on_shutdown [Constant] Kafka::tag_json +[Constant] Kafka::json_timestamps [Constant] Kafka::debug === % cat .stderr 1 of 10 tests failed error: http://github.com/jonzeolla/metron-bro-plugin-kafka tests failed, inspect contents of /root/.bro-pkg/testing/metron-bro-plugin-kafka for details Proceed to install anyway? [N/y] ``` > Allow metron-bro-plugin-kafka to include or exclude logs > > > Key: METRON-1304 > URL: https://issues.apache.org/jira/browse/METRON-1304 > Project: Metron > Issue Type: Improvement >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > > Right now, you must specify which logs you want to send to kafka via > metron-bro-plugin-kafka. This would allow the additional feature of > excluding certain logs, and sending everything else. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1866) Improve metron-bro-plugin-kafka documentation
[ https://issues.apache.org/jira/browse/METRON-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682025#comment-16682025 ] ASF GitHub Bot commented on METRON-1866: Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/17#discussion_r232414752 --- Diff: README.md --- @@ -145,23 +217,35 @@ event bro_init() =-5 Notes * `logs_to_send` is mutually exclusive with `$pred`, thus for each log you want to set `$pred` on, you must individually setup a `Log::add_filter` and refrain from including that log in `logs_to_send`. + * In Bro 2.5.x the bro project introduced a [logger function](https://www.bro.org/sphinx/cluster/index.html#logger) which removes the logging functions from the manager thread, and taking advantage of that is highly recommended. If you are running this plugin on Bro 2.4.x, you may encounter issues where the manager thread is taking on too much responsibility and pinning a single CPU core without the ability to spread the load across additional cores. In this case, it may be in your best interest to prefer using a bro logging predicate over filtering in your Metron cluster [using Stellar](https://github.com/apache/metron/tree/master/metron-stellar/stellar-common) in order to lessen the load of that thread. * You can also filter IPv6 logs from within your Metron cluster [using Stellar](https://github.com/apache/metron/tree/master/metron-stellar/stellar-common#is_ip). In that case, you wouldn't apply a predicate in your bro configuration, and instead Stellar would filter the logs out before they were processed by the enrichment layer of Metron. * It is also possible to use the `is_v6_subnet()` bro function in your predicate, as of their [2.5 release](https://www.bro.org/sphinx-git/install/release-notes.html#bro-2-5), however the above example should work on [bro 2.4](https://www.bro.org/sphinx-git/install/release-notes.html#bro-2-4) and newer, which has been the focus of the kafka plugin. ## Settings -### `kafka_conf` +### `logs_to_send` --- End diff -- We don't currently have `logs_to_send` that comes later. Some of these are remnants of #2 perhaps. If it's too much pain to clean-up what should be in #2 versus in this PR, I can be flexible. Just let me know. > Improve metron-bro-plugin-kafka documentation > - > > Key: METRON-1866 > URL: https://issues.apache.org/jira/browse/METRON-1866 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron-bro-plugin-kafka issue #2: METRON-1304: Allow metron-bro-plugin-kafka...
Github user nickwallen commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 I am getting a test failure. ``` [root@localhost ~]# bro-pkg install http://github.com/jonzeolla/metron-bro-plugin-kafka --version=METRON-1304 The following packages will be INSTALLED: http://github.com/jonzeolla/metron-bro-plugin-kafka (METRON-1304) Verify the following REQUIRED external dependencies: (Ensure their installation on all relevant systems before proceeding): from http://github.com/jonzeolla/metron-bro-plugin-kafka (METRON-1304): librdkafka ~0.9.4 Proceed? [Y/n] Y http://github.com/jonzeolla/metron-bro-plugin-kafka asks for LIBRDKAFKA_ROOT (Path to librdkafka installation tree) ? [/usr/local/lib] Saved answers to config file: /root/.bro-pkg/config Running unit tests for "http://github.com/jonzeolla/metron-bro-plugin-kafka; [ 90%] kafka.show-plugin ... failed % 'btest-diff output' failed unexpectedly (exit code 1) % cat .diag == File === Apache::Kafka - Writes logs to Kafka (dynamic, version 0.3) [Writer] KafkaWriter (Log::WRITER_KAFKAWRITER) [Constant] Kafka::kafka_conf [Constant] Kafka::topic_name [Constant] Kafka::max_wait_on_shutdown [Constant] Kafka::tag_json [Constant] Kafka::json_timestamps [Constant] Kafka::debug == Diff === --- /tmp/test-diff.22832.output.baseline.tmp 2018-11-09 22:51:27.714922560 + +++ /tmp/test-diff.22832.output.tmp 2018-11-09 22:51:27.719922348 + @@ -4,5 +4,6 @@ [Constant] Kafka::topic_name [Constant] Kafka::max_wait_on_shutdown [Constant] Kafka::tag_json +[Constant] Kafka::json_timestamps [Constant] Kafka::debug === % cat .stderr 1 of 10 tests failed error: http://github.com/jonzeolla/metron-bro-plugin-kafka tests failed, inspect contents of /root/.bro-pkg/testing/metron-bro-plugin-kafka for details Proceed to install anyway? [N/y] ``` ---
[jira] [Commented] (METRON-1865) Fix metron-bro-plugin-kafka tests
[ https://issues.apache.org/jira/browse/METRON-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682030#comment-16682030 ] ASF GitHub Bot commented on METRON-1865: Github user nickwallen commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/16 +1 Thanks! I was able to test like this... ``` bro-pkg install http://github.com/jonzeolla/metron-bro-plugin-kafka --version=METRON-1865 ``` > Fix metron-bro-plugin-kafka tests > - > > Key: METRON-1865 > URL: https://issues.apache.org/jira/browse/METRON-1865 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron-bro-plugin-kafka issue #16: METRON-1865: Fix metron-bro-plugin-kafka ...
Github user nickwallen commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/16 +1 Thanks! I was able to test like this... ``` bro-pkg install http://github.com/jonzeolla/metron-bro-plugin-kafka --version=METRON-1865 ``` ---
[GitHub] metron-bro-plugin-kafka pull request #17: METRON-1866: Improve metron-bro-pl...
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/17#discussion_r232414752 --- Diff: README.md --- @@ -145,23 +217,35 @@ event bro_init() =-5 Notes * `logs_to_send` is mutually exclusive with `$pred`, thus for each log you want to set `$pred` on, you must individually setup a `Log::add_filter` and refrain from including that log in `logs_to_send`. + * In Bro 2.5.x the bro project introduced a [logger function](https://www.bro.org/sphinx/cluster/index.html#logger) which removes the logging functions from the manager thread, and taking advantage of that is highly recommended. If you are running this plugin on Bro 2.4.x, you may encounter issues where the manager thread is taking on too much responsibility and pinning a single CPU core without the ability to spread the load across additional cores. In this case, it may be in your best interest to prefer using a bro logging predicate over filtering in your Metron cluster [using Stellar](https://github.com/apache/metron/tree/master/metron-stellar/stellar-common) in order to lessen the load of that thread. * You can also filter IPv6 logs from within your Metron cluster [using Stellar](https://github.com/apache/metron/tree/master/metron-stellar/stellar-common#is_ip). In that case, you wouldn't apply a predicate in your bro configuration, and instead Stellar would filter the logs out before they were processed by the enrichment layer of Metron. * It is also possible to use the `is_v6_subnet()` bro function in your predicate, as of their [2.5 release](https://www.bro.org/sphinx-git/install/release-notes.html#bro-2-5), however the above example should work on [bro 2.4](https://www.bro.org/sphinx-git/install/release-notes.html#bro-2-4) and newer, which has been the focus of the kafka plugin. ## Settings -### `kafka_conf` +### `logs_to_send` --- End diff -- We don't currently have `logs_to_send` that comes later. Some of these are remnants of #2 perhaps. If it's too much pain to clean-up what should be in #2 versus in this PR, I can be flexible. Just let me know. ---
[GitHub] metron-bro-plugin-kafka pull request #17: METRON-1866: Improve metron-bro-pl...
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/17#discussion_r232413911 --- Diff: README.md --- @@ -12,24 +12,72 @@ This software is a part of the [Apache Metron](http://metron.apache.org/) projec ## Installation +### `bro-pkg` Installation + +`bro-pkg` is the preferred mechanism for installing this plugin, as it will dynamically retrieve, build, test, and load the plugin. Note, that you will still need to [activate](#activation) and configure the plugin after your installation. + +1. Install [librdkafka](https://github.com/edenhill/librdkafka), a native client library for Kafka. This plugin has been tested against the latest release of librdkafka, which at the time of this writing is v0.9.4. + +In order to use this plugin within a kerberized Kafka environment, you will also need `libsasl2` installed and will need to pass `--enable-sasl` to the `configure` script. + +``` +$ curl -L https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz | tar xvz +$ cd librdkafka-0.9.4/ +$ ./configure --enable-sasl +$ make +$ sudo make install +``` + +1. Configure `bro-pkg` by following the quickstart guide [here](https://bro-package-manager.readthedocs.io/en/stable/quickstart.html). + +1. Install the plugin using `bro-pkg install`. + +``` +$ bro-pkg install apache/metron-bro-plugin-kafka --version master +The following packages will be INSTALLED: + bro/apache/metron-bro-plugin-kafka (master) + +Verify the following REQUIRED external dependencies: +(Ensure their installation on all relevant systems before proceeding): + from bro/apache/metron-bro-plugin-kafka (master): +librdkafka ~0.9.4 + +Proceed? [Y/n] +bro/apache/metron-bro-plugin-kafka asks for LIBRDKAFKA_ROOT (Path to librdkafka installation tree) ? [/usr/local/lib] +Saved answers to config file: /home/jonzeolla/.bro-pkg/config +Running unit tests for "bro/apache/metron-bro-plugin-kafka" +all 10 tests successful + + +Installing "bro/apache/metron-bro-plugin-kafka" +Installed "bro/apache/metron-bro-plugin-kafka" (master) +Loaded "bro/apache/metron-bro-plugin-kafka" +``` + --- End diff -- I think it would be good to add the same sanity check that we have at the end of the manual installation instructions, as the last step here. ``` [root@localhost ~]# bro -N Apache::Kafka Apache::Kafka - Writes logs to Kafka (dynamic, version 0.3) ``` ---
[jira] [Commented] (METRON-1740) Improve Palo Alto parser to handle CONFIG and SYSTEM syslog messages
[
https://issues.apache.org/jira/browse/METRON-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681996#comment-16681996
]
ASF GitHub Bot commented on METRON-1740:
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/1171
> @JonZeolla: +0 only performed functional testing.
What testing did you do? Did you run this up against some Panos logs?
> Improve Palo Alto parser to handle CONFIG and SYSTEM syslog messages
>
>
> Key: METRON-1740
> URL: https://issues.apache.org/jira/browse/METRON-1740
> Project: Metron
> Issue Type: Improvement
>Reporter: Yi Liu
>Priority: Major
>
> As a Metron's user (security analyst)
> I would like Metron's Palo Alto parser be able to parse CONFIG and SYSTEM
> PanOS syslog messages
> so that I can know what, when how the system configuration has been changed
> and how the system has been running.
>
> The current PaloAlto parser (BasicPaloAltoFirewallParser) only supports
> THREAT and TRAFFIC log messages. The task is to extend it to support CONFIG
> and SYSTEM log messages. The supported PanOS versions are 6.1, 7.0 and 8.0.
> The sample of CONFIG log (PanOS 7.0)
> {code:java}
> 1,2017/08/11 11:23:36,,CONFIG,0,0,2017/08/11
> 11:23:36,192.168.14.162,,edit,admin,Web,Succeeded, vsys vsys4 rule X
> rules dev-to-dev-ext-http-https,1336,0x0,0,0,0,0,,dev-something200-01
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (METRON-1740) Improve Palo Alto parser to handle CONFIG and SYSTEM syslog messages
[
https://issues.apache.org/jira/browse/METRON-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681990#comment-16681990
]
ASF GitHub Bot commented on METRON-1740:
Github user JonZeolla commented on the issue:
https://github.com/apache/metron/pull/1171
+0 only performed functional testing.
> Improve Palo Alto parser to handle CONFIG and SYSTEM syslog messages
>
>
> Key: METRON-1740
> URL: https://issues.apache.org/jira/browse/METRON-1740
> Project: Metron
> Issue Type: Improvement
>Reporter: Yi Liu
>Priority: Major
>
> As a Metron's user (security analyst)
> I would like Metron's Palo Alto parser be able to parse CONFIG and SYSTEM
> PanOS syslog messages
> so that I can know what, when how the system configuration has been changed
> and how the system has been running.
>
> The current PaloAlto parser (BasicPaloAltoFirewallParser) only supports
> THREAT and TRAFFIC log messages. The task is to extend it to support CONFIG
> and SYSTEM log messages. The supported PanOS versions are 6.1, 7.0 and 8.0.
> The sample of CONFIG log (PanOS 7.0)
> {code:java}
> 1,2017/08/11 11:23:36,,CONFIG,0,0,2017/08/11
> 11:23:36,192.168.14.162,,edit,admin,Web,Succeeded, vsys vsys4 rule X
> rules dev-to-dev-ext-http-https,1336,0x0,0,0,0,0,,dev-something200-01
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] metron issue #1171: METRON-1740 make parser support CONFIG and SYSTEM log ty...
Github user JonZeolla commented on the issue: https://github.com/apache/metron/pull/1171 +0 only performed functional testing. ---
[jira] [Commented] (METRON-1304) Allow metron-bro-plugin-kafka to include or exclude logs
[ https://issues.apache.org/jira/browse/METRON-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681987#comment-16681987 ] ASF GitHub Bot commented on METRON-1304: Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/2#discussion_r232405800 --- Diff: README.md --- @@ -144,23 +194,35 @@ event bro_init() =-5 Notes * `logs_to_send` is mutually exclusive with `$pred`, thus for each log you want to set `$pred` on, you must individually setup a `Log::add_filter` and refrain from including that log in `logs_to_send`. + * In Bro 2.5.x the bro project introduced a [logger function](https://www.bro.org/sphinx/cluster/index.html#logger) which removes the logging functions from the manager thread, and taking advantage of that is highly recommended. If you are running this plugin on Bro 2.4.x, you may encounter issues where the manager thread is taking on too much responsibility and pinning a single CPU core without the ability to spread the load across additional cores. In this case, it may be in your best interest to prefer using a bro logging predicate over filtering in your Metron cluster [using Stellar](https://github.com/apache/metron/tree/master/metron-stellar/stellar-common) in order to lesson the load of that thread. --- End diff -- As a part of the review process, this improvement was split into #17 > Allow metron-bro-plugin-kafka to include or exclude logs > > > Key: METRON-1304 > URL: https://issues.apache.org/jira/browse/METRON-1304 > Project: Metron > Issue Type: Improvement >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > > Right now, you must specify which logs you want to send to kafka via > metron-bro-plugin-kafka. This would allow the additional feature of > excluding certain logs, and sending everything else. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1304) Allow metron-bro-plugin-kafka to include or exclude logs
[ https://issues.apache.org/jira/browse/METRON-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681985#comment-16681985 ] ASF GitHub Bot commented on METRON-1304: Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/2#discussion_r232405751 --- Diff: README.md --- @@ -11,6 +11,32 @@ This software is a part of the [Apache Metron](http://metron.apache.org/) projec ## Installation +### `bro-pkg` Installation + +`bro-pkg` is the preferred mechanism for installing this plugin, as it will dynamically retrieve, build, test, and load the plugin. Note, that you will still need to [activate](#activation) and configure the plugin after your installation. + --- End diff -- As a part of the review process, this improvement was split into #17 > Allow metron-bro-plugin-kafka to include or exclude logs > > > Key: METRON-1304 > URL: https://issues.apache.org/jira/browse/METRON-1304 > Project: Metron > Issue Type: Improvement >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > > Right now, you must specify which logs you want to send to kafka via > metron-bro-plugin-kafka. This would allow the additional feature of > excluding certain logs, and sending everything else. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron-bro-plugin-kafka pull request #2: METRON-1304: Allow metron-bro-plugi...
Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/2#discussion_r232405844 --- Diff: README.md --- @@ -11,6 +11,32 @@ This software is a part of the [Apache Metron](http://metron.apache.org/) projec ## Installation +### `bro-pkg` Installation + --- End diff -- As a part of the review process, this improvement was split into #17 ---
[GitHub] metron-bro-plugin-kafka pull request #2: METRON-1304: Allow metron-bro-plugi...
Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/2#discussion_r232405751 --- Diff: README.md --- @@ -11,6 +11,32 @@ This software is a part of the [Apache Metron](http://metron.apache.org/) projec ## Installation +### `bro-pkg` Installation + +`bro-pkg` is the preferred mechanism for installing this plugin, as it will dynamically retrieve, build, test, and load the plugin. Note, that you will still need to [activate](#activation) and configure the plugin after your installation. + --- End diff -- As a part of the review process, this improvement was split into #17 ---
[jira] [Commented] (METRON-1304) Allow metron-bro-plugin-kafka to include or exclude logs
[ https://issues.apache.org/jira/browse/METRON-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681981#comment-16681981 ] ASF GitHub Bot commented on METRON-1304: Github user JonZeolla commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 Suggested order of review: #16 -> #2 -> #17 -> #13 > Allow metron-bro-plugin-kafka to include or exclude logs > > > Key: METRON-1304 > URL: https://issues.apache.org/jira/browse/METRON-1304 > Project: Metron > Issue Type: Improvement >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > > Right now, you must specify which logs you want to send to kafka via > metron-bro-plugin-kafka. This would allow the additional feature of > excluding certain logs, and sending everything else. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron-bro-plugin-kafka issue #2: METRON-1304: Allow metron-bro-plugin-kafka...
Github user JonZeolla commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 Suggested order of review: #16 -> #2 -> #17 -> #13 ---
[GitHub] metron-bro-plugin-kafka pull request #17: METRON-1866: Improve metron-bro-pl...
GitHub user JonZeolla opened a pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/17 METRON-1866: Improve metron-bro-plugin-kafka documentation ## Contributor Comments This depends on METRON-1304. Pure documentation change You can merge this pull request into a Git repository by running: $ git pull https://github.com/JonZeolla/metron-bro-plugin-kafka METRON-1866 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron-bro-plugin-kafka/pull/17.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #17 commit d0e59108238d7f36c707e7f44a0f00bbbdd73c74 Author: Jon Zeolla Date: 2018-11-09T21:49:01Z METRON-1866 ---
[jira] [Commented] (METRON-1866) Improve metron-bro-plugin-kafka documentation
[ https://issues.apache.org/jira/browse/METRON-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681977#comment-16681977 ] ASF GitHub Bot commented on METRON-1866: GitHub user JonZeolla opened a pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/17 METRON-1866: Improve metron-bro-plugin-kafka documentation ## Contributor Comments This depends on METRON-1304. Pure documentation change You can merge this pull request into a Git repository by running: $ git pull https://github.com/JonZeolla/metron-bro-plugin-kafka METRON-1866 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron-bro-plugin-kafka/pull/17.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #17 commit d0e59108238d7f36c707e7f44a0f00bbbdd73c74 Author: Jon Zeolla Date: 2018-11-09T21:49:01Z METRON-1866 > Improve metron-bro-plugin-kafka documentation > - > > Key: METRON-1866 > URL: https://issues.apache.org/jira/browse/METRON-1866 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1304) Allow metron-bro-plugin-kafka to include or exclude logs
[ https://issues.apache.org/jira/browse/METRON-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681962#comment-16681962 ] ASF GitHub Bot commented on METRON-1304: Github user JonZeolla commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 Note that this one now depends on apache/metron-bro-plugin-kafka#16 to work properly because I split out the btest bugfix. > Allow metron-bro-plugin-kafka to include or exclude logs > > > Key: METRON-1304 > URL: https://issues.apache.org/jira/browse/METRON-1304 > Project: Metron > Issue Type: Improvement >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > > Right now, you must specify which logs you want to send to kafka via > metron-bro-plugin-kafka. This would allow the additional feature of > excluding certain logs, and sending everything else. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron-bro-plugin-kafka issue #2: METRON-1304: Allow metron-bro-plugin-kafka...
Github user JonZeolla commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 Note that this one now depends on apache/metron-bro-plugin-kafka#16 to work properly because I split out the btest bugfix. ---
[jira] [Commented] (METRON-1304) Allow metron-bro-plugin-kafka to include or exclude logs
[ https://issues.apache.org/jira/browse/METRON-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681959#comment-16681959 ] ASF GitHub Bot commented on METRON-1304: Github user JonZeolla commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 Good to go on this one @nickwallen > Allow metron-bro-plugin-kafka to include or exclude logs > > > Key: METRON-1304 > URL: https://issues.apache.org/jira/browse/METRON-1304 > Project: Metron > Issue Type: Improvement >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > > Right now, you must specify which logs you want to send to kafka via > metron-bro-plugin-kafka. This would allow the additional feature of > excluding certain logs, and sending everything else. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron-bro-plugin-kafka issue #2: METRON-1304: Allow metron-bro-plugin-kafka...
Github user JonZeolla commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 Good to go on this one @nickwallen ---
[jira] [Commented] (METRON-1865) Fix metron-bro-plugin-kafka tests
[ https://issues.apache.org/jira/browse/METRON-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681946#comment-16681946 ] ASF GitHub Bot commented on METRON-1865: GitHub user JonZeolla opened a pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/16 METRON-1865: Fix metron-bro-plugin-kafka tests ## Contributor Comments This fixes the btests which were unintentionally broken as a part of apache/metron-bro-plugin-kafka#6. An easy way to test this is to use `bro-pkg` to install the plugin (which runs all of the tests by default), or you can go on a properly configured box (wrt `btest`) and run `cd metron-bro-plugin-kafka/tests/ && btest -d`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/JonZeolla/metron-bro-plugin-kafka METRON-1865 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron-bro-plugin-kafka/pull/16.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #16 commit af396e13be872a14bc89856e33c2395ed53adeef Author: Jon Zeolla Date: 2018-11-09T21:10:03Z METRON-1865: Fix metron-bro-plugin-kafka tests > Fix metron-bro-plugin-kafka tests > - > > Key: METRON-1865 > URL: https://issues.apache.org/jira/browse/METRON-1865 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron-bro-plugin-kafka pull request #16: METRON-1865: Fix metron-bro-plugin...
GitHub user JonZeolla opened a pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/16 METRON-1865: Fix metron-bro-plugin-kafka tests ## Contributor Comments This fixes the btests which were unintentionally broken as a part of apache/metron-bro-plugin-kafka#6. An easy way to test this is to use `bro-pkg` to install the plugin (which runs all of the tests by default), or you can go on a properly configured box (wrt `btest`) and run `cd metron-bro-plugin-kafka/tests/ && btest -d`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/JonZeolla/metron-bro-plugin-kafka METRON-1865 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron-bro-plugin-kafka/pull/16.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #16 commit af396e13be872a14bc89856e33c2395ed53adeef Author: Jon Zeolla Date: 2018-11-09T21:10:03Z METRON-1865: Fix metron-bro-plugin-kafka tests ---
[jira] [Created] (METRON-1865) Fix metron-bro-plugin-kafka tests
Jon Zeolla created METRON-1865: -- Summary: Fix metron-bro-plugin-kafka tests Key: METRON-1865 URL: https://issues.apache.org/jira/browse/METRON-1865 Project: Metron Issue Type: Task Reporter: Jon Zeolla Assignee: Jon Zeolla -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1862) Cant Load Metron Bro Plugin
[ https://issues.apache.org/jira/browse/METRON-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681939#comment-16681939 ] ASF GitHub Bot commented on METRON-1862: Github user JonZeolla commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/15 This should be resolved by apache/metron-bro-plugin-kafka#2 as it currently stands, since it adds the `bro-pkg` example in the documentation and `btest` fix which isn't otherwise obvious. Based on feedback I will be splitting that out into a new PR, but I expect this to be fixed in the near term as a part of an official 0.3 release of the plugin. > Cant Load Metron Bro Plugin > --- > > Key: METRON-1862 > URL: https://issues.apache.org/jira/browse/METRON-1862 > Project: Metron > Issue Type: Bug >Affects Versions: 0.4.1 >Reporter: Mustafa Akmal >Priority: Blocker > Labels: Bro, metron, plugin > > I received the following error when running 'broctl deploy' : can't find > packages/metron-bro-plugin-kafka/Apache/Kafka > This happens when I load the metron bro plugin by making changes to > 'local.bro' by inserting the following line "[@load|https://github.com/load] > packages/metron-bro-plugin-kafka/Apache/Kafka" > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron-bro-plugin-kafka issue #15: METRON-1862: Cant Load Metron Bro Plugin
Github user JonZeolla commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/15 This should be resolved by apache/metron-bro-plugin-kafka#2 as it currently stands, since it adds the `bro-pkg` example in the documentation and `btest` fix which isn't otherwise obvious. Based on feedback I will be splitting that out into a new PR, but I expect this to be fixed in the near term as a part of an official 0.3 release of the plugin. ---
[GitHub] metron-bro-plugin-kafka pull request #15: METRON-1862: Cant Load Metron Bro ...
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron-bro-plugin-kafka/pull/15#discussion_r232298989
--- Diff: README.md ---
@@ -50,7 +51,7 @@ The goal in this example is to send all HTTP and DNS
records to a Kafka topic na
* By defining `topic_name` all records will be sent to the same Kafka
topic.
* Defining `logs_to_send` will ensure that only HTTP and DNS records are
sent.
```
-@load packages/metron-bro-plugin-kafka/Apache/Kafka
+@load
'PathToDirContainingTheDownloadedMetronPlugin'/metron-bro-plugin-kafka/scripts/Apache/Kafka
--- End diff --
These examples are intended to be defined within a location in your Bro
installation like `${BRO_HOME}/share/bro/site/local.bro`. And this has to
occur after you install the plugin in your Bro instance.
You should not be loading from the path to the plugin source code. So
this is incorrect.
---
[jira] [Commented] (METRON-1304) Allow metron-bro-plugin-kafka to include or exclude logs
[ https://issues.apache.org/jira/browse/METRON-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681608#comment-16681608 ] ASF GitHub Bot commented on METRON-1304: Github user nickwallen commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 @JonZeolla Yes, I would definitely appreciate if you could break this into separate, focused PRs. This PR itself has been open since Nov 2017 and it seems like it has been accumulating changes. Would be better to just open a PR when you're ready for a review. You might consider closing some of the other ones that you have open (#12, #8, ?), since those don't seem to be ready for review. It makes it difficult to track what needs reviewed if we have PRs just hanging out there. > Allow metron-bro-plugin-kafka to include or exclude logs > > > Key: METRON-1304 > URL: https://issues.apache.org/jira/browse/METRON-1304 > Project: Metron > Issue Type: Improvement >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > > Right now, you must specify which logs you want to send to kafka via > metron-bro-plugin-kafka. This would allow the additional feature of > excluding certain logs, and sending everything else. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron-bro-plugin-kafka issue #2: METRON-1304: Allow metron-bro-plugin-kafka...
Github user nickwallen commented on the issue: https://github.com/apache/metron-bro-plugin-kafka/pull/2 @JonZeolla Yes, I would definitely appreciate if you could break this into separate, focused PRs. This PR itself has been open since Nov 2017 and it seems like it has been accumulating changes. Would be better to just open a PR when you're ready for a review. You might consider closing some of the other ones that you have open (#12, #8, ?), since those don't seem to be ready for review. It makes it difficult to track what needs reviewed if we have PRs just hanging out there. ---
[jira] [Commented] (METRON-1563) Initial Feature Branch Work
[ https://issues.apache.org/jira/browse/METRON-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681536#comment-16681536 ] ASF GitHub Bot commented on METRON-1563: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1014 @ottobackwards If it's a feature branch, the PR on Github doesn't automatically close. It only happens for merges into a master. It's a bit annoying, but I'm not sure there's a whole lot to do about it > Initial Feature Branch Work > --- > > Key: METRON-1563 > URL: https://issues.apache.org/jira/browse/METRON-1563 > Project: Metron > Issue Type: Sub-task >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1014: METRON-1563 : Base Stellar assign for feature branch
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1014 @ottobackwards If it's a feature branch, the PR on Github doesn't automatically close. It only happens for merges into a master. It's a bit annoying, but I'm not sure there's a whole lot to do about it ---
