[GitHub] [metron-bro-plugin-kafka] JonZeolla opened a new pull request #34: METRON-2045: Pass a version argument to the bro plugin docker scripts
JonZeolla opened a new pull request #34: METRON-2045: Pass a version argument to the bro plugin docker scripts URL: https://github.com/apache/metron-bro-plugin-kafka/pull/34 ## Contributor Comments `bro-pkg` recently released version 1.5.4, and now we can specify commit hashes as versions (in addition to branches and tags). This could be helpful when troubleshooting the plugin at specific points in time, and came up as a part of [METRON-2044](https://issues.apache.org/jira/browse/METRON-2044). ### Testing You will need to rebuild your bro container in order to get the latest version of `bro-pkg`. **Test 1** Make sure it still installs the currently checked out branch by default. ``` $ git checkout METRON-2045 $ cd docker/ $ ./run_end_to_end.sh --skip-docker-build $ ./scripts/docker_execute_shell.sh # cd code/ # git status # On branch METRON-2045 nothing to commit, working directory clean # bro-pkg list /root/code (installed: METRON-2045) - A Bro log writer plugin that sends logging output to Kafka. ``` **Test 2** Make sure it will install a provided commit hash, when provided. ``` $ git checkout METRON-2045 $ cd docker/ $ ./run_end_to_end.sh --skip-docker-build --plugin-version=b360b85e00a8d6b8db9c790c44a767a54c81eb2b $ ./scripts/docker_execute_shell.sh # cd code/ # git status # Expect METRON-2045 is checked out # On branch METRON-2045 nothing to commit, working directory clean # bro-pkg list # bro-pkg should actually install the provided commit hash /root/code (installed: b360b85e00a8d6b8db9c790c44a767a54c81eb2b) - A Bro log writer plugin that sends logging output to Kafka. ``` Don’t forget to `./finish_end_to_end.sh` in between your testing. ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron's Bro kafka writer plugin. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [X] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [X] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [X] Have you included steps or a guide to how the change may be verified and tested manually? - [X] Have you ensured that the full suite of tests and checks have been executed via: ``` ./run_end_to_end.sh ``` - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Apache Metron's [Vagrant full-dev environment](https://github.com/apache/metron/tree/master/metron-deployment/development/centos6) or the equivalent? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [metron] nickwallen commented on issue #1363: IS_EMPTY with empty map should work
nickwallen commented on issue #1363: IS_EMPTY with empty map should work URL: https://github.com/apache/metron/pull/1363#issuecomment-475719997 +1 Thanks for the addition. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (METRON-2044) Error during bro kafka plugin installation
[ https://issues.apache.org/jira/browse/METRON-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799105#comment-16799105 ] Jon Zeolla edited comment on METRON-2044 at 3/22/19 5:46 PM: - Okay [bro-pkg v1.5.4|https://github.com/zeek/package-manager/commit/f4160fc75ce20adafa5a495e0dfcd30d92962816] was recently released which supports specifying commit hashes for the version. This means that if you update your bro-pkg you can now easily take the safest/simplest approach which is: {{bro-pkg install bro/apache/metron-bro-plugin-kafka --version b360b85e00a8d6b8db9c790c44a767a54c81eb2b}} This will fix two issues in the plugin - a segfault when bro shuts down, and the btest issue you were encountering. Does that help in the interim? Note that this does change the recommended version of librdkafka, but the old version should work just fine. was (Author: zeo...@gmail.com): Okay [bro-pkg v1.5.4|https://github.com/zeek/package-manager/commit/f4160fc75ce20adafa5a495e0dfcd30d92962816] has been released which now supports specifying commit hashes for the version. This means that if you update your bro-pkg you can now easily take the safest/simplest approach which is: {{bro-pkg install bro/apache/metron-bro-plugin-kafka --version b360b85e00a8d6b8db9c790c44a767a54c81eb2b}} This will fix two issues in the plugin - a segfault when bro shuts down, and the btest issue you were encountering. Does that help in the interim? Note that this does change the recommended version of librdkafka, but the old version should work just fine. > Error during bro kafka plugin installation > -- > > Key: METRON-2044 > URL: https://issues.apache.org/jira/browse/METRON-2044 > Project: Metron > Issue Type: Bug >Affects Versions: 0.2.1BETA > Environment: metron-bro-plugin-kafka 0.2 > bro version 2.5.4 > Linux 3.13.0-164-generic #214-Ubuntu SMP Wed Dec 5 10:42:33 UTC 2018 x86_64 > x86_64 x86_64 GNU/Linux >Reporter: gimmic >Assignee: Otto Fowler >Priority: Minor > > Running unit tests for "bro/apache/metron-bro-plugin-kafka" > [ 0%] kafka.show-plugin ... failed > % 'btest-diff output' failed unexpectedly (exit code 1) > % cat .diag > == File === > Apache::Kafka - Writes logs to Kafka (dynamic, version 0.2) > [Writer] KafkaWriter (Log::WRITER_KAFKAWRITER) > [Constant] Kafka::kafka_conf > [Constant] Kafka::topic_name > [Constant] Kafka::max_wait_on_shutdown > [Constant] Kafka::tag_json > [Constant] Kafka::json_timestamps > [Constant] Kafka::debug > > == Diff === > --- /tmp/test-diff.15227.output.baseline.tmp 2019-03-20 21:14:47.295246231 > + > +++ /tmp/test-diff.15227.output.tmp 2019-03-20 21:14:47.303246125 + > @@ -4,5 +4,6 @@ > [Constant] Kafka::topic_name > [Constant] Kafka::max_wait_on_shutdown > [Constant] Kafka::tag_json > + [Constant] Kafka::json_timestamps > [Constant] Kafka::debug > > === > % cat .stderr > 1 of 1 test failed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [metron] anandsubbu opened a new pull request #1363: IS_EMPTY with empty map should work
anandsubbu opened a new pull request #1363: IS_EMPTY with empty map should work URL: https://github.com/apache/metron/pull/1363 ## Contributor Comments Problem: https://issues.apache.org/jira/browse/METRON-2046 Solution: `IS_EMPTY` should work for empty map. **Testing** Spin up `stellar` shell and run the following commands. Output should be as below viz. the `IS_EMPTY` function should return `true` ``` [Stellar]>>> val := GEO_GET("10.1.1.1") {} [Stellar]>>> IS_EMPTY( val) true ``` ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [x] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [x] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh ``` - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (METRON-2046) IS_EMPTY stellar functions fails on empty map
[ https://issues.apache.org/jira/browse/METRON-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Subramanian updated METRON-2046: -- Description: The following case is not working as expected: {code:java} [Stellar]>>> val := GEO_GET("10.1.1.1") {} [Stellar]>>> IS_EMPTY( val) false{code} was: The following case is not working as expected: {code:java} [Stellar]>>> val := GEO_GET("10.1.1.1") [Stellar]>>> IS_EMPTY( val) false{code} > IS_EMPTY stellar functions fails on empty map > - > > Key: METRON-2046 > URL: https://issues.apache.org/jira/browse/METRON-2046 > Project: Metron > Issue Type: Bug >Reporter: Anand Subramanian >Assignee: Anand Subramanian >Priority: Major > > The following case is not working as expected: > {code:java} > [Stellar]>>> val := GEO_GET("10.1.1.1") > {} > [Stellar]>>> IS_EMPTY( val) > false{code} > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (METRON-2044) Error during bro kafka plugin installation
[ https://issues.apache.org/jira/browse/METRON-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16798360#comment-16798360 ] Jon Zeolla edited comment on METRON-2044 at 3/22/19 4:30 PM: - Ever since this became a package it has had support of >= 2.5.0 so I wouldn't expect that would be an issue. After the upcoming release I'll be soliciting to make that >= 2.6.0 but we need a clean release before that. My first code block is still the preferred fix for now until we get a new release out. was (Author: zeo...@gmail.com): Ever since I migrated this to be a it has had support of >= 2.5.0 so I wouldn't expect that would be an issue. After the upcoming release I'll be soliciting to make that >= 2.6.0 but we need a clean release before that. My first code block is still the preferred fix for now until we get a new release out. > Error during bro kafka plugin installation > -- > > Key: METRON-2044 > URL: https://issues.apache.org/jira/browse/METRON-2044 > Project: Metron > Issue Type: Bug >Affects Versions: 0.2.1BETA > Environment: metron-bro-plugin-kafka 0.2 > bro version 2.5.4 > Linux 3.13.0-164-generic #214-Ubuntu SMP Wed Dec 5 10:42:33 UTC 2018 x86_64 > x86_64 x86_64 GNU/Linux >Reporter: gimmic >Assignee: Otto Fowler >Priority: Minor > > Running unit tests for "bro/apache/metron-bro-plugin-kafka" > [ 0%] kafka.show-plugin ... failed > % 'btest-diff output' failed unexpectedly (exit code 1) > % cat .diag > == File === > Apache::Kafka - Writes logs to Kafka (dynamic, version 0.2) > [Writer] KafkaWriter (Log::WRITER_KAFKAWRITER) > [Constant] Kafka::kafka_conf > [Constant] Kafka::topic_name > [Constant] Kafka::max_wait_on_shutdown > [Constant] Kafka::tag_json > [Constant] Kafka::json_timestamps > [Constant] Kafka::debug > > == Diff === > --- /tmp/test-diff.15227.output.baseline.tmp 2019-03-20 21:14:47.295246231 > + > +++ /tmp/test-diff.15227.output.tmp 2019-03-20 21:14:47.303246125 + > @@ -4,5 +4,6 @@ > [Constant] Kafka::topic_name > [Constant] Kafka::max_wait_on_shutdown > [Constant] Kafka::tag_json > + [Constant] Kafka::json_timestamps > [Constant] Kafka::debug > > === > % cat .stderr > 1 of 1 test failed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-2046) IS_EMPTY stellar functions fails on empty map
Anand Subramanian created METRON-2046: - Summary: IS_EMPTY stellar functions fails on empty map Key: METRON-2046 URL: https://issues.apache.org/jira/browse/METRON-2046 Project: Metron Issue Type: Bug Reporter: Anand Subramanian Assignee: Anand Subramanian The following case is not working as expected: {code:java} [Stellar]>>> val := GEO_GET("10.1.1.1") [Stellar]>>> IS_EMPTY( val) false{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-2045) Pass a version argument to the bro plugin docker scripts
Jon Zeolla created METRON-2045: -- Summary: Pass a version argument to the bro plugin docker scripts Key: METRON-2045 URL: https://issues.apache.org/jira/browse/METRON-2045 Project: Metron Issue Type: Bug Reporter: Jon Zeolla Assignee: Jon Zeolla As of bro-pkg 1.5.4 we can now specify commit hashes as versions (in addition to branches and tags). This could be helpful when working with users to troubleshoot. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (METRON-2045) Pass a version argument to the bro plugin docker scripts
[ https://issues.apache.org/jira/browse/METRON-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Zeolla updated METRON-2045: --- Issue Type: Improvement (was: Bug) > Pass a version argument to the bro plugin docker scripts > > > Key: METRON-2045 > URL: https://issues.apache.org/jira/browse/METRON-2045 > Project: Metron > Issue Type: Improvement >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > > As of bro-pkg 1.5.4 we can now specify commit hashes as versions (in addition > to branches and tags). This could be helpful when working with users to > troubleshoot. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-2044) Error during bro kafka plugin installation
[ https://issues.apache.org/jira/browse/METRON-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799105#comment-16799105 ] Jon Zeolla commented on METRON-2044: Okay [bro-pkg v1.5.4|https://github.com/zeek/package-manager/commit/f4160fc75ce20adafa5a495e0dfcd30d92962816] has been released which now supports specifying commit hashes for the version. This means that if you update your bro-pkg you can now easily take the safest/simplest approach which is: {{bro-pkg install bro/apache/metron-bro-plugin-kafka --version b360b85e00a8d6b8db9c790c44a767a54c81eb2b}} This will fix two issues in the plugin - a segfault when bro shut down, and the btest issue you were encountering. Does that help in the interim? Note that this does change the recommended version of librdkafka, but the old version should work just fine. > Error during bro kafka plugin installation > -- > > Key: METRON-2044 > URL: https://issues.apache.org/jira/browse/METRON-2044 > Project: Metron > Issue Type: Bug >Affects Versions: 0.2.1BETA > Environment: metron-bro-plugin-kafka 0.2 > bro version 2.5.4 > Linux 3.13.0-164-generic #214-Ubuntu SMP Wed Dec 5 10:42:33 UTC 2018 x86_64 > x86_64 x86_64 GNU/Linux >Reporter: gimmic >Assignee: Otto Fowler >Priority: Minor > > Running unit tests for "bro/apache/metron-bro-plugin-kafka" > [ 0%] kafka.show-plugin ... failed > % 'btest-diff output' failed unexpectedly (exit code 1) > % cat .diag > == File === > Apache::Kafka - Writes logs to Kafka (dynamic, version 0.2) > [Writer] KafkaWriter (Log::WRITER_KAFKAWRITER) > [Constant] Kafka::kafka_conf > [Constant] Kafka::topic_name > [Constant] Kafka::max_wait_on_shutdown > [Constant] Kafka::tag_json > [Constant] Kafka::json_timestamps > [Constant] Kafka::debug > > == Diff === > --- /tmp/test-diff.15227.output.baseline.tmp 2019-03-20 21:14:47.295246231 > + > +++ /tmp/test-diff.15227.output.tmp 2019-03-20 21:14:47.303246125 + > @@ -4,5 +4,6 @@ > [Constant] Kafka::topic_name > [Constant] Kafka::max_wait_on_shutdown > [Constant] Kafka::tag_json > + [Constant] Kafka::json_timestamps > [Constant] Kafka::debug > > === > % cat .stderr > 1 of 1 test failed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (METRON-2044) Error during bro kafka plugin installation
[ https://issues.apache.org/jira/browse/METRON-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799105#comment-16799105 ] Jon Zeolla edited comment on METRON-2044 at 3/22/19 3:22 PM: - Okay [bro-pkg v1.5.4|https://github.com/zeek/package-manager/commit/f4160fc75ce20adafa5a495e0dfcd30d92962816] has been released which now supports specifying commit hashes for the version. This means that if you update your bro-pkg you can now easily take the safest/simplest approach which is: {{bro-pkg install bro/apache/metron-bro-plugin-kafka --version b360b85e00a8d6b8db9c790c44a767a54c81eb2b}} This will fix two issues in the plugin - a segfault when bro shuts down, and the btest issue you were encountering. Does that help in the interim? Note that this does change the recommended version of librdkafka, but the old version should work just fine. was (Author: zeo...@gmail.com): Okay [bro-pkg v1.5.4|https://github.com/zeek/package-manager/commit/f4160fc75ce20adafa5a495e0dfcd30d92962816] has been released which now supports specifying commit hashes for the version. This means that if you update your bro-pkg you can now easily take the safest/simplest approach which is: {{bro-pkg install bro/apache/metron-bro-plugin-kafka --version b360b85e00a8d6b8db9c790c44a767a54c81eb2b}} This will fix two issues in the plugin - a segfault when bro shut down, and the btest issue you were encountering. Does that help in the interim? Note that this does change the recommended version of librdkafka, but the old version should work just fine. > Error during bro kafka plugin installation > -- > > Key: METRON-2044 > URL: https://issues.apache.org/jira/browse/METRON-2044 > Project: Metron > Issue Type: Bug >Affects Versions: 0.2.1BETA > Environment: metron-bro-plugin-kafka 0.2 > bro version 2.5.4 > Linux 3.13.0-164-generic #214-Ubuntu SMP Wed Dec 5 10:42:33 UTC 2018 x86_64 > x86_64 x86_64 GNU/Linux >Reporter: gimmic >Assignee: Otto Fowler >Priority: Minor > > Running unit tests for "bro/apache/metron-bro-plugin-kafka" > [ 0%] kafka.show-plugin ... failed > % 'btest-diff output' failed unexpectedly (exit code 1) > % cat .diag > == File === > Apache::Kafka - Writes logs to Kafka (dynamic, version 0.2) > [Writer] KafkaWriter (Log::WRITER_KAFKAWRITER) > [Constant] Kafka::kafka_conf > [Constant] Kafka::topic_name > [Constant] Kafka::max_wait_on_shutdown > [Constant] Kafka::tag_json > [Constant] Kafka::json_timestamps > [Constant] Kafka::debug > > == Diff === > --- /tmp/test-diff.15227.output.baseline.tmp 2019-03-20 21:14:47.295246231 > + > +++ /tmp/test-diff.15227.output.tmp 2019-03-20 21:14:47.303246125 + > @@ -4,5 +4,6 @@ > [Constant] Kafka::topic_name > [Constant] Kafka::max_wait_on_shutdown > [Constant] Kafka::tag_json > + [Constant] Kafka::json_timestamps > [Constant] Kafka::debug > > === > % cat .stderr > 1 of 1 test failed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [metron] asfgit closed pull request #1356: METRON-2029: Configure Table should have filter
asfgit closed pull request #1356: METRON-2029: Configure Table should have filter URL: https://github.com/apache/metron/pull/1356 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [metron] asfgit closed pull request #1354: METRON-2032: Create summary table having list of stellar functions in README
asfgit closed pull request #1354: METRON-2032: Create summary table having list of stellar functions in README URL: https://github.com/apache/metron/pull/1354 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services