[jira] [Commented] (METRON-1665) Move hosting of Alerts and Config UIs from Nodejs to Spring Boot
[ https://issues.apache.org/jira/browse/METRON-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595959#comment-16595959 ] ASF GitHub Bot commented on METRON-1665: Github user simonellistonball closed the pull request at: https://github.com/apache/metron/pull/ > Move hosting of Alerts and Config UIs from Nodejs to Spring Boot > > > Key: METRON-1665 > URL: https://issues.apache.org/jira/browse/METRON-1665 > Project: Metron > Issue Type: Sub-task >Reporter: Simon Elliston Ball >Assignee: Simon Elliston Ball >Priority: Major > > The current UIs are served up by very lightweight nodejs applications, which > serve the static bundle files produced by the angular build process, and > proxies the rest api. > The proposal is to use a spring boot application, allowing us to harmonise > the security implementation across the UI static servers and the REST layer, > and to provide a routing platform for later microservices. > The UIs currently proxy to the REST API to avoid CORS issues, this will be > achieved with Zuul. > Spring Security will also be extended to use a Knox SSO authenticator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball closed the pull request at: https://github.com/apache/metron/pull/ ---
[jira] [Commented] (METRON-1757) Storm Profiler Serialization Exception
[
https://issues.apache.org/jira/browse/METRON-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595807#comment-16595807
]
ASF GitHub Bot commented on METRON-1757:
Github user JonZeolla commented on the issue:
https://github.com/apache/metron/pull/1178
While you're at it, what do you think about making a tweak to the related
instructions? Updating
[this](https://github.com/apache/metron/tree/master/metron-analytics/metron-profiler#deploying-profiles-with-the-stellar-shell)
to reference `source /etc/default/metron` and then use `$ZOOKEEPER` instead of
`ZK:2181` so the steps more copy/pastable.
> Storm Profiler Serialization Exception
> --
>
> Key: METRON-1757
> URL: https://issues.apache.org/jira/browse/METRON-1757
> Project: Metron
> Issue Type: Bug
>Reporter: Nick Allen
>Assignee: Nick Allen
>Priority: Major
>
> When running the Storm Profiler with multiple workers this serialization
> error can occur.
> Even in an environment with multiple workers It seems to not occur all the
> time. It may depend on how the individual bolts are executed across the
> workers in a cluster. This likely occurs only when the HBaseBolt is executed
> on a separate worker from the ProfileBuilderBolt.
> It will never occur when running with a single worker.
> {code}
> 2018-08-28 15:24:10.002 o.a.s.m.n.StormServerHandler
> Netty-server-localhost-6700-worker-1 [ERROR] server errors in handling the
> request
> com.esotericsoftware.kryo.KryoException: Class cannot be created (missing
> no-arg constructor):
> org.apache.metron.common.configuration.profiler.ProfileResult
> Serialization trace:
> result (org.apache.metron.common.configuration.profiler.ProfileConfig)
> definition (org.apache.metron.profiler.ProfileMeasurement)
> at
> com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInstantiatorOf(Kryo.java:1272)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1078)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1087)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldSerializer.java:570)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:546)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:689)
> ~[kryo-3.0.3.jar:?]
> at
> org.apache.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:37)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.serialization.KryoTupleDeserializer.deserialize(KryoTupleDeserializer.java:50)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.messaging.DeserializingConnectionCallback.recv(DeserializingConnectionCallback.java:56)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at org.apache.storm.messaging.netty.Server.enqueue(Server.java:133)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at org.apache.storm.messaging.netty.Server.received(Server.java:254)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.messaging.netty.StormServerHandler.messageReceived(StormServerHandler.java:61)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
>
[GitHub] metron issue #1178: METRON-1757 Storm Profiler Serialization Exception
Github user JonZeolla commented on the issue: https://github.com/apache/metron/pull/1178 While you're at it, what do you think about making a tweak to the related instructions? Updating [this](https://github.com/apache/metron/tree/master/metron-analytics/metron-profiler#deploying-profiles-with-the-stellar-shell) to reference `source /etc/default/metron` and then use `$ZOOKEEPER` instead of `ZK:2181` so the steps more copy/pastable. ---
[GitHub] metron pull request #1179: METRON-1758: Add support for Ansible 2.6 in dev
GitHub user JonZeolla opened a pull request: https://github.com/apache/metron/pull/1179 METRON-1758: Add support for Ansible 2.6 in dev ## Contributor Comments Spin up full-dev with any version of ansible >= 2.2.2.0. ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [X] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [X] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh ``` - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/JonZeolla/metron METRON-1758 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/1179.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1179 commit a749b0a5a97e2fa0fc6f8f632a67f43ef0986124 Author: Jon Zeolla Date: 2018-08-29T00:09:22Z METRON-1758: Add support for Ansible 2.6 in dev ---
[jira] [Commented] (METRON-1758) Add support for Ansible 2.6 in dev
[ https://issues.apache.org/jira/browse/METRON-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595771#comment-16595771 ] ASF GitHub Bot commented on METRON-1758: GitHub user JonZeolla opened a pull request: https://github.com/apache/metron/pull/1179 METRON-1758: Add support for Ansible 2.6 in dev ## Contributor Comments Spin up full-dev with any version of ansible >= 2.2.2.0. ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [X] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [X] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh ``` - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/JonZeolla/metron METRON-1758 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/1179.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1179 commit a749b0a5a97e2fa0fc6f8f632a67f43ef0986124 Author: Jon Zeolla Date: 2018-08-29T00:09:22Z METRON-1758: Add support for Ansible 2.6 in dev > Add support for Ansible 2.6 in dev > -- > > Key: METRON-1758 > URL: https://issues.apache.org/jira/browse/METRON-1758 > Project: Metron > Issue Type: Improvement >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Major > > Add support for Ansible 2.6 in dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1758) Add support for Ansible 2.6 in dev
Jon Zeolla created METRON-1758: -- Summary: Add support for Ansible 2.6 in dev Key: METRON-1758 URL: https://issues.apache.org/jira/browse/METRON-1758 Project: Metron Issue Type: Improvement Reporter: Jon Zeolla Assignee: Jon Zeolla Add support for Ansible 2.6 in dev -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1715) Create DEB Packaging for Batch Profiler
[ https://issues.apache.org/jira/browse/METRON-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595730#comment-16595730 ] ASF GitHub Bot commented on METRON-1715: Github user nickwallen closed the pull request at: https://github.com/apache/metron/pull/1167 > Create DEB Packaging for Batch Profiler > --- > > Key: METRON-1715 > URL: https://issues.apache.org/jira/browse/METRON-1715 > Project: Metron > Issue Type: Sub-task >Reporter: Nick Allen >Assignee: Nick Allen >Priority: Major > > Create DEB packaging for the Batch Profiler -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1715) Create DEB Packaging for Batch Profiler
[ https://issues.apache.org/jira/browse/METRON-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595729#comment-16595729 ] ASF GitHub Bot commented on METRON-1715: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1167 Thanks again @merrimanr I have merged this PR into the feature branch. > Create DEB Packaging for Batch Profiler > --- > > Key: METRON-1715 > URL: https://issues.apache.org/jira/browse/METRON-1715 > Project: Metron > Issue Type: Sub-task >Reporter: Nick Allen >Assignee: Nick Allen >Priority: Major > > Create DEB packaging for the Batch Profiler -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1167: METRON-1715 Create DEB Packaging for Batch Profiler [Fea...
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1167 Thanks again @merrimanr I have merged this PR into the feature branch. ---
[GitHub] metron pull request #1167: METRON-1715 Create DEB Packaging for Batch Profil...
Github user nickwallen closed the pull request at: https://github.com/apache/metron/pull/1167 ---
[jira] [Commented] (METRON-1715) Create DEB Packaging for Batch Profiler
[ https://issues.apache.org/jira/browse/METRON-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595613#comment-16595613 ] ASF GitHub Bot commented on METRON-1715: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/1167 I ran through the testing instructions and everything worked fine. +1 > Create DEB Packaging for Batch Profiler > --- > > Key: METRON-1715 > URL: https://issues.apache.org/jira/browse/METRON-1715 > Project: Metron > Issue Type: Sub-task >Reporter: Nick Allen >Assignee: Nick Allen >Priority: Major > > Create DEB packaging for the Batch Profiler -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1167: METRON-1715 Create DEB Packaging for Batch Profiler [Fea...
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/1167 I ran through the testing instructions and everything worked fine. +1 ---
[jira] [Commented] (METRON-1736) Enhance Batch Profiler Integration Test
[ https://issues.apache.org/jira/browse/METRON-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595570#comment-16595570 ] ASF GitHub Bot commented on METRON-1736: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1162 Thanks for the review. This has been merged. > Enhance Batch Profiler Integration Test > --- > > Key: METRON-1736 > URL: https://issues.apache.org/jira/browse/METRON-1736 > Project: Metron > Issue Type: Sub-task >Reporter: Nick Allen >Assignee: Nick Allen >Priority: Major > > The integration test for the Batch Profiler should use the Profiler Client > API and `PROFILE_GET` to validate the values that are produced. This is more > effective end-to-end validation that the Batch Profiler is working. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1736) Enhance Batch Profiler Integration Test
[ https://issues.apache.org/jira/browse/METRON-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595571#comment-16595571 ] ASF GitHub Bot commented on METRON-1736: Github user nickwallen closed the pull request at: https://github.com/apache/metron/pull/1162 > Enhance Batch Profiler Integration Test > --- > > Key: METRON-1736 > URL: https://issues.apache.org/jira/browse/METRON-1736 > Project: Metron > Issue Type: Sub-task >Reporter: Nick Allen >Assignee: Nick Allen >Priority: Major > > The integration test for the Batch Profiler should use the Profiler Client > API and `PROFILE_GET` to validate the values that are produced. This is more > effective end-to-end validation that the Batch Profiler is working. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1162: METRON-1736 Enhance Batch Profiler Integration Test [Fea...
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1162 Thanks for the review. This has been merged. ---
[GitHub] metron pull request #1162: METRON-1736 Enhance Batch Profiler Integration Te...
Github user nickwallen closed the pull request at: https://github.com/apache/metron/pull/1162 ---
[jira] [Commented] (METRON-1757) Storm Profiler Serialization Exception
[ https://issues.apache.org/jira/browse/METRON-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595553#comment-16595553 ] ASF GitHub Bot commented on METRON-1757: GitHub user nickwallen opened a pull request: https://github.com/apache/metron/pull/1178 METRON-1757 Storm Profiler Serialization Exception ### Description * When using Kryo serialization, specifically in Storm, the class must have a no-arg constructor. * The existing unit tests that validated Kryo serialization for the Profiler classes, used the SerDeUtils class to perform Kryo serialization. This class sets up Kryo in a slightly different way from how Storm does internally. This made the tests not catch the missing no-arg constructor. * The `ProfileMeasurementTest`'s setup method did not have the correct annotation so it was never called. This caused the test to validate serialization of a null, rather than an actual object. * Removed the 'asm' dependency from 'metron-profiler-common'. Conflicting version of this library was causing issues with the Kryo serialization tests. There should be no dependency to Storm from `metron-profiler-common`. ### Work Around If you are experiencing this issue, you can work around the problem by changing the following values in your `profiler.properties`. This will cause Storm to fall-back to Java serialization when Kryo serialization fails in Storm for `org.apache.metron.common.configuration.profiler.ProfileResult`. ``` topology.fall.back.on.java.serialization=true topology.kryo.register=[ org.apache.metron.profiler.ProfileMeasurement, \ org.apache.metron.profiler.ProfilePeriod, \ org.apache.metron.common.configuration.profiler.ProfileResultExpressions, \ org.apache.metron.common.configuration.profiler.ProfileTriageExpressions, \ org.apache.metron.common.configuration.profiler.ProfilerConfig, \ org.apache.metron.common.configuration.profiler.ProfileConfig, \ org.json.simple.JSONObject, \ org.json.simple.JSONArray, \ java.util.LinkedHashMap, \ org.apache.metron.statistics.OnlineStatisticsProvider ] ``` ### Testing 1. Spin-up Full Dev 1. Create a profile by following the Profiler README as a guide. 1. Spin-up this change on a multi-node cluster. Run the profiler with multiple workers. Create the same profile as before and ensure that values can be persisted in HBase. ## Pull Request Checklist - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [ ] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [ ] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? You can merge this pull request into a Git repository by running: $ git pull https://github.com/nickwallen/metron METRON-1757 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/1178.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1178 commit 457aa3cf73506901a2b43ffcb2674bfacd2dd635 Author: Nick Allen Date: 2018-08-28T18:12:01Z metron-profiler-common should not depend on Storm commit 3098270599d1f1cb24211fc981c1f116da2c85e4 Author: Nick Allen Date: 2018-08-28T18:14:27Z Serialization tests were not working as measurement always null. Also had to exclude asm for tests to pass commit d2d40bf0f7e9a9f7073126bab78b3cd001b40458 Author: Nick Allen Date: 2018-08-28T19:47:13Z Updated tests to catch bug with Kryo serialization commit eccd79908c980b211a9c39848c1977de5be0b51b Author: Nick Allen Date: 2018-08-28T19:52:56Z Added default constructor which is required by Kryo
[GitHub] metron pull request #1178: METRON-1757 Storm Profiler Serialization Exceptio...
GitHub user nickwallen opened a pull request: https://github.com/apache/metron/pull/1178 METRON-1757 Storm Profiler Serialization Exception ### Description * When using Kryo serialization, specifically in Storm, the class must have a no-arg constructor. * The existing unit tests that validated Kryo serialization for the Profiler classes, used the SerDeUtils class to perform Kryo serialization. This class sets up Kryo in a slightly different way from how Storm does internally. This made the tests not catch the missing no-arg constructor. * The `ProfileMeasurementTest`'s setup method did not have the correct annotation so it was never called. This caused the test to validate serialization of a null, rather than an actual object. * Removed the 'asm' dependency from 'metron-profiler-common'. Conflicting version of this library was causing issues with the Kryo serialization tests. There should be no dependency to Storm from `metron-profiler-common`. ### Work Around If you are experiencing this issue, you can work around the problem by changing the following values in your `profiler.properties`. This will cause Storm to fall-back to Java serialization when Kryo serialization fails in Storm for `org.apache.metron.common.configuration.profiler.ProfileResult`. ``` topology.fall.back.on.java.serialization=true topology.kryo.register=[ org.apache.metron.profiler.ProfileMeasurement, \ org.apache.metron.profiler.ProfilePeriod, \ org.apache.metron.common.configuration.profiler.ProfileResultExpressions, \ org.apache.metron.common.configuration.profiler.ProfileTriageExpressions, \ org.apache.metron.common.configuration.profiler.ProfilerConfig, \ org.apache.metron.common.configuration.profiler.ProfileConfig, \ org.json.simple.JSONObject, \ org.json.simple.JSONArray, \ java.util.LinkedHashMap, \ org.apache.metron.statistics.OnlineStatisticsProvider ] ``` ### Testing 1. Spin-up Full Dev 1. Create a profile by following the Profiler README as a guide. 1. Spin-up this change on a multi-node cluster. Run the profiler with multiple workers. Create the same profile as before and ensure that values can be persisted in HBase. ## Pull Request Checklist - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [ ] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [ ] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? You can merge this pull request into a Git repository by running: $ git pull https://github.com/nickwallen/metron METRON-1757 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/1178.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1178 commit 457aa3cf73506901a2b43ffcb2674bfacd2dd635 Author: Nick Allen Date: 2018-08-28T18:12:01Z metron-profiler-common should not depend on Storm commit 3098270599d1f1cb24211fc981c1f116da2c85e4 Author: Nick Allen Date: 2018-08-28T18:14:27Z Serialization tests were not working as measurement always null. Also had to exclude asm for tests to pass commit d2d40bf0f7e9a9f7073126bab78b3cd001b40458 Author: Nick Allen Date: 2018-08-28T19:47:13Z Updated tests to catch bug with Kryo serialization commit eccd79908c980b211a9c39848c1977de5be0b51b Author: Nick Allen Date: 2018-08-28T19:52:56Z Added default constructor which is required by Kryo commit a9ceb97b6d2167392225ee8768c69b33f788cff8 Author: Nick Allen Date: 2018-08-28T20:06:10Z Corrected comment ---
[jira] [Created] (METRON-1757) Storm Profiler Serialization Exception
Nick Allen created METRON-1757:
--
Summary: Storm Profiler Serialization Exception
Key: METRON-1757
URL: https://issues.apache.org/jira/browse/METRON-1757
Project: Metron
Issue Type: Bug
Reporter: Nick Allen
Assignee: Nick Allen
When running the Storm Profiler with multiple workers this serialization error
can occur.
Even in an environment with multiple workers It seems to not occur all the
time. It may depend on how the individual bolts are executed across the workers
in a cluster. This likely occurs only when the HBaseBolt is executed on a
separate worker from the ProfileBuilderBolt.
It will never occur when running with a single worker.
{code}
2018-08-28 15:24:10.002 o.a.s.m.n.StormServerHandler
Netty-server-localhost-6700-worker-1 [ERROR] server errors in handling the
request
com.esotericsoftware.kryo.KryoException: Class cannot be created (missing
no-arg constructor):
org.apache.metron.common.configuration.profiler.ProfileResult
Serialization trace:
result (org.apache.metron.common.configuration.profiler.ProfileConfig)
definition (org.apache.metron.profiler.ProfileMeasurement)
at
com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInstantiatorOf(Kryo.java:1272)
~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1078)
~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1087)
~[kryo-3.0.3.jar:?]
at
com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldSerializer.java:570)
~[kryo-3.0.3.jar:?]
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:546)
~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711) ~[kryo-3.0.3.jar:?]
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
~[kryo-3.0.3.jar:?]
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711) ~[kryo-3.0.3.jar:?]
at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
~[kryo-3.0.3.jar:?]
at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
~[kryo-3.0.3.jar:?]
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
~[kryo-3.0.3.jar:?]
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:689) ~[kryo-3.0.3.jar:?]
at
org.apache.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:37)
~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at
org.apache.storm.serialization.KryoTupleDeserializer.deserialize(KryoTupleDeserializer.java:50)
~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at
org.apache.storm.messaging.DeserializingConnectionCallback.recv(DeserializingConnectionCallback.java:56)
~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at org.apache.storm.messaging.netty.Server.enqueue(Server.java:133)
~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at org.apache.storm.messaging.netty.Server.received(Server.java:254)
~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at
org.apache.storm.messaging.netty.StormServerHandler.messageReceived(StormServerHandler.java:61)
~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at
org.apache.storm.shade.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at
org.apache.storm.shade.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at
org.apache.storm.shade.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at
org.apache.storm.shade.org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at
org.apache.storm.shade.org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at
org.apache.storm.shade.org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at
org.apache.storm.shade.org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:310)
[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
at
[jira] [Commented] (METRON-1736) Enhance Batch Profiler Integration Test
[ https://issues.apache.org/jira/browse/METRON-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595501#comment-16595501 ] ASF GitHub Bot commented on METRON-1736: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/1162 I feel like this makes it easier to understand what this is testing. Nice improvement! +1 > Enhance Batch Profiler Integration Test > --- > > Key: METRON-1736 > URL: https://issues.apache.org/jira/browse/METRON-1736 > Project: Metron > Issue Type: Sub-task >Reporter: Nick Allen >Assignee: Nick Allen >Priority: Major > > The integration test for the Batch Profiler should use the Profiler Client > API and `PROFILE_GET` to validate the values that are produced. This is more > effective end-to-end validation that the Batch Profiler is working. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1162: METRON-1736 Enhance Batch Profiler Integration Test [Fea...
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/1162 I feel like this makes it easier to understand what this is testing. Nice improvement! +1 ---
[jira] [Commented] (METRON-1665) Move hosting of Alerts and Config UIs from Nodejs to Spring Boot
[ https://issues.apache.org/jira/browse/METRON-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595485#comment-16595485 ] ASF GitHub Bot commented on METRON-1665: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/ This has been merged into the feature branch. Can you close @simonellistonball? > Move hosting of Alerts and Config UIs from Nodejs to Spring Boot > > > Key: METRON-1665 > URL: https://issues.apache.org/jira/browse/METRON-1665 > Project: Metron > Issue Type: Sub-task >Reporter: Simon Elliston Ball >Assignee: Simon Elliston Ball >Priority: Major > > The current UIs are served up by very lightweight nodejs applications, which > serve the static bundle files produced by the angular build process, and > proxies the rest api. > The proposal is to use a spring boot application, allowing us to harmonise > the security implementation across the UI static servers and the REST layer, > and to provide a routing platform for later microservices. > The UIs currently proxy to the REST API to avoid CORS issues, this will be > achieved with Zuul. > Spring Security will also be extended to use a Knox SSO authenticator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1111: METRON-1665 Host UIs on Spring Boot and add SSO authenti...
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/ This has been merged into the feature branch. Can you close @simonellistonball? ---
[jira] [Created] (METRON-1756) REST tests should use Embedded LDAP in metron-security
Ryan Merriman created METRON-1756: - Summary: REST tests should use Embedded LDAP in metron-security Key: METRON-1756 URL: https://issues.apache.org/jira/browse/METRON-1756 Project: Metron Issue Type: Sub-task Reporter: Ryan Merriman Tests in metron-rest are still using JDBC-based authentication. These tests (and all security-related tests) should use the testing infrastructure introduced in METRON-1665. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (METRON-1755) Create an architecture document
Ryan Merriman created METRON-1755: - Summary: Create an architecture document Key: METRON-1755 URL: https://issues.apache.org/jira/browse/METRON-1755 Project: Metron Issue Type: Sub-task Reporter: Ryan Merriman Since this is such a significant shift to our security strategy, we should provide some kind of architectural documentation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1665) Move hosting of Alerts and Config UIs from Nodejs to Spring Boot
[ https://issues.apache.org/jira/browse/METRON-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595459#comment-16595459 ] ASF GitHub Bot commented on METRON-1665: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/ Thanks @simonellistonball. I think this is good enough to start with. +1 > Move hosting of Alerts and Config UIs from Nodejs to Spring Boot > > > Key: METRON-1665 > URL: https://issues.apache.org/jira/browse/METRON-1665 > Project: Metron > Issue Type: Sub-task >Reporter: Simon Elliston Ball >Assignee: Simon Elliston Ball >Priority: Major > > The current UIs are served up by very lightweight nodejs applications, which > serve the static bundle files produced by the angular build process, and > proxies the rest api. > The proposal is to use a spring boot application, allowing us to harmonise > the security implementation across the UI static servers and the REST layer, > and to provide a routing platform for later microservices. > The UIs currently proxy to the REST API to avoid CORS issues, this will be > achieved with Zuul. > Spring Security will also be extended to use a Knox SSO authenticator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1111: METRON-1665 Host UIs on Spring Boot and add SSO authenti...
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/ Thanks @simonellistonball. I think this is good enough to start with. +1 ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/#discussion_r213434094
--- Diff:
metron-interface/metron-alerts-host/src/main/scripts/metron-alerts.sh ---
@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+if [ -z "${METRON_SSL_PASSWORD}" ]; then
+echo "METRON_SSL_PASSWORD unset."
+fi
+
+METRON_VERSION=${project.version}
+METRON_HOME="${METRON_HOME:-/usr/metron/${METRON_VERSION}}"
+METRON_SYSCONFIG="${METRON_SYSCONFIG:-/etc/default/metron}"
+
+echo "METRON_VERSION=${METRON_VERSION}"
+echo "METRON_HOME=${METRON_HOME}"
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+
+if [ -f "$METRON_SYSCONFIG" ]; then
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+set -a
+. "$METRON_SYSCONFIG"
+fi
+
+echo "METRON_SPRING_PROFILES_ACTIVE=${METRON_SPRING_PROFILES_ACTIVE}"
+
+METRON_CONFIG_LOCATION="
--spring.config.location=classpath:/application.yml,$METRON_HOME/config/alerts_ui.yml"
+echo "METRON_CONFIG_LOCATION=${METRON_CONFIG_LOCATION}"
+METRON_SPRING_OPTIONS+=${METRON_CONFIG_LOCATION}
--- End diff --
Sure that's fine.
---
[jira] [Commented] (METRON-1665) Move hosting of Alerts and Config UIs from Nodejs to Spring Boot
[
https://issues.apache.org/jira/browse/METRON-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595442#comment-16595442
]
ASF GitHub Bot commented on METRON-1665:
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/#discussion_r213434094
--- Diff:
metron-interface/metron-alerts-host/src/main/scripts/metron-alerts.sh ---
@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+if [ -z "${METRON_SSL_PASSWORD}" ]; then
+echo "METRON_SSL_PASSWORD unset."
+fi
+
+METRON_VERSION=${project.version}
+METRON_HOME="${METRON_HOME:-/usr/metron/${METRON_VERSION}}"
+METRON_SYSCONFIG="${METRON_SYSCONFIG:-/etc/default/metron}"
+
+echo "METRON_VERSION=${METRON_VERSION}"
+echo "METRON_HOME=${METRON_HOME}"
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+
+if [ -f "$METRON_SYSCONFIG" ]; then
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+set -a
+. "$METRON_SYSCONFIG"
+fi
+
+echo "METRON_SPRING_PROFILES_ACTIVE=${METRON_SPRING_PROFILES_ACTIVE}"
+
+METRON_CONFIG_LOCATION="
--spring.config.location=classpath:/application.yml,$METRON_HOME/config/alerts_ui.yml"
+echo "METRON_CONFIG_LOCATION=${METRON_CONFIG_LOCATION}"
+METRON_SPRING_OPTIONS+=${METRON_CONFIG_LOCATION}
--- End diff --
Sure that's fine.
> Move hosting of Alerts and Config UIs from Nodejs to Spring Boot
>
>
> Key: METRON-1665
> URL: https://issues.apache.org/jira/browse/METRON-1665
> Project: Metron
> Issue Type: Sub-task
>Reporter: Simon Elliston Ball
>Assignee: Simon Elliston Ball
>Priority: Major
>
> The current UIs are served up by very lightweight nodejs applications, which
> serve the static bundle files produced by the angular build process, and
> proxies the rest api.
> The proposal is to use a spring boot application, allowing us to harmonise
> the security implementation across the UI static servers and the REST layer,
> and to provide a routing platform for later microservices.
> The UIs currently proxy to the REST API to avoid CORS issues, this will be
> achieved with Zuul.
> Spring Security will also be extended to use a Knox SSO authenticator.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (METRON-1476) Update angular
[
https://issues.apache.org/jira/browse/METRON-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595160#comment-16595160
]
ASF GitHub Bot commented on METRON-1476:
Github user sardell commented on a diff in the pull request:
https://github.com/apache/metron/pull/1096#discussion_r213364339
--- Diff:
metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.spec.ts
---
@@ -42,17 +42,18 @@ describe('TableViewComponent', () => {
let fixture: ComponentFixture;
beforeEach(async(() => {
-// FIXME: mock all the unnecessary dependencies
+// FIXME: mock all the unnecessary dependencies
TestBed.configureTestingModule({
- imports: [ HttpModule ],
+ imports: [ HttpClientModule ],
providers: [
SearchService,
UpdateService,
GlobalConfigService,
MetaAlertService,
MetronDialogBox,
+HttpClient,
--- End diff --
You're right, `HttpClientModule` should be enough here. We should remove
this line and the import. Thanks!
> Update angular
> --
>
> Key: METRON-1476
> URL: https://issues.apache.org/jira/browse/METRON-1476
> Project: Metron
> Issue Type: Improvement
>Reporter: Daniel Toth
>Assignee: Daniel Toth
>Priority: Major
>
> Update angular to speed up development
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] metron pull request #1096: METRON-1476: Update angular
Github user sardell commented on a diff in the pull request:
https://github.com/apache/metron/pull/1096#discussion_r213364339
--- Diff:
metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.spec.ts
---
@@ -42,17 +42,18 @@ describe('TableViewComponent', () => {
let fixture: ComponentFixture;
beforeEach(async(() => {
-// FIXME: mock all the unnecessary dependencies
+// FIXME: mock all the unnecessary dependencies
TestBed.configureTestingModule({
- imports: [ HttpModule ],
+ imports: [ HttpClientModule ],
providers: [
SearchService,
UpdateService,
GlobalConfigService,
MetaAlertService,
MetronDialogBox,
+HttpClient,
--- End diff --
You're right, `HttpClientModule` should be enough here. We should remove
this line and the import. Thanks!
---
[jira] [Commented] (METRON-1476) Update angular
[ https://issues.apache.org/jira/browse/METRON-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595151#comment-16595151 ] ASF GitHub Bot commented on METRON-1476: Github user sardell commented on a diff in the pull request: https://github.com/apache/metron/pull/1096#discussion_r213361764 --- Diff: metron-interface/metron-alerts/.nvmrc --- @@ -0,0 +1 @@ +v9.11.1 --- End diff -- Thanks for pointing this out! The maven configuration is fine because we specify the npm version to install in the pom.xml (6.2.0). However, this will effect developers working on Metron. As far as I can tell, there isn't a way for us to include an npm version in our .nvmrc, so I think the next best thing we can do is add a step to the README that instructs developers to install npm 6.2.0. > Update angular > -- > > Key: METRON-1476 > URL: https://issues.apache.org/jira/browse/METRON-1476 > Project: Metron > Issue Type: Improvement >Reporter: Daniel Toth >Assignee: Daniel Toth >Priority: Major > > Update angular to speed up development -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #1096: METRON-1476: Update angular
Github user sardell commented on a diff in the pull request: https://github.com/apache/metron/pull/1096#discussion_r213361764 --- Diff: metron-interface/metron-alerts/.nvmrc --- @@ -0,0 +1 @@ +v9.11.1 --- End diff -- Thanks for pointing this out! The maven configuration is fine because we specify the npm version to install in the pom.xml (6.2.0). However, this will effect developers working on Metron. As far as I can tell, there isn't a way for us to include an npm version in our .nvmrc, so I think the next best thing we can do is add a step to the README that instructs developers to install npm 6.2.0. ---
[jira] [Commented] (METRON-1476) Update angular
[
https://issues.apache.org/jira/browse/METRON-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594873#comment-16594873
]
ASF GitHub Bot commented on METRON-1476:
Github user ruffle1986 commented on a diff in the pull request:
https://github.com/apache/metron/pull/1096#discussion_r213278817
--- Diff:
metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.spec.ts
---
@@ -42,17 +42,18 @@ describe('TableViewComponent', () => {
let fixture: ComponentFixture;
beforeEach(async(() => {
-// FIXME: mock all the unnecessary dependencies
+// FIXME: mock all the unnecessary dependencies
TestBed.configureTestingModule({
- imports: [ HttpModule ],
+ imports: [ HttpClientModule ],
providers: [
SearchService,
UpdateService,
GlobalConfigService,
MetaAlertService,
MetronDialogBox,
+HttpClient,
--- End diff --
Isn't it enough to import the `HttpClientModule` only? Is it necessary to
explicitly set `HttpClient` as a provider?
As far as I know, you only have to set the HttpClient when you want to set
a mock instead of the original http module.
> Update angular
> --
>
> Key: METRON-1476
> URL: https://issues.apache.org/jira/browse/METRON-1476
> Project: Metron
> Issue Type: Improvement
>Reporter: Daniel Toth
>Assignee: Daniel Toth
>Priority: Major
>
> Update angular to speed up development
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] metron pull request #1096: METRON-1476: Update angular
Github user ruffle1986 commented on a diff in the pull request:
https://github.com/apache/metron/pull/1096#discussion_r213278817
--- Diff:
metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.spec.ts
---
@@ -42,17 +42,18 @@ describe('TableViewComponent', () => {
let fixture: ComponentFixture;
beforeEach(async(() => {
-// FIXME: mock all the unnecessary dependencies
+// FIXME: mock all the unnecessary dependencies
TestBed.configureTestingModule({
- imports: [ HttpModule ],
+ imports: [ HttpClientModule ],
providers: [
SearchService,
UpdateService,
GlobalConfigService,
MetaAlertService,
MetronDialogBox,
+HttpClient,
--- End diff --
Isn't it enough to import the `HttpClientModule` only? Is it necessary to
explicitly set `HttpClient` as a provider?
As far as I know, you only have to set the HttpClient when you want to set
a mock instead of the original http module.
---
[GitHub] metron pull request #1096: METRON-1476: Update angular
Github user ruffle1986 commented on a diff in the pull request: https://github.com/apache/metron/pull/1096#discussion_r213283494 --- Diff: metron-interface/metron-alerts/.nvmrc --- @@ -0,0 +1 @@ +v9.11.1 --- End diff -- This version of node includes npm v5.6.0 which hasn't got `ci` command yet. Should `npm` be updated manually to the latest version? If yes, we should add the update command to the maven config as well. ---
[jira] [Created] (METRON-1754) Add users to role mapping for ldap based Metron REST
Simon Elliston Ball created METRON-1754: --- Summary: Add users to role mapping for ldap based Metron REST Key: METRON-1754 URL: https://issues.apache.org/jira/browse/METRON-1754 Project: Metron Issue Type: Sub-task Reporter: Simon Elliston Ball We need a means of mapping users to roles in Spring Security for Metron REST. This should be done via the new LDAP group lookups, and be similar to the way Hadoop groups are mapped for consistency. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1665) Move hosting of Alerts and Config UIs from Nodejs to Spring Boot
[ https://issues.apache.org/jira/browse/METRON-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594599#comment-16594599 ] ASF GitHub Bot commented on METRON-1665: Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r213198753 --- Diff: metron-interface/metron-alerts-host/src/main/java/org/apache/metron/ui/AlertsApplication.java --- @@ -0,0 +1,31 @@ +/** --- End diff -- Done > Move hosting of Alerts and Config UIs from Nodejs to Spring Boot > > > Key: METRON-1665 > URL: https://issues.apache.org/jira/browse/METRON-1665 > Project: Metron > Issue Type: Sub-task >Reporter: Simon Elliston Ball >Assignee: Simon Elliston Ball >Priority: Major > > The current UIs are served up by very lightweight nodejs applications, which > serve the static bundle files produced by the angular build process, and > proxies the rest api. > The proposal is to use a spring boot application, allowing us to harmonise > the security implementation across the UI static servers and the REST layer, > and to provide a routing platform for later microservices. > The UIs currently proxy to the REST API to avoid CORS issues, this will be > achieved with Zuul. > Spring Security will also be extended to use a Knox SSO authenticator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1665) Move hosting of Alerts and Config UIs from Nodejs to Spring Boot
[ https://issues.apache.org/jira/browse/METRON-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594601#comment-16594601 ] ASF GitHub Bot commented on METRON-1665: Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r213198883 --- Diff: metron-interface/metron-alerts-host/pom.xml --- @@ -0,0 +1,143 @@ + --- End diff -- Done, and agreed, it should be a formal style guide addition, and part of the formatting configs on the contributor wiki really. > Move hosting of Alerts and Config UIs from Nodejs to Spring Boot > > > Key: METRON-1665 > URL: https://issues.apache.org/jira/browse/METRON-1665 > Project: Metron > Issue Type: Sub-task >Reporter: Simon Elliston Ball >Assignee: Simon Elliston Ball >Priority: Major > > The current UIs are served up by very lightweight nodejs applications, which > serve the static bundle files produced by the angular build process, and > proxies the rest api. > The proposal is to use a spring boot application, allowing us to harmonise > the security implementation across the UI static servers and the REST layer, > and to provide a routing platform for later microservices. > The UIs currently proxy to the REST API to avoid CORS issues, this will be > achieved with Zuul. > Spring Security will also be extended to use a Knox SSO authenticator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r213198883 --- Diff: metron-interface/metron-alerts-host/pom.xml --- @@ -0,0 +1,143 @@ + --- End diff -- Done, and agreed, it should be a formal style guide addition, and part of the formatting configs on the contributor wiki really. ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r213198753 --- Diff: metron-interface/metron-alerts-host/src/main/java/org/apache/metron/ui/AlertsApplication.java --- @@ -0,0 +1,31 @@ +/** --- End diff -- Done ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r213198659 --- Diff: metron-interface/metron-ui-host/pom.xml --- @@ -0,0 +1,142 @@ + + +http://maven.apache.org/POM/4.0.0; +xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; +xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + +metron-ui-host +jar + +Metron Generic UI Host +Spring Server to host config ui + + +org.apache.metron +metron-interface +0.5.1 + + + --- End diff -- Done, I moved the spring versions up to interface. ---
[jira] [Commented] (METRON-1665) Move hosting of Alerts and Config UIs from Nodejs to Spring Boot
[ https://issues.apache.org/jira/browse/METRON-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594597#comment-16594597 ] ASF GitHub Bot commented on METRON-1665: Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r213198659 --- Diff: metron-interface/metron-ui-host/pom.xml --- @@ -0,0 +1,142 @@ + + +http://maven.apache.org/POM/4.0.0; +xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; +xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + +metron-ui-host +jar + +Metron Generic UI Host +Spring Server to host config ui + + +org.apache.metron +metron-interface +0.5.1 + + + --- End diff -- Done, I moved the spring versions up to interface. > Move hosting of Alerts and Config UIs from Nodejs to Spring Boot > > > Key: METRON-1665 > URL: https://issues.apache.org/jira/browse/METRON-1665 > Project: Metron > Issue Type: Sub-task >Reporter: Simon Elliston Ball >Assignee: Simon Elliston Ball >Priority: Major > > The current UIs are served up by very lightweight nodejs applications, which > serve the static bundle files produced by the angular build process, and > proxies the rest api. > The proposal is to use a spring boot application, allowing us to harmonise > the security implementation across the UI static servers and the REST layer, > and to provide a routing platform for later microservices. > The UIs currently proxy to the REST API to avoid CORS issues, this will be > achieved with Zuul. > Spring Security will also be extended to use a Knox SSO authenticator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
