Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/526
Also, the commit history looks strange in this PR. Any idea why? Did you
start this branch from master?
---
If your project is set up for it, you can reply to this email and have your
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/526
I am not able to get this to run. Here are the steps I followed:
- Deploy and start the REST app on node1:8082
- Run "npm install" in metron-config
- Start t
Github user merrimanr closed the pull request at:
https://github.com/apache/incubator-metron/pull/535
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
GitHub user merrimanr reopened a pull request:
https://github.com/apache/incubator-metron/pull/535
METRON-859: Use REST application with Kerberos
## Contributor Comments
This PR enables the REST application to function in a Kerberized cluster.
Testing instructions are as
Github user merrimanr closed the pull request at:
https://github.com/apache/incubator-metron/pull/535
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
GitHub user merrimanr reopened a pull request:
https://github.com/apache/incubator-metron/pull/535
METRON-859: Use REST application with Kerberos
## Contributor Comments
This PR enables the REST application to function in a Kerberized cluster.
Testing instructions are as
GitHub user merrimanr opened a pull request:
https://github.com/apache/incubator-metron/pull/535
METRON-859: Use REST application with Kerberos
## Contributor Comments
This PR enables the REST application to function in a Kerberized cluster.
Testing instructions are as follows
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/533
No this fixes a regression.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user merrimanr opened a pull request:
https://github.com/apache/incubator-metron/pull/533
METRON-856: Ansible rpm build wipes out prior binary build
## Contributor Comments
Tested this on full dev and verified the correct profile is being used. I
was able to follow the
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/500
That's a really good idea @mattf-horton! I think that would work but It's
probably a moot point now.
The PR that kerberizes the REST application will be going up soon so
Github user merrimanr closed the pull request at:
https://github.com/apache/incubator-metron/pull/308
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/500
Just pushed out a commit that runs the REST service as the metron user and
added configurable log and pid directories.
@justinleet, I tried setting cardinality 0+ and changed the
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/500
Just merged in master and pushed changes out to address the feedback so
far.
One question I have: which account should this service run as? My
assumption would be the
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/500#discussion_r110772976
--- Diff: metron-interface/metron-rest/src/main/scripts/metron-rest ---
@@ -0,0 +1,128 @@
+#!/usr/bin/env bash
+#
+# Licensed to the
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/500#discussion_r110771928
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/rest_master.py
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/500#discussion_r110771854
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/rest_master.py
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/500#discussion_r110771846
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/rest_commands.py
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/489
Master has been merged in. I tested this on full dev but ran into an issue
with the RPMs being created before the archives were ready, resulting in the
error you see. If I include the
GitHub user merrimanr opened a pull request:
https://github.com/apache/incubator-metron/pull/522
METRON-839: RPM build should happen after archives are built
## Contributor Comments
This PR separates the build process into 2 sequential tasks: Archive build
and RPM build. This
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/489
The latest commits should address most of the feedback that has been given
so far. There were 2 exceptions that I propose we separate into additional
pull requests:
Refactor
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/489#discussion_r110036136
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/GrokController.java
---
@@ -53,4 +54,11
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/489#discussion_r110034628
--- Diff:
metron-interface/metron-config/src/app/login/login.component.spec.ts ---
@@ -0,0 +1,65 @@
+/**
+ * Licensed to the Apache
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/489#discussion_r109464805
--- Diff: metron-interface/metron-config/README.md ---
@@ -0,0 +1,67 @@
+# Metron Management UI
+
+This module provides a user
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/489#discussion_r109464464
--- Diff: metron-interface/metron-config/README.md ---
@@ -0,0 +1,67 @@
+# Metron Management UI
+
+This module provides a user
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/489#discussion_r109211360
--- Diff:
metron-interface/metron-config/e2e/sensor-config-readonly/sensor-config-readonly.po.ts
---
@@ -0,0 +1,125
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/489#discussion_r109198348
--- Diff:
metron-interface/metron-config/src/app/general-settings/general-settings.component.ts
---
@@ -0,0 +1,82 @@
+/**
+ * Licensed
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/489#discussion_r109196352
--- Diff:
metron-interface/metron-config/e2e/sensor-config-readonly/sensor-config-readonly.po.ts
---
@@ -0,0 +1,125
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/489#discussion_r109191525
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/GrokController.java
---
@@ -53,4 +54,11
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/489#discussion_r109186326
--- Diff:
metron-interface/metron-config/src/app/model/threat-triage-config.ts ---
@@ -0,0 +1,23 @@
+import {RiskLevelRule} from '.
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/489#discussion_r109185069
--- Diff:
metron-interface/metron-config/src/app/model/threat-triage-config.ts ---
@@ -0,0 +1,23 @@
+import {RiskLevelRule} from '.
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/489#discussion_r109185011
--- Diff:
metron-interface/metron-config/src/app/model/parse-message-request.ts ---
@@ -0,0 +1,23 @@
+/**
+ * Licensed to the Apache
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/500#discussion_r109159403
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/templates/application.yml.j2
GitHub user merrimanr opened a pull request:
https://github.com/apache/incubator-metron/pull/500
METRON-795: Install Metron REST with Ambari MPack
## Contributor Comments
This PR adds the metron-rest module to the Ambari MPack. This can be
tested by building the rpms and mpack
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/472
Looks good. I like the packages approach instead of listing out individual
modules. +1
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/491
Ran it several times. Looks good. +1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/493
+1 tested this against quick dev and was able to parse sample asa messages
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/472
Bump. I think this is ready to go @kylerichardson.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user merrimanr opened a pull request:
https://github.com/apache/incubator-metron/pull/489
METRON-623: Management UI
## Contributor Comments
This PR includes the latest version of the Management UI. Instructions for
building and installing are located in
metron
Github user merrimanr closed the pull request at:
https://github.com/apache/incubator-metron/pull/484
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
GitHub user merrimanr reopened a pull request:
https://github.com/apache/incubator-metron/pull/484
METRON-623: Management UI
## Contributor Comments
This PR includes the latest version of the Management UI. Instructions for
building and installing are located in
metron
Github user merrimanr closed the pull request at:
https://github.com/apache/incubator-metron/pull/484
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
GitHub user merrimanr opened a pull request:
https://github.com/apache/incubator-metron/pull/484
METRON-623: Management UI
## Contributor Comments
This PR includes the latest version of the Management UI. Instructions for
building and installing are located in
metron
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/472
If you don't mind adding the indexing configs in this PR that would be
great. I had to make 4 changes to get it to work:
- updated the pom.xml to copy the indexing archive
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/472
While reviewing this I had an idea for another PR that would make this
easier to use. The slowest part of spinning this up is downloading the kafka
and hbase distributions. What would
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/472
Looks good. The "Run sensor data end to end" didn't work for me and I had
to make a couple changes to get data flowing through the indexing topology.
When I start
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/474
Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
GitHub user merrimanr opened a pull request:
https://github.com/apache/incubator-metron/pull/474
METRON-758: HdfsServiceImplTest should sort files for list test
Simple fix for the same issue that caused
https://issues.apache.org/jira/browse/METRON-743. I verified that metron-rest
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
Does anyone have any outstanding comments that they need addressed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/468
I can't think of anything else
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
Just pushed out a commit to address recent comments. Commit includes:
- unit test and javadoc for MessageGetters
- error index template with the "ignore_above": 819
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/316
All integration tests are passing. I was able to start up the application
against quick dev and all the endpoints look like they are working.
---
If your project is set up for it, you
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
I think you're right, that could happen (didn't this happen to you at one
point?). So what is the correct approach then? Do we leave off the raw
message field if the error
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
Technically generating an error message and sending it back through the
indexing topology should be ok now because the original message is serialized
into a string and shouldn't
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
Default in this case is JSON_FROM_FIELD without specifying the field
(defaults to "message"). I couldn't think of an ideal way to do this so open
to ideas. We could ju
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
The latest commit includes the changes discussed. Most error messages now
go to the indexing topic by default so there is no need for another error
indexing topology. The error topic
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/316
Finally figured this one out. I wasn't able to reproduce the issue because
my maven version was a couple minor releases behind (on 3.3.9 now for what it's
worth). Once I got
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/441
+1 from me
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/316
Hmm I just tried deploying on quick-dev the same way and was able to hit
the stellar endpoints without issue. I will continue testing to see if I can
get it to break tomorrow.
---
If
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/316
Just merged in the latest from master including the maxmind fix that was
breaking the build.
I am not able to recreate the NoSuchError exception. Here is what I'm
doing to
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/462
+1 on committing based on Justin's Travis results
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project doe
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/441
Works great. I have just one very small request. Can you update the Usage
section of the README (where it lists the UI addresses) to point to the
Elasticsearch head plugin instead of
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
You need a source.type to look up a parser config right? I don't think you
can parse a message without it. I'm relying on the bolts to supply the
source.type and I think th
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
That wasn't an exact structure. Take a look at the ErrorFields enum in
https://github.com/apache/incubator-metron/pull/453/files#diff-19fcef3f36d5353a8ad399a128f40f3e.
It's
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
The error message will look something like this:
{
"exception": "java.lang.Exception: there was an error",
"hostname": "host&qu
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
So the consensus is to use a single indexing topology for both error and
normal messages by default. I will remove the scripts and configuration (flux
and properties) files and remove
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/441
@kylerichardson I will get this reviewed today
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
Those scripts and configurations have already been created and are part of
this PR so the question is should we remove them.
---
If your project is set up for it, you can reply to this
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
I would prefer a separate error topology but you and others have provided
good reasons for not doing that. Single indexing approach by default is a good
compromise and fine with me
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
After thinking about it a few minutes more, maybe the answer to the
deployment question is to deploy all the necessary scripts, configurations, etc
but just not start it by default
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
I think a configurable error topic is a reasonable request. The
requirements need a little clarification though.
How granular should this configuration property be? Should it
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/316
Removing the org.reflections dependency from the metron-rest pom and
inheriting it from metron-common seems to have resolved the
"java.lang.NoSuchMethodError" issue with t
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/316#discussion_r102089127
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/StellarController.java
---
@@ -0,0 +1,80
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/316
Just pushed out commits to address these bugs. In regards to this:
"Another thing that may be nice is to describe what is all needed for the
REST API to run. For examp
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/316#discussion_r102077984
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/SensorEnrichmentConfigController.java
---
@@ -0,0 +1,97
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/316#discussion_r102077888
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/StellarController.java
---
@@ -0,0 +1,80
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/316#discussion_r102077714
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/GrokController.java
---
@@ -0,0 +1,56
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/316#discussion_r102076697
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/HdfsController.java
---
@@ -0,0 +1,89
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/316#discussion_r102076188
--- Diff: metron-interface/metron-rest/pom.xml ---
@@ -0,0 +1,335 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi=
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/316#discussion_r102076204
--- Diff: metron-interface/pom.xml ---
@@ -0,0 +1,90 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/316#discussion_r102076172
--- Diff: metron-interface/metron-rest-client/pom.xml ---
@@ -0,0 +1,40 @@
+
+
+
+http://maven.apache.org/POM/4.0.0";
xmln
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/316#discussion_r102076209
--- Diff: metron-interface/metron-rest/src/test/resources/README.vm ---
@@ -0,0 +1,142 @@
+#[[#]]# Metron REST
+
+This module
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
@ottobackwards to answer some of your questions (sorry was at RSA all week):
- An extra topology will take up a couple worker slots but the performance
implications should be
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
I think removing the error indexing topology is fine as long as we're
careful to avoid error messages getting stuck in a loop in the case of a failed
index write. I agree that it
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/316
The latest commit includes removing MySQL as the security credential store
and replacing with H2. I also updated the documentation with instructions on
configuring a production database
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/453#discussion_r100898119
--- Diff:
metron-platform/metron-elasticsearch/src/main/config/elasticsearch_error.properties
---
@@ -0,0 +1,69 @@
+# Licensed to the
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/453#discussion_r100897917
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/error/MetronError.java
---
@@ -0,0 +1,200
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/453#discussion_r100897656
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-env.xml
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/453#discussion_r100894071
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-env.xml
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/453#discussion_r100892848
--- Diff:
metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java
---
@@ -74,7 +81,11 @@ public
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/453#discussion_r100891429
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/error/MetronError.java
---
@@ -0,0 +1,200
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
On the topic of invalid messages, they are now treated as error messages.
They can still be distinguished as invalid message though. Is there any reason
they should be treated
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
In response to "Is there any reason we didn't just use the normal indexing
topology". Here are the issues I see with doing that. First, I think we
should be carefu
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/453#discussion_r100848948
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/error/MetronError.java
---
@@ -0,0 +1,200
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/453#discussion_r100848754
--- Diff:
metron-platform/metron-indexing/src/main/config/zookeeper/indexing/error.json
---
@@ -0,0 +1,17 @@
+{
+ "
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
Sorry I should have included this in the original description. I still
need to update the various READMEs, that task is outstanding and this should
not be merged until that is done
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
No you are correct, we need a more comprehensive test plan. I'm still
thinking about it. Triggering errors at each point in the topologies is not
straightforward. Sending in a me
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/453
METRON-694 includes both the topology changes and the Ambari MPack changes.
I started on METRON-695 but decided to include both in a single PR, hence the
branch being named METRON-695
GitHub user merrimanr opened a pull request:
https://github.com/apache/incubator-metron/pull/453
METRON-694: Index Errors from Topologies
This PR addresses METRON-695, including updates to the Ambari MPack. A
summary of the changes:
- Defaulted FieldValidator.input to
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/316#discussion_r99714082
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/GrokServiceImpl.java
---
@@ -0,0 +1,112
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/316
Just added a controller for the HdfsService and renamed
TransformationService to StellarService. Let me know what you think of those
changes.
---
If your project is set up for it, you
1 - 100 of 317 matches
Mail list logo
