[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread jjmeyer0
Github user jjmeyer0 commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126589803 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/HashFunctions.java --- @@ -0,0 +1,85 @@ +/* + * Licen

[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-10 Thread mraliagha
Github user mraliagha commented on the issue: https://github.com/apache/metron/pull/620 @iraghumitra I am going to test your latest code to make sure the mentioned issues haven't been resolved yet. I was using an older version of your build, so it might be outdated. --- If your proj

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126579811 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/HashFunctions.java --- @@ -0,0 +1,85 @@ +/* + *

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126577935 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/HashFunctions.java --- @@ -0,0 +1,85 @@ +/* + *

Re: [Request for Consensus Approval] dev branch for Stellar additional work

2017-07-10 Thread Matt Foley
Hey all, it appears that we’re not ready to do speculative branches yet, so I’ll proceed with the next chunk of Stellar separation work in my private branch, currently in https://github.com/mattf-horton/metron/tree/stellar-mod4 There is of course nothing “private” about it; anyone is most welcom

Re: [VOTE][PROPOSAL] minor changes to release process

2017-07-10 Thread Matt Foley
Vote passes with +1 : 4 votes (3 binding, 1 non-binding) 0 : none -1 : none I’ll edit the doc to reflect the change. Thanks, --Matt On 7/6/17, 10:53 AM, "Matt Foley" wrote: Thanks, all. That’s 3 binding +1’s, so I’m going to proceed with METRON-1021. Vote needs to stay open 72 hours

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread mattf-horton
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126573313 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/HashFunctions.java --- @@ -0,0 +1,85 @@ +/* + * L

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126566101 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/HashFunctions.java --- @@ -0,0 +1,85 @@ +/* + *

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread mattf-horton
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126560557 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/HashFunctionsTest.java --- @@ -0,0 +1,169 @@ +/*

[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-10 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/620 I just added some comments related to abstracting the search service. Much improved with the latest commits. I think we're almost there. --- If your project is set up for it, you can reply to th

[GitHub] metron pull request #620: Metron-988: UI for viewing alerts generated by Met...

2017-07-10 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/620#discussion_r126534924 --- Diff: metron-interface/metron-alerts/src/app/utils/elasticsearch-utils.ts --- @@ -0,0 +1,72 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] metron pull request #620: Metron-988: UI for viewing alerts generated by Met...

2017-07-10 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/620#discussion_r126534489 --- Diff: metron-interface/metron-alerts/src/app/service/elasticsearch-localstorage-impl.ts --- @@ -0,0 +1,291 @@ +/** + * Licensed to the Apache

[GitHub] metron pull request #620: Metron-988: UI for viewing alerts generated by Met...

2017-07-10 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/620#discussion_r126532937 --- Diff: metron-interface/metron-alerts/src/app/service/data-source.ts --- @@ -0,0 +1,62 @@ +/** + * Licensed to the Apache Software Foundation (AS

[GitHub] metron pull request #620: Metron-988: UI for viewing alerts generated by Met...

2017-07-10 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/620#discussion_r126532195 --- Diff: metron-interface/metron-alerts/src/app/model/search-request.ts --- @@ -0,0 +1,7 @@ +export class SearchRequest { + _source: string[];

[GitHub] metron pull request #620: Metron-988: UI for viewing alerts generated by Met...

2017-07-10 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/620#discussion_r126531609 --- Diff: metron-interface/metron-alerts/src/app/alerts/saved-searches/saved-searches.component.ts --- @@ -0,0 +1,124 @@ +import { Component, OnInit }

[GitHub] metron pull request #620: Metron-988: UI for viewing alerts generated by Met...

2017-07-10 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/620#discussion_r126531076 --- Diff: metron-interface/metron-alerts/src/app/alerts/alerts-list/query-builder.ts --- @@ -0,0 +1,139 @@ +/** + * Licensed to the Apache Software

[GitHub] metron pull request #620: Metron-988: UI for viewing alerts generated by Met...

2017-07-10 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/620#discussion_r126530918 --- Diff: metron-interface/metron-alerts/src/app/alerts/alerts-list/query-builder.ts --- @@ -0,0 +1,139 @@ +/** + * Licensed to the Apache Software

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/636 If the es or solr code was packaged as a bundle ( it with it's dependencies -> the es and solr libs ), the bundle system let you load each implementation as a plugin, and will create a new clas

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread jjmeyer0
Github user jjmeyer0 commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126524234 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/HashFunctionsTest.java --- @@ -0,0 +1,169 @@ +/* + *

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/636 The core issue is that both Solr and Elasticsearch depend on Lucene. Our version of Elasticsearch (2.3.3) is old so there are lots of Lucene class version conflicts when adding Solr as a dependenc

[GitHub] metron issue #644: METRON-1027: Errant Log Message When No GeoIP Hit on Vali...

2017-07-10 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/644 Oh, nevermind. Got it. Sorry, Travis. Thank you for your service. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] metron pull request #644: METRON-1027: Errant Log Message When No GeoIP Hit ...

2017-07-10 Thread nickwallen
GitHub user nickwallen reopened a pull request: https://github.com/apache/metron/pull/644 METRON-1027: Errant Log Message When No GeoIP Hit on Valid, External IPv4 Address When a valid public IPv4 address does not match a record in the Maxmind GeoIP database, a log message indicate

[GitHub] metron issue #644: METRON-1027: Errant Log Message When No GeoIP Hit on Vali...

2017-07-10 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/644 Thanks for the review guys. Just trying to understand why Travis is reporting a unit test failure. Its like it is not running the latest code. Bah. --- If your project is set up for it, you c

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/636 What kind of isolation we do need? The 777 introduces bundle loading gives classloader isolation If they were loaded as plugins then it may work. --- If your project is set up for it, you

[GitHub] metron pull request #644: METRON-1027: Errant Log Message When No GeoIP Hit ...

2017-07-10 Thread nickwallen
Github user nickwallen closed the pull request at: https://github.com/apache/metron/pull/644 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/636 @ottobackwards just created a discuss thread on the general approach here. @cestella you're absolutely correct. We need a way for solr and ES to coexist. A precedent has already been set

[GitHub] metron issue #644: METRON-1027: Errant Log Message When No GeoIP Hit on Vali...

2017-07-10 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/644 +1 by inspection. @nickwallen is right that the log already occurs in the GeoLiteDatabase, so I'm okay with dropping it. --- If your project is set up for it, you can reply to this email and hav

[GitHub] metron pull request #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/636#discussion_r126511864 --- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/writer/ElasticsearchWriter.java --- @@ -60,96 +61,8 @@ public

[GitHub] metron pull request #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/636#discussion_r126511838 --- Diff: metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/ElasticsearchServiceImpl.java --- @@ -0,0 +1,74 @@ +/** +

[DISCUSS] Search in REST

2017-07-10 Thread Ryan Merriman
This discussion is an attempt to clarify some questions and discuss design decisions related to METRON-1022. The primary purpose of METRON-1022 is to provide a foundation for building Metron-specific Elasticsearch (or other search engine implementations) functions in our REST application. This tr

[GitHub] metron pull request #644: METRON-1027: Errant Log Message When No GeoIP Hit ...

2017-07-10 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/644#discussion_r126510446 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/adapters/geo/GeoAdapter.java --- @@ -46,7 +46,6 @@ public JSONObje

[GitHub] metron issue #644: METRON-1027: Errant Log Message When No GeoIP Hit on Vali...

2017-07-10 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/644 +1 by inspection. I agree that the log should be moved in level and not dropped --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] metron issue #644: METRON-1027: Errant Log Message When No GeoIP Hit on Vali...

2017-07-10 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/644 Specifically, I'd say add it in ``` if (isIneligibleAddress(ip, addr)) { return Optional.of(new HashMap()); } ``` --- If your project is set up for it, you can r

[GitHub] metron issue #644: METRON-1027: Errant Log Message When No GeoIP Hit on Vali...

2017-07-10 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/644 @nickwallen Off the top of my head, I think it could be Optional.empty(), but I'd like to see a debug log added so we can easily tell the difference between "IP not found" and "It wasn't even some

[GitHub] metron pull request #644: METRON-1027: Errant Log Message When No GeoIP Hit ...

2017-07-10 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/metron/pull/644#discussion_r126507756 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/adapters/geo/GeoAdapter.java --- @@ -46,7 +46,6 @@ public JSONObje

[GitHub] metron issue #644: METRON-1027: Errant Log Message When No GeoIP Hit on Vali...

2017-07-10 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/644 @justinleet Another thing I noticed while in the code is that a lookup of a private IP address is expected to return an empty map. Since we're using an Optional, wouldn't it make more sense for i

[GitHub] metron pull request #644: METRON-1027: Errant Log Message When No GeoIP Hit ...

2017-07-10 Thread nickwallen
GitHub user nickwallen opened a pull request: https://github.com/apache/metron/pull/644 METRON-1027: Errant Log Message When No GeoIP Hit on Valid, External … When a valid public IPv4 address does not match a record in the Maxmind GeoIP database, a log message indicates that an er

[GitHub] metron issue #639: METRON-1013 add command line verification to stellar shel...

2017-07-10 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/639 I don't know what is up with the ProfilerIntegrationTest. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] metron pull request #639: METRON-1013 add command line verification to stell...

2017-07-10 Thread ottobackwards
GitHub user ottobackwards reopened a pull request: https://github.com/apache/metron/pull/639 METRON-1013 add command line verification to stellar shell Stellar does no verification of parameters passed on the command line. It should check if the files don't exist for -v, -p, irc,

[GitHub] metron pull request #639: METRON-1013 add command line verification to stell...

2017-07-10 Thread ottobackwards
Github user ottobackwards closed the pull request at: https://github.com/apache/metron/pull/639 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread mattf-horton
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126474912 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/HashFunctions.java --- @@ -0,0 +1,85 @@ +/* + * L

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread mattf-horton
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126481330 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/HashFunctionsTest.java --- @@ -0,0 +1,169 @@ +/*

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread mattf-horton
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126475943 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/HashFunctionsTest.java --- @@ -0,0 +1,169 @@ +/*

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread mattf-horton
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126476908 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/HashFunctionsTest.java --- @@ -0,0 +1,169 @@ +/*

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread mattf-horton
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126474167 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/HashFunctionsTest.java --- @@ -0,0 +1,169 @@ +/*

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread mattf-horton
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/641#discussion_r126473615 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/HashFunctionsTest.java --- @@ -0,0 +1,169 @@ +/*

Re: threatintel_taxii_load.sh throws exception

2017-07-10 Thread Casey Stella
Thanks Vladimir, https://github.com/apache/metron/pull/643 was submitted a few moments ago. On Mon, Jul 10, 2017 at 3:35 PM, Vladimir Shlyakhtin < vladimir.shlyakh...@sstech.us> wrote: > Filed METRON-1026 > > - Vladimir > > From: Otto Fowler [ottobackwa...@gmail.c

[GitHub] metron pull request #643: METRON-1026: threatintel_taxii_load.sh throws exce...

2017-07-10 Thread cestella
GitHub user cestella opened a pull request: https://github.com/apache/metron/pull/643 METRON-1026: threatintel_taxii_load.sh throws exception ## Contributor Comments The delegation wrapping done to enable Stellar transformations on data as it is imported broke the taxii loader.

RE: threatintel_taxii_load.sh throws exception

2017-07-10 Thread Vladimir Shlyakhtin
Filed METRON-1026 - Vladimir From: Otto Fowler [ottobackwa...@gmail.com] Sent: Monday, July 10, 2017 9:13 AM To: dev@metron.apache.org; Vladimir Shlyakhtin Subject: Re: threatintel_taxii_load.sh throws exception After upgrading to 0.4.0? I am not sure you should

[GitHub] metron pull request #642: METRON-984 Stellar functions to decode encoded fie...

2017-07-10 Thread ottobackwards
GitHub user ottobackwards opened a pull request: https://github.com/apache/metron/pull/642 METRON-984 Stellar functions to decode encoded fields or strings This functionality involves ( as a start ) encodings based on the Apache Commons Codec library ( binary ) Support exis

[GitHub] metron pull request #641: METRON-539: added HASH function for stellar.

2017-07-10 Thread jjmeyer0
GitHub user jjmeyer0 opened a pull request: https://github.com/apache/metron/pull/641 METRON-539: added HASH function for stellar. ## Contributor Comments Added a hash function to stellar. Many unit tests were created. Along with these tests I opened the Stellar terminal and did

RE: threatintel_taxii_load.sh throws exception

2017-07-10 Thread Vladimir Shlyakhtin
We have both version installed (0.3.1 and 0.4.0), the same exception for both version. The same for latest (0.4.1) - Vladimir From: Otto Fowler [ottobackwa...@gmail.com] Sent: Monday, July 10, 2017 9:13 AM To: dev@metron.apache.org; Vladimir Shlyakhtin Subject: Re

[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

2017-07-10 Thread iraghumitra
Github user iraghumitra commented on the issue: https://github.com/apache/metron/pull/620 @mraliagha - For 1 can you check with the latest code base. I am unable to simulate this. - For 2 I am slightly confused here. I wanted to know the result of the GET rest call` http:

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/636 This is good work! Thanks, Ryan. We desperately need this abstraction. I think the confusion may be that this PR is the general abstraction as well as a concrete implementation for one index. I

Re: threatintel_taxii_load.sh throws exception

2017-07-10 Thread Otto Fowler
After upgrading to 0.4.0? I am not sure you should be running the threatintell_taxii_load.sh out of /usr/metron/0.3.1. What happens when you run it out of /usr/metron/0.4.1 ? On July 10, 2017 at 08:19:18, Vladimir Shlyakhtin ( vladimir.shlyakh...@sstech.us) wrote: Hello, After upgrading from

Re: threatintel_taxii_load.sh throws exception

2017-07-10 Thread Casey Stella
This is absolutely a bug and you should open a JIRA. On Mon, Jul 10, 2017 at 1:11 PM, Vladimir Shlyakhtin < vladimir.shlyakh...@sstech.us> wrote: > Hello, > > After upgrading from 0.3.0 version we noticed that taxii loader does not > work. > > Here is details: > > # /usr/metron/0.3.1/bin/threatin

Re: threatintel_taxii_load.sh throws exception

2017-07-10 Thread Casey Stella
Also, please reply here with the JIRA and I'll submit a PR. It's a very easy fix. On Mon, Jul 10, 2017 at 1:24 PM, Casey Stella wrote: > This is absolutely a bug and you should open a JIRA. > > On Mon, Jul 10, 2017 at 1:11 PM, Vladimir Shlyakhtin < > vladimir.shlyakh...@sstech.us> wrote: > >> H

threatintel_taxii_load.sh throws exception

2017-07-10 Thread Vladimir Shlyakhtin
Hello, After upgrading from 0.3.0 version we noticed that taxii loader does not work. Here is details: # /usr/metron/0.3.1/bin/threatintel_taxii_load.sh -c /usr/local/opentaxii/etc/connections.conf/guest.phishtank_com.json -e /usr/local/opentaxii/etc/connections.conf/extractor.json -p 1 17

[GitHub] metron issue #520: METRON-833: Update MaaS documentation to explain how it i...

2017-07-10 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/520 +1, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] metron issue #520: METRON-833: Update MaaS documentation to explain how it i...

2017-07-10 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/520 Deconflicted; thanks for the patience. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] metron issue #638: METRON-933 New stellar function for regex group capture

2017-07-10 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/638 I love it. The pattern cache is great! +1 by inspection --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] metron pull request #621: METRON-1001: Allow metron to ingest parser metadat...

2017-07-10 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/621#discussion_r126377527 --- Diff: metron-platform/metron-storm-kafka/src/main/java/org/apache/metron/storm/kafka/flux/SimpleStormKafkaBuilder.java --- @@ -176,16 +182,48 @@ public