[GitHub] metron issue #481: METRON-322 Global Batching and Flushing

2017-07-07 Thread dlyle65535
Github user dlyle65535 commented on the issue: https://github.com/apache/metron/pull/481 Awesome! I can run it up on a live system and see what's what. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] metron issue #481: METRON-322 Global Batching and Flushing

2017-07-07 Thread mattf-horton
Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/481 Hi @dlyle65535 , I would welcome help. I think this weekend I can update to current master, and integrate the (already done) fixes for @cestella 's suggestions, including stateful testing with

[GitHub] metron pull request #637: METRON-1021 increment metron version number to 0.4...

2017-07-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/637 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Matt Foley
I’d be happy to collaborate in specifying and implementing this solution, if you guys like. The stuff currently in Stellar is a good starting point, so it really wouldn’t be a big job. --Matt On 7/7/17, 12:01 PM, "Otto Fowler" wrote: No, you are right, I miss read Matt’s suggestion.

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Otto Fowler
No, you are right, I miss read Matt’s suggestion. On July 7, 2017 at 14:53:02, Nick Allen (n...@nickallen.org) wrote: Otto - My original understanding from reading the JIRA was that you were suggesting have the REPL call the REST API. That is the idea that I am not fond of. I must have misunders

[GitHub] metron pull request #638: METRON-933 New stellar function for regex group ca...

2017-07-07 Thread ottobackwards
GitHub user ottobackwards opened a pull request: https://github.com/apache/metron/pull/638 METRON-933 New stellar function for regex group capture New Stellar function for capturing a group from a Regex Pattern. I created a new RegExFunctions in Stellar for this, and moved t

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Matt Foley
Nick and Otto, Yes, I would view this as the correct solution here. Thanks, --Matt On 7/7/17, 11:50 AM, "Nick Allen" wrote: Yep, I think you are right, Matt. Create a Configuration API that is called from wherever it is needed; the REST API or a Stellar function. The logic is m

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Nick Allen
Otto - My original understanding from reading the JIRA was that you were suggesting have the REPL call the REST API. That is the idea that I am not fond of. I must have misunderstood. My bad. On Fri, Jul 7, 2017 at 2:47 PM, Otto Fowler wrote: > This was my original inclination and Casey’s as

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Nick Allen
Yep, I think you are right, Matt. Create a Configuration API that is called from wherever it is needed; the REST API or a Stellar function. The logic is more cohesive, simplifies testing. That's what we've done in most places. On Fri, Jul 7, 2017 at 2:36 PM, Matt Foley wrote: > Hi all, > At t

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Otto Fowler
Wait, Matt are you saying the answer is ‘neither’ and we should have a 3rd layer? That both call? On July 7, 2017 at 14:47:06, Otto Fowler (ottobackwa...@gmail.com) wrote: This was my original inclination and Casey’s as well when we spoke. I think Nick has some good points however, so I creat

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Otto Fowler
This was my original inclination and Casey’s as well when we spoke. I think Nick has some good points however, so I created this thread. Thanks for replying! On July 7, 2017 at 14:36:51, Matt Foley (ma...@apache.org) wrote: Hi all, At the risk of getting suddenly unpopular (:-) I would like to

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Otto Fowler
The issue ( if we agree that there is one ) is that we are going to face dual implementation of management type functions in both stellar and rest. *If* we think it is important to have one base API we should pick Stellar or Rest. On July 7, 2017 at 13:01:18, Nick Allen (n...@nickallen.org) wrot

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Matt Foley
Hi all, At the risk of getting suddenly unpopular (:-) I would like to argue the other side. Architecturally I disagree with having REST invoke Stellar, or in general making Stellar the single point of contact for management functionality. Several reasons: 1. The architectural component proper

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Nick Allen
> Are we talking about exposing an endpoint that just executes a stellar statement? No, that is not the case AFAIK, Ryan. I see Otto's PR as more of a discussion around how to go about implementing Management-ish functionality in the REST API. I think the goal here is just to get consensus on an

[GitHub] metron issue #637: METRON-1021 increment metron version number to 0.4.1 in p...

2017-07-07 Thread mattf-horton
Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/637 @ottobackwards , thanks for the check. Re your question, that's correct, the "site" directory and the current-book it contains, should be for the current public site only, which is at 0.4.0.

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Ryan Merriman
Makes sense to me. Are we talking about exposing an endpoint that just executes a stellar statement? We already have one but it's scope is limited to applying stellar transformations to a sample message. Assuming we would just add on to that controller. What Jiras are going to come out of th

Stellar: Is there a reason we don’t use ConvertUtils?

2017-07-07 Thread Otto Fowler
Is there a reason that we don’t use the ConversionUtils conversion function in stellar, as opposed to casting and checking null everywhere? This is wrt the args.

[GitHub] metron issue #608: METRON-986 Enhance Fastcapa to Support Intel X520

2017-07-07 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/608 > That sounds good; can you create a follow-on JIRA to investigate how to make the properties specified at run-time rather than as macros in code? FYI - Created https://issues.apache.org/j

[GitHub] metron pull request #608: METRON-986 Enhance Fastcapa to Support Intel X520

2017-07-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/608 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Otto Fowler
Anyone else have feelings? On July 7, 2017 at 11:06:32, Nick Allen (n...@nickallen.org) wrote: Like you mentioned, Otto, I think it makes more sense to have a REST API that is backed by Stellar functions executed in a JVM. That is, the REST API simply executes the right Stellar functions in a JV

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Nick Allen
Like you mentioned, Otto, I think it makes more sense to have a REST API that is backed by Stellar functions executed in a JVM. That is, the REST API simply executes the right Stellar functions in a JVM. This makes it very simple to reuse the same implementation (Stellar functions) across multipl

Re: Post-parsing and Enrichment test framework

2017-07-07 Thread Nick Allen
> > Is there any other approach to check > that through writing Java test-cases? Righting test-cases would be easier > for keeping track of changes. While the Shell is great, it does not serve as an automated, repeatable test case. An alternative approach along these lines, is to create your own

Re: Post-parsing and Enrichment test framework

2017-07-07 Thread Nick Allen
For experimenting or validating specific Stellar expressions, the Stellar Shell is perfect. To do this, you just have to remember than when your Stellar expressions execute all of the fields of the message are in-scope. For example, here is a quick session where I mock-up some logic that sends a

[GitHub] metron issue #637: METRON-1021 increment metron version number to 0.4.1 in p...

2017-07-07 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/637 +1 - searched, built, tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] metron pull request #621: METRON-1001: Allow metron to ingest parser metadat...

2017-07-07 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/metron/pull/621#discussion_r126156447 --- Diff: metron-platform/metron-storm-kafka/src/main/java/org/apache/metron/storm/kafka/flux/SimpleStormKafkaBuilder.java --- @@ -176,16 +182,48 @@ publ

[GitHub] metron issue #608: METRON-986 Enhance Fastcapa to Support Intel X520

2017-07-07 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/608 That sounds good; can you create a follow-on JIRA to investigate how to make the properties specified at run-time rather than as macros in code? --- If your project is set up for it, you can reply

[GitHub] metron issue #608: METRON-986 Enhance Fastcapa to Support Intel X520

2017-07-07 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/608 +1 by inspection --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] metron pull request #550: METRON-890: Intermittent unit test errors in shutt...

2017-07-07 Thread cestella
Github user cestella closed the pull request at: https://github.com/apache/metron/pull/550 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] metron issue #599: METRON-975: Normalize logging and switch to common idiom ...

2017-07-07 Thread simonellistonball
Github user simonellistonball commented on the issue: https://github.com/apache/metron/pull/599 Is it worth us getting this in sooner rather than later, before we get too many other bits of logging that will need to be backported? --- If your project is set up for it, you can reply

Re: UI pivotting / aggregation backend

2017-07-07 Thread Casey Stella
I just want to chime in and support the notion of an abstraction layer between the UI and the indexed stores. I think that having an API that people can conform to is going to be important as people want to plug in their own backing indices in the future. Casey On Thu, Jul 6, 2017 at 2:11 PM, Ju