[GitHub] incubator-metron pull request #345: METRON-532 Define Profile Period When Ca...

2017-01-03 Thread mattf-horton
Github user mattf-horton closed the pull request at: https://github.com/apache/incubator-metron/pull/345 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] incubator-metron issue #410: METRON-645 Unable to Start Fastcapa Test Enviro...

2017-01-03 Thread JonZeolla
Github user JonZeolla commented on the issue: https://github.com/apache/incubator-metron/pull/410 I tested successfully without merging METRON-635. LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] incubator-metron issue #407: METRON-643: Stellar function documentation need...

2017-01-03 Thread JonZeolla
Github user JonZeolla commented on the issue: https://github.com/apache/incubator-metron/pull/407 Note: `BIN` and `STATS_BIN` may be added via [METRON-637](https://github.com/apache/incubator-metron/pull/401) but the documentation in metron-common was not updated, potentially

Re: METRON-648 GrokWebSphereParserTest and BasicAsaParserTest are not 2017-safe

2017-01-03 Thread Michael Miklavcic
I also introduced a Clock object and testing mechanism back in METRON-235 - https://github.com/apache/incubator-metron/pull/156 Sample test utilizing the Clock object here -

Re: Tests failing due to new year

2017-01-03 Thread Matt Foley
Heh, darn it, crossed in Jira – see METRON-648. You win :-) On 1/3/17, 1:13 PM, "Nick Allen" wrote: Thanks, Kyle. I am seeing the same issue. Happy Y2K... I mean 2017. On Tue, Jan 3, 2017 at 4:09 PM, Kyle Richardson wrote:

[GitHub] incubator-metron issue #408: METRON-608 Mpack to install a single-node test ...

2017-01-03 Thread mattf-horton
Github user mattf-horton commented on the issue: https://github.com/apache/incubator-metron/pull/408 Please note the Travis failures are not relevant. All nine are due to https://issues.apache.org/jira/browse/METRON-648 --- If your project is set up for it, you can reply to this

[GitHub] incubator-metron issue #410: METRON-645 Unable to Start Fastcapa Test Enviro...

2017-01-03 Thread JonZeolla
Github user JonZeolla commented on the issue: https://github.com/apache/incubator-metron/pull/410 +1 (nonbinding) pending a successful travis build. It appears there is a general issue that Kyle Richardson is investigating (dev mailing list titled "Test failing due to new year").

Re: Tests failing due to new year

2017-01-03 Thread Nick Allen
Thanks, Kyle. I am seeing the same issue. Happy Y2K... I mean 2017. On Tue, Jan 3, 2017 at 4:09 PM, Kyle Richardson wrote: > Created METRON-647 for tracking. > > -Kyle > > On Tue, Jan 3, 2017 at 3:49 PM, Kyle Richardson > > wrote: > > >

Re: Tests failing due to new year

2017-01-03 Thread Kyle Richardson
Created METRON-647 for tracking. -Kyle On Tue, Jan 3, 2017 at 3:49 PM, Kyle Richardson wrote: > ** This is causing all new PRs to fail Travis CI ** > > The rollover to the new year is causing unit test failures for some of our > parser classes. It looks like the

Tests failing due to new year

2017-01-03 Thread Kyle Richardson
** This is causing all new PRs to fail Travis CI ** The rollover to the new year is causing unit test failures for some of our parser classes. It looks like the issue in the same in all cases... We have hard coded a timestamp assertion but the original message does not contain the year and is now

Re: Custom Storm Topologies

2017-01-03 Thread Carolyn Duby
Also please consider the security of the scripts and script injection attacks. For example, we should probably restrict file access. Thanks Carolyn On 1/3/17, 3:25 PM, "Otto Fowler" wrote: >A script bolt would still allow them to write the script the way they want

Re: Custom Storm Topologies

2017-01-03 Thread Otto Fowler
A script bolt would still allow them to write the script the way they want to, but would avoid having to write all the scaffolding. The matter then would be how to integrate that script bolt into the topologies. On January 3, 2017 at 15:17:59, zeo...@gmail.com (zeo...@gmail.com) wrote: Right,

Re: Custom Storm Topologies

2017-01-03 Thread zeo...@gmail.com
Right, that definitely is more efficient, but part of the point here is to lower the barrier of entry to using Metron. It makes Metron's triage abilities more flexible and allows a user to reuse existing code quickly and easily. Having this available for PoC, prototyping, and low volume

[GitHub] incubator-metron pull request #410: METRON-645 Unable to Start Fastcapa Test...

2017-01-03 Thread nickwallen
GitHub user nickwallen opened a pull request: https://github.com/apache/incubator-metron/pull/410 METRON-645 Unable to Start Fastcapa Test Environment The Fastcapa Test Environment could not be started. There were three problems that needed addressed. 1. The version of

Re: Custom Storm Topologies

2017-01-03 Thread Matt Foley
Well, yes :-) And clearly it should always be more efficient to write a custom bolt in Java than to invoke a script and manage it. --Matt From: Otto Fowler Date: Tuesday, January 3, 2017 at 7:08 AM To: "dev@metron.incubator.apache.org"

[GitHub] incubator-metron issue #393: METRON-622: Create a Metron Docker Compose appl...

2017-01-03 Thread kylerichardson
Github user kylerichardson commented on the issue: https://github.com/apache/incubator-metron/pull/393 I'll created METRON-646 for the elasticsearch image customizations. I already have a start on those changes. --- If your project is set up for it, you can reply to this email and

[GitHub] incubator-metron issue #393: METRON-622: Create a Metron Docker Compose appl...

2017-01-03 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/393 Maybe we can create jiras for these, and someone else might help? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] incubator-metron issue #393: METRON-622: Create a Metron Docker Compose appl...

2017-01-03 Thread kylerichardson
Github user kylerichardson commented on the issue: https://github.com/apache/incubator-metron/pull/393 I've run this up and successfully tested it using the examples provided in the README. It works as documented on docker-machine/boot2docker. Nice job. One showstopper for

[GitHub] incubator-metron issue #393: METRON-622: Create a Metron Docker Compose appl...

2017-01-03 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/393 ok, I was not using the updated examples in the read me to verify the pr. It was not clear to me, after starting everything up, what I should expect to be working and what should

[GitHub] incubator-metron issue #393: METRON-622: Create a Metron Docker Compose appl...

2017-01-03 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/393 If the indices are not installed, then the topologies are moot. I am sorry, I don't mean to be difficult, I am just not sure if it is working and I don't understand what the result

Re: Long-term storage for enriched data

2017-01-03 Thread zeo...@gmail.com
For those interested, I ended up finding a recording of the talk itself when doing some Avro research - https://www.youtube.com/watch?v=tB28rPTvRiI Jon On Sun, Jan 1, 2017 at 8:41 PM Matt Foley wrote: > I’m not an expert on these things, but my understanding is that Avro and

[GitHub] incubator-metron issue #393: METRON-622: Create a Metron Docker Compose appl...

2017-01-03 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/393 I would consider the topologies installed, just not running. But yes, no data flowing end to end by default. --- If your project is set up for it, you can reply to this email and

Re: Long-term storage for enriched data

2017-01-03 Thread zeo...@gmail.com
Right, I second that. That was kind of my intent with my initial question (although I did a bad job at making it clear) - Metron specific benefits/details for Avro use. Sound like it may make sense for someone to throw together a proposal doc? Not volunteering though =) Jon On Tue, Jan 3, 2017

Re: [GitHub] incubator-metron issue #393: METRON-622: Create a Metron Docker Compose appl...

2017-01-03 Thread Ryan Merriman
I would consider the topologies installed, just not running. But yes, no data flowing end to end by default. Ryan On Thu, Dec 22, 2016 at 11:42 AM, ottobackwards wrote: > Github user ottobackwards commented on the issue: > >

Re: Custom Storm Topologies

2017-01-03 Thread Otto Fowler
Wouldn’t that be a bolt? On January 2, 2017 at 14:39:34, Matt Foley (ma...@apache.org) wrote: Should we consider a script calling capability that can launch a streaming script and keep it alive and fed, long-term, rather than launching the script anew every time the Stellar function is invoked?

Re: Long-term storage for enriched data

2017-01-03 Thread Otto Fowler
What I would like to see is something on using avro with a non-static model, such as would be required with metron should new enrichments or threat - intelligence, Stellar capabilities, or source changes. On January 2, 2017 at 11:41:32, Carolyn Duby (cd...@hortonworks.com) wrote: Avro is a

[GitHub] incubator-metron issue #397: METRON-627: Add HyperLogLogPlus implementation ...

2017-01-03 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/397 @nickwallen Thanks for the feedback! - I'm currently working on finishing up some performance metrics (p/sp vals vs accuracy, cardinality, serialized mem consumption) in addition to an

Re: Confluence write access to a space

2017-01-03 Thread Matt Foley
Hi Dima, I think it would make a lot of sense to make 642 a sub-task of 634. Certainly it’s a needed improvement and there’s no need to wait for the laundry list of other stuff in 634. In fact, shorter is better. But gathering the pieces as subtasks will make it easier to track and avoid