Re: [DISCUSS] Next Release Name

+ 1 On November 9, 2016 at 17:15:16, James Sirota (jsir...@apache.org) wrote: Guys, You know, looking at the release I think the changes were significant enough due to the storm & kafka upgrade to justify moving it to a non-point release. Generally point releases are reserved for patches or

Re: [DISCUSS] Next Release Name

Makes sense to me. +1 -Kyle > On Nov 9, 2016, at 5:50 PM, Nick Allen wrote: > > Me likey. +1 > >> On Wed, Nov 9, 2016 at 5:15 PM, James Sirota wrote: >> >> Guys, >> >> You know, looking at the release I think the changes were significant >> enough

Re: [DISCUSS] Next Release Name

Me likey. +1 On Wed, Nov 9, 2016 at 5:15 PM, James Sirota wrote: > Guys, > > You know, looking at the release I think the changes were significant > enough due to the storm & kafka upgrade to justify moving it to a non-point > release. Generally point releases are reserved

Re: [DISCUSS] Next Release Name

+1 On Wed, Nov 9, 2016 at 4:30 PM, Casey Stella wrote: > Agreed, +1 to 0.3.0 > > On Wed, Nov 9, 2016 at 5:28 PM, zeo...@gmail.com wrote: > > > That sounds very reasonable to me. > > > > Jon > > > > On Wed, Nov 9, 2016, 17:15 James Sirota

Re: [DISCUSS] Next Release Name

Agreed, +1 to 0.3.0 On Wed, Nov 9, 2016 at 5:28 PM, zeo...@gmail.com wrote: > That sounds very reasonable to me. > > Jon > > On Wed, Nov 9, 2016, 17:15 James Sirota wrote: > > Guys, > > You know, looking at the release I think the changes were significant

Re: [DISCUSS] Next Release Name

That sounds very reasonable to me. Jon On Wed, Nov 9, 2016, 17:15 James Sirota wrote: Guys, You know, looking at the release I think the changes were significant enough due to the storm & kafka upgrade to justify moving it to a non-point release. Generally point releases

Re: [DISCUSS] Next Release Name

Guys, You know, looking at the release I think the changes were significant enough due to the storm & kafka upgrade to justify moving it to a non-point release. Generally point releases are reserved for patches or maintenance releases. I think this release is more than just a maintenance

Re: [DISCUSS] Next Release Name

Agreed. I could also contribute to that doc. On Sat, Nov 5, 2016, 11:41 Kyle Richardson wrote: > Thanks, James. Very helpful information. Based on that, I agree the path is > there and I have no issues with it being manual at this point. I would > suggest we add a

Re: [DISCUSS] Next Release Name

Thanks, James. Very helpful information. Based on that, I agree the path is there and I have no issues with it being manual at this point. I would suggest we add a simple UPGRADING.md outining the steps you have with a little more detail to make it easy for the user. I'd be happy to take this on

Re: [DISCUSS] Next Release Name

I agree. I think the upgrade path is clear however manual right now. Going forward we will need to prioritize making it more automated, but I think the path is there. On Sat, Nov 5, 2016 at 00:26 James Sirota wrote: > Hi Kyle, > > The HDP upgrade guide can be found here: > >

Re: [DISCUSS] Next Release Name

Metron builds against Apache artifacts by default (storm 1.x, hbase 1.x, Kafka 0.10), so the bits can run on other Hadoop installations that conform to those versions, but our ansible uses HDP 2.5 as a base Hadoop. What James meant was that upgrade instructions for Metron start with Hadoop

Re: [DISCUSS] Next Release Name

Hello James, Does that mean Metron 0.2.2 goes with HDP 2.5 by default? - Dima On 11/05/2016 06:26 AM, James Sirota wrote: > Hi Kyle, > > The HDP upgrade guide can be found here: > https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_command-line-upgrade/content/ch_upgrade_2_4.html > >

Re: [DISCUSS] Next Release Name

Hi Kyle, The HDP upgrade guide can be found here: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_command-line-upgrade/content/ch_upgrade_2_4.html After executing these instructions you get to HDP 2.5 with no data loss. After that, upgrading Metron is as simple as saving the old

Re: [DISCUSS] Next Release Name

I'm a little late to the party but thought I would go ahead and throw my two cents into the mix. I share the concern around an upgrade / migration path. While I would love to see the BETA dropped sooner than later, to me, this is a game changer for people implementing Metron. I think there is a

Re: [DISCUSS] Next Release Name

Jon, Thank you for your thoughts; they are appreciated and you should keep them coming. This kind of discussion is exactly why I sent out this thread. I think it's safe to say that the entire community shares your desire for Metron to be as easy to use as possible and a "data analysis platform

Re: [DISCUSS] Next Release Name

RE- METRON-485 I believe that there are a couple of issues here. 1. We don’t use the -w timeout parameter when killing the topologies, which means technically we may not get out cleanly. We should change this. 2. Beyond the storm timeouts monit itself has timeouts and will ‘kill’ the scripts

Re: [DISCUSS] Next Release Name

Please understand that my points mostly relate to perception and ease of use, not what's technically possible or available. I'm coming at this as Metron should be a data analysis platform for the masses. METRON-517/542 - While I'm willing to let this one go it depends on your definition of

Re: [DISCUSS] Next Release Name

Hi Jon, Here are my thoughts around your objections. METRON-517/METRON-542 I thin the mechanism currently exists within Metron to make this a non-issue. I believe you can solve it with a combination of a Stellar statement and ES templates. As you mentioned, we can truncate the string and

Re: [DISCUSS] Next Release Name

I agree that we can split METRON-517 into a short term and long term fix. I have attempted to organize my thoughts regarding the long term fix into METRON-542 and can get a PR out for METRON-517 soon to close that out. This leaves cluster tuning and a valid upgrade path for users, the later of

Re: [DISCUSS] Next Release Name

Ok, regarding METRON-517, I've thought about this a bit having read your really great and detailed JIRA as well as the discussion around this on the dev list between you and Matt Foley. I want to separate the discussion between what is the correct long-term solution for this issue versus what is

Re: [DISCUSS] Next Release Name

Hi, This is in interesting discussion. Would you mind moving it to the jira or it's own DISCUSS thread? Thanks! -D... On Thu, Nov 3, 2016 at 7:26 AM, zeo...@gmail.com wrote: > To clarify, it only needs to truncate fields > 32766 which need a > full/exact string match

Re: [DISCUSS] Next Release Name

To clarify, it only needs to truncate fields > 32766 which need a full/exact string match search to be run on them (analyzed fields generally would not hit this limitation but I guess in theory they could). However, that's probably every field which can get > 32766 because I'm assuming those will

Re: [DISCUSS] Next Release Name

That would break searching on uri entirely unless you queried and knew to truncate at 32766 because it's not analyzed. I don't like pushing that complication to the end user. I would suggest truncation in the indexingBolt (not using stellar because you'd want this across the board) for all

Re: [DISCUSS] Next Release Name

Jon, For METRON-517 would it suffice to have a stellar statement to take a URI string and truncate it to length of 32766 in the ES writer? But still write the actual string to HDFS? You can then search against ES on the truncated portion, but retrieve the actual timestamp from HDFS. It's

Re: [DISCUSS] Next Release Name

Sounds to me that we should close METRON-370 as non-discrepant. If we want a feature later on to be able to replay enrichments we should add a Jira on that instead. I don't see why you would do this, though. If the enrichment fails you want to pass the message through unenriched. Otherwise

Re: [DISCUSS] Next Release Name

Hi Jon, I have commented on 370 - https://issues.apache.org/jira/browse/METRON-370 Best, Mike On Wed, Nov 2, 2016 at 3:11 PM, zeo...@gmail.com wrote: > I personally would like to see the following things done before things > leave BETA: > (1) Address data integrity concerns

Re: [DISCUSS] Next Release Name

I'm +1 dropping the BETA. Correct me if I'm wrong, but there are users in production. I think leaving it in the 0 major is appropriate until the concerns Jon mentioned are addressed. -D... On Wed, Nov 2, 2016 at 5:11 PM, zeo...@gmail.com wrote: > I personally would like to

Re: [DISCUSS] Next Release Name

I personally would like to see the following things done before things leave BETA: (1) Address data integrity concerns (Specifically thinking of METRON-370, METRON-517) (2) Make cluster tuning easier and more consistent (METRON-485, METRON-470, and the "[DISCUSS] moving parsers back to flux" which

[DISCUSS] Next Release Name

Hello Everyone, Now that the discussion around the next release has started, it has been proposed and I think it's a good time to discuss what to name this next release. Before, we have adopted the BETA suffix. I think it might be time to drop it and call the next release 0.2.2 Thoughts?