Re: Heterogeneous indexing batch size for different Metron feeds

2017-12-10 Thread Ali Nazemian
This seems not the same as our observations. Whenever there are some messages in the indexing or enrichments backlog, the new configurations (at least related to the batch size) won't be applied to the new messages. It will remain as the previous state until it processes all the old messages. This

[GitHub] metron issue #858: METRON-1344: Externalize the infrastructural components u...

2017-12-10 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/858 I would be for +1 and landing this pr on the feature branch, then we can discuss next steps, proposals and start new pr's against FB ---

Re: Heterogeneous indexing batch size for different Metron feeds

2017-12-10 Thread Michael Miklavcic
I know there are a number of questions in this thread, but I have some specific comments on dealing with your failed tuples in the indexing topology. For starters, I can almost guarantee that your problem is going to be with tuning ES and not HDFS. Not to say you won't need to tune HDFS, but look a

Re: [DISCUSS] Upcoming Release

2017-12-10 Thread Matt Foley
Yes, but let’s see if anyone else find other issues. From: Otto Fowler Date: Saturday, December 9, 2017 at 6:16 AM To: Matt Foley , "dev@metron.apache.org" Subject: Re: [DISCUSS] Upcoming Release So RC2 then? On December 8, 2017 at 20:43:21, Matt Foley (mfo...@hortonworks.com) wrote: