I'll see what I can do re batch sizes.
I can pretty much gather any metric you want. Which one do you need?
On 2017-08-18 03:22, [email protected] wrote:
I'm grasping at straws but maybe play with your batch sizes? I hear
now
there's a time based option too that I haven't played with.
It would be helpful to have more metrics about resource utilization
over
time in your cluster, are you able to gather anything and maybe put it
into
a visualization tool?
Jon
On Thu, Aug 17, 2017, 16:35 Laurens Vets <[email protected]> wrote:
That seems close to the issue that I'm having... But there's a part I
don't quite understand in my case.
Everything's working fine for days and then suddenly, Java throws an
error (see other mail to the mailing list) and I can't get Metron
stable
again. It's always either the indexingBolt or hdfsIndexingBolt. The
error I see the most after this is "java.lang.OutOfMemoryError: GC
overhead limit exceeded"...
It seems that somewhere there's a big pileup of messages which Metron
suddenly can't process.
Any ideas on how to further troubleshoot this?
On 2017-08-17 11:10, [email protected] wrote:
> I used to run into similar issues when my environment was resource
> constrained but never ran it to root cause. It has been a long time
> since
> I was in this scenario to re-test.
>
> https://issues.apache.org/jira/projects/METRON/issues/METRON-485
>
> Jon
>
> On Thu, Aug 17, 2017 at 12:49 PM Laurens Vets <[email protected]>
> wrote:
>
>> Hi,
>>
>> Simple question, when I stop Metron, Kafka & Storm via Ambari, I still
>> see the storm worker processes running, is this expected?
>>
