Re: Indexing topology keep crashing

> Is there a way to better handle big bursts? You will want to tune your cluster based on your expected burst rate, not your average rate. We include some tools [1] in Metron to generate synthetic data at your

Re: Indexing topology keep crashing

This should be handled fine by Storm. You can set topology.max.spout.pending as back pressure. https://github.com/apache/metron/blob/master/metron-platform/Performance-tuning-guide.md#storm-tuning When there are bursts you may have some backup in the Kafka topics, but the topologies will catch

Re: Indexing topology keep crashing

1. worker.childopts: -Xmx2048m 2. As in individual messages? Just small(-ish) JSON messages. A few KBytes? On 13-Sep-18 11:21, Casey Stella wrote: > Two questions: > 1. How much memory are you giving the workers for the indexing topology? > 2. how large are the messages you're sending through? >

Re: Indexing topology keep crashing

Two questions: 1. How much memory are you giving the workers for the indexing topology? 2. how large are the messages you're sending through? On Thu, Sep 13, 2018 at 2:00 PM Vets, Laurens wrote: > Hello list, > > I've installed OS updates on my Metron 0.4.2 yesterday, restarted all > nodes and

Indexing topology keep crashing

Hello list, I've installed OS updates on my Metron 0.4.2 yesterday, restarted all nodes and now my indexing topology keeps crashing. This is what I see in the Storm UI for the indexing topology: Topology stats: 10m 0s    1304380    1953520    12499.833    1320    3h 0m 0s    1304380    1953520