Re: Where to find reason why message fails in topology

Hi Margus, See which bolts have high 'Capacity' values in the Storm UI, and whether any of those are close to / above 1 to get a clue about where the bottleneck might be. On Tue, Jan 20, 2015 at 3:38 AM, Margus Roo wrote: > Hi > > I have simple topology Kafka consumer (Spout) and HBase writer

Re: worker logs

Even in local mode, logs are generated per worker. On Mon, Jan 19, 2015 at 1:44 PM, Parth Brahmbhatt < pbrahmbh...@hortonworks.com> wrote: > I don’t think local mode will honor your changes. Local mode does not > spawn new worker processes. Your worker.xml changes will only take effect > in distr

Re: worker logs

I don’t think local mode will honor your changes. Local mode does not spawn new worker processes. Your worker.xml changes will only take effect in distributed mode. Thanks Parth On Jan 19, 2015, at 11:34 AM, clay teahouse wrote: > I am using 0.9.3. I did set it in storm.yaml, and the console

Re: worker logs

I am using 0.9.3. I did set it in storm.yaml, and the console log shows it as set, but the worker logs still end up in /tmp. I am running storm in local cluster mode. thanks, Clay On Mon, Jan 19, 2015 at 12:39 PM, Parth Brahmbhatt < pbrahmbh...@hortonworks.com> wrote: > What version of storm ar

Re: worker logs

What version of storm are you using? I just tried this locally and it seem to be working on the master branch right now. You can try to set the “storm.log.dir” in your storm config and see if that takes effect or not. Thanks Parth On Jan 19, 2015, at 9:55 AM, clay teahouse wrote: > Thanks P

Re: Meaning of "Solts"

Armando, slots means worker slots. In this case it looks like you assigned 3 workers to your topology. -Harsha On Mon, Jan 19, 2015, at 09:38 AM, Armando Martinez Briones wrote: > > Hi. > > I'm rebalancing a topology, on the log of nimbus I can see the line: > > b.s.d.nimbus [INFO] Reassigning >

Re: worker logs

Thanks Parth. I did set storm.log.dir in storm.yaml, but it seems that it is being ignored. . On Mon, Jan 19, 2015 at 10:46 AM, Parth Brahmbhatt < pbrahmbh...@hortonworks.com> wrote: > you can specify the worker log location in worker.xml. The default > is ${storm.log.dir}/${logfile.name}. You ca

Meaning of "Solts"

Hi. I'm rebalancing a topology, on the log of nimbus I can see the line: b.s.d.nimbus [INFO] Reassigning topologyCancellationShedulerTask-1-1421687906 to 3 slots But, I don't know what is the meaning of the "slots" for a topology. Thanks. -- [image: Tralix][image: 1]José Armando Martínez Br

Topology dead even with synchronised output collector

Hi Guys I have fight over this for several days now. 34493 [Thread-17-disruptor-executor[3 8]-send-queue] ERROR backtype.storm.util - Async loop died! java.lang.RuntimeException: java.lang.NullPointerException at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)

Re: worker logs

you can specify the worker log location in worker.xml. The default is ${storm.log.dir}/${logfile.name}. You can find the file under STORM_HOME/logback/. Thanks Parth On Jan 19, 2015, at 7:34 AM, Susana González wrote: > I'm afraid I haven't specified them. That location is where the logs wer

Where to find reason why message fails in topology

Hi I have simple topology Kafka consumer (Spout) and HBase writer (bolt) Sometimes some messages getting failed status. Most of time topology works well but I'd like to know what is exact reason why message fails. As much I have read there is timeout (default 30s) when message fails. But this

Re: How are you managing cluster upgrades?

I haven’t had to do this but I think currently storm does not support a rolling upgrade/downgrade. The process I have seen goes something like this Ensure you have a backup of all your topology jars and confs, everything that is necessary to resubmit the topologies. Kill al the topologies. Perfor

Re: worker logs

I'm afraid I haven't specified them. That location is where the logs were created by default. On Mon, Jan 19, 2015 at 4:29 PM, clay teahouse wrote: > Thanks Susana. > Where do you specify the location of the logs? > > -Clay > > On Mon, Jan 19, 2015 at 9:09 AM, Susana González > wrote: > >> Hi C

Re: worker logs

Thanks Susana. Where do you specify the location of the logs? -Clay On Mon, Jan 19, 2015 at 9:09 AM, Susana González wrote: > Hi Clay, > > I use "logrotate" to rotate the storm worker logs (in my case they are > generated by default in $STORM_HOME/logs). You can create e.g. a file > called "sto

Re: worker logs

Hi Clay, I use "logrotate" to rotate the storm worker logs (in my case they are generated by default in $STORM_HOME/logs). You can create e.g. a file called "storm" in the folder /etc/logrotate.d with something like the following: /opt/storm/logs/worker-*.log { daily rotate 5 compress

Waiting for all bolts to prepare

Hey everybody We have an issue with bolts that take a long time to prepare. Since Storm doesn't wait for the entire topology to be ready before emitting tuples we get tuples in the topology before they can actually be processed. We have a low latency situation going and we need to be able to s

worker logs

Hi All, Sorry if these questions have asked before. I cannot find the answers. 1) Where do I specify the location of the worker logs? The default seems to be /tmp. 2) How do I rotate these logs? thanks, Clay