Dell - Internal Use - Confidential
Dell - Internal Use - Confidential
What logs are you seeing the out of memory error, can you post some stack
traces.
I would turn the -Xmx argument on the local machine to a much lower value after
enabling heapdumps on OOM. Then analyze the heapdump to see who is holding on
to the memory and if
Dell - Internal Use - Confidential
The previous reply was from March 10th so Bill may have resolved this issue but
here is some info that can be useful to other folks in the future.
You can set the JVM flags to do a heapdump on OOM using
-XX:+HeapDumpOnOutOfMemoryError.
You can then analyze th
Dell - Internal Use - Confidential
Thanks Bobby, for the detailed answer.
So it sounds like , it is better not to combine Storm with batch workloads at
this point (yarn, mesos or ec2), due to the network saturation and timeout
threats.
Is this behavior also seen in other streaming frameworks l
Dell - Internal Use - Confidential
Never mind found this on a previous thread right after I sent this email.
https://xumingming.sinaapp.com/885/twitter-storm-how-to-develop-a-pluggable-scheduler/
From: Kalluri, Rajesh - Dell Team
Sent: Friday, May 22, 2015 5:17 PM
To: user@storm.apache.org
Subje
Dell - Internal Use - Confidential
Nathan, can you point to some documentation on custom schedulers to use with
Storm.
From: Nathan Leung [mailto:ncle...@gmail.com]
Sent: Friday, May 22, 2015 5:07 PM
To: user
Subject: Re: Is it supported to set different JVM options for different
tasks(workers)?
Dell - Internal Use - Confidential
Mark,
Can you give us some information like how many workers (each with 48G of RAM)
you have in your cluster and how many bolts in your topology.
Tasks run on executor threads inside a worker JVM and based on how many workers
you have tasks share the same JVM.
Dell - Internal Use - Confidential
I am aware of the local mode, was looking for alternatives of on the cluster
testing.
From: Mark Reddy [mailto:mark.l.re...@gmail.com]
Sent: Thursday, May 21, 2015 2:37 AM
To: user@storm.apache.org
Subject: Re: Whats the alternative to creating a fat jar for top
Dell - Internal Use - Confidential
I am tired of waiting to build and upload my fat jar to the storm cluster.
If I am only changing logic and not dependencies, during debugging. What are my
options.
Can I create two jars, one for dependencies and one for the topology code.
Anyone done this befo
Dell - Internal Use - Confidential
If you learn better by reading a book try http://www.manning.com/sallen/.
It starts with basics and ramps up to reliable message processing and beyond.
From: Jeffery Maass [mailto:maas...@gmail.com]
Sent: Friday, May 15, 2015 10:18 AM
To: user@storm.apache.org
Dell - Internal Use - Confidential
Thanks Jeffery, great to see enthusiastic community around storm.
Do you have any suggestions on the log retention and indexing mechanisms
(logstash etc?)
From: Jeffery Maass [mailto:maas...@gmail.com]
Sent: Thursday, May 14, 2015 4:42 PM
To: user@storm.apache.
Dell - Internal Use - Confidential
Assuming your parallelism hints default to 1, you have combined parallelism of
6 and with 6 workers each would get 1 executor each.
So each of the bolt and task will run on a different worker(if you have >6
available slots on your cluster)
Checkout the awesome
Dell - Internal Use - Confidential
Nathan,
Can you explain in a little more detail what you mean by “When you have more
tasks than executors, the spout thread does the same logic, it just does it for
more tasks during its main loop.” I thought the spout thread emits tuples
based on the max spo
Dell - Internal Use - Confidential
Regarding the NotSerializableException:
I would like to see what grouping you have defined between the two bolts that
you have experienced the NotSerializableException with.
I am guessing it is localOrShuffleGrouping, can you change it to
shuffleGrouping and r
Dell - Internal Use - Confidential
what is every one using to process the storm logs from all over the cluster for
troubleshooting and debugging purposes.
The accessing storm logs via the storm UI or tailing worker logs is getting to
be pretty painful.
I guess this could be said of a lot of oth
Dell - Internal Use - Confidential
Using the same zk cluster for both storm and kafka.
From: Harsha [mailto:st...@harsha.io]
Sent: Wednesday, May 13, 2015 11:58 AM
To: Kalluri, Rajesh - Dell Team; user@storm.apache.org
Subject: Re: Strom Kafka spout has sporadic issue with figuring out partition
Dell - Internal Use - Confidential
Strom Kafka spout has sporadic issue with figuring out partition info from ZK
on startup.
Here is the stack trace. Any clues on how to troubleshoot this error?
Did not find much help on Stackoverflow and Google.
b.s.d.executor [ERROR]
java.lang.RuntimeExcepti
17 matches
Mail list logo
