Re: Discrepancy in Storm while Job Distribution on available nodes

2015-02-25 Thread Nathan Leung
It looks like you configured your topology to only use one worker process. On Wed, Feb 25, 2015 at 6:39 AM, Vineet Mishra clearmido...@gmail.com wrote: Hi All, I am running a 3 node storm cluster with a Topology which is subscribed to Kafka using Kafka-Storm Spout. The Cluster is hosting

Re: Discrepancy in Storm while Job Distribution on available nodes

2015-02-25 Thread Nathan Leung
In practice it's very difficult to fully maximize cpu usage, especially when each thread has to do things like wait for i/o in any way (network, disk, etc). Since you're cpu limited, or very close to it, you should try adding more workers. On Feb 25, 2015 7:48 AM, Vineet Mishra

Re: Urgent - Some workers stop processing after a few seconds

2015-02-25 Thread Harsha
Hi Martin, Can you share your storm.zookeeper.session.timeout and storm.zookeeper.connection.timeout and supervisor.worker.timeout.secs. By looking at the supervisor logs I see Error when processing event java.io.FileNotFoundException: File

Re: Discrepancy in Storm while Job Distribution on available nodes

2015-02-25 Thread Nathan Leung
If you have only one node on the cluster then yes I would recommend to add more nodes. If you have more than one I would recommend you increase the worker processes in the topology configuration to use the other hardware. On Feb 25, 2015 10:38 AM, Vineet Mishra clearmido...@gmail.com wrote: If I

Re: Discrepancy in Storm while Job Distribution on available nodes

2015-02-25 Thread Vineet Mishra
Hi Nathan, I guess you missed the first mail of this thread. As mentioned before I m having 3 node cluster out of which node 1 is for nimbus and ui while other 2 nodes are for worker and supervisor. The specification of machine is 4 core intel xeon processor which currently is running with more