Re: Storm cluster with time

HI sam, when you have a bottleneck issue your spout sats in UI shows the failed tuples or your bolts will turn into red colour in visualisation view of your topology. when you note such changes in UI then your topology is having bottleneck. To avoid this bottle-necks you just need to increase pa

Re: Storm 1.0.0 upgrade Serialization issue

Yes sure, here is storm-core jar 1.0.2 snapshot which just applies STORM-1773 patch into Storm 1.0.1. http://people.apache.org/~kabhwan/storm-core-1.0.2-SNAPSHOT.jar It just changes the version of common-io from 2.4 to 2.5. Please let me know if this works so that I can submit pull request. Thank

Is Storm visualization enough for performance ?

Iam researcher and my goal was in algorithm in project using storm to make data more accuracy .I finished the project and submitted topology but need to make comparison between old I have and new i made . Need to know if I made a valuable change or not ? Is storm visualization in storm 0.9.6 enoug

Re: Storm cluster with time

Thanks for replying . But how can I find bottleneck from storm ui . And what can i do if I find it ? On Wednesday, May 11, 2016, cogumelosmaravilha wrote: > You can check where is the bottleneck using the ui of Storm. > > On 10-05-2016 22:20, sam mohel wrote: > > Iam a researcher and I hope yo

Re: Storm cluster with time

You can check where is the bottleneck using the ui of Storm. On 10-05-2016 22:20, sam mohel wrote: Iam a researcher and I hope you can help me on this questions my goal is to update algorithm that coder used to make data more accuracy. I have one Java process that make me need one supervisor

Storm cluster with time

Iam a researcher and I hope you can help me on this questions my goal is to update algorithm that coder used to make data more accuracy. I have one Java process that make me need one supervisor right ? When I made machine core i5 with 8 ram as supervisor and machine core2duo with 2 ram as nimbus

Custom resource aware scheduler

Hello all , Anyone, please help me in getting the logic . While using the new scheduler for storm 1.0.0 I am getting the following error on decreasing the number of sup. slots from 8 to 4 on 4 core machine . *(Without changing any other setting in YAML file on supervisors it is working fine f

Storm 1.0.0 simple UI auth

I have the following configured in my storm.yaml file: ui.filter: “org.apache.hadoop.security.authentication.server.AuthenticationFilter" ui.filter.params: "type": "simple" "user.name": “stormtest" When I try to access the UI passing the parameter ?user.name=stormtest all is well. I’m g

Re: Getting Kafka Offset in Storm Bolt

Anybody ? Anything about this ? On Wed, May 4, 2016 at 11:31 AM, Milind Vaidya wrote: > Is there any way I can know what Kafka offset corresponds to current tuple > I am processing in a bolt ? > > Use case : Need to batch events from Kafka, persists them to a local file > and eventually upload i

Re: Storm 1.0.0 upgrade Serialization issue

Thanks for this update. Actually our topology is built on multiple wrappers on top of Storm. I'll try to create simple topology to reproduce the problem. Meanwhile would it be possible to create a snapshot release with the fix. I'll test and let you know. Please let me know. Regards, On Tue, May

FTPSpout implementation

Is there an FTPSpout implementation somewhere, similar to the HDFS spout? If not, I'll look to implement one, using HdfsSpout as a model. Thanks. Manish

Re: How to let a topology know that it's time to stop?

Hmm...yes that's a better idea. On Tue, May 10, 2016 at 3:12 PM, Matthias J. Sax wrote: > I am not sure if NimbusClient works well with LocalCluster. My > suggestion was based on the assumption, that you run in a real cluster. > > There would be LocalCluster.killTopology(); maybe you should use

Re: How to let a topology know that it's time to stop?

I am not sure if NimbusClient works well with LocalCluster. My suggestion was based on the assumption, that you run in a real cluster. There would be LocalCluster.killTopology(); maybe you should use this method instead of NimbusClient.kill(). Using LocalCluster, I usually use the following patte

storm 0.10, supervisor constantly dies when it tries to start task.

Hi, don't know what to do. What is the right debug the problem? Supervisor constantly dies with provided execution on the same machine... 2016-05-06 23:12:15.271 b.s.event [ERROR] Error when processing event java.lang.RuntimeException: org.apache.thrift7.protocol.TProtocolException: Required field

Re: How to let a topology know that it's time to stop?

Turns out, using nimbus.seeds was sufficient. *import org.apache.storm.utils.NimbusClient;import org.apache.storm.utils.Utils;Map conf = Utils.readStormConfig();conf.put("nimbus.seeds", "localhost");NimbusClient cc = NimbusClient.getConfiguredClient(conf);

Re: Spout Thread Waiting

> I haven't set topology.max.spout.pending. Using the default value, but > I'm watching a internal Set to check the size of beeing processed tuple. > Found it in the snippet you posted earlier. This duplicates what topology.max.spout.pending does so unless you have another use for that internal se

Re: How to let a topology know that it's time to stop?

My bad. The parameter is called "nimbus.seeds" (former "nimbus.host") and not "nimbus.leader". And I guess, "build/libs" is not your working directory. (See you IDE setting of your run configuration.) In doubt, include a "System.out.println(new File().getAbsolutePath());" (or similar) in your bo