Re: Multilang, C and binary data

Hi, You may have a look at: http://demeter.inf.ed.ac.uk/cross/stormcpp.html It may help you to run storm with binary c code. I am a storm beginner, and cannot help you with other questions... --- Xiang Wang PhD Candidate Database Research Group School of Computer

Re: Multilang, C and binary data

I have provided an implementation `MessagePackSerializer` for improving muti-lang performance. (PR: https://github.com/apache/storm/pull/1136). You can take a look at this. It's not merged yet. Thanks, Xin 2016-03-22 18:37 GMT+08:00 Brian Candler : > Hello, > > I have some questions about exte

Re: define a cycle in storm topology

Hi Sai, Thank you for your kind reply. The problem has been solved based your suggestion. Actually, there is no need to add bolt C. We can simply redirect the output of B to A by adding B as one of A's incoming stream sources. Best, Xiang --- Xiang Wang PhD Candidate

Re: define a cycle in storm topology

Hi Ravi, Thank you for your kind reply. I have solved this problem based your suggestion. Thanks:) Best, Xiang --- Xiang Wang PhD Candidate Database Research Group School of Computer Science and Engineering The University of New South Wales Sydney, Australia On Tue,

Re: external/storm-elasticsearch - upgrade requested

No, we hadn't looked at that, but once we are streaming into elastic at scale (not there yet) I would be interested to compare. From: Tech Id mailto:tech.login@gmail.com>> Reply-To: "user@storm.apache.org" mailto:user@storm.apache.org>> Date: Monday, March 21, 2

Re: [HELP!] Storm Network Performance?

On 22/03/2016 20:13, Matthew Lowe wrote: The maxTuplePending is set to 35 (During testing we found 3 ackers and 35 max pending was “good") Have you tried increasing this now you have built your spout-bolt-bolt topology? When testing with only the Spout (no bolts) I was emitting about 70,000 pe

[HELP!] Storm Network Performance?

Hello, I seems to be getting decreasing performance for each additional bolt I add to a topology. I would like to know what performance decreases to expect when making longer topologies. Here is my current topology: Spout —shuffleGrouping--> Bolt1 —fieldGrouping--> Bolt2 1 x Spout, 1 x Bolt1, 1

Re: define a cycle in storm topology

Hi Ziang, I think you should add one more bolt C which takes the stream updates from B and give those to update streams to A. *Best regards,* *K.Sai Dilip Reddy* On Tue, Mar 22, 2016 at 3:40 PM, Ravi Sharma wrote: > Hi Ziang, > I think you should be able to define it. But you will have to

How Storm take topic data which is more than the default value of 1MB.

I'm Trying to get the fb pages data using graph api. The size each post is more than 1MB where kafka default fetch.message is 1MB. I have changed the kafka properties from 1MB to 3MB by adding the below lines in kafa consumer.properties and server.properties file. fetch.message.max.bytes=3048576 (

How to change default kafka SpoutConfig class

hai., I am getting the message stream of 3MB from kafka topic but the default value is 1MB. Now I have changed the kafka properties from 1MB to 3MB by adding the below lines in kafa consumer.properties and server.properties file. fetch.message.max.bytes=2048576 ( consumer.properties )

Multilang, C and binary data

Hello, I have some questions about external workers and the multi-lang protocol. We have a bunch of existing C code for running processing steps over binary data and I'm looking to see how feasible it is to hook it into Storm. (1) Is it possible to handle binary data with multi-lang? Or is

Re: define a cycle in storm topology

Hi Ziang, I think you should be able to define it. But you will have to make sure that you wont go into infinite loop. Ravi. On Tue, Mar 22, 2016 at 3:44 AM, Xiang Wang wrote: > ​Does anyone could help? > Thanks. ​ > > > --- > Xiang Wang PhD Candidate > Database Rese

Re: using storm-kafka-0.10.0.jar in storm 0.10.0

Hello All, Some more updates here.. I checked the "bin/storm classpath" and it is properly referring to kafka jar file. But once uber jar is ready I do not see any kafka class inside the jar file which is generated in target directory. Inside jar there is an empty kafka folder. I was expecting

Re: using storm-kafka-0.10.0.jar in storm 0.10.0

I guess I am already using uber jar (shade plugin) in my project. For more information this is the default pom.xml file which is provided in storm starter project. Still no success. Any suggestion? Attached is the pom.xml file. Thanks.. On Tue, Mar 22, 2016 at 3:21 PM, Abhishek Agarwal wrote:

Re: using storm-kafka-0.10.0.jar in storm 0.10.0

https://maven.apache.org/plugins/maven-shade-plugin/ On Tue, Mar 22, 2016 at 12:46 PM, Neeraj Kumar wrote: > Hi Abhishek, > > I am sorry I am new to storm and I don't know about uber jar. > > Could you please provide some more information? > > Thanks for your time. > > Neeraj > > > On Tue, Mar

Re: using storm-kafka-0.10.0.jar in storm 0.10.0

Hi Abhishek, I am sorry I am new to storm and I don't know about uber jar. Could you please provide some more information? Thanks for your time. Neeraj On Tue, Mar 22, 2016 at 3:11 PM, Abhishek Agarwal wrote: > why not shade it in your application uber jar? > > On Tue, Mar 22, 2016 at 11:2

Re: using storm-kafka-0.10.0.jar in storm 0.10.0

why not shade it in your application uber jar? On Tue, Mar 22, 2016 at 11:25 AM, Neeraj Kumar wrote: > Hi, > > Could somebody let me know how to use storm-kafka-0.10.0.jar file which > resides in > external directory to storm starter project. > > I copied the storm-kafka-0.10.0.jar in lib direct