Re: Streaming DRPC?

Hi Carl, DRPC is inherently synchronous in the way it works so if I understand what you are trying to do correctly then I suggest you stick to non-DRPC trident or even vanilla storm. You can setup some messaging queues to handle the input (request) and output (streaming result). Include a field

Measuring Latency and Throughput in Trident Topologies

Hi, I am new user to storm-trident, and struggling to understand how i can measure the throughput and latency on a trident topology. For example, if we take the TridentWorkCount Topology, and run it in a storm cluster, in *Topology Stats* of Storm UI, i see a big difference in the emitted transfe

Re: Measuring Latency and Throughput in Trident Topologies

Trident is actually a high level API over the regular Storm spout/bolts API. Upon submitting a Trident topology, Storm reorganizes the various topology components (spouts, functions, filters, etc) into as few Storm bolts as possible. How it does so is described on Storm's wiki. Yes, that means this

Reading state of a streaming topology via DRPC?

Hi, I want to create a bolt that keeps some kind of state (aggregate) about the messages it has seen so far (i.e. web click stream). Once such a bolt has gathered information I would like to get to that information for an application I designing. So far I've come up with two way of getting at thi

collecting global statistics in a topology

Hi I have a question regarding collecting of stats data from running topology for example in a simple scenario having let say counters storm_read bolt_A_read bolt_B_read and topology Spout -> Bolt A -> Bolt B Spout -> increments storm_read counter on every nextTuple Bolt A -> increments bol

Re: [Need help] [How to benchmark storm topologies]

Thanks for the reply :) On Sat, Feb 15, 2014 at 1:17 PM, Mark Hu wrote: > We instrument our spout and bolts using statsd and graphite due to its > ease of use and flexibility. Its UDP and won't add additional risk or > vulnerabilities to your topologies. > On Feb 15, 2014 1:05 AM, "Abhishek Bha

Re: How to identify Transaction success in IPartitionedTridentSpout?

Hi, Can someone please attempt to answer this question? Its a crucial need for my project. Without a deterministic way to find out when a transaction is completed, I cant trigger the related actions happening outside of the trident topology. If you feel more details are required on the question, pl

Re: collecting global statistics in a topology

One option includes statsd and graphite for safe metrics gathering and easy graphing over time. On Feb 15, 2014 4:09 PM, "Andrew Milkowski" wrote: > Hi > > I have a question regarding collecting of stats data from running topology > > for example in a simple scenario > > having let say counters >

Re: Streaming DRPC?

Hey Adam, Actually, that's quite a good idea. I'm glad you responded, this is a better approach than what I was going to attempt (aka, mega hacks). I understand how your approach could be done with non-DRPC trident. The one drawback that I can think of would be that the request message would need

Re: collecting global statistics in a topology

Thanks Mark with custom consumer metric however spout can not record anything and beans are not serialized like JMX but for bolt operation and graphs certainly ok So if there is a stat in a spout that never made to a bolt that metric can not be recorded in the consumer at least in the most recent