Re: groupBy / persistentAggregate and "distribution"

looks like this article https://svendvanderveken.wordpress.com/2013/07/30/scalable-real-time-state-update-with-storm/ gave me some answers. On Wed, Apr 1, 2015 at 5:15 PM, Olivier Mallassi wrote: > Hi all > > I would like to better understand how groupBy/persistentAggregate works in > a "distrib

KafkaSpout : How to recover from corrupted segment

Hi all, We are into a situation where our topology generated java.io.Exception : FAILED_TO_UNCOMPRESS (5) when it reached a given offset. It is no longer consuming data from any partition. We identified the broker / partition / offset and ran a tool to find out the range of offset that are un

Re: Enabling SSL on Storm UI

I haven’t added two-way SSL in the current PR. I can add that as part of this PR. --  Harsha On April 1, 2015 at 10:36:44 AM, Mike Thomsen (mikerthom...@gmail.com) wrote: That's what I was afraid of. Any idea when that PR is going to be merged? Also, will it support two-way SSL? Some of our u

Problem with an storm cluster and nimbus continuos rebalancing

Hi, I'm having some problems with an storm cluster that produces that nimbus rebalance the topology too often because thinks that some workers are down. My setup is using apache-storm 0.9.3: 2 storm nodes running storm-supervisor (storm-1 and storm-2) 1 server running storm-nimbus and storm-ui a

Re: Enabling SSL on Storm UI

That's what I was afraid of. Any idea when that PR is going to be merged? Also, will it support two-way SSL? Some of our users need support for logging in via client cert. Thanks, Mike On Wed, Apr 1, 2015 at 12:58 PM, Harsha wrote: > Hi Mike, > Current release versions don't have a supp

Re: Enabling SSL on Storm UI

Hi Mike, Current release versions don't have a support for HTTPS for UI. I have a PR open here https://github.com/apache/storm/pull/479 to add that support. -Harsha On Wed, Apr 1, 2015, at 09:49 AM, Mike Thomsen wrote: > Is it possible to have Storm UI run over HTTPS? I've done a decent > bit of

Re: Using Storm with IBM MQ Series

Hey Taylor : I would love to collaborate with you on this! However, I don't have Storm up and running yet. Right now, I'm just investigating whether or not we can use Storm for this project, or if I need to pick a different technology. It sounds like we may be able to use Storm if we put some de

Enabling SSL on Storm UI

Is it possible to have Storm UI run over HTTPS? I've done a decent bit of research on Google, but couldn't find any references on how to do it. Thanks, Mike

Re: Using Storm with IBM MQ Series

Memory slightly refreshed, but I don’t have access to an IBM-MQ instance at the moment, so I can’t verify… In the storm-jms spout[1], we receive a JMS message in the JMS `onMessage()` method (the thread that invokes that method is a JMS thread) and put it in a queue so we can perform the JMS me

Re: Using Storm with IBM MQ Series

Thank you, Alesky. A couple of questions : 1) How well do MQ Series and Storm work together? Have you run into any performance issues? You mentioned that you are not happy with the current solution. What do you dislike about it? 2) You mention that you need to scale topology throughput adding

RE: Using Storm with IBM MQ Series

Hello Parth, We are using JMS API, means in spout I use Connection, Session, Destination, MessageConsumer, etc. Implementation is MQ provided, and, as I know, it's not open source. Thanks, Aleksey. From: Parth Brahmbhatt [mailto:pbrahmbh...@hortonworks.com] Sent: Wednesday, April 01,

Re: Using Storm with IBM MQ Series

Hi Jeremy, I was the engineer who gave that webinar. And the statement you quoted is misleading (IIRC, that’s not the wording I used when the question was asked). Nathan is correct, the nextTuple(), ack() and fail() methods are all called from the same thread. At the moment I don’t have access

Re: Using Storm with IBM MQ Series

Hi Aleksey, I am guessing you are using proprietary IBM-MQ Apis instead of using the JMS API. Is that right? Is this code open sourced? Thanks Parth From: "aleksey.di...@ubs.com" mailto:aleksey.di...@ubs.com>> Reply-To: "user@storm.apache.org

Re: Using Storm with IBM MQ Series

All good. Thank you for your help! I really appreciate it. Please keep us updated as to your findings. Likewise, I'll let you know if I uncover anything. --Jeremy On Wed, Apr 1, 2015 at 11:37 AM, Parth Brahmbhatt < pbrahmbh...@hortonworks.com> wrote: > Jeremy, > > Taylor and I work for the

RE: Using Storm with IBM MQ Series

Hello Jeremy, I'm using Storm with IBM MQ for my project a half of a year. In MQ you have no per message acknowledgment – that the problem. You could ack all messages received in current session only altogether. That makes Parth to make his trick with pending list committing only when

Re: Using Storm with IBM MQ Series

Jeremy, Taylor and I work for the same company and the webinar contains the same info because it is coming from the same source. We will investigate further what caused the IBM-MQ integration failure but we can not promise a timeline. Once again apologies for providing inaccurate info in first

Re: Using Storm with IBM MQ Series

Nathan : thanks for the response! It appears that the engineer who gave this webinar ran into the same problem as Parth : http://hortonworks.com/blog/discover-hdp-2-2-apache-kafka-apache-storm-stream-data-processing/ He reports that he was unable to make MQ Series work with Storm because : "We ran

groupBy / persistentAggregate and "distribution"

Hi all I would like to better understand how groupBy/persistentAggregate works in a "distributed environment" I got the following topology (just an extract) Stream currStream = topology.newStream(this.getTopologyName(), jmsspout) ... currStream.groupBy(new Fields(A,B)) .persistentAggregate(myD

答复: How to check program log when the program is submitted?

Sorry for some mistakes. In default I output log to console, and redirect it to a dedicated file, which I called log files in the question. So my question should be, how can I see the console output of my program? 发件人: Junfeng Chen [mailto:k-2f...@hotmail.com] 发送时间: 2015年4月1日 16:19 收件人: user@

How to check program log when the program is submitted?

I meet a tcp connection problem which I didn't meet on LocalCluster mode. Now I need to find the program log(I print it to console with log4j ), how to find there log files? Thanks