Re:

[cutoutsy Feng]

try shutdown the firewall...,maybe it works.

2015-07-21 23:27 GMT+08:00 Harsha :

> By default it runs on 8080 if not you can look at storm.yaml for
> configured “ui.port"
>
> --
> Harsha
>
>
> On July 21, 2015 at 7:01:02 AM, Vamsikrishna Vinjam (
> vamsikrishna.vin...@inndata.in) wrote:
>
> storm web port number :
>
> im trying with 8772 but it is not working
>
>

Is storm 0.8 compatible with kafka 0.8?

[S. Zhou]

We are using storm 0.8 with Kafka 0.6. Now we want to upgrade Kafka to 0.8 Is 
Storm 0.8 compatible with Kafka 0.8?
Thanks!

Re: Getting Metrics from MetricsConsumer

[John Reilly]

fwiw, I can confirm it did not work for me either in 0.9.3.

On Tue, Jul 21, 2015 at 12:46 PM Hong Jeon  wrote:

> Hi,
>
> Is there any way for the MetricsConsumer to send Metrics about itself to
> itself?
> Basically, I want to be able to track the % thread usage for a
> MetricsConsumer task so that I can figure out if I need to increase
> parallelism etc. I tried registering a Metric in the MetricConsumer's
> prepare method, but the MetricsConsumer does not seem to be receiving it.
>
> Any thoughts on this would be appreciated. Thanks.
>
> Best,
> Hong
>

Re: Storm topology freeze and slow down

[Manish Nema]

Hi Nikos
Yes I am acking tuples in all bolt ...from all path in code
Regards
Manish
On 21 Jul 2015 20:31, "Nick R. Katsipoulakis"  wrote:

> Hello Manish,
>
> Do you make sure that you are ACK-ing tuples after processing?
>
> Nikos
>
> 2015-07-21 10:57 GMT-04:00 Manish Nema :
>
>> Hi,
>>  I am running a topology in the 6 node cluster on RHEL 5 and Oracle JDK
>> 1.7. Schematic for topology as attached, there are 6 workers for spout and
>> bolts. Topology picks up message from JMS and sends out message to bolts.
>> Each bolt does some processing and sends out next set of tuple to
>> downstream for further processing. All bolts are in shuffle grouping. There
>> are two special bolts
>> 1. Log
>> 2. Ticket
>> Log bolt receive same tuple from each bolt for logging purpose but these
>> tuples are unanchored. Similarly Ticket bolt receive tuple that are failed
>> due to business logic, these are also unanchored. Bolt1 to Bolt4 and
>> anchored tuple since we wanted message guarantee for these tuples. Each
>> bolt receive one tuple and emit one data tuple from Bolt1 to Bolt4.
>>
>> I have observed that we were acking some unanchored tuple in Log and
>> Ticket bolts, and it was causing topology freeze after sometime (10 mins
>> when worker jvm heap size is set to 2 GB , 2 days when worker jvm heap was
>> set to 16 GB). These freeze essentially slows down message read from JMS.
>> After commenting out acking I didnt observe any slow down/freeze.
>>
>>
>> I would like to ask group that would acking unanchored tuple is the
>> reason for the topology slow down or freeze ?
>>
>> Regards,
>> Manish
>>
>>
>
>
> --
> Nikolaos Romanos Katsipoulakis,
> University of Pittsburgh, PhD candidate
>

Getting Metrics from MetricsConsumer

[Hong Jeon]

Hi,

Is there any way for the MetricsConsumer to send Metrics about itself to
itself?
Basically, I want to be able to track the % thread usage for a
MetricsConsumer task so that I can figure out if I need to increase
parallelism etc. I tried registering a Metric in the MetricConsumer's
prepare method, but the MetricsConsumer does not seem to be receiving it.

Any thoughts on this would be appreciated. Thanks.

Best,
Hong

Kafka Consumer Producer authentication

[Kushan Maskey]

Is there a way we can set authentication for storm processes to
consume/produce messages to/from Kafka? For example by passing a token key
or something like that? Or is there any other way to do it?

--
Kushan Maskey

Getting Metrics from MetricsConsumer

[Hong Jeon]

Hi,

Is there any way for the MetricsConsumer to send Metrics about itself to
itself?
Basically, I want to be able to track the % thread usage for a
MetricsConsumer task so that I can figure out if I need to increase
parallelism etc. I tried registering a Metric in the MetricConsumer's
prepare method, but the MetricsConsumer does not seem to be receiving it.

Any thoughts on this would be appreciated. Thanks.

Best,
Hong

Re:

[Harsha]

By default it runs on 8080 if not you can look at storm.yaml for configured 
“ui.port"

-- 
Harsha


On July 21, 2015 at 7:01:02 AM, Vamsikrishna Vinjam 
(vamsikrishna.vin...@inndata.in) wrote:

storm web port number :

im trying with 8772 but it is not working

Re: Storm topology freeze and slow down

[Nick R. Katsipoulakis]

Hello Manish,

Do you make sure that you are ACK-ing tuples after processing?

Nikos

2015-07-21 10:57 GMT-04:00 Manish Nema :

> Hi,
>  I am running a topology in the 6 node cluster on RHEL 5 and Oracle JDK
> 1.7. Schematic for topology as attached, there are 6 workers for spout and
> bolts. Topology picks up message from JMS and sends out message to bolts.
> Each bolt does some processing and sends out next set of tuple to
> downstream for further processing. All bolts are in shuffle grouping. There
> are two special bolts
> 1. Log
> 2. Ticket
> Log bolt receive same tuple from each bolt for logging purpose but these
> tuples are unanchored. Similarly Ticket bolt receive tuple that are failed
> due to business logic, these are also unanchored. Bolt1 to Bolt4 and
> anchored tuple since we wanted message guarantee for these tuples. Each
> bolt receive one tuple and emit one data tuple from Bolt1 to Bolt4.
>
> I have observed that we were acking some unanchored tuple in Log and
> Ticket bolts, and it was causing topology freeze after sometime (10 mins
> when worker jvm heap size is set to 2 GB , 2 days when worker jvm heap was
> set to 16 GB). These freeze essentially slows down message read from JMS.
> After commenting out acking I didnt observe any slow down/freeze.
>
>
> I would like to ask group that would acking unanchored tuple is the reason
> for the topology slow down or freeze ?
>
> Regards,
> Manish
>
>


-- 
Nikolaos Romanos Katsipoulakis,
University of Pittsburgh, PhD candidate

Fwd: Storm topology freeze and slow down

[Manish Nema]

Hi,
 I am running a topology in the 6 node cluster on RHEL 5 and Oracle JDK
1.7. Schematic for topology as attached, there are 6 workers for spout and
bolts. Topology picks up message from JMS and sends out message to bolts.
Each bolt does some processing and sends out next set of tuple to
downstream for further processing. All bolts are in shuffle grouping. There
are two special bolts
1. Log
2. Ticket
Log bolt receive same tuple from each bolt for logging purpose but these
tuples are unanchored. Similarly Ticket bolt receive tuple that are failed
due to business logic, these are also unanchored. Bolt1 to Bolt4 and
anchored tuple since we wanted message guarantee for these tuples. Each
bolt receive one tuple and emit one data tuple from Bolt1 to Bolt4.

I have observed that we were acking some unanchored tuple in Log and Ticket
bolts, and it was causing topology freeze after sometime (10 mins when
worker jvm heap size is set to 2 GB , 2 days when worker jvm heap was set
to 16 GB). These freeze essentially slows down message read from JMS. After
commenting out acking I didnt observe any slow down/freeze.


I would like to ask group that would acking unanchored tuple is the reason
for the topology slow down or freeze ?

Regards,
Manish

[no subject]

[Vamsikrishna Vinjam]

storm web port number :

im trying with 8772 but it is not working

Getting Metrics from MetricsConsumer

[Hong Jeon]

Hi,

Is there any way for the MetricsConsumer to send Metrics about itself to
itself?
Basically, I want to be able to track the % thread usage for a
MetricsConsumer task so that I can figure out if I need to increase
parallelism etc. I tried registering a Metric in the MetricConsumer's
prepare method, but the MetricsConsumer does not seem to be receiving it.

Any thoughts on this would be appreciated. Thanks.

Best,
Hong

Re: What happens when a message times out?

[Matthias J. Sax]

If the call to "execute()" does loop infinitely, yes; the whole
computation stops (or might fail completely with an exception if all
buffers are full -> OutOfMemoryException)

-Matthias


On 07/21/2015 02:05 PM, Ganesh Chandrasekaran wrote:
> So let’s say we have a single threaded topology with single worker.
> Doesn’t this mean that if a my computation gets stuck at some point all
> the messages in the queue get blocked too? Because the fail method will
> be called on the queue but there is no thread to process the next
> message. Is that correct?
> 
>  
> 
> *From:*Nathan Leung [mailto:ncle...@gmail.com]
> *Sent:* Monday, July 20, 2015 4:07 PM
> *To:* user
> *Subject:* Re: What happens when a message times out?
> 
>  
> 
> The computation (and the tuple) continue on.  The fail() method will be
> called on the spout with the message id.
> 
>  
> 
> On Mon, Jul 20, 2015 at 1:51 PM, Mark Tomko  > wrote:
> 
> Suppose we have a simple topology with a spout and a single bolt, and
> the tuple timeout is set to some value. When a message exceeds the
> processing time allotted, even if the bolt is still working, what
> happens to that computation? Does the framework kill the bolt or
> interrupt the executing thread and reallocate it to some other message?
> Does it leave the bolt working to finish its work, starting a new thread
> in its place?
> 
>  
> 
> Thanks,
> 
> Mark
> 
>  
> 



signature.asc
Description: OpenPGP digital signature

RE: What happens when a message times out?

[Ganesh Chandrasekaran]

So let’s say we have a single threaded topology with single worker. Doesn’t 
this mean that if a my computation gets stuck at some point all the messages in 
the queue get blocked too? Because the fail method will be called on the queue 
but there is no thread to process the next message. Is that correct?

From: Nathan Leung [mailto:ncle...@gmail.com]
Sent: Monday, July 20, 2015 4:07 PM
To: user
Subject: Re: What happens when a message times out?

The computation (and the tuple) continue on.  The fail() method will be called 
on the spout with the message id.

On Mon, Jul 20, 2015 at 1:51 PM, Mark Tomko 
mailto:mto...@broadinstitute.org>> wrote:
Suppose we have a simple topology with a spout and a single bolt, and the tuple 
timeout is set to some value. When a message exceeds the processing time 
allotted, even if the bolt is still working, what happens to that computation? 
Does the framework kill the bolt or interrupt the executing thread and 
reallocate it to some other message? Does it leave the bolt working to finish 
its work, starting a new thread in its place?

Thanks,
Mark

Storrm worker issue .daemon.supervisor still hasn't started when using apostrophe char in launcher command

[Spico Florin]