Re: Re: Kafka spout stops emmiting messages

[fanxi...@travelsky.com]

Hi, Kuzmenko：

250 is a small number, when the bolt can not process the tuple in time, the 
spout will stop emitting. But the kafka cosumer coodinator has a timeout 
parameter, like 30s, if the spout has not fetched message from kafka in 30s, 
this spout as a consumer will be kicked out by consumer coodinator thinking the 
consumer is dead. So, even the bolt processed the pending tuple, the spout can 
not  get message anymore. You can assure this by checking the worker log.



Josh
 
From: Igor Kuzmenko
Date: 2017-01-24 17:57
To: user
Subject: Re: Kafka spout stops emmiting messages
Thanks for reply, Josh.
My maxUncommitedOffset was 250, increasing uncommited offset helped me, but I 
still don't understand why spout completly stoped emiting tuples. You said, 
that eventualy spout will produce new tuples, after old one will be acked, but 
in my case didn't.




On Tue, Jan 24, 2017 at 4:24 AM, fanxi...@travelsky.com 
<fanxi...@travelsky.com> wrote:
Hi, Kuzmenko：

please pay attention to the number about setMaxUncommittedOffsets, if this 
number is too small, the spout may stop  emitting until the pending tuple is 
acked by the down bolt. You can change the number to a large number.



Josh

From: Igor Kuzmenko
Date: 2017-01-24 02:28
To: user
Subject: Kafka spout stops emmiting messages
Hello, I'm trying to upgrade my topology from old kafka spout (storm-kafka 
project) to new one (storm-kafka-client) version 1.0.1. I've configured new 
spout to work with my topology. After deploy it processes and acks few hundreds 
of tuples and then stops. Kafka topic definitely have new messages, and in 
storm UI I can see  kafka spout lag increasing. What could be the problem?

The acker does not work well.

[fanxi...@travelsky.com]

Hi  user,

Does anyone know why it happen? The acker does not send message to spout, and 
almost all of the message is timeout.



Joshua
2016-12-20 19:19:45

Re: Re: Urgent help! kafka-spout stops fetching data after running for a while

[fanxi...@travelsky.com]

Hi Junfeng：

Thanks for your help.
Yes, I use a kafka-spout written by myself.  I use storm-kafka-client because I 
want to implement the SSL and ACL with Kafka topic.
Now, I just test the kafka-spout, and the volume is small.



Joshua


 
From: Junfeng Chen
Date: 2016-09-29 15:27
To: user@storm apache. org
Subject: Re: Re: Urgent help! kafka-spout stops fetching data after running for 
a while
I have ever met the similar problem before, but if I only use kafka spout and 
print bolt, it runs smoothly without any error. i notice you are using 
storm-kafka-client, so you are using the kafka spout written by yourself? if 
yes, try kafka spout provided by Storm. And I am not sure how much of your data 
volume. if your data volume is large, increase the maxspoutpending value and  
try to use localOrShuffleGrouping between kafka spout and print bolt to reduce 
network pressure. 

Regard,
Junfeng Chen

On Thu, Sep 29, 2016 at 3:17 PM, fanxi...@travelsky.com 
<fanxi...@travelsky.com> wrote:
Hi Junfeng,

Actually, the topology just has two components, one is kafkaspout, and the 
other is the a print-bolt.

the bolt does nothing but print the tuple message itself got.

Can you suggest what kind of config will raise suche a problem? I will check 
the kafka-spout config.



Joshua


 
From: Junfeng Chen
Date: 2016-09-29 15:05
To: user@storm apache. org
Subject: Re: Re: Urgent help! kafka-spout stops fetching data after running for 
a while
I suggest you to run the topology containing kafka spout only, aiming to find 
out if it is the kafka config problem. If kafka spout is running well, it means 
your bolts drag the whole topology. 

Regard,
Junfeng Chen

On Thu, Sep 29, 2016 at 2:52 PM, fanxi...@travelsky.com 
<fanxi...@travelsky.com> wrote:
Hi Junfeng：

The usage of the worker seems ok too, the %cpu is about 10, and the %MEM is 
about 5.



Joshua


 
From: Junfeng Chen
Date: 2016-09-29 13:41
To: user@storm apache. org
Subject: Re: Re: Urgent help! kafka-spout stops fetching data after running for 
a while
Did you check the cpu usage of processes which run the kafkaspout. 

Regard,
Junfeng Chen

On Thu, Sep 29, 2016 at 12:46 PM, fanxi...@travelsky.com 
<fanxi...@travelsky.com> wrote:
Hi Junfeng ：

Yes, I checked the worker log, it seems ok, no error or warning is printed.



Joshua


 
From: Junfeng Chen
Date: 2016-09-29 11:29
To: user@storm apache. org
Subject: Re: Urgent help! kafka-spout stops fetching data after running for a 
while
Have you checked your worker log file? Frequent worker accident may cause kafka 
spout not working. 

Regard,
Junfeng Chen

On Thu, Sep 29, 2016 at 10:54 AM, fanxi...@travelsky.com 
<fanxi...@travelsky.com> wrote:
Hi  user,

My kafka-spout is based on storm-kafka-client.

It can read existed data from kafka.

It also can read data that I sent to kafka when the topology just runs for a 
while.

But when there is no data, the topology remains processing nothing for a long 
while, like 1 hour, I begin to send data to the topic, the topology just can 
not read the data.

I read the mails and find a way to set topology.backpressure.enable to false.

But the problem remains.

Great apprication for any advice.



Joshua
2016-09-29 10:43:11

Re: Re: Urgent help! kafka-spout stops fetching data after running for a while

[fanxi...@travelsky.com]

Hi Junfeng,

Actually, the topology just has two components, one is kafkaspout, and the 
other is the a print-bolt.

the bolt does nothing but print the tuple message itself got.

Can you suggest what kind of config will raise suche a problem? I will check 
the kafka-spout config.



Joshua


 
From: Junfeng Chen
Date: 2016-09-29 15:05
To: user@storm apache. org
Subject: Re: Re: Urgent help! kafka-spout stops fetching data after running for 
a while
I suggest you to run the topology containing kafka spout only, aiming to find 
out if it is the kafka config problem. If kafka spout is running well, it means 
your bolts drag the whole topology. 

Regard,
Junfeng Chen

On Thu, Sep 29, 2016 at 2:52 PM, fanxi...@travelsky.com 
<fanxi...@travelsky.com> wrote:
Hi Junfeng：

The usage of the worker seems ok too, the %cpu is about 10, and the %MEM is 
about 5.



Joshua


 
From: Junfeng Chen
Date: 2016-09-29 13:41
To: user@storm apache. org
Subject: Re: Re: Urgent help! kafka-spout stops fetching data after running for 
a while
Did you check the cpu usage of processes which run the kafkaspout. 

Regard,
Junfeng Chen

On Thu, Sep 29, 2016 at 12:46 PM, fanxi...@travelsky.com 
<fanxi...@travelsky.com> wrote:
Hi Junfeng ：

Yes, I checked the worker log, it seems ok, no error or warning is printed.



Joshua


 
From: Junfeng Chen
Date: 2016-09-29 11:29
To: user@storm apache. org
Subject: Re: Urgent help! kafka-spout stops fetching data after running for a 
while
Have you checked your worker log file? Frequent worker accident may cause kafka 
spout not working. 

Regard,
Junfeng Chen

On Thu, Sep 29, 2016 at 10:54 AM, fanxi...@travelsky.com 
<fanxi...@travelsky.com> wrote:
Hi  user,

My kafka-spout is based on storm-kafka-client.

It can read existed data from kafka.

It also can read data that I sent to kafka when the topology just runs for a 
while.

But when there is no data, the topology remains processing nothing for a long 
while, like 1 hour, I begin to send data to the topic, the topology just can 
not read the data.

I read the mails and find a way to set topology.backpressure.enable to false.

But the problem remains.

Great apprication for any advice.



Joshua
2016-09-29 10:43:11

Re: Re: Urgent help! kafka-spout stops fetching data after running for a while

[fanxi...@travelsky.com]

Hi Junfeng：

The usage of the worker seems ok too, the %cpu is about 10, and the %MEM is 
about 5.



Joshua


 
From: Junfeng Chen
Date: 2016-09-29 13:41
To: user@storm apache. org
Subject: Re: Re: Urgent help! kafka-spout stops fetching data after running for 
a while
Did you check the cpu usage of processes which run the kafkaspout. 

Regard,
Junfeng Chen

On Thu, Sep 29, 2016 at 12:46 PM, fanxi...@travelsky.com 
<fanxi...@travelsky.com> wrote:
Hi Junfeng ：

Yes, I checked the worker log, it seems ok, no error or warning is printed.



Joshua


 
From: Junfeng Chen
Date: 2016-09-29 11:29
To: user@storm apache. org
Subject: Re: Urgent help! kafka-spout stops fetching data after running for a 
while
Have you checked your worker log file? Frequent worker accident may cause kafka 
spout not working. 

Regard,
Junfeng Chen

On Thu, Sep 29, 2016 at 10:54 AM, fanxi...@travelsky.com 
<fanxi...@travelsky.com> wrote:
Hi  user,

My kafka-spout is based on storm-kafka-client.

It can read existed data from kafka.

It also can read data that I sent to kafka when the topology just runs for a 
while.

But when there is no data, the topology remains processing nothing for a long 
while, like 1 hour, I begin to send data to the topic, the topology just can 
not read the data.

I read the mails and find a way to set topology.backpressure.enable to false.

But the problem remains.

Great apprication for any advice.



Joshua
2016-09-29 10:43:11

Re: Re: Urgent help! kafka-spout stops fetching data after running for a while

[fanxi...@travelsky.com]

Hi Junfeng ：

Yes, I checked the worker log, it seems ok, no error or warning is printed.



Joshua


 
From: Junfeng Chen
Date: 2016-09-29 11:29
To: user@storm apache. org
Subject: Re: Urgent help! kafka-spout stops fetching data after running for a 
while
Have you checked your worker log file? Frequent worker accident may cause kafka 
spout not working. 

Regard,
Junfeng Chen

On Thu, Sep 29, 2016 at 10:54 AM, fanxi...@travelsky.com 
<fanxi...@travelsky.com> wrote:
Hi  user,

My kafka-spout is based on storm-kafka-client.

It can read existed data from kafka.

It also can read data that I sent to kafka when the topology just runs for a 
while.

But when there is no data, the topology remains processing nothing for a long 
while, like 1 hour, I begin to send data to the topic, the topology just can 
not read the data.

I read the mails and find a way to set topology.backpressure.enable to false.

But the problem remains.

Great apprication for any advice.



Joshua
2016-09-29 10:43:11

Urgent help! kafka-spout stops fetching data after running for a while

[fanxi...@travelsky.com]

Hi  user,

My kafka-spout is based on storm-kafka-client.

It can read existed data from kafka.

It also can read data that I sent to kafka when the topology just runs for a 
while.

But when there is no data, the topology remains processing nothing for a long 
while, like 1 hour, I begin to send data to the topic, the topology just can 
not read the data.

I read the mails and find a way to set topology.backpressure.enable to false.

But the problem remains.

Great apprication for any advice.



Joshua
2016-09-29 10:43:11

Re: Re: How will storm replay the tuple tree?

[fanxi...@travelsky.com]

Yes, only the failed tuple are replayed, but the whole batch will be held.

So, if the tuple failed forever, the batch will be held forever?

I am just not clear  the tuple itself or the batch which owns the tuple will be 
held in spout.




Josh


 
From: Ambud Sharma
Date: 2016-09-14 09:10
To: user
Subject: Re: Re: How will storm replay the tuple tree?
No as per the code only individual messages are replayed.

On Sep 13, 2016 6:09 PM, "fanxi...@travelsky.com" <fanxi...@travelsky.com> 
wrote:
Hi：

I'd like to make clear on something about Kafka-spout referring to ack.

For example, kafka-spout fetches offset 5000-6000 from Kafka server, but one 
tuple whose offset is 5101 is failed by a bolt, the whole batch of 5000-6000 
will be remain in kafka-spout until the 5101 tuple will be acked. If the 5101 
tuple can not be acked for a long time, the batch 5000-6000 will remain for a 
long time, and the kafka-spout will stop to fetch data from kafka in these time.

Am I right?




Josh


 
From: Tech Id
Date: 2016-09-14 06:26
To: user
Subject: Re: How will storm replay the tuple tree?
I agree with this statement about code/architecture but in case of some system 
outages, like one of the end-points (Solr, Couchbase, Elastic-Search etc.) 
being down temporarily, a very large number of other fully-functional and 
healthy systems will receive a large number of duplicate replays (especially in 
heavy throughput topologies).

If you can elaborate a little more on the performance cost of tracking tuples 
or point to a document reflecting the same, that will be of great help.

Best,
T.I.

On Tue, Sep 13, 2016 at 12:26 PM, Hart, James W. <jwh...@seic.com> wrote:
Failures should be very infrequent, if they are not then rethink the code and 
architecture.  The performance cost of tracking tuples in the way that would be 
required to replay at the failure is large, basically that method would slow 
everything way down for very infrequent failures.
 
From: S G [mailto:sg.online.em...@gmail.com] 
Sent: Tuesday, September 13, 2016 3:17 PM
To: user@storm.apache.org
Subject: Re: How will storm replay the tuple tree?
 
Hi,
 
I am a little curious to know why we begin at the spout level for case 1.
If we replay at the failing bolt's parent level (BoltA in this case), then it 
should be more performant due to a decrease in duplicate processing (as 
compared to whole tuple tree replays).
 
If BoltA crashes due to some reason while replaying, only then the Spout should 
receive this as a failure and whole tuple tree should be replayed.
 
This saving in duplicate processing will be more visible with several layers of 
bolts.
 
I am sure there is a good reason to replay the whole tuple-tree, and want to 
know the same.
 
Thanks
SG
 
On Tue, Sep 13, 2016 at 10:22 AM, P. Taylor Goetz <ptgo...@gmail.com> wrote:
Hi Cheney,
 
Replays happen at the spout level. So if there is a failure at any point in the 
tuple tree (the tuple tree being the anchored emits, unanchored emits don’t 
count), the original spout tuple will be replayed. So the replayed tuple will 
traverse the topology again, including unanchored points.
 
If an unanchored tuple fails downstream, it will not trigger a replay.
 
Hope this helps.
 
-Taylor
 
 
On Sep 13, 2016, at 4:42 AM, Cheney Chen <tbcql1...@gmail.com> wrote:
 
Hi there, 
 
We're using storm 1.0.1, and I'm checking through 
http://storm.apache.org/releases/1.0.1/Guaranteeing-message-processing.html
 
Got questions for below two scenarios.
Assume topology: S (spout) --> BoltA --> BoltB
1. S: anchored emit, BoltA: anchored emit
Suppose BoltB processing failed w/ ack, what will the replay be, will it 
execute both BoltA and BoltB or only failed BoltB processing?
 
2. S: anchored emit, BoltA: unanchored emit
Suppose BoltB processing failed w/ ack, replay will not happen, correct?
 
-- 
Regards,
Qili Chen (Cheney)

E-mail: tbcql1...@gmail.com 
MP: (+1) 4086217503
 
 

Re: Re: How will storm replay the tuple tree?

[fanxi...@travelsky.com]

Hi：

I'd like to make clear on something about Kafka-spout referring to ack.

For example, kafka-spout fetches offset 5000-6000 from Kafka server, but one 
tuple whose offset is 5101 is failed by a bolt, the whole batch of 5000-6000 
will be remain in kafka-spout until the 5101 tuple will be acked. If the 5101 
tuple can not be acked for a long time, the batch 5000-6000 will remain for a 
long time, and the kafka-spout will stop to fetch data from kafka in these time.

Am I right?




Josh


 
From: Tech Id
Date: 2016-09-14 06:26
To: user
Subject: Re: How will storm replay the tuple tree?
I agree with this statement about code/architecture but in case of some system 
outages, like one of the end-points (Solr, Couchbase, Elastic-Search etc.) 
being down temporarily, a very large number of other fully-functional and 
healthy systems will receive a large number of duplicate replays (especially in 
heavy throughput topologies).

If you can elaborate a little more on the performance cost of tracking tuples 
or point to a document reflecting the same, that will be of great help.

Best,
T.I.

On Tue, Sep 13, 2016 at 12:26 PM, Hart, James W.  wrote:
Failures should be very infrequent, if they are not then rethink the code and 
architecture.  The performance cost of tracking tuples in the way that would be 
required to replay at the failure is large, basically that method would slow 
everything way down for very infrequent failures.
 
From: S G [mailto:sg.online.em...@gmail.com] 
Sent: Tuesday, September 13, 2016 3:17 PM
To: user@storm.apache.org
Subject: Re: How will storm replay the tuple tree?
 
Hi,
 
I am a little curious to know why we begin at the spout level for case 1.
If we replay at the failing bolt's parent level (BoltA in this case), then it 
should be more performant due to a decrease in duplicate processing (as 
compared to whole tuple tree replays).
 
If BoltA crashes due to some reason while replaying, only then the Spout should 
receive this as a failure and whole tuple tree should be replayed.
 
This saving in duplicate processing will be more visible with several layers of 
bolts.
 
I am sure there is a good reason to replay the whole tuple-tree, and want to 
know the same.
 
Thanks
SG
 
On Tue, Sep 13, 2016 at 10:22 AM, P. Taylor Goetz  wrote:
Hi Cheney,
 
Replays happen at the spout level. So if there is a failure at any point in the 
tuple tree (the tuple tree being the anchored emits, unanchored emits don’t 
count), the original spout tuple will be replayed. So the replayed tuple will 
traverse the topology again, including unanchored points.
 
If an unanchored tuple fails downstream, it will not trigger a replay.
 
Hope this helps.
 
-Taylor
 
 
On Sep 13, 2016, at 4:42 AM, Cheney Chen  wrote:
 
Hi there, 
 
We're using storm 1.0.1, and I'm checking through 
http://storm.apache.org/releases/1.0.1/Guaranteeing-message-processing.html
 
Got questions for below two scenarios.
Assume topology: S (spout) --> BoltA --> BoltB
1. S: anchored emit, BoltA: anchored emit
Suppose BoltB processing failed w/ ack, what will the replay be, will it 
execute both BoltA and BoltB or only failed BoltB processing?
 
2. S: anchored emit, BoltA: unanchored emit
Suppose BoltB processing failed w/ ack, replay will not happen, correct?
 
-- 
Regards,
Qili Chen (Cheney)

E-mail: tbcql1...@gmail.com 
MP: (+1) 4086217503
 
 

Re: Re: how to insert into EDB by storm-jdbc

[fanxi...@travelsky.com]

Hi, Satish：

Thank you for your advice.
 I changed the dataSourceClassName to com.edb.ds.PGSimpleDataSource, and the 
ClassCastException is gone. 
But there happened another error "Property url does not exist on target class 
com.edb.ds.PGSimpleDataSource". 
I changed the property to PGPoolingDataSource, the same error remains.

Am I using a wrong edb-jdbc lib? I googled, and nearly all of the example is 
referring to edb-jdbc16.jar.
Does the HikariDataSource work well with EDB?

Appreciate for anyone's kindly help.

Best wished.
Josh

 
From: Satish Duggana
Date: 2016-09-08 19:12
To: User
Subject: Re: how to insert into EDB by storm-jdbc
You should give datasource class name like 
`org.postgresql.ds.PGSimpleDataSource` instead of Driver class.

Thanks,
Satish.


On Thu, Sep 8, 2016 at 3:45 PM, fanxi...@travelsky.com <fanxi...@travelsky.com> 
wrote:
Hi  user,

Recently I am doing a job using the storm-jdbc to insert into EDB.

I use the edb-jdbc16.jar.

I config for the ConnectionProvider like below:
map.put("dataSourceClassName", "com.edb.Driver");
map.put("dataSource.url", "jdbc:edb://localhost:5444/test");
map.put("dataSource.user", "test");
map.put("dataSource.password", "likeThe");

but when I run the topology, the jdbc-bolt throws an exception 
"java.lang.RuntimeException: java.lang.ClassCastException: Cannot cast 
com.edb.Driver to javax.sql.DataSource
at com.zaxxer.hikari.util.UtilityElf.createInstance(UtilityElf.java:90) 
~[HikariCP-2.4.3.jar:?]
at com.zaxxer.hikari.pool.PoolBase.initializeDataSource(PoolBase.java:292) 
~[HikariCP-2.4.3.jar:?]
at com.zaxxer.hikari.pool.PoolBase.(PoolBase.java:84) 
~[HikariCP-2.4.3.jar:?]
at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:102) 
~[HikariCP-2.4.3.jar:?]
at com.zaxxer.hikari.HikariDataSource.(HikariDataSource.java:71) 
~[HikariCP-2.4.3.jar:?]"

I realized that the com.edb.Driver which implments java.sql.Driver may be is 
not a proper driver to use.

I googled, nothing useful found.

Can anyone give me some advice on how to insert into EDB by storm-jdbc?

Thanks very much.


Joshua
2016-09-08 18:06:56

how to insert into EDB by storm-jdbc

[fanxi...@travelsky.com]

Hi  user,

Recently I am doing a job using the storm-jdbc to insert into EDB.

I use the edb-jdbc16.jar.

I config for the ConnectionProvider like below:
map.put("dataSourceClassName", "com.edb.Driver");
map.put("dataSource.url", "jdbc:edb://localhost:5444/test");
map.put("dataSource.user", "test");
map.put("dataSource.password", "likeThe");

but when I run the topology, the jdbc-bolt throws an exception 
"java.lang.RuntimeException: java.lang.ClassCastException: Cannot cast 
com.edb.Driver to javax.sql.DataSource
at com.zaxxer.hikari.util.UtilityElf.createInstance(UtilityElf.java:90) 
~[HikariCP-2.4.3.jar:?]
at com.zaxxer.hikari.pool.PoolBase.initializeDataSource(PoolBase.java:292) 
~[HikariCP-2.4.3.jar:?]
at com.zaxxer.hikari.pool.PoolBase.(PoolBase.java:84) 
~[HikariCP-2.4.3.jar:?]
at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:102) 
~[HikariCP-2.4.3.jar:?]
at com.zaxxer.hikari.HikariDataSource.(HikariDataSource.java:71) 
~[HikariCP-2.4.3.jar:?]"

I realized that the com.edb.Driver which implments java.sql.Driver may be is 
not a proper driver to use.

I googled, nothing useful found.

Can anyone give me some advice on how to insert into EDB by storm-jdbc?

Thanks very much.


Joshua
2016-09-08 18:06:56

Re: STORM REBALACING

[fanxi...@travelsky.com]

hi Junquk：

1. first, you should make sure your cluster has more than 10 slots.
2. second, you should make sure your topology has been assigned more than 10 
task. because the rebalacing is based on the task to do more workers. If you 
did not assigne 10 tasks(at least) to the topology explicitly, the rebalancing 
can not work out as you want.







 
From: Junguk Cho
Date: 2016-08-11 11:09
To: user
Subject: STORM REBALACING
Hi, All. 

I tried to use STORM rebalance 
(http://storm.apache.org/releases/1.0.1/Command-line-client.html).

I used Storm 1.0.1 version.
I first ran 3 workers (one spout, one split and one count.) for WordCount 
example.

After running it, In commandline, I typed
./storm rebalance WordCount -w 10 -n 6 -e spout=1 -e split=2 -e count=2

I saw something in nimbus.log and UI like
 Delaying event :do-rebalance for 10 secs for WordCount-1-1470884051.
However, when I checked UI, it was still 3 workers.


Are there some requirements to use rebalance feature?
Also, in UI, there are no input interfaces to specify # of workers and # of 
components except for waiting time for rebalace.


Thanks in advance.
- Junguk

Re: Re: Problem to write into HBase

[fanxi...@travelsky.com]

Hi Wenwei

Actually, I did not new any thread in my bolt. The error reported just comes 
from storm core itself.




I t seems you created too many threads, that
cause no available thread resource.

Sent from my iPhone

On Jun 12, 2016, at 10:18, "fanxi...@travelsky.com" <fanxi...@travelsky.com> 
wrote:

Hi  user,

I have a topology to write into HBase. Every time I submitted the topology, it 
runned well. But after a well, for example, one or two days, the topology 
always reports an execption like below:

java.lang.OutOfMemoryError: unable to create new native thread at 
java.lang.Thread.start0(Native Method) at 
java.lang.Thread.start(Thread.java:714) at 
org.apache.zookeeper.ClientCnxn.start(ClientCnxn.java:406) at 
org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:450) at 
org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:380) at 
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.checkZk(RecoverableZooKeeper.java:140)
 at 
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:127)
 at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:132) at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.(ZooKeeperWatcher.java:165)
 at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.(ZooKeeperWatcher.java:134)
 at 
org.apache.hadoop.hbase.catalog.CatalogTracker.(CatalogTracker.java:179) 
at 
org.apache.hadoop.hbase.catalog.CatalogTracker.(CatalogTracker.java:153) 
at 
org.apache.hadoop.hbase.catalog.CatalogTracker.(CatalogTracker.java:135) 
at 
org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:234)
 at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:306) 
at 
com.travelsky.roc.hbase.utils.HBaseUtils.isTableAvailable(HBaseUtils.java:22) 
at com.travelsky.roc.hbase.bolt.HBaseSinkBolt.execute(HBaseSinkBolt.java:279) 
at 
backtype.storm.daemon.executor$fn__5641$tuple_action_fn__5643.invoke(executor.clj:631)
 at 
backtype.storm.daemon.executor$mk_task_receiver$fn__5564.invoke(executor.clj:399)
 at 
backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58) 
at 
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
 at 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
 at 
backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) 
at 
backtype.storm.daemon.executor$fn__5641$fn__5653$fn__5700.invoke(executor.clj:746)
 at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431) at 
clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745)
then the topology runs very slow.
I took a look at the log, it is full of the information like below:
2016-06-12 10:16:04 o.a.h.h.z.RecoverableZooKeeper [INFO] Process 
identifier=catalogtracker-on-hconnection-0x5ade861c connecting to ZooKeeper 
ensemble=r720m6-hdp:2181,r720m8-hdp:2181,r720n5-hdp:2181 
2016-06-12 10:16:04 o.a.z.ZooKeeper [INFO] Initiating client connection, 
connectString=r720m6-hdp:2181,r720m8-hdp:2181,r720n5-hdp:2181 
sessionTimeout=12 watcher=catalogtracker-on-hconnection-0x5ade861c, 
quorum=r720m6-hdp:2181,r720m8-hdp:2181,r720n5-hdp:2181, baseZNode=/hbasenew 
2016-06-12 10:16:04 o.a.z.ClientCnxn [INFO] Opening socket connection to server 
r720m8-hdp/10.6.116.3:2181. Will not attempt to authenticate using SASL 
(unknown error) 
2016-06-12 10:16:04 o.a.z.ClientCnxn [INFO] Socket connection established to 
r720m8-hdp/10.6.116.3:2181, initiating session 
2016-06-12 10:16:04 o.a.z.ClientCnxn [INFO] Session establishment complete on 
server r720m8-hdp/10.6.116.3:2181, sessionid = 0x15138b0b2df471f, negotiated 
timeout = 12 
2016-06-12 10:16:04 o.a.z.ZooKeeper [INFO] Session: 0x15138b0b2df471f closed 
2016-06-12 10:16:04 o.a.z.ClientCnxn [INFO] EventThread shut down 
2016-06-12 10:16:07 o.a.h.h.z.RecoverableZooKeeper [INFO] Process 
identifier=catalogtracker-on-hconnection-0x5ade861c connecting to ZooKeeper 
ensemble=r720m6-hdp:2181,r720m8-hdp:2181,r720n5-hdp:2181 
2016-06-12 10:16:07 o.a.z.ZooKeeper [INFO] Initiating client connection, 
connectString=r720m6-hdp:2181,r720m8-hdp:2181,r720n5-hdp:2181 
sessionTimeout=12 watcher=catalogtracker-on-hconnection-0x5ade861c, 
quorum=r720m6-hdp:2181,r720m8-hdp:2181,r720n5-hdp:2181, baseZNode=/hbasenew 
2016-06-12 10:16:07 o.a.z.ClientCnxn [INFO] Opening socket connection to server 
r720m8-hdp/10.6.116.3:2181. Will not attempt to authenticate using SASL 
(unknown error) 
2016-06-12 10:16:07 o.a.z.ClientCnxn [INFO] Socket connection established to 
r720m8-hdp/10.6.116.3:2181, initiating session 
2016-06-12 10:16:07 o.a.z.ClientCnxn [INFO] Session establishment complete on 
server r720m8-hdp/10.6.116.3:2181, sessionid = 0x15138b0b2df473f, negotiated 
timeout = 12 
2016-06-12 10:16:07 o.a.z.ZooKeeper [INFO] Session: 0x15138b0b2df473f closed 
2016-06-12 10:16:07 o.a.z.ClientCnxn [INFO] EventThread shut down 

Anyone comes across this problem? Thanks for your hints.



Joshua
2016-06-12 10:12:33

Problem to write into HBase

[fanxi...@travelsky.com]

Hi  user,

I have a topology to write into HBase. Every time I submitted the topology, it 
runned well. But after a well, for example, one or two days, the topology 
always reports an execption like below:

java.lang.OutOfMemoryError: unable to create new native thread at 
java.lang.Thread.start0(Native Method) at 
java.lang.Thread.start(Thread.java:714) at 
org.apache.zookeeper.ClientCnxn.start(ClientCnxn.java:406) at 
org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:450) at 
org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:380) at 
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.checkZk(RecoverableZooKeeper.java:140)
 at 
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:127)
 at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:132) at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.(ZooKeeperWatcher.java:165)
 at 
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.(ZooKeeperWatcher.java:134)
 at 
org.apache.hadoop.hbase.catalog.CatalogTracker.(CatalogTracker.java:179) 
at 
org.apache.hadoop.hbase.catalog.CatalogTracker.(CatalogTracker.java:153) 
at 
org.apache.hadoop.hbase.catalog.CatalogTracker.(CatalogTracker.java:135) 
at 
org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:234)
 at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:306) 
at 
com.travelsky.roc.hbase.utils.HBaseUtils.isTableAvailable(HBaseUtils.java:22) 
at com.travelsky.roc.hbase.bolt.HBaseSinkBolt.execute(HBaseSinkBolt.java:279) 
at 
backtype.storm.daemon.executor$fn__5641$tuple_action_fn__5643.invoke(executor.clj:631)
 at 
backtype.storm.daemon.executor$mk_task_receiver$fn__5564.invoke(executor.clj:399)
 at 
backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58) 
at 
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
 at 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
 at 
backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) 
at 
backtype.storm.daemon.executor$fn__5641$fn__5653$fn__5700.invoke(executor.clj:746)
 at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431) at 
clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745)
then the topology runs very slow.
I took a look at the log, it is full of the information like below:
2016-06-12 10:16:04 o.a.h.h.z.RecoverableZooKeeper [INFO] Process 
identifier=catalogtracker-on-hconnection-0x5ade861c connecting to ZooKeeper 
ensemble=r720m6-hdp:2181,r720m8-hdp:2181,r720n5-hdp:2181 
2016-06-12 10:16:04 o.a.z.ZooKeeper [INFO] Initiating client connection, 
connectString=r720m6-hdp:2181,r720m8-hdp:2181,r720n5-hdp:2181 
sessionTimeout=12 watcher=catalogtracker-on-hconnection-0x5ade861c, 
quorum=r720m6-hdp:2181,r720m8-hdp:2181,r720n5-hdp:2181, baseZNode=/hbasenew 
2016-06-12 10:16:04 o.a.z.ClientCnxn [INFO] Opening socket connection to server 
r720m8-hdp/10.6.116.3:2181. Will not attempt to authenticate using SASL 
(unknown error) 
2016-06-12 10:16:04 o.a.z.ClientCnxn [INFO] Socket connection established to 
r720m8-hdp/10.6.116.3:2181, initiating session 
2016-06-12 10:16:04 o.a.z.ClientCnxn [INFO] Session establishment complete on 
server r720m8-hdp/10.6.116.3:2181, sessionid = 0x15138b0b2df471f, negotiated 
timeout = 12 
2016-06-12 10:16:04 o.a.z.ZooKeeper [INFO] Session: 0x15138b0b2df471f closed 
2016-06-12 10:16:04 o.a.z.ClientCnxn [INFO] EventThread shut down 
2016-06-12 10:16:07 o.a.h.h.z.RecoverableZooKeeper [INFO] Process 
identifier=catalogtracker-on-hconnection-0x5ade861c connecting to ZooKeeper 
ensemble=r720m6-hdp:2181,r720m8-hdp:2181,r720n5-hdp:2181 
2016-06-12 10:16:07 o.a.z.ZooKeeper [INFO] Initiating client connection, 
connectString=r720m6-hdp:2181,r720m8-hdp:2181,r720n5-hdp:2181 
sessionTimeout=12 watcher=catalogtracker-on-hconnection-0x5ade861c, 
quorum=r720m6-hdp:2181,r720m8-hdp:2181,r720n5-hdp:2181, baseZNode=/hbasenew 
2016-06-12 10:16:07 o.a.z.ClientCnxn [INFO] Opening socket connection to server 
r720m8-hdp/10.6.116.3:2181. Will not attempt to authenticate using SASL 
(unknown error) 
2016-06-12 10:16:07 o.a.z.ClientCnxn [INFO] Socket connection established to 
r720m8-hdp/10.6.116.3:2181, initiating session 
2016-06-12 10:16:07 o.a.z.ClientCnxn [INFO] Session establishment complete on 
server r720m8-hdp/10.6.116.3:2181, sessionid = 0x15138b0b2df473f, negotiated 
timeout = 12 
2016-06-12 10:16:07 o.a.z.ZooKeeper [INFO] Session: 0x15138b0b2df473f closed 
2016-06-12 10:16:07 o.a.z.ClientCnxn [INFO] EventThread shut down 

Anyone comes across this problem? Thanks for your hints.



Joshua
2016-06-12 10:12:33

tune up storm

[fanxi...@travelsky.com]

Hi  All,

Is there any recommended articles or books about tuning up performance of 
Storm? 

Tried hard in google but found less.



Joshua
2016-05-23 15:41:57

Re: Re: to remove the kill button in storm ui

[fanxi...@travelsky.com]

Hi Edwin

Did you ever succeed by this way?

I tried again, and open the ui in a clean browser, it does not work.




it should work, tried to view the  page(topology-page-template.html) directly 
on the browser , and if it does not work well again , tried to clear browser 
cache 

2016-04-11 17:34 GMT+08:00 fanxi...@travelsky.com <fanxi...@travelsky.com>:
Hi, Edwin

Thank you for your reply.
I have tried that way, but it is useless.




范新普

中国民航信息网络股份有限公司
研发中心  数据服务部
地址：北京市朝阳区天辰东路7号国家会议中心北区七层


 
From: Edwin Liu
Date: 2016-04-11 17:14
To: user
Subject: Re: to remove the kill button in storm ui
try this way:

tried to change topology-page-template.html

try to comment the line 





2016-04-11 16:52 GMT+08:00 fanxi...@travelsky.com <fanxi...@travelsky.com>:
user，您好：

Hi,

Any one  can help me?

I would like to disable the "kill" button  in the storm UI. My storm version is 
0.9.2.

I tried to revise the topologyActions staff in the src/ui/topology.html, it 
turned out to be no use.
I tried to revise the topology-actions-template staff in the 
src/ui/template/topology-page-template.html, it turned out to be no use.
I tried to revise the confirmAction staff in the src/ui/js/script.js, it turned 
out to be no use.
I tried to revise the main-routes staff in the 
src/clj/backtype/storm/ui/core.clj, it turned out to be  useful. But it is 
weird that there is a button which does nothing when being clicked.

So, is there a way to remove just the button in the topology-page in the UI?



范新普

中国民航信息网络股份有限公司
研发中心  数据服务部
地址：北京市朝阳区天辰东路7号国家会议中心北区七层

Re: Re: to remove the kill button in storm ui

[fanxi...@travelsky.com]

Hi, Edwin

Thank you for your reply.
I have tried that way, but it is useless.




范新普

中国民航信息网络股份有限公司
研发中心  数据服务部
地址：北京市朝阳区天辰东路7号国家会议中心北区七层


 
From: Edwin Liu
Date: 2016-04-11 17:14
To: user
Subject: Re: to remove the kill button in storm ui
try this way:

tried to change topology-page-template.html

try to comment the line 





2016-04-11 16:52 GMT+08:00 fanxi...@travelsky.com <fanxi...@travelsky.com>:
user，您好：

Hi,

Any one  can help me?

I would like to disable the "kill" button  in the storm UI. My storm version is 
0.9.2.

I tried to revise the topologyActions staff in the src/ui/topology.html, it 
turned out to be no use.
I tried to revise the topology-actions-template staff in the 
src/ui/template/topology-page-template.html, it turned out to be no use.
I tried to revise the confirmAction staff in the src/ui/js/script.js, it turned 
out to be no use.
I tried to revise the main-routes staff in the 
src/clj/backtype/storm/ui/core.clj, it turned out to be  useful. But it is 
weird that there is a button which does nothing when being clicked.

So, is there a way to remove just the button in the topology-page in the UI?



范新普

中国民航信息网络股份有限公司
研发中心  数据服务部
地址：北京市朝阳区天辰东路7号国家会议中心北区七层

any one can help?

[fanxi...@travelsky.com]

Hi,

Any one  can help me?

I would like to disable the "delete" button  in the storm UI. My storm version 
is 0.9.2.

I tried to revise the topologyActions staff in the src/ui/topology.html, it 
turned out to be no use.
I tried to revise the topology-actions-template staff in the 
src/ui/template/topology-page-template.html, it turned out to be no use.
I tried to revise the confirmAction staff in the src/ui/js/script.js, it turned 
out to be no use.
I tried to revise the main-routes staff in the 
src/clj/backtype/storm/ui/core.clj, it turned out to be  useful. But it is 
weird that there is a button which does nothing when being clicked.

So, is there a way to remove just the button in the topology-page in the UI?



范新普

中国民航信息网络股份有限公司
研发中心  数据服务部
地址：北京市朝阳区天辰东路7号国家会议中心北区七层