[
https://issues.apache.org/jira/browse/STORM-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972094#comment-14972094
]
ASF GitHub Bot commented on STORM-350:
--------------------------------------
Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/797#issuecomment-150714706
@kishorvpatil @binhnv @revans2
Here's my storm.yaml.
```
worker.childopts: "-Dfile.encoding=UTF-8 -Xmx768m
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/daum/logs/storm/heapdump.hprof
-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=1%ID%
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false"
topology.backpressure.enable: false
topology.transfer.buffer.size: 32
topology.executor.send.buffer.size: 16384
topology.executor.receive.buffer.size: 16384
```
I used topology.backpressure.enable, topology.transfer.buffer.size,
topology.executor.send.buffer.size, topology.executor.receive.buffer.size as
default but no luck.
I changed these values as
http://www.michael-noll.com/blog/2013/06/21/understanding-storm-internal-message-buffers
but also no luck.
Failed tuples appear within 3 minutes, which is same as we were seen from
#134 (origin STORM-350).
I'm using VMs (IaaS on my office), but without Disruptor 3.3.2 it doesn't
show failed tuples.
I ran perftest on command line.
> storm jar
storm_perf_test-1.0.0-SNAPSHOT-jar-for-0.10.0-rc-with-dependencies.jar
com.yahoo.storm.perftest.Main --ack --ackers 3 --bolt 4 --name test -l 1 -n 1
--workers 3 --spout 3 --testTimeSec 900 -c topology.max.spout.pending=1092
--messageSize 10
Here's log.
```
1034 [main] INFO c.y.s.p.Main - Adding in 3 spouts
1081 [main] INFO c.y.s.p.Main - Adding in 4 bolts
1231 [main] INFO b.s.StormSubmitter - Generated ZooKeeper secret payload
for MD5-digest: -6427640641721598663:-8827787475099997913
1236 [main] INFO b.s.s.a.AuthUtils - Got AutoCreds []
1297 [main] INFO b.s.StormSubmitter - Uploading topology jar
storm_perf_test-1.0.0-SNAPSHOT-jar-for-0.10.0-rc-with-dependencies.jar to
assigned location:
/data1/storm/nimbus/inbox/stormjar-38afac0d-a0d8-49a6-8529-7705d89280e9.jar
1326 [main] INFO b.s.StormSubmitter - Successfully uploaded topology jar
to assigned location:
/data1/storm/nimbus/inbox/stormjar-38afac0d-a0d8-49a6-8529-7705d89280e9.jar
1326 [main] INFO b.s.StormSubmitter - Submitting topology test_0 in
distributed mode with conf
{"storm.zookeeper.topology.auth.scheme":"digest","storm.zookeeper.topology.auth.payload":"-6427640641721598663:-8827787475099997913","topology.workers":3,"topology.acker.executors":3,"topology.debug":false,"topology.max.spout.pending":1092}
1520 [main] INFO b.s.StormSubmitter - Finished submitting topology: test_0
status topologies totalSlots slotsUsed totalExecutors
executorsWithMetrics time time-diff ms transferred throughput
(MB/s)
WAITING 1 12 0 13 0 1445640974611 0
0 0.0
WAITING 1 12 3 13 13 1445641004611 30000
72600 0.02307891845703125
WAITING 1 12 3 13 13 1445641034612 30001
608340 0.1933796318931296
RUNNING 1 12 3 13 13 1445641064611 29999
783820 0.24917797349429877
RUNNING 1 12 3 13 13 1445641094611 30000
200520 0.06374359130859375
RUNNING 1 12 3 13 13 1445641124611 30000
17140 0.005448659261067708
RUNNING 1 12 3 13 13 1445641154612 30001
0 0.0
RUNNING 1 12 3 13 13 1445641184611 29999
17980 0.0057158785989480905
RUNNING 1 12 3 13 13 1445641214611 30000
0 0.0
RUNNING 1 12 3 13 13 1445641244611 30000
18180 0.005779266357421875
RUNNING 1 12 3 13 13 1445641274611 30000
0 0.0
RUNNING 1 12 3 13 13 1445641304611 30000
17620 0.005601247151692708
RUNNING 1 12 3 13 13 1445641334611 30000
0 0.0
RUNNING 1 12 3 13 13 1445641364611 30000
16800 0.005340576171875
RUNNING 1 12 3 13 13 1445641394611 30000
0 0.0
RUNNING 1 12 3 13 13 1445641424611 30000
17500 0.005563100179036458
RUNNING 1 12 3 13 13 1445641454611 30000
0 0.0
RUNNING 1 12 3 13 13 1445641484611 30000
17840 0.005671183268229167
RUNNING 1 12 3 13 13 1445641514611 30000
0 0.0
RUNNING 1 12 3 13 13 1445641544611 30000
18260 0.005804697672526042
RUNNING 1 12 3 13 13 1445641574611 30000
0 0.0
RUNNING 1 12 3 13 13 1445641604612 30001
17980 0.0057154975530763555
RUNNING 1 12 3 13 13 1445641634611 29999
0 0.0
RUNNING 1 12 3 13 13 1445641664611 30000
18000 0.0057220458984375
RUNNING 1 12 3 13 13 1445641694611 30000
0 0.0
RUNNING 1 12 3 13 13 1445641724611 30000
17720 0.005633036295572917
RUNNING 1 12 3 13 13 1445641754611 30000
0 0.0
RUNNING 1 12 3 13 13 1445641784611 30000
18720 0.005950927734375
RUNNING 1 12 3 13 13 1445641814611 30000
0 0.0
RUNNING 1 12 3 13 13 1445641844612 30001
18640 0.005925298909307189
RUNNING 1 12 3 13 13 1445641874611 29999
0 0.0
RUNNING 1 12 3 13 13 1445641904612 30001
18160 0.005772716104775674
RUNNING 1 12 3 13 13 1445641934611 29999
0 0.0
```
It nearly stops functioning but I couldn't find any warning / error from
worker logs.
It may be nice if we can share perftest jar / binary dist file which
applies STORM-350 (with Disruptor 3.3.2) / binary dist file which doesn't apply
STORM-350 (with Disruptor 2.10.4).
> Update disruptor to latest version (3.2.1)
> ------------------------------------------
>
> Key: STORM-350
> URL: https://issues.apache.org/jira/browse/STORM-350
> Project: Apache Storm
> Issue Type: Dependency upgrade
> Components: storm-core
> Reporter: Boris Aksenov
> Assignee: Boris Aksenov
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments:
> 20141117-0.9.3-rc1-3-worker-separate-1-spout-and-2-bolts-failing-tuples.png,
> 20141117-0.9.3-rc1-one-worker-failing-tuples.png,
> 20141117-0.9.3-rc1-three-workers-1-spout-3-bolts-failing-tuples.png,
> 20141118-0.9.3-branch-3-worker-separate-1-spout-and-2-bolts-ok.png,
> 20141118-0.9.3-branch-one-worker-ok.png,
> 20141118-0.9.3-branch-three-workers-1-spout-3-bolts-ok.png, Storm UI1.pdf,
> Storm UI2.pdf, storm-0.9.3-rc1-failing-tuples.png,
> storm-0_9_2-incubating-failing-tuples.png,
> storm-0_9_2-incubating-no-failing-tuples.png,
> storm-failed-tuples-multi-node.png, storm-multi-node-without-350.png
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
