[
https://issues.apache.org/jira/browse/STORM-886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14715365#comment-14715365
]
ASF GitHub Bot commented on STORM-886:
--------------------------------------
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/700#discussion_r38023281
--- Diff: storm-core/src/clj/backtype/storm/daemon/worker.clj ---
@@ -114,12 +114,34 @@
(fast-list-iter [[task tuple :as pair] tuple-batch]
(.serialize serializer tuple)))
+(defn- mk-backpressure-handler [executors]
+ "make a handler that checks and updates worker's backpressure flag"
+ (disruptor/backpressure-handler
+ (fn [worker]
+ (let [storm-id (:storm-id worker)
+ assignment-id (:assignment-id worker)
+ port (:port worker)
+ storm-cluster-state (:storm-cluster-state worker)]
+ (if executors
+ (if (reduce #(or %1 %2) (map #(.get-backpressure-flag %1)
executors))
+ (reset! (:backpressure worker) true) ;; at least one
executor has set backpressure
+ (reset! (:backpressure worker) false))) ;; no executor has
backpressure set
+ ;; update the worker's backpressure flag to zookeeper here
+ (.worker-backpressure! storm-cluster-state storm-id assignment-id
port @(:backpressure worker))
--- End diff --
I think we have an opportunity here to not put so much load on ZK. We
should be able to keep cached if we think back-pressure is on or not for this
worker, and only update if we think it is changing, instead of reading the
state each time and updating it, if it is different. This is being called at
least ever 100 ms, which is going to put more load on ZK then I like.
> Automatic Back Pressure
> -----------------------
>
> Key: STORM-886
> URL: https://issues.apache.org/jira/browse/STORM-886
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Robert Joseph Evans
> Assignee: Zhuo Liu
> Attachments: an simple example for backpressure.png, backpressure.png
>
>
> This new feature is aimed for automatic flow control through the topology DAG
> since different components may have unmatched tuple processing speed.
> Currently, the tuples may get dropped if the downstream components can not
> process as quickly, thereby causing a waste of network bandwidth and
> processing capability. In addition, it is difficult to tune the
> max.spout.pending parameter for best backpressure performance. Therefore, an
> automatic back pressure scheme is highly desirable.
> Heron proposed a form of back pressure that does not rely on acking or max
> spout pending. Instead spouts throttle not only when max.spout.pending is
> hit, but also if any bolt has gone over a high water mark in their input
> queue, and has not yet gone below a low water mark again. There is a lot of
> room for potential improvement here around control theory and having spouts
> only respond to downstream bolts backing up, but a simple bang-bang
> controller like this is a great start.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
