Repository: samza Updated Branches: refs/heads/master edce6b76d -> 9fb9a7673
SAMZA-1482: add config documentation for auto-restart/fail behavior o⦠â¦n partition count changes Author: Yi Pan (Data Infrastructure) <nickpa...@gmail.com> Reviewers: Jagadish <jagad...@apache.org> Closes #363 from nickpan47/partition-change-docsite and squashes the following commits: 5342ea0 [Yi Pan (Data Infrastructure)] SAMZA-1482: fix configuration table in documentation site 7c9f326 [Yi Pan (Data Infrastructure)] SAMZA-1482: add config documentation for auto-restart/fail behavior on partition count changes Project: http://git-wip-us.apache.org/repos/asf/samza/repo Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/9fb9a767 Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/9fb9a767 Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/9fb9a767 Branch: refs/heads/master Commit: 9fb9a76732642c1004cf60b28defdb95a2b4e052 Parents: edce6b7 Author: Yi Pan (Data Infrastructure) <nickpa...@gmail.com> Authored: Mon Nov 20 11:48:14 2017 -0800 Committer: Yi Pan (Data Infrastructure) <nickpa...@gmail.com> Committed: Mon Nov 20 11:48:14 2017 -0800 ---------------------------------------------------------------------- .../versioned/jobs/configuration-table.html | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/samza/blob/9fb9a767/docs/learn/documentation/versioned/jobs/configuration-table.html ---------------------------------------------------------------------- diff --git a/docs/learn/documentation/versioned/jobs/configuration-table.html b/docs/learn/documentation/versioned/jobs/configuration-table.html index fb8a97e..b6ae0d9 100644 --- a/docs/learn/documentation/versioned/jobs/configuration-table.html +++ b/docs/learn/documentation/versioned/jobs/configuration-table.html @@ -231,11 +231,7 @@ <td class="property" id="job-coordinator-monitor-partition-change">job.coordinator.<br />monitor-partition-change</td> <td class="default">false</td> <td class="description"> - If you are using Kafka for coordinator stream, this configuration enables the Job Coordinator to - detect partition count difference in Kafka input topics. On detection, it updates a Gauge - metric of format <span class="system">system-name</span>.<span class="stream">stream-name</span>.partitionCount, - which indicates the difference in the partition count from the initial state. Please note that currently this - feature only works for Kafka-based systems. + This is deprecated in favor of <a href="#job-coordinator-monitor-partition-change-frequency-ms">job.coordinator.monitor-partition-change.frequency.ms</a>. </td> </tr> @@ -243,8 +239,12 @@ <td class="property" id="job-coordinator-monitor-partition-change-frequency-ms">job.coordinator.<br />monitor-partition-change.frequency.ms</td> <td class="default">300000</td> <td class="description"> - The frequency at which the input streams' partition count change should be detected. This check - can be tuned to be pretty low as partition increase is not a common event. + The frequency at which the input streams' partition count change should be detected. When the input + partition count change is detected, Samza will automatically restart a stateless job or fail a + stateful job. A longer time interval is recommended for jobs w/ large number of input system stream + partitions, since gathering partition count may incur measurable overhead to the job. You can + completely disable partition count monitoring by setting this value to 0 or a negative integer, + which will also disable auto-restart/failing behavior of a Samza job on partition count changes. </td> </tr>
