[
https://issues.apache.org/jira/browse/HELIX-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15050218#comment-15050218
]
ASF GitHub Bot commented on HELIX-617:
--------------------------------------
Github user kishoreg commented on the pull request:
https://github.com/apache/helix/pull/40#issuecomment-163518949
LGTM, thanks for the description. Merging it.
> Job IdealState is generated even the job is not running and not removed when
> it is completed.
> ---------------------------------------------------------------------------------------------
>
> Key: HELIX-617
> URL: https://issues.apache.org/jira/browse/HELIX-617
> Project: Apache Helix
> Issue Type: Task
> Reporter: Lei Xia
> Assignee: Lei Xia
>
> -- Problem:
> In current task framework implementation, one IdealState and one ExternalView
> znode are generated for each job once the job is queued (not when it is
> started). These znodes will stay in the zookeeper until the specified data
> expiry time (usually one day in Espresso backup setting). This is even worse
> for recurrent job queue. For recurrent job queue, one
> idealstate/externalview is created for original job template, and one
> idealstate/externalview will be created for each scheduled run of the job.
> This usually generates a significant amounts of znodes in IdealState. For
> example, to schedule a queue with 100 jobs, 100 "template" idealstates are
> created and will stay there for ever. While a new set of 100 "scheduling"
> idealstates are created for every time all these jobs are scheduled to run,
> which will stay until the expiry time setup in the job configuration has
> passed (usually a relative long time).
> -- Proposed Changes:
> The IdealState of a job will be generated only when it is scheduled to run,
> and will be removed immediately once the job is completed. In practice,
> there will be only one job running at the same time for each queue, thus only
> one IdealState will exist for each queue at any given time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
