[ https://issues.apache.org/jira/browse/KYLIN-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925772#comment-16925772 ]
Xiaoxiang Yu edited comment on KYLIN-4162 at 9/9/19 3:26 PM: ------------------------------------------------------------- If a discard job found in Coordinator, it maybe be detected but dropped . https://github.com/apache/kylin/blob/1cd284bcd38c4787e13b102a28df7a533ef82de6/stream-coordinator/src/main/java/org/apache/kylin/stream/coordinator/Coordinator.java#L1186 {code:java} else if (ExecutableState.DISCARDED.equals(jobState)) { // if the job has been discard manually, just think that the segment is not in building logger.info("job:{} is discard, reset the job state in metaStore", jobId); SegmentBuildState.BuildState state = new SegmentBuildState.BuildState(); state.setBuildStartTime(0); state.setState(SegmentBuildState.BuildState.State.WAIT); state.setJobId(cubingJob.getId()); streamMetadataStore.updateSegmentBuildState(cubeName, segmentState.getSegmentName(), state); segmentState.setState(state); } {code} In above code, if coordinator find the state of job is *DISCARDED*, coordinator will change the state of current segment in stream metadata from *building* to *wait*, and try to add to *result* to mark it need to be rebuilt. But, before add the segment the *result*, coordinator will first check it has enough quota. If not, coordinator won't add it into *result*, thus that segment will be lost. When check in next time, it will be ignore because the state is *wait*. https://github.com/apache/kylin/blob/1cd284bcd38c4787e13b102a28df7a533ef82de6/stream-coordinator/src/main/java/org/apache/kylin/stream/coordinator/Coordinator.java#L1160 {code:java} if (segmentState.isInBuilding()) { {code} Did I understand it in right way? Happy to hear your reply. was (Author: hit_lacus): If a discard job found in Coordinator, it will be detected at https://github.com/apache/kylin/blob/1cd284bcd38c4787e13b102a28df7a533ef82de6/stream-coordinator/src/main/java/org/apache/kylin/stream/coordinator/Coordinator.java#L1186 . {code:java} else if (ExecutableState.DISCARDED.equals(jobState)) { // if the job has been discard manually, just think that the segment is not in building logger.info("job:{} is discard, reset the job state in metaStore", jobId); SegmentBuildState.BuildState state = new SegmentBuildState.BuildState(); state.setBuildStartTime(0); state.setState(SegmentBuildState.BuildState.State.WAIT); state.setJobId(cubingJob.getId()); streamMetadataStore.updateSegmentBuildState(cubeName, segmentState.getSegmentName(), state); segmentState.setState(state); } {code} In above code, if coordinator find the state of job is *DISCARDED*, coordinator will change the entry in stream metadata from building to wait, and try to add to *result* to mark it need to be rebuilt. But, before add the segment the *result*, coordinator will first check it has enough quota. If not, coordinator won't add it into *result*, thus that segment will be lost. When check in next time, it will be ignore because the state is *wait*. {code:java} if (segmentState.isInBuilding()) { {code} > After drop the build task on the monitor page, subsequent segments cannot be > constructed. > ----------------------------------------------------------------------------------------- > > Key: KYLIN-4162 > URL: https://issues.apache.org/jira/browse/KYLIN-4162 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming > Affects Versions: v3.0.0-alpha2 > Reporter: Yaqian Zhang > Priority: Major > Fix For: Future > > -- This message was sent by Atlassian Jira (v8.3.2#803003)