[
https://issues.apache.org/jira/browse/KYLIN-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16044250#comment-16044250
]
kangkaisen commented on KYLIN-2547:
-----------------------------------
Shaofeng, Thanks very much for your review and comment.
1 I think whether or not adding lock is not related the job step;
The single job engine still use ZK lock;
I think this issue is the cube metadata consistency essentially, which isn't
the job scheduler' responsibility.
2 Yes. I think the network isn't always reliable for http sync.
3 I don't understand this point. what's your mean is {{UpdateCubeInfoStep}}
relay on {{CubeService}}?
> Fix multi-process concurrence bug in UpdateCubeInfoXXStep
> ---------------------------------------------------------
>
> Key: KYLIN-2547
> URL: https://issues.apache.org/jira/browse/KYLIN-2547
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
> Affects Versions: v2.0.0
> Reporter: kangkaisen
> Assignee: kangkaisen
> Priority: Minor
>
> There is a minor bug in "Update Cube Info" step when build a cube in
> distributed env, which will make this job failed, but the next auto merge
> job won't be affected and will succeed. So the cube data still is consistent
> and correct.
> {code:java}
> Caused by: java.lang.IllegalStateException: Segments overlap:
> waimai_dolphin_topic_flow_activity_expose_food_d_cube[20170405000000_20170412000000]
> and
> waimai_dolphin_topic_flow_activity_expose_food_d_cube[20170405000000_20170412000000]
> at org.apache.kylin.cube.CubeValidator.validate(CubeValidator.java:85)
> at
> org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:359)
> at
> org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:386)
> at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:302)
> at
> org.apache.kylin.cube.CubeManager.mergeSegments(CubeManager.java:533)
> at
> org.apache.kylin.rest.service.CubeService.mergeCubeSegment(CubeService.java:635)
> at
> org.apache.kylin.rest.service.CubeService.updateOnNewSegmentReady(CubeService.java:587)
> at
> org.apache.kylin.rest.service.CubeServiceFastClassBySpringCGLIB17a07c0e.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxyDynamicAdvisedInterceptor.intercept(CglibAopProxy.java:629)
> at
> org.apache.kylin.rest.service.CubeServiceEnhancerBySpringCGLIB$$c6fabb3f.updateOnNewSegmentReady()
> at
> org.apache.kylin.rest.service.CacheService.rebuildCubeCache(CacheService.java:237)
> at
> org.apache.kylin.rest.service.CacheService.access$000(CacheService.java:62)
> at
> org.apache.kylin.rest.service.CacheService$1.afterCubeUpdate(CacheService.java:86)
> at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:305)
> at
> org.apache.kylin.cube.CubeManager.promoteNewlyBuiltSegments(CubeManager.java:735)
> at
> org.apache.kylin.engine.mr.steps.UpdateCubeInfoAfterBuildStep.doWork(UpdateCubeInfoAfterBuildStep.java:62)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
> ... 6 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
