[ https://issues.apache.org/jira/browse/KYLIN-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849702#comment-15849702 ]
Shaofeng SHI commented on KYLIN-2242: ------------------------------------- The patch with fix has been merge to master branch; this can be marked as resolved now. > Directly write hdfs file in reducer is dangerous > ------------------------------------------------ > > Key: KYLIN-2242 > URL: https://issues.apache.org/jira/browse/KYLIN-2242 > Project: Kylin > Issue Type: Bug > Components: Job Engine > Affects Versions: v1.6.0 > Reporter: kangkaisen > Assignee: kangkaisen > Fix For: v2.0.0 > > Attachments: consoleFull.html, KYLIN-2242.patch > > > currently, Kylin directly write hdfs file in {{FactDistinctColumnsReducer}}, > which is dangerous because the MapReduce Speculative Execution will result in > more than one reducers write the same hdfs file at the same time. > After KYLIN-2217, I think this issue will occur with higher probability. we > should output the value by {{context.wirte}} in reducer. -- This message was sent by Atlassian JIRA (v6.3.15#6346)