[ https://issues.apache.org/jira/browse/KYLIN-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004079#comment-17004079 ]
zhaoguoxian commented on KYLIN-2654: ------------------------------------ I encountered the same problem. Then your yarn environment is cdh and your 'mapreduce.job.map.output.collector.class' param is org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator. you can not use your Custom writable type or comparator,example,kylin`s org.apache.kylin.engine.mr.steps.SelfDefineSortableKey. my solution was by setting the param mapreduce.job.map.output.collector.class=org.apache.hadoop.mapred.MapTask$MapOutputBuffer. btw. my environment was cdh5.5.2 > Kylin failed to build cube at 3rd step on Hadoop 2.6.0-cdh5.4.4 + jdk 1.8 > ------------------------------------------------------------------------- > > Key: KYLIN-2654 > URL: https://issues.apache.org/jira/browse/KYLIN-2654 > Project: Kylin > Issue Type: Bug > Components: Job Engine > Affects Versions: v2.0.0 > Reporter: Shao Feng Shi > Assignee: Dong Li > Priority: Major > > From community user qiumingming: > When I use kylin 2.0 to build cube, I got a error at Extract Fact Table > Distinct Columns step, mr job failed. My Hadoop version is 2.6.0-cdh5.4.4. > Below is my error info: > {code} > Error: java.io.IOException: Initialization of all the collectors failed. > Error in last collector was :Cannot find serializer for > org.apache.kylin.engine.mr.steps.SelfDefineSortableKey at > org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:423) at > org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:82) at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:718) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:790) at > org.apache.hadoop.mapred.MapTask.run(MapTask.java:350) at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at > java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:422) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: > java.io.IOException: Cannot find serializer for > org.apache.kylin.engine.mr.steps.SelfDefineSortableKey at > org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator.init(NativeMapOutputCollectorDelegator.java:147) > at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:411) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)