[
https://issues.apache.org/jira/browse/KYLIN-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17433547#comment-17433547
]
Yaqian Zhang edited comment on KYLIN-5107 at 10/25/21, 5:48 AM:
----------------------------------------------------------------
Hi [~mzz_q]:
I checked the source code of FSUtils class in HBase 2.3.4 and found that the
method in error reporting actually exists, which may not be the problem of
HBase version.
If you want to solve this problem quickly, you can try to use the package of
apache-kylin-3.1.0-bin-hadoop3.tar.gz. Kylin 3.1.0 does not call the method
FSUtils.setStoragePolicy.
was (Author: zhangyaqian):
Hi [~mzz_q]:
I checked the source code of FSUtils class in HBase 2.3.4 and found that the
method in error reporting actually exists, which may not be the problem of
HBase version.
If you want to solve this problem quickly, you can try to use the package of
apache-kylin-3.1.0-bin-hadoop3.tar.gz. Kylin 3.1.0 does not call the method in
the error report.
> when excuting the cube job at "Convert Cuboid Data to HFile" step：Error
> running child
> --------------------------------------------------------------------------------------
>
> Key: KYLIN-5107
> URL: https://issues.apache.org/jira/browse/KYLIN-5107
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Affects Versions: v3.1.2
> Environment: hadoop3.2.1
> habse2.3.4
> Reporter: mzz
> Priority: Blocker
>
> As title, when excuting the cube job at "Convert Cuboid Data to HFile"
> step,throw an exception:
> {code:java}
> 2021-10-21 10:15:38,928 ERROR [main] org.apache.kylin.engine.mr.KylinReducer:
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.util.FSUtils.setStoragePolicy(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/Path;Ljava/lang/String;)V
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3.configureStoragePolicy(HFileOutputFormat3.java:468)
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:287)
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:243)
> at
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:559)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
> at
> org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:141)
> at
> org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:135)
> at org.apache.kylin.engine.mr.KylinReducer.reduce(KylinReducer.java:77)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:628)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1732)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> 2021-10-21 10:15:38,929 INFO [main] org.apache.kylin.engine.mr.KylinReducer:
> Do cleanup, available memory: 2773m
> 2021-10-21 10:15:38,929 INFO [main] org.apache.kylin.engine.mr.KylinReducer:
> Total rows: 1
> 2021-10-21 10:15:38,929 ERROR [main] org.apache.hadoop.mapred.YarnChild:
> Error running child : java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.util.FSUtils.setStoragePolicy(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/Path;Ljava/lang/String;)V
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3.configureStoragePolicy(HFileOutputFormat3.java:468)
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:287)
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:243)
> at
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:559)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
> at
> org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:141)
> at
> org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:135)
> at org.apache.kylin.engine.mr.KylinReducer.reduce(KylinReducer.java:77)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:628)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1732)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> {code}
> I try move all of hbase libs to {KYLIN_HOME}/lib ,but this problem not
> solved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
