[jira] [Commented] (KYLIN-4105) use spark engine to build cube on step Convert Cuboid Data to HFile failed
[ https://issues.apache.org/jira/browse/KYLIN-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17065241#comment-17065241 ] Gaurav Rawat commented on KYLIN-4105: - [~xiangakun] [~dockj] how was this resolved do I need to copy the props and set KYLIN_HOME and KYLIN_CONF on each core node of the cluster ? Please help bit stuck here > use spark engine to build cube on step Convert Cuboid Data to HFile failed > -- > > Key: KYLIN-4105 > URL: https://issues.apache.org/jira/browse/KYLIN-4105 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.4.0, v2.4.1, v2.5.0, v2.6.0, v2.6.1, v2.6.2 >Reporter: dockj >Assignee: wangrupeng >Priority: Minor > > use spark engine to build cube on step Convert Cuboid Data to HFile。visit > yarn logs has this exception: > 19/05/05 17:36:42 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 0.0 > (TID 0, spslave4.bigdata.ly, executor 1): > java.lang.ExceptionInInitializerError > at org.apache.kylin.metadata.datatype.DataType.(DataType.java:134) > at java.io.ObjectStreamClass.hasStaticInitializer(Native Method) > at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1787) > at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:72) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:253) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:251) > at java.security.AccessController.doPrivileged(Native Method) > at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:250) > at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:611) > at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630) > at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1714) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373) > at java.util.HashSet.readObject(HashSet.java:333) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputS
[jira] [Commented] (KYLIN-4105) use spark engine to build cube on step Convert Cuboid Data to HFile failed
[ https://issues.apache.org/jira/browse/KYLIN-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16902855#comment-16902855 ] xiangakun commented on KYLIN-4105: -- Thanks dockj, as you advised, it works fine for me. > use spark engine to build cube on step Convert Cuboid Data to HFile failed > -- > > Key: KYLIN-4105 > URL: https://issues.apache.org/jira/browse/KYLIN-4105 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.4.0, v2.4.1, v2.5.0, v2.6.0, v2.6.1, v2.6.2 >Reporter: dockj >Priority: Minor > > use spark engine to build cube on step Convert Cuboid Data to HFile。visit > yarn logs has this exception: > 19/05/05 17:36:42 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 0.0 > (TID 0, spslave4.bigdata.ly, executor 1): > java.lang.ExceptionInInitializerError > at org.apache.kylin.metadata.datatype.DataType.(DataType.java:134) > at java.io.ObjectStreamClass.hasStaticInitializer(Native Method) > at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1787) > at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:72) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:253) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:251) > at java.security.AccessController.doPrivileged(Native Method) > at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:250) > at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:611) > at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630) > at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1714) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373) > at java.util.HashSet.readObject(HashSet.java:333) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStrea
[jira] [Commented] (KYLIN-4105) use spark engine to build cube on step Convert Cuboid Data to HFile failed
[ https://issues.apache.org/jira/browse/KYLIN-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16899722#comment-16899722 ] dockj commented on KYLIN-4105: -- you can check your yarn cluster every host has KYLIN_HOME or KYLIN_CONF environment variable.if so, you should remove the environment variable. > use spark engine to build cube on step Convert Cuboid Data to HFile failed > -- > > Key: KYLIN-4105 > URL: https://issues.apache.org/jira/browse/KYLIN-4105 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.4.0, v2.4.1, v2.5.0, v2.6.0, v2.6.1, v2.6.2 >Reporter: dockj >Priority: Minor > > use spark engine to build cube on step Convert Cuboid Data to HFile。visit > yarn logs has this exception: > 19/05/05 17:36:42 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 0.0 > (TID 0, spslave4.bigdata.ly, executor 1): > java.lang.ExceptionInInitializerError > at org.apache.kylin.metadata.datatype.DataType.(DataType.java:134) > at java.io.ObjectStreamClass.hasStaticInitializer(Native Method) > at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1787) > at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:72) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:253) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:251) > at java.security.AccessController.doPrivileged(Native Method) > at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:250) > at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:611) > at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630) > at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1714) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373) > at java.util.HashSet.readObject(HashSet.java:333) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInp
[jira] [Commented] (KYLIN-4105) use spark engine to build cube on step Convert Cuboid Data to HFile failed
[ https://issues.apache.org/jira/browse/KYLIN-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896677#comment-16896677 ] xiangakun commented on KYLIN-4105: -- Could anyone help answer "KYLIN_HOME or KYLIN_CONF set in yarn cluster" is a must? And how to resolve the error "fail to locate kylin.properties" ? > use spark engine to build cube on step Convert Cuboid Data to HFile failed > -- > > Key: KYLIN-4105 > URL: https://issues.apache.org/jira/browse/KYLIN-4105 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.4.0, v2.4.1, v2.5.0, v2.6.0, v2.6.1, v2.6.2 >Reporter: dockj >Priority: Minor > > use spark engine to build cube on step Convert Cuboid Data to HFile。visit > yarn logs has this exception: > 19/05/05 17:36:42 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 0.0 > (TID 0, spslave4.bigdata.ly, executor 1): > java.lang.ExceptionInInitializerError > at org.apache.kylin.metadata.datatype.DataType.(DataType.java:134) > at java.io.ObjectStreamClass.hasStaticInitializer(Native Method) > at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1787) > at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:72) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:253) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:251) > at java.security.AccessController.doPrivileged(Native Method) > at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:250) > at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:611) > at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630) > at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1714) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373) > at java.util.HashSet.readObject(HashSet.java:333) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.i
[jira] [Commented] (KYLIN-4105) use spark engine to build cube on step Convert Cuboid Data to HFile failed
[ https://issues.apache.org/jira/browse/KYLIN-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16893299#comment-16893299 ] xiangakun commented on KYLIN-4105: -- Hi dockj, I met the same error as you posted, Could you please suggest how to resolve the problem? thanks > use spark engine to build cube on step Convert Cuboid Data to HFile failed > -- > > Key: KYLIN-4105 > URL: https://issues.apache.org/jira/browse/KYLIN-4105 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.4.0, v2.4.1, v2.5.0, v2.6.0, v2.6.1, v2.6.2 >Reporter: dockj >Priority: Minor > > use spark engine to build cube on step Convert Cuboid Data to HFile。visit > yarn logs has this exception: > 19/05/05 17:36:42 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 0.0 > (TID 0, spslave4.bigdata.ly, executor 1): > java.lang.ExceptionInInitializerError > at org.apache.kylin.metadata.datatype.DataType.(DataType.java:134) > at java.io.ObjectStreamClass.hasStaticInitializer(Native Method) > at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1787) > at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:72) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:253) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:251) > at java.security.AccessController.doPrivileged(Native Method) > at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:250) > at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:611) > at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630) > at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1714) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373) > at java.util.HashSet.readObject(HashSet.java:333) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryO
[jira] [Commented] (KYLIN-4105) use spark engine to build cube on step Convert Cuboid Data to HFile failed
[ https://issues.apache.org/jira/browse/KYLIN-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889982#comment-16889982 ] dockj commented on KYLIN-4105: -- use spark engine build cube on step Convert Cuboid Data to HFile ，must invoke MeasureTypeFactory.init() to get custom MesureType。method call link: +MeasureTypeFactory.init()->KylinConfig.getInstanceFromEnv()->KylinConfig.buildSiteProperties()->KylinConfig.buildSiteOrderedProps()->KylinConfig.getSitePropertiesFile()+ when yarn cluster set KYLIN_HOME or KYLIN_CONF environment variable，method KylinConfig.getSitePropertiesFile() will according to KYLIN_HOME or KYLIN_CONF to find kylin.properties，but yarn cluster host just set KYLIN_HOME or KYLIN_CONF,no kylin.properties file.method KylinConfig.buildSiteOrderedProps() will throws java.lang.RuntimeException: fail to locate kylin.properties。but method MeasureTypeFactory.init() just catch KylinConfigCannotInitException,finally cause use spark engine to build cube on step Convert Cuboid Data to HFile fail,maybe can expand method MeasureTypeFactory.init() KylinConfigCannotInitException to RuntimeException. > use spark engine to build cube on step Convert Cuboid Data to HFile failed > -- > > Key: KYLIN-4105 > URL: https://issues.apache.org/jira/browse/KYLIN-4105 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.4.0, v2.4.1, v2.5.0, v2.6.0, v2.6.1, v2.6.2 >Reporter: dockj >Priority: Minor > > use spark engine to build cube on step Convert Cuboid Data to HFile。visit > yarn logs has this exception: > 19/05/05 17:36:42 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 0.0 > (TID 0, spslave4.bigdata.ly, executor 1): > java.lang.ExceptionInInitializerError > at org.apache.kylin.metadata.datatype.DataType.(DataType.java:134) > at java.io.ObjectStreamClass.hasStaticInitializer(Native Method) > at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1787) > at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:72) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:253) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:251) > at java.security.AccessController.doPrivileged(Native Method) > at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:250) > at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:611) > at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630) > at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1714) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373) > at java.util.HashSet.readObject(HashSet.java:333) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) > at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) > at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) > at java.io.ObjectInputStream.readObj
