hello
when I run command as follow
kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete true
and I got the exception msg as follow :
2017-06-28 10:08:17,432 INFO [main StorageCleanupJob:145]: options: '
delete=true'
2017-06-28 10:08:17,432 INFO [main StorageCleanupJob:146]: delete option
value: 'true'
2017-06-28 10:08:17,432 INFO [main StorageCleanupJob:147]: force option
value: 'null'
2017-06-28 10:08:17,759 WARN [main HiveConf:3683]: HiveConf of name
hive.multigroupby.singlemar does not exist
2017-06-28 10:08:17,872 INFO [main HiveMetaStore:564]: 0: Opening raw store
with implementation class:org.apache.hadoop.hive.metastore.ObjectStore
2017-06-28 10:08:17,956 INFO [main ObjectStore:325]: ObjectStore,
initialize called
2017-06-28 10:08:18,208 INFO [main Persistence:77]: Property
datanucleus.fixedDatastore unknown - will be ignored
2017-06-28 10:08:18,209 INFO [main Persistence:77]: Property
hive.metastore.integral.jdo.pushdown unknown - will be ignored
2017-06-28 10:08:18,209 INFO [main Persistence:77]: Property
datanucleus.cache.level2 unknown - will be ignored
2017-06-28 10:08:19,617 WARN [main HiveConf:3683]: HiveConf of name
hive.multigroupby.singlemar does not exist
2017-06-28 10:08:19,619 INFO [main ObjectStore:429]: Setting MetaStore
object pin classes with
hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
2017-06-28 10:08:21,953 INFO [main MetaStoreDirectSql:145]: Using direct
SQL, underlying DB is MYSQL
2017-06-28 10:08:21,957 INFO [main ObjectStore:308]: Initialized
ObjectStore
2017-06-28 10:08:22,455 INFO [main HiveMetaStore:651]: Added admin role in
metastore
2017-06-28 10:08:22,459 INFO [main HiveMetaStore:660]: Added public role in
metastore
2017-06-28 10:08:22,532 INFO [main HiveMetaStore:700]: No user is added in
admin role, since config is empty
2017-06-28 10:08:22,808 INFO [main HiveMetaStore:734]: 0: get_all_tables:
db=default
2017-06-28 10:08:22,811 INFO [main audit:295]: ugi=root
ip=unknown-ip-addr cmd=get_all_tables: db=default
2017-06-28 10:08:22,947 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID 0363e422-1be2-4652-89e5-24a727ce98ba, ignore it
2017-06-28 10:08:22,954 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID 05990f76-b2e7-463c-bd5f-9b6906db4a63, ignore it
2017-06-28 10:08:22,958 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID 198969b3-dd27-4bd6-b5eb-476d963cc59c, ignore it
2017-06-28 10:08:22,964 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID 1f97122e-1e38-4007-a8a2-705b8af1521a, ignore it
2017-06-28 10:08:22,969 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID 39f59489-aaea-43a6-be2e-436447d67dae, ignore it
2017-06-28 10:08:22,974 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID 42de30ea-82a6-455c-9cbc-2c6df04812cd, ignore it
2017-06-28 10:08:22,978 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID 6849dbc5-6a48-41e3-b258-b248c77e0a52, ignore it
2017-06-28 10:08:22,983 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID 6c9aa465-ae82-402f-ba7b-cbbac2adca3f, ignore it
2017-06-28 10:08:22,989 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID 82994592-ff8b-4863-aa42-dc0bab66d47d, ignore it
2017-06-28 10:08:22,994 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID 84cc3f86-29fc-438a-be45-5ec46792c568, ignore it
2017-06-28 10:08:22,998 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID 90219e38-3b2a-4ec8-8d3a-7c86930d5caa, ignore it
2017-06-28 10:08:23,003 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID 9dc10d6b-d1b4-4470-adf2-dedbe1f0f90e, ignore it
2017-06-28 10:08:23,008 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID b4d99f1d-bc13-45eb-8433-84de238279a1, ignore it
2017-06-28 10:08:23,013 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID c0f24c74-44f9-4171-b56e-8056d19e3ae1, ignore it
2017-06-28 10:08:23,017 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID ca7a8143-e676-4c9f-8ffe-fdadcc84724a, ignore it
2017-06-28 10:08:23,024 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID dd8f4ea5-5c3c-4165-88e1-f06a796ca095, ignore it
2017-06-28 10:08:23,029 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID e57934e1-1398-457f-9070-6bc5010f461a, ignore it
2017-06-28 10:08:23,032 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID e9a845c5-5aba-4c83-a589-99984b1b07ee, ignore it
2017-06-28 10:08:23,037 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID eadcc797-b9ef-439c-891c-9ac912cc7ce6, ignore it
2017-06-28 10:08:23,043 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID ef9e5b30-d8fa-498d-ad38-48494a15fb22, ignore it
2017-06-28 10:08:23,048 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID f1c5089a-b6e1-4118-b58e-48164833de91, ignore it
2017-06-28 10:08:23,052 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID f469d883-d066-4f48-ad4a-098dede4a74b, ignore it
2017-06-28 10:08:23,056 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID f4ddb713-be4e-4c30-a24a-645d87847f01, ignore it
2017-06-28 10:08:23,060 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID f6579632-3b05-4546-a826-96e761c4e678, ignore it
2017-06-28 10:08:23,065 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID f9ae3fd4-9f74-4c59-9b47-d604388e1970, ignore it
2017-06-28 10:08:23,069 WARN [main StorageCleanupJob:285]: Failed to find
segment ID from job ID fbe1e38d-c4f0-47ad-a93a-7489f5638cdf, ignore it
2017-06-28 10:08:23,069 INFO [main StorageCleanupJob:289]: Working jobIDs:
[0363e422-1be2-4652-89e5-24a727ce98ba, 05990f76-b2e7-463c-bd5f-9b6906db4a63,
198969b3-dd27-4bd6-b5eb-476d963cc59c, 1f97122e-1e38-4007-a8a2-705b8af1521a,
39f59489-aaea-43a6-be2e-436447d67dae, 42de30ea-82a6-455c-9cbc-2c6df04812cd,
6849dbc5-6a48-41e3-b258-b248c77e0a52, 6c9aa465-ae82-402f-ba7b-cbbac2adca3f,
82994592-ff8b-4863-aa42-dc0bab66d47d, 84cc3f86-29fc-438a-be45-5ec46792c568,
90219e38-3b2a-4ec8-8d3a-7c86930d5caa, 9dc10d6b-d1b4-4470-adf2-dedbe1f0f90e,
b4d99f1d-bc13-45eb-8433-84de238279a1, c0f24c74-44f9-4171-b56e-8056d19e3ae1,
ca7a8143-e676-4c9f-8ffe-fdadcc84724a, dd8f4ea5-5c3c-4165-88e1-f06a796ca095,
e57934e1-1398-457f-9070-6bc5010f461a, e9a845c5-5aba-4c83-a589-99984b1b07ee,
eadcc797-b9ef-439c-891c-9ac912cc7ce6, ef9e5b30-d8fa-498d-ad38-48494a15fb22,
f1c5089a-b6e1-4118-b58e-48164833de91, f469d883-d066-4f48-ad4a-098dede4a74b,
f4ddb713-be4e-4c30-a24a-645d87847f01, f6579632-3b05-4546-a826-96e761c4e678,
f9ae3fd4-9f74-4c59-9b47-d604388e1970, fbe1e38d-c4f0-47ad-a93a-7489f5638cdf]
2017-06-28 10:08:23,096 INFO [main StorageCleanupJob:292]: Checking table
kylin_intermediate_biam_busi_anl_day_kind_p_29fd1441_f7c4_4625_af15_58a39cbdd08a
Exception in thread "main" java.lang.RuntimeException: error execute
org.apache.kylin.tool.StorageCleanupJob
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
at
org.apache.kylin.tool.StorageCleanupJob.main(StorageCleanupJob.java:394)
Caused by: java.lang.NullPointerException
at
org.apache.kylin.common.util.ClassUtil.forRenamedClass(ClassUtil.java:84)
at org.apache.kylin.common.util.ClassUtil.forName(ClassUtil.java:76)
at
org.apache.kylin.job.execution.ExecutableManager.parseTo(ExecutableManager.java:439)
at
org.apache.kylin.job.execution.ExecutableManager.getJob(ExecutableManager.java:129)
at
org.apache.kylin.tool.StorageCleanupJob.getSegmentIdFromJobId(StorageCleanupJob.java:375)
at
org.apache.kylin.tool.StorageCleanupJob.isTableInUse(StorageCleanupJob.java:382)
at
org.apache.kylin.tool.StorageCleanupJob.cleanUnusedIntermediateHiveTable(StorageCleanupJob.java:311)
at
org.apache.kylin.tool.StorageCleanupJob.execute(StorageCleanupJob.java:153)
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
... 1 more
2017-06-28 10:08:23,110 INFO [Thread-0
ConnectionManager$HConnectionImplementation:2139]: Closing master protocol:
MasterService
2017-06-28 10:08:23,142 INFO [Thread-0
ConnectionManager$HConnectionImplementation:1710]: Closing zookeeper
sessionid=0x685caabe66b20097
2017-06-28 10:08:23,146 INFO [Thread-0 ZooKeeper:684]: Session:
0x685caabe66b20097 closed
2017-06-28 10:08:23,146 INFO [main-EventThread ClientCnxn:512]: EventThread
shut down
My software version as follow:
hadoop-2.6.0-cdh5.4.0
hbase-1.2.3-bin
apache-hive-2.1.0-bin
kylin.version:2.0.0
commit:375fd807c281d8c5deff0620747c806be2019782;
os.name:Linux
os.arch:amd64
os.version:3.10.0-229.el7.x86_64
java.version:1.8.0_102
java.vendor:Oracle Corporation
Thank you !
--
View this message in context:
http://apache-kylin.74782.x6.nabble.com/errors-when-run-StorageCleanupJob-tp8321.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
