MyLanPangzi opened a new issue #2717: URL: https://github.com/apache/hudi/issues/2717
**_Tips before filing an issue_** - Have you gone through our [FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)? - Join the mailing list to engage in conversations and get faster support at dev-subscr...@hudi.apache.org. - If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly. **Describe the problem you faced** **can we modify the run_sync_tool.sh ?** **i have a hoodie table t1 on hadoop, and i want import into hive. when i execute import shell, get an error:** ./run_sync_tool.sh \ --user hive \ --pass '' \ --jdbc-url jdbc:hive2:\/\/yh001:10000 \ --partitioned-by partition \ --base-path hdfs://yh001:9820/hudi/t1 \ --database default \ --table t1 2021-03-25 18:28:27,011 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:<init>(87)) - Creating hive connection jdbc:hive2://yh001:10000 2021-03-25 18:28:27,283 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:createHiveConnection(437)) - Successfully established Hive connection to jdbc:hive2://yh001:10000 Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/calcite/plan/RelOptRule at org.apache.hudi.hive.HoodieHiveClient.<init>(HoodieHiveClient.java:91) at org.apache.hudi.hive.HiveSyncTool.<init>(HiveSyncTool.java:69) at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:249) Caused by: java.lang.ClassNotFoundException: org.apache.calcite.plan.RelOptRule at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 3 more **after update the shell** HIVE_CALCITE=`ls ${HIVE_HOME}/lib/calcite-*.jar | tr '\n' ':'` HIVE_JARS=$HIVE_METASTORE:$HIVE_SERVICE:$HIVE_EXEC:$HIVE_JDBC:$HIVE_JACKSON:$HIVE_CALCITE **i get anthor error** 1-03-25 18:30:51,256 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:createHiveConnection(437)) - Successfully established Hive connection to jdbc:hive2://yh001:10000 Exception in thread "main" java.lang.NoClassDefFoundError: com/facebook/fb303/FacebookService$Iface at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) at java.net.URLClassLoader.access$100(URLClassLoader.java:73) at java.net.URLClassLoader$1.run(URLClassLoader.java:368) at java.net.URLClassLoader$1.run(URLClassLoader.java:362) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:361) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.hadoop.hive.metastore.utils.JavaUtils.getClass(JavaUtils.java:52) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:146) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:4299) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4367) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4347) at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4603) at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:291) at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:274) at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:435) at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:375) at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:355) at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:331) at org.apache.hudi.hive.HoodieHiveClient.<init>(HoodieHiveClient.java:91) at org.apache.hudi.hive.HiveSyncTool.<init>(HiveSyncTool.java:69) at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:249) Caused by: java.lang.ClassNotFoundException: com.facebook.fb303.FacebookService$Iface at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 30 more **and then i modify the shell:** HIVE_CALCITE=`ls ${HIVE_HOME}/lib/calcite-*.jar | tr '\n' ':'` HIVE_LIBFB303=`ls ${HIVE_HOME}/lib/libfb303-*.jar | tr '\n' ':'` HIVE_JARS=$HIVE_METASTORE:$HIVE_SERVICE:$HIVE_EXEC:$HIVE_JDBC:$HIVE_JACKSON:$HIVE_CALCITE:$HIVE_LIBFB303 **it is ok.** 2021-03-25 18:33:03,158 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:createTable(262)) - Creating table with CREATE EXTERNAL TABLE IF NOT EXISTS `default`.`t2`( `_hoodie_commit_time` string, `_hoodie_commit_seqno` string, `_hoodie_record_key` string, `_hoodie_partition_path` string, `_hoodie_file_name` string, `uuid` string, `name` string, `age` int, `ts` bigint) PARTITIONED BY (`partition` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hudi.hadoop.HoodieParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 'hdfs://yh001:9820/hudi/t1' 2021-03-25 18:33:03,158 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL CREATE EXTERNAL TABLE IF NOT EXISTS `default`.`t2`( `_hoodie_commit_time` string, `_hoodie_commit_seqno` string, `_hoodie_record_key` string, `_hoodie_partition_path` string, `_hoodie_file_name` string, `uuid` string, `name` string, `age` int, `ts` bigint) PARTITIONED BY (`partition` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hudi.hadoop.HoodieParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 'hdfs://yh001:9820/hudi/t1' 2021-03-25 18:33:03,276 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(156)) - Schema sync complete. Syncing partitions for t2 2021-03-25 18:33:03,276 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(162)) - Last commit time synced was found to be null 2021-03-25 18:33:03,276 INFO [main] common.AbstractSyncHoodieClient (AbstractSyncHoodieClient.java:getPartitionsWrittenToSince(128)) - Last commit time synced is not known, listing all partitions in hdfs://yh001:9820/hudi/t1,FS :DFS[DFSClient[clientName=DFSClient_NONMAPREDUCE_-410852585_1, ugi=hdp (auth:SIMPLE)]] 2021-03-25 18:33:03,298 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(164)) - Storage partitions scan complete. Found 3 2021-03-25 18:33:03,354 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncPartitions(223)) - New Partitions [2021/03/23, 2021/03/24, 2021/03/25] 2021-03-25 18:33:03,355 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:addPartitionsToTable(120)) - Adding partitions 3 to table t2 2021-03-25 18:33:03,355 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL ALTER TABLE `default`.`t2` ADD IF NOT EXISTS PARTITION (`partition`='2021-03-23') LOCATION 'hdfs://yh001:9820/hudi/t1/2021/03/23' PARTITION (`partition`='2021-03-24') LOCATION 'hdfs://yh001:9820/hudi/t1/2021/03/24' PARTITION (`partition`='2021-03-25') LOCATION 'hdfs://yh001:9820/hudi/t1/2021/03/25' 2021-03-25 18:33:03,525 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncPartitions(226)) - Changed Partitions [] 2021-03-25 18:33:03,526 INFO [main] hive.HoodieHiveClient (HoodieHiveClient.java:updatePartitionsToTable(131)) - No partitions to change for t2 2021-03-25 18:33:03,608 INFO [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(169)) - Sync complete for t2 2021-03-25 18:33:03,620 INFO [main] metastore.HiveMetaStoreClient (HiveMetaStoreClient.java:close(600)) - Closed a connection to metastore, current connections: 0 **To Reproduce** Steps to reproduce the behavior: 1. 2. 3. 4. **Expected behavior** A clear and concise description of what you expected to happen. **Environment Description** * Hudi version : 0.8.0-SNAPSHOT * Spark version : * Hive version : 3.1.2 * Hadoop version : 3.1.4 * Storage (HDFS/S3/GCS..) : HDFS * Running on Docker? (yes/no) : no **Additional context** Add any other context about the problem here. **Stacktrace** ```Add the stacktrace of the error.``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
