wangsheng has posted comments on this change. ( http://gerrit.cloudera.org:8080/15797 )
Change subject: IMPALA-9688: Support create iceberg table by impala ...................................................................... Patch Set 19: (1 comment) Hi Zoltan, here is some of my ideas about this question http://gerrit.cloudera.org:8080/#/c/15797/17/fe/src/main/java/org/apache/impala/util/IcebergUtil.java File fe/src/main/java/org/apache/impala/util/IcebergUtil.java: http://gerrit.cloudera.org:8080/#/c/15797/17/fe/src/main/java/org/apache/impala/util/IcebergUtil.java@24 PS17, Line 24: import org.apache.iceberg.hadoop.HadoopTableOperations; : import org.apache.iceberg.hadoop.HadoopTables; > "if we use HiveCatalog to create a metastore table, we cannot create extern Just do an "invalidate metadata" when create external table for Iceberg metastore table is indeed a good solution, and I understand your description about "external purge table". But I'm not familiar with S3 in fact, do you think that we should support both file system table and metastore table, and file system table in only for hdfs? Or just use metastore table to replace file system table in this patch? I've already use HiveCatalog to replace HadoopTables in my test enviroment today, and when I create iceberg by HiveCatalog.createTable(), I got an error: "NoSuchMethodError: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(Lorg/apache/hadoop/hive/conf/HiveConf;)V", and this error maybe caused by lacking of related jar, I'm not sure if there is a constructor in hive-standalone-metastore-*.jar, if so, we may need to add some other hive dependency in pom.xml When I write a demo to test HiveCatalog and HadoopTables api several days ago, I found that lots of hive related jars, such as hive-exec/hive-metastore are necessary, so I finally put the whole $hive_client/lib/*.jar to my project CLASSPATH, I'm not sure if this is a problem when we use HiveCatalog in impala code. By the way, HadoopTables and HiveCatalog are different, which means if you create a table by HiveCatalog, you cannot load this table by HadoopTables.load(location). -- To view, visit http://gerrit.cloudera.org:8080/15797 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8d85db4c904a8c758c4cfb4f19cfbdab7e6ea284 Gerrit-Change-Number: 15797 Gerrit-PatchSet: 19 Gerrit-Owner: wangsheng <sky...@163.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Sahil Takiar <stak...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: wangsheng <sky...@163.com> Gerrit-Comment-Date: Wed, 13 May 2020 13:24:51 +0000 Gerrit-HasComments: Yes
