MrDannyWu commented on issue #6021: URL: https://github.com/apache/iceberg/issues/6021#issuecomment-1296575962
> Seems the primiary key can't be derived from the iceberg's HiveCatalog even though you have defined it using Hive Cli . > > 1. you can define with the following using Flink SQL > > ``` > CREATE TABLE ods_data_1_1 ( > xxx, > xxx, > PRIMARY KEY(`id`) NOT ENFORCED > ) WITH ( > 'connector'='iceberg', > 'catalog-name'='hive_prod', > 'catalog-database'='default', > 'catalog-table'='ods_data_1_1', > 'uri'='hdfs://nn-1:8020/user/hive/warehouse/', > 'warehouse'='hdfs://nn:8020/path/to/warehouse' > ); > ``` > > Note: it won't create a table, it's just a mapping to the table crated before in Hive. Refer the [flink-connector](https://iceberg.apache.org/docs/latest/flink-connector/) for more details. Then write the data using the sql normally. > > 2. You can use Flink DDL to create the table, the Hive should have the ability to read the table. Have you ever followed the document [enable Iceberg support](https://iceberg.apache.org/docs/latest/hive/#enabling-iceberg-support-in-hive)? Ok, so far I have solved this problem. When I created the Iceberg table (Hive Ctatlog) in the Flink SQL Client, I lost the parameters. After carefully reviewing the Iceberg and Flink documents and trying many times, Upsert can be implemented. Thanks to every helper! ### The correct steps are as follows: 1. create catalog ``` CREATE CATALOG hive_iceberg WITH ( 'clients'='5', 'type'='iceberg', 'catalog-type'='hive', 'property-version'='1', 'uri'='thrift://xxxx:9083', 'warehouse'='hdfs://xxxx/user/hive/warehouse/' ); ``` 2. create db ``` CREATE DATABASE ods; ``` 3. create table ``` CREATE TABLE IF NOT EXISTS hive_iceberg.ods.table( id BIGINT, ... PRIMARY KEY (id) NOT ENFORCED ) WITH ( 'format-version'='2', 'iceberg.mr.catalog'='hive', 'engine.hive.enabled'='true', 'write.upsert.enabled'='true', 'write.metadata.previous-versions-max'='3', 'hive.vectorized.execution.enabled'='false', 'write.metadata.delete-after-commit.enabled'='true', 'warehouse'='hdfs://xxxx/user/hive/warehouse/ods/table' ) ; ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
