MohamedAdelHsn opened a new issue, #6448:
URL: https://github.com/apache/iceberg/issues/6448
### Query engine
Flink Sql
### Question
i am doing project using flink sql to streaming data from kafka to hive
tables with iceberg format .
i working several days in this issue with no solution , your support please !
i wrote table in flink sql as kafka source as below :
CREATE TABLE flink_table_source (
id BIGINT
,data VARCHAR
) WITH (
'connector' = 'kafka'
,'topic' = 'flink'
,'properties.bootstrap.servers' = 'localhost:9092'
,'properties.group.id' = 'user_log_x'
,'scan.startup.mode' = 'earliest-offset'
,'format' = 'json'
);
this table use default_catalog and default_database and when i start publish
data to kafka topic it is working fine with
select * from flink_table_source;
i want to insert data into hive table based on iceberg from that table and i
am facing with below sql
insert into hive_iceberg_table select * from flink_table_source;
my hive_iceberg_table created as below
CREATE CATALOG ice WITH (
'type'='iceberg',
'catalog-type'='hive',
'uri'='thrift://localhost:9083',
'clients'='5',
'property-version'='2',
'warehouse'='hdfs://localhost:9000/user/hive/warehouse'
);
USE ice;
CREATE DATABASE iceberg;
USE iceberg;
/*hive table definition */
CREATE TABLE ice.ice.user_log_sink (
user_id INT,
item_id STRING,
PRIMARY KEY (user_id) NOT ENFORCED
)WITH(
'engine.hive.enabled' = 'true',
);
and when i start querying table i have no data also i changed execution mode
to batch and the same issue
also an abnormal behavior when i canceled the job datafiles loaded to
warehouse directory else the streaming job continue executed without datafiles
loaded to warehouse and hive table
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
