[GitHub] [hudi] davidshtian commented on issue #7591: [SUPPORT] Kinesis Data Analytics Flink1.13 to HUDI

[GitBox]

davidshtian commented on issue #7591:
URL: https://github.com/apache/hudi/issues/7591#issuecomment-1382693602

   > @davidshtian what are recommend setting for querying data in Athena AKA 
Hive SYNC?
   > 
   > ```
   > %flink.ssql(type=update)
   > DROP TABLE if exists orders;
   > CREATE TABLE orders(
   >     orderid VARCHAR PRIMARY KEY NOT ENFORCED,
   >     customer_id VARCHAR,
   >     ts TIMESTAMP(3),
   >     order_value DOUBLE,
   >     priority VARCHAR
   > )
   > WITH (
   >     'connector' = 'hudi',
   >     'path' = 's3a://soumilshah-hudi-demos/tmp/',
   >     'table.type' = 'COPY_ON_WRITE' ,
   >     'hoodie.embed.timeline.server' = 'false',
   >     'hive_sync.enable' = 'true',  
   >     'hive_sync.mode' = 'hms'     
   > 
   > );
   > ```
   > 
   > @davidshtian also curious why did you use %ssql instead of 
%flink.ssql(type=update)
   > 
   > Also follow up Question at times KDA throws Error is that normal looks 
like its very unstable what are your honest feedback on that
   
   @soumilshah1995  %ssql is also configured by default.
   
   <img width="464" alt="image" 
src="https://user-images.githubusercontent.com/14228056/212458273-c02f5fc9-b508-4c88-8049-319ff998a0bb.png";>
   
   For Glue catalog sync, yes it should be using hive_sync but it is required 
to upload a new version of hudi-flink jar file (and others related), as [hudi 
docs](https://hudi.apache.org/docs/0.10.1/syncing_metastore) shows:
   
   > hudi-flink-bundle module pom.xml sets the scope related to hive as 
provided by default. If you want to use hive sync, you need to use the profile 
flink-bundle-shade-hive during packaging. 
   
   ```
   'hive_sync.enable'='true',
   'hive_sync.db'='<your db>',
   'hive_sync.table' = '<your table>',
   'hive_sync.mode' = 'hms',
   ```
   
   I've tried it but failed to sync, not sure if it is due to this part:
   <img width="832" alt="image" 
src="https://user-images.githubusercontent.com/14228056/212463692-f06859a6-3b3c-4794-b356-06209338808e.png";>
   
   And I used manual way to create the table for Athena. 
   
   Cool content on Youtube~ 👍


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org