[ https://issues.apache.org/jira/browse/FLINK-30326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644312#comment-17644312 ]
Alex Sorokoumov commented on FLINK-30326: ----------------------------------------- I used the following example to conclude that the SQL client creates the table structure. h2. Setup I used a docker-compose with 1 JM and 1 TM that do not have shared volumes as well as an SQL client started on the JM. h3. Step 1: Create FTS catalog and schema via a SQL client on JM and see where the schema files appear {noformat} Flink SQL> CREATE CATALOG my_catalog WITH ( > 'type'='table-store', > 'warehouse'='file:/tmp/table_store' > ); [INFO] Execute statement succeed.Flink SQL> USE CATALOG my_catalog; > Flink SQL> CREATE TABLE word_count ( > word STRING PRIMARY KEY NOT ENFORCED, > cnt BIGINT > ); [INFO] Execute statement succeed.{noformat} h3. Step 2: Check where the directory was created JM: {noformat} root@flink:/opt/flink# ls /tmp/table_store/default.db/word_count/schema/schema-0 /tmp/table_store/default.db/word_count/schema/schema-0 {noformat} TM: {noformat} ls /tmp/table_store/default.db/ ls: cannot access '/tmp/table_store/default.db/': No such file or directory {noformat} h3. Step 3: Do a single insert and see where the new files appear {noformat} Flink SQL> INSERT INTO word_count (word, cnt) VALUES ('foo', 1); [INFO] Submitting SQL update statement to the cluster... WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.apache.flink.api.java.ClosureCleaner (file:/opt/flink/lib/flink-dist-1.17-SNAPSHOT.jar) to field java.lang.Class.ANNOTATION WARNING: Please consider reporting this to the maintainers of org.apache.flink.api.java.ClosureCleaner WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release [INFO] SQL update statement has been successfully submitted to the cluster: Job ID: c9c175d0137280aeeba83cc7b2ce8454 {noformat} On JM: nothing changed On TM: {noformat} $ ls /tmp/table_store/default.db/word_count/ bucket-0 manifest snapshot {noformat} > SQL Client should not create table directory structure > ------------------------------------------------------ > > Key: FLINK-30326 > URL: https://issues.apache.org/jira/browse/FLINK-30326 > Project: Flink > Issue Type: Bug > Components: Table Store > Reporter: Alex Sorokoumov > Priority: Major > > The SQL client creates the schema part of the directory structure for new > tables. This behavior is incorrect because the SQL client could be attached > to a SQL gateway that does not necessarily have access to DFSes for all > catalogs/tables. > I propose to change the behavior such that FTS sink creates the schema, > either on the fly if `auto-create=true` or just as a separate job if > `auto-create=false`. -- This message was sent by Atlassian Jira (v8.20.10#820010)