Caizhi Weng created FLINK-15570:
-----------------------------------
Summary: Support inserting into non-empty csv tables
Key: FLINK-15570
URL: https://issues.apache.org/jira/browse/FLINK-15570
Project: Flink
Issue Type: Improvement
Components: Connectors / FileSystem, Formats (JSON, Avro, Parquet,
ORC, SequenceFile)
Reporter: Caizhi Weng
Currently we only support inserting into empty csv tables. It would be nice for
a new user from traditional database to try out Flink if we support inserting
into non-empty csv tables.
In other words, it would be nice if the following SQL produces a valid result:
{code:java}
CREATE TABLE myTable(
a INT,
b DOUBLE
) WITH (
'connector.type' = 'filesystem',
'connector.path' = an existing csv file,
'format.type' = 'csv',
'format.derive-schema' = 'true'
);
SELECT * FROM myTable; // produces 3 rows
INSERT INTO myTable VALUES (4, 4.4);
SELECT * FROM myTable; // produces 4 rows{code}
We currently only have two write modes, namely NO_OVERWRITE and OVERWRITE. In
NO_OVERWRITE mode we can only insert into empty csv tables, while in OVERWRITE
mode inserting into a csv table will wipe all existing data in it (in the above
example, the last SELECT will produce only 1 instead of 4 rows) which is really
strange for a mere INSERT operation. We need to add a new APPEND write mode, or
change the behavior for OVERWRITE mode to something like appending to files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
