Yibo Dong created FLINK-41551:
---------------------------------
Summary: Filesystem table cannot read a NULL partition written by
its own sink
Key: FLINK-41551
URL: https://issues.apache.org/jira/browse/FLINK-41551
Project: Flink
Issue Type: Bug
Environment: Apache Flink 2.3.0 (stock), batch Table API, local
filesystem connector with CSV format. No optimizer rules disabled.
Reporter: Yibo Dong
h2. Description
The filesystem sink successfully writes a row with a NULL partition key,
creating a default partition directory. However, reading the same partitioned
table fails with a {{{}TableException{}}}. The same rows can be read correctly
from an unpartitioned filesystem table.
h2. How to Reproduce
Run the following SQL in a fresh session, using fresh filesystem paths:
{code:sql}
SET 'execution.runtime-mode' = 'BATCH';
CREATE VIEW input_rows AS
SELECT * FROM (VALUES
(1, CAST(NULL AS STRING), 11),
(2, 'ordinary', 12)
) AS t(id, k, v);
CREATE TABLE flat (id INT, k STRING, v INT)
WITH (
'connector' = 'filesystem',
'format' = 'csv',
'csv.null-literal' = 'NULL',
'path' = 'file:///tmp/flink-null-partition-repro/flat'
);
CREATE TABLE part (id INT, k STRING, v INT)
PARTITIONED BY (k)
WITH (
'connector' = 'filesystem',
'format' = 'csv',
'csv.null-literal' = 'NULL',
'path' = 'file:///tmp/flink-null-partition-repro/partitioned'
);
INSERT INTO flat SELECT * FROM input_rows;
INSERT INTO part SELECT * FROM input_rows;
SELECT id, k, v FROM input_rows ORDER BY id;
SELECT id, k, v FROM flat ORDER BY id;
SELECT id, k, v FROM part ORDER BY id;
{code}
h2. Expected Result
All three queries return the same two rows:
{code}
(1, NULL, 11)
(2, ordinary, 12)
{code}
h2. Actual Result
The input view and unpartitioned filesystem table return both rows.
The partitioned filesystem sink creates the directories {{k=DEFAULT_PARTITION}}
and {{{}k=ordinary{}}}, but reading the partitioned table fails before
returning any rows:
{code}
TableException: Path should not be null or empty: null
{code}
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)