Nikolaus Schuetz created FLINK-40236:
----------------------------------------
Summary: _infer_type infers ARRAY<NULL> for a list whose first
element is None
Key: FLINK-40236
URL: https://issues.apache.org/jira/browse/FLINK-40236
Project: Flink
Issue Type: Bug
Components: API / Python
Reporter: Nikolaus Schuetz
_infer_type in flink-python/pyflink/table/types.py infers a list's element type
from obj[0] instead of the first non-None element it scans for. When the first
element is None, the
array element type collapses to NULL, so schema inference becomes
order-dependent:
{code}
_infer_type([1, None]) -> ArrayType(BigIntType) # correct
_infer_type([None, 1]) -> ArrayType(NullType) # wrong; should be BigIntType
{code}
The dict branch just above handles this correctly (it infers from the found
non-None value). from_elements([Row(c=[None, 1])]) without an explicit schema
infers ARRAY<NULL>, which
then typically errors downstream.
Fix: infer from the scanned non-None element v rather than obj[0]. Verified
against the released apache-flink 2.3.0 wheel: the array element type is
NullType before the fix and
BigIntType after.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
