auroflow opened a new pull request, #28924:
URL: https://github.com/apache/flink/pull/28924
## What is the purpose of the change
Creating a PyFlink literal with `lit(value, data_type)` requires the value
to first cross the Py4J boundary and then pass Flink's Java-side literal
validation. The current implementation can fail at either stage:
1. Some Python values cannot be serialized by Py4J. In particular,
`datetime.date`, `datetime.time`, `datetime.datetime`, and `datetime.timedelta`
are not supported by the Py4J protocol directly. Therefore, literals declared
as `DATE`, `TIME`, `TIMESTAMP`, `TIMESTAMP_LTZ`, or day-time `INTERVAL` fail
before the Java Table API is invoked.
2. Python numeric values are implicitly converted by Py4J into a limited set
of Java boxed classes. A Python `int` becomes `Integer` when it fits into 32
bits and `Long` otherwise, while a Python `float` always becomes `Double`.
Consequently:
- `TINYINT` expects `Byte` but receives `Integer`;
- `SMALLINT` expects `Short` but receives `Integer`;
- `BIGINT` expects `Long` but receives `Integer` for values within the
32-bit range;
- `FLOAT` expects `Float` but receives `Double`.
These values are rejected by
`ValueLiteralExpression.validateValueDataType()`.
The same issue affected nested values in arrays, maps, multisets, and rows.
This change converts Python-only values before transport, normalizes boxed
numeric values in Java, and creates the literal in the same JVM call so that
Py4J cannot convert the normalized value back to a Python primitive.
## Brief change log
- Convert Python date, time, timestamp, interval, and composite literal
values into Java-compatible representations.
- Add type-directed Java conversion for boxed numeric values and recursively
convert arrays, maps, multisets, and rows.
- Create the converted value and typed Table API literal within the same JVM
call.
- Remove the DataFrame API's special-case workaround for small `BIGINT`
values.
- Add Table API, DataFrame API, and Java converter tests.
## Verifying this change
This change added tests and can be verified as follows:
- Added `PythonTableUtilsTest` coverage for primitive numeric types and
composite values, including `MULTISET<SMALLINT>`.
- Extended Table API tests for explicitly typed numeric, temporal, array,
map, multiset, and row literals.
- Extended DataFrame API tests for explicitly typed `SMALLINT`, `FLOAT`, and
`DATE` literals.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: OpenAI Codex (GPT-5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
