beetle0915 opened a new pull request, #29513: URL: https://github.com/apache/flink/pull/29513
## What is the purpose of the change Implement [FLINK-40192](https://issues.apache.org/jira/browse/FLINK-40192), adding `DataFrame.join_asof()` and `pf.proctime()` as described in [FLIP-591](https://cwiki.apache.org/confluence/spaces/FLINK/pages/430409184/FLIP-591+Introducing+Python+DataFrame+API+in+PyFlink). This lets DataFrame users perform processing-time lookup joins and event-time temporal joins without converting to Table API or writing the corresponding SQL themselves. ## Brief change log - Add inner and left ASOF joins with an explicit left time attribute, shared keys or paired left/right keys, and expression predicates. - Reuse the existing join preparation and SQL serialization helpers to generate `FOR SYSTEM_TIME AS OF` queries. Keep source lookup capability, primary-key and watermark requirements in the Planner; no Planner or runtime implementation changes. - Validate time attributes and row-wise keys, require a cross-input equality, reject OR conditions, and preserve direct key references for temporal primary-key recognition. - Export `proctime()` and document the new APIs. - Add unit and execution tests. Make the existing collection test fixture's lookup function class public so it meets function validation requirements. ## Verifying this change Added eight unit/Planner test methods covering laziness, key forms, SQL expressions, inline functions, invalid arguments, quoted identifiers, and primary-key preservation. Two execution tests cover lookup and event-time temporal joins, including inner/left results, unmatched rows and historical versions. Each execution test combines its variants into one collection. Local validation: - `python -m unittest pyflink.dataframe.tests.test_dataframe`: 307 tests passed, including the new tests and existing join regressions. - Flake8 for all changed Python files and mypy for `dataframe.py` passed. - The affected Java artifacts were built locally; `flink-python` Spotless and Checkstyle checks passed. - Sphinx HTML documentation build with warnings treated as errors passed. - `git diff --check` passed. Tests used JDK 17 and Python 3.12 with locally built Java artifacts from this worktree. Full-repository `mvn clean verify` and community CI have not been run locally. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: yes; adds `DataFrame.join_asof()` and `proctime()` - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? Python API docstrings and the DataFrame reference documentation. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: OpenAI Codex (codex-cli 0.160.0) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
