Sxnan opened a new pull request, #567: URL: https://github.com/apache/flink-agents/pull/567
Linked issue: N/A ### Purpose of change The `flink-agents` Python wheel includes JAR files (~110 MB), which exceeds PyPI's default 100 MB upload limit. This PR adds a [PEP 517](https://peps.python.org/pep-0517/) custom build backend so that JARs are downloaded from Maven Central during `pip install` (from sdist), keeping the published package well under the limit. **Changes:** - **Custom build backend** (`python/_build_backend/backend.py`): Wraps `setuptools.build_meta`, overrides `build_wheel()` to download JARs per `jar_manifest.json` before the standard build, with SHA-256 verification. - **`jar_manifest.json`**: Declares Maven coordinates, destination paths, and checksums for 5 JARs (1 common + 4 Flink-version-specific thin JARs). - **`pyproject.toml`**: Points `build-backend` to the custom backend. - **Maven POM changes**: - `dist/common/pom.xml`: Remove `maven-deploy-plugin` skip to enable Maven Central deployment. - `dist/flink-{1.20,2.0,2.1,2.2}/pom.xml`: Set `shadedArtifactAttached=true` with `thin` classifier so thin JARs are published as attached artifacts. - **Release scripts**: `create_release_branch.sh` and `update_branch_version.sh` now update `jar_manifest.json` version alongside `pyproject.toml`. - **`tools/ut.sh`**: Run `_build_backend/tests/` before the per-Flink-version test loop. - **`.gitignore`**: Ignore transient `python/flink_agents/lib/`. **Environment variables:** | Variable | Description | |----------|-------------| | `FLINK_AGENTS_SKIP_JAR_DOWNLOAD` | Skip JAR download (for local dev with `tools/build.sh`) | | `FLINK_AGENTS_MAVEN_MIRROR` | Override Maven repository base URL | ### Tests - 11 unit/integration tests in `python/_build_backend/tests/test_backend.py` covering manifest loading, checksum verification, skip logic, HTTP download with a local test server, checksum mismatch detection, and end-to-end `build_wheel` producing a wheel with JARs. ### API No public API changes. The custom build backend is an internal build infrastructure component. ### Documentation - [ ] `doc-needed` <!-- Your PR changes impact docs --> - [ ] `doc-not-needed` <!-- Your PR changes do not impact docs --> - [x] `doc-included` <!-- Your PR already contains the necessary documentation updates --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
