alnzng commented on code in PR #567:
URL: https://github.com/apache/flink-agents/pull/567#discussion_r2944386841
##########
python/jar_manifest.json:
##########
@@ -0,0 +1,36 @@
+{
+ "maven_base_url": "https://repo1.maven.org/maven2";,
+ "group_id": "org.apache.flink",
+ "version": "0.3-SNAPSHOT",
Review Comment:
Just want to confirm - We have a tool to maintain this version in `main`
branch, right?
##########
python/_build_backend/backend.py:
##########
@@ -0,0 +1,183 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#################################################################################
+
+"""Custom PEP 517 build backend that downloads JARs from Maven Central.
+
+Wraps ``setuptools.build_meta`` and overrides ``build_wheel()`` to download
+JAR files before the standard setuptools build runs. This ensures that
+``pip install flink-agents`` (from sdist) produces a wheel that already
Review Comment:
To make sure I fully understand the UX impact for both devs and users, I
have a few clarifying questions:
- Local Development Flow: How does ./tools/build.sh behave on the local main
branch now? Since this change seems to expect JARs to be available in a Maven
repo, how should a developer test local Java changes before they are published?
- Distribution: Just to confirm, will we only be providing the sdist
(.tar.gz) on PyPI moving forward, with no wheel files available?
- Dependency Management: Since dependencies will now be resolved on the
user's side at install time, they may end up with different library versions
than what we tested at release time. If a newer dependency breaks the
framework, is there a way for us to control or 'pin' these versions from the
framework side, or is the burden now on the user to manage those conflicts?
##########
python/_build_backend/backend.py:
##########
@@ -0,0 +1,183 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#################################################################################
+
+"""Custom PEP 517 build backend that downloads JARs from Maven Central.
+
+Wraps ``setuptools.build_meta`` and overrides ``build_wheel()`` to download
+JAR files before the standard setuptools build runs. This ensures that
+``pip install flink-agents`` (from sdist) produces a wheel that already
+contains the required JARs.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import logging
+import os
+import urllib.request
+from pathlib import Path
+
+from setuptools.build_meta import (
+ build_sdist,
+ get_requires_for_build_sdist,
+ get_requires_for_build_wheel,
+ prepare_metadata_for_build_wheel,
+)
+from setuptools.build_meta import (
+ build_wheel as _setuptools_build_wheel,
+)
+
+__all__ = [
+ "build_sdist",
+ "build_wheel",
+ "get_requires_for_build_sdist",
+ "get_requires_for_build_wheel",
+ "prepare_metadata_for_build_wheel",
+]
+
+# PEP 660 editable install hooks (setuptools >= 64)
+try:
+ from setuptools.build_meta import (
+ build_editable,
+ get_requires_for_build_editable,
+ prepare_metadata_for_build_editable,
+ )
+
+ __all__ += [
+ "build_editable",
+ "get_requires_for_build_editable",
+ "prepare_metadata_for_build_editable",
+ ]
+except ImportError:
+ pass
+
+logger = logging.getLogger(__name__)
+
+_MANIFEST_FILE = "jar_manifest.json"
+_SKIP_ENV_VAR = "FLINK_AGENTS_SKIP_JAR_DOWNLOAD"
+_MIRROR_ENV_VAR = "FLINK_AGENTS_MAVEN_MIRROR"
+_TRUTHY_VALUES = frozenset({"1", "true", "yes", "on"})
+
+
+# ---------------------------------------------------------------------------
+# Public PEP 517 hook
+# ---------------------------------------------------------------------------
+
+
+def build_wheel(
+ wheel_directory: str,
+ config_settings: dict | None = None,
+ metadata_directory: str | None = None,
+) -> str:
+ """Build wheel after downloading JARs from Maven Central."""
+ _ensure_jars()
+ return _setuptools_build_wheel(
+ wheel_directory,
+ config_settings=config_settings,
+ metadata_directory=metadata_directory,
+ )
+
+
+# ---------------------------------------------------------------------------
+# Internal helpers
+# ---------------------------------------------------------------------------
+
+
+def _ensure_jars() -> None:
+ """Download JARs listed in *jar_manifest.json* if they are not present."""
+ manifest_path = Path(_MANIFEST_FILE)
+ if not manifest_path.exists():
+ logger.info("No %s found - skipping JAR download.", _MANIFEST_FILE)
+ return
+
+ if os.environ.get(_SKIP_ENV_VAR, "").lower() in _TRUTHY_VALUES:
+ logger.info(
+ "%s is set - skipping JAR download.",
+ _SKIP_ENV_VAR,
+ )
+ return
+
+ manifest = _load_manifest(manifest_path)
+ maven_base_url = os.environ.get(_MIRROR_ENV_VAR) or
manifest["maven_base_url"]
+ maven_base_url = maven_base_url.rstrip("/")
+ group_path = manifest["group_id"].replace(".", "/")
+ version = manifest["version"]
+
+ for jar_entry in manifest["jars"]:
Review Comment:
Technically, when a user run `pip install`, they should already have the
Flink version decided. Ideally, the users only need to download a jar. But I
guess "download all 5 jars" is not a big deal, since this only happen once
during each build time.
##########
docs/content/docs/get-started/installation.md:
##########
@@ -170,6 +170,15 @@ After building:
- The Python package is installed and ready to use
- The distribution JAR is located at:
`dist/flink-${FLINK_VERSION%.*}/target/flink-agents-dist-*.jar`
+### Build Environment Variables
+
+The following environment variables can be used to control how JARs are
resolved during `pip install flink-agents` (from sdist) or `python -m build`:
+
+| Variable | Description |
+|----------|-------------|
+| `FLINK_AGENTS_SKIP_JAR_DOWNLOAD` | Set to `1`, `true`, `yes`, or `on` to
skip downloading JARs from Maven Central. Useful when building from source with
`tools/build.sh`, which copies JARs from the local Maven build. |
Review Comment:
Are these values case sensitive?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
