alberttwong opened a new issue, #40919: URL: https://github.com/apache/beam/issues/40919
### What would you like to happen? Please validate Apache Beam's Python SDK with the latest published PyArrow, **26.0.0**, and raise the supported upper bound to include it once the relevant tests pass. Apache Beam 2.77.0 currently declares `pyarrow>=14.0.1,<26.0.0`. The same cap is present in [sdks/python/setup.py on current master](https://github.com/apache/beam/blob/b88840e0d3624dc2bec79a5b6f0292b640761ef2/sdks/python/setup.py#L152). The nearby comment asks that the PyArrow test matrix in `sdks/python/tox.ini` be updated together with this bound. ### Use case / motivation We are testing native Arrow Database Connectivity (ADBC) data workflows alongside Beam. Our other current analytics integrations use PyArrow 26, but Beam's cap forces a separate environment or a downgrade to PyArrow 25. This request concerns dependency interoperability; it does not assume PyArrow 26 runtime compatibility is already proven. As a related upstream example, MLflow has already accepted this upgrade: [mlflow/mlflow#27201](https://github.com/mlflow/mlflow/issues/27201), fixed on its main branch by [mlflow/mlflow#27216](https://github.com/mlflow/mlflow/pull/27216). Beam still needs its own validation and dependency update. ### Reproduction On macOS arm64, Python 3.12.15: ```sh uv venv --python 3.12 /tmp/beam-pyarrow26 uv pip install --dry-run --python /tmp/beam-pyarrow26/bin/python \ 'apache-beam==2.77.0' 'pyarrow==26.0.0' ``` Observed resolver result: ```text No solution found when resolving dependencies apache-beam>=2.77.0 depends on pyarrow>=14.0.1,<26.0.0 you require pyarrow==26.0.0 your requirements are unsatisfiable ``` Verified October 30, 2026. PyPI lists PyArrow 26.0.0 as the latest release and requires Python >=3.11, so that Python-version boundary should be considered when updating supported environments. ### Suggested scope - Exercise the Arrow conversion/schema, Parquet I/O, and applicable batch/dataframe paths on PyArrow 26. - Update the SDK dependency range and corresponding PyArrow CI/tox environments after validation. - Retain compatible resolution for supported older Python versions and document any version-specific limits. I have reproduced the package-resolution restriction; I have not bypassed the constraint or run Beam's upstream test suite against PyArrow 26. ### Issue Priority Priority: 3 (dependency interoperability improvement). ### Issue Component Component: sdk-py-core. Prepared with AI assistance (OpenAI Codex). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
