Hi everyone, I am writing to propose that we disable build isolation in boot.go of the Beam Python Containers by default, starting with the upcoming Beam 2.75 release.
*Context* - Starting with pip version 25.3 ( https://discuss.python.org/t/announcement-pip-25-3-release/104550), build isolation became the default mechanism for package installation. This means pip now creates a *temporary, isolated* environment to build packages, which often requires fetching build-time dependencies (like setuptools) from external repositories. - In restricted network environments—such as Dataflow jobs configured without public IPs—this change can cause pipelines to hang or fail with "network unreachable" errors during container initialization. We have observed these failures in use cases where upgrading to a newer image version (containing a more recent pip) broke previously working pipelines. - To address this, we already introduced an experimental flag in Beam 2.72: `--experiments=pip_no_build_isolation` ( https://github.com/apache/beam/pull/37331). This flag instructs the Beam SDK's container boot process to disable build isolation during package installation, preventing external network calls. *Proposal* I propose making this disabled state the default behavior for Beam 2.75 (Beam 2.74 is currently in RC). Here are some rationales: - Before upgrading to pip 25.3, Beam did not use build isolation, and users rarely reported issues regarding package builds. - Build isolation requires network access, which fundamentally limits the ability to run pipelines in secure or restricted network environments (See the above Dataflow runner example). Disabling it by default removes our reliance on external networks and PyPI availability, especially for users who already include their dependencies in custom containers. - To accommodate any edge cases, we can introduce a new flag (`--experiments=pip_use_build_isolation`) to turn build isolation back on if users truly need it, though we have not yet encountered a use case requiring it. Please let me know your thoughts or if you have any concerns. Thanks, Shunping
