This is an automated email from the ASF dual-hosted git repository.
tqchen pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm-ffi.git
The following commit(s) were added to refs/heads/main by this push:
new 347d71fe [FIX] Support free-threaded Python cross-builds (#706)
347d71fe is described below
commit 347d71fef17ad77dc9810424113a92e560b4f643
Author: Alfredo Luque <[email protected]>
AuthorDate: Wed Sep 2 14:16:24 2026 -0400
[FIX] Support free-threaded Python cross-builds (#706)
## Summary
The conda-forge Python 3.14 free-threading migration is blocked for TVM
FFI on platforms that rely on cross-compilation. Linux PPC64LE and macOS
ARM64 builds fail during CMake's Python development lookup even though
the target 3.14t headers are installed, while native 3.14t builds and
GIL-enabled Python 3.14 cross-builds succeed.
This PR makes Python discovery distinguish the host interpreter used to
run Cython from the target development artifacts used to build the
extension. That lets downstream packagers build Python 3.14t wheels for
those architectures without changing native builds or GIL-enabled
cross-build behavior.
## Background
Cross-compiling against Python 3.14t currently finds the host
interpreter and detects
`Py_GIL_DISABLED`, but the second `FindPython` call combines that host
interpreter with target
`Development.Module` artifacts. CMake 4.4 also enforces the
`gil_disabled` ABI default of `OFF`,
so it rejects the installed `python3.14t` target headers and reports
`missing: Development.Module`.
The failing configurations are visible in
[conda-forge/apache-tvm-ffi-feedstock#20](https://github.com/conda-forge/apache-tvm-ffi-feedstock/pull/20):
Linux x86_64 to PPC64LE and macOS x86_64 to ARM64. Native Python 3.14t
and cross-built GIL-enabled
Python 3.14 already pass.
This change keeps the first interpreter for free-thread detection and
Cython, performs a
development-only target lookup when cross-compiling, and uses
`SKBUILD_SOABI` for the target
extension suffix when available. Native discovery remains unchanged.
Fixes #705.
## Testing
- configured with CMake 4.4.2 against the exact PPC64LE Python 3.14t
target headers; generated
`core.cpython-314t-powerpc64le-linux-gnu.so`
- configured with CMake 4.4.2 against the exact macOS ARM64 Python 3.14t
target headers; generated
`core.cpython-314t-darwin.so`
- verified the cross path on CMake 3.26.4 and 3.30.5 with developer
warnings treated as errors
- built and imported an editable native Python 3.14t package; verified
the extension retained its
`cpython-314t` suffix
- `pytest -n0 -vvs tests/python/test_free_threaded_python_helpers.py
tests/python/test_build.py::test_build_cpp`
- `cmake-format --check CMakeLists.txt`
- `cmake-lint CMakeLists.txt`
- ASF header, file-type, and whitespace checks
## AI Disclosure
Was AI used in the creation of this PR?
- [ ] No. This was fully written by a human.
- [ ] Limited (uses other than code generation, such as ideation or
debugging, or very limited code generation)
- [ ] Moderate (code generated by an LLM with continuous human review
and intervention)
- [x] High (key decisions made by a human with active steering and
feedback; final code reviewed by the human)
- [ ] Vibe Coded (generated largely autonomously from an initial prompt
with limited human steering or review)
The human identified the failing downstream migration, set the upstream
issue and PR scope, and
made the key workflow decisions. OpenAI Codex (GPT-5), including
delegated Codex agents,
investigated the CMake behavior, implemented the change, ran the native
and cross-build validation
matrix, reviewed the final diff, and prepared this PR. The human
reviewed and approved the final
diff; agent review did not replace that review.
---------
Co-authored-by: tqchen <[email protected]>
---
CMakeLists.txt | 34 ++++++++++++++++++++++++++++------
1 file changed, 28 insertions(+), 6 deletions(-)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 2e76a758..b89ab290 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -247,22 +247,34 @@ if (TVM_FFI_BUILD_PYTHON_MODULE)
# Run a Python script to check for free-threaded build
execute_process(
- COMMAND ${Python_EXECUTABLE} -c
+ COMMAND "${Python_EXECUTABLE}" -c
"import sysconfig;
print(sysconfig.get_config_var('Py_GIL_DISABLED') == 1)"
OUTPUT_VARIABLE PYTHON_IS_FREE_THREADED
OUTPUT_STRIP_TRAILING_WHITESPACE
)
if (PYTHON_IS_FREE_THREADED)
message(STATUS "Free-threaded Python detected.")
+ if (CMAKE_CROSSCOMPILING
+ AND CMAKE_VERSION VERSION_GREATER_EQUAL "4.4"
+ AND NOT DEFINED Python_FIND_ABI
+ )
+ # CMake 4.4 enforces a default gil_disabled value of OFF, excluding
target 3.14t headers.
+ set(Python_FIND_ABI "ANY;ANY;ANY;ON") # cmake-lint: disable=C0103
+ endif ()
endif ()
# The PyObject-tying module builds against the per-version ABI (WITH_SOABI
below), so it never
# needs the limited-API (abi3) SABIModule component -- a plain
Development.Module find suffices
- # for every version, free-threaded or not.
- set(_tvm_ffi_python_version ${Python_VERSION})
+ # for every version, free-threaded or not. Cross builds run Cython with the
host interpreter while
+ # compiling against target headers.
+ if (CMAKE_CROSSCOMPILING)
+ set(_tvm_ffi_python_components Development.Module)
+ else ()
+ set(_tvm_ffi_python_components Interpreter Development.Module)
+ endif ()
find_package(
- Python ${_tvm_ffi_python_version} EXACT
- COMPONENTS Interpreter Development.Module
+ Python ${Python_VERSION} EXACT
+ COMPONENTS ${_tvm_ffi_python_components}
REQUIRED
)
set(_core_cpp ${CMAKE_CURRENT_BINARY_DIR}/core.cpp)
@@ -283,7 +295,7 @@ if (TVM_FFI_BUILD_PYTHON_MODULE)
)
add_custom_command(
OUTPUT ${_core_cpp}
- COMMAND ${Python_EXECUTABLE} -m cython --cplus ${_core_pyx} -o
${_core_cpp} --module-name
+ COMMAND "${Python_EXECUTABLE}" -m cython --cplus ${_core_pyx} -o
${_core_cpp} --module-name
"tvm_ffi.core"
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMMENT "Transpiling ${_core_pyx} to ${_core_cpp}"
@@ -295,6 +307,16 @@ if (TVM_FFI_BUILD_PYTHON_MODULE)
# PyObject_GC_Del, atomic header reads), so we build against the per-version
ABI rather than the
# limited (abi3) ABI.
python_add_library(tvm_ffi_cython MODULE "${_core_cpp}" WITH_SOABI)
+ # A target-only FindPython lookup may not derive SOABI without running the
target interpreter.
+ if (CMAKE_CROSSCOMPILING AND SKBUILD_SOABI)
+ if (WIN32)
+ set_property(TARGET tvm_ffi_cython PROPERTY SUFFIX
".${SKBUILD_SOABI}.pyd")
+ else ()
+ set_property(
+ TARGET tvm_ffi_cython PROPERTY SUFFIX
".${SKBUILD_SOABI}${CMAKE_SHARED_MODULE_SUFFIX}"
+ )
+ endif ()
+ endif ()
set_target_properties(tvm_ffi_cython PROPERTIES OUTPUT_NAME "core")
target_include_directories(
tvm_ffi_cython PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/python/tvm_ffi/cython
