This is an automated email from the ASF dual-hosted git repository.
tqchen pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm-tirx.git
The following commit(s) were added to refs/heads/main by this push:
new fc4ebf822 [REFACTOR] Rename examples/extension (#190)
fc4ebf822 is described below
commit fc4ebf822e0bb3eaebfcc3e024d1477e1ea93b41
Author: Linzhang Li <[email protected]>
AuthorDate: Wed Nov 4 13:57:49 2026 -0500
[REFACTOR] Rename examples/extension (#190)
## Summary
This PR renames `examples/extension` to `examples/extension_guides`,
matching `examples/ir_guides`
and `examples/programming_guides`, and splits the two versions of the
extension into sibling
Python projects, `rust/` and `python/`. Each has its own
`pyproject.toml` and installs the same
distribution, `my-ext`, with the same package, `my_ext`, so
`run_example.py` runs unchanged with
whichever one is installed.
### Layout
```text
examples/extension_guides/
├── README.md
├── run_example.py # shared by both versions
├── rust/ # the extension in Rust
│ ├── Cargo.toml, CMakeLists.txt, rustfmt.toml, .gitignore
│ ├── pyproject.toml # scikit-build-core, wheel.packages = ["my_ext"]
│ ├── src/lib.rs
│ └── my_ext/__init__.py # was python/my_ext/__init__.py
└── python/ # the same extension in Python
├── pyproject.toml # setuptools, pure Python
└── my_ext/__init__.py # was my_ext_py/my_ext.py
```
All files are moved as they were, except:
- `rust/Cargo.toml`: the path dependencies on `rust/` and
`3rdparty/tvm-ffi` gain one `../`.
- `rust/pyproject.toml`: `wheel.packages` is `["my_ext"]`.
- `python/pyproject.toml` (new): a setuptools project with the package
`my_ext`.
- `python/my_ext/__init__.py`: the module docstring points to
`rust/my_ext`.
Because both projects are named `my-ext`, installing one replaces the
other. The Python version is
now installed instead of being put on `PYTHONPATH`:
```bash
uv pip install --no-deps ./examples/extension_guides/python
python examples/extension_guides/run_example.py
```
---
docs/advanced/defining_ir_extensions.rst | 42 +++++++++++-----------
docs/advanced/extending_ir_in_rust.rst | 26 +++++++-------
docs/advanced/using_ir_extensions.rst | 14 ++++----
examples/{extension => extension_guides}/README.md | 40 ++++++++++++---------
.../python/my_ext/__init__.py} | 2 +-
.../python/pyproject.toml} | 15 ++++++--
.../{extension => extension_guides}/run_example.py | 0
.../rust}/.gitignore | 0
.../rust}/CMakeLists.txt | 0
.../rust}/Cargo.toml | 4 +--
.../rust}/my_ext/__init__.py | 0
.../rust}/pyproject.toml | 2 +-
.../rust}/rustfmt.toml | 0
.../rust}/src/lib.rs | 0
tests/python/ir/test_op_rust.py | 14 ++++----
15 files changed, 91 insertions(+), 68 deletions(-)
diff --git a/docs/advanced/defining_ir_extensions.rst
b/docs/advanced/defining_ir_extensions.rst
index 3f7991dbf..7af329cac 100644
--- a/docs/advanced/defining_ir_extensions.rst
+++ b/docs/advanced/defining_ir_extensions.rst
@@ -21,20 +21,22 @@ Defining Types, Operators, and Passes
In this chapter, we define the extension set up in :doc:`extending_ir_in_rust`:
a complex type, four operators, and a pass that lowers them to scalar
arithmetic. Each step is shown in Rust, from the library
-`src/lib.rs
<https://github.com/apache/tvm-tirx/blob/main/examples/extension/src/lib.rs>`_,
-and in Python, from the module
-`my_ext_py/my_ext.py
<https://github.com/apache/tvm-tirx/blob/main/examples/extension/my_ext_py/my_ext.py>`_.
+`rust/src/lib.rs
<https://github.com/apache/tvm-tirx/blob/main/examples/extension_guides/rust/src/lib.rs>`_,
+and in Python, from the package
+`python/my_ext/__init__.py
<https://github.com/apache/tvm-tirx/blob/main/examples/extension_guides/python/my_ext/__init__.py>`_.
Both files also contain the validation and traversal helpers that the excerpts
use.
-The two versions define the same names, so the example program runs with
-either. The Python version is a single module that needs no build, which makes
-it a quick way to try out an extension before writing it in Rust. To run the
-example with it, put its directory on ``PYTHONPATH``:
+The two versions are Python projects that install the same package,
+``my_ext``, with the same names, so the example program runs with either. The
+Python version needs no build, which makes it a quick way to try out an
+extension before writing it in Rust. To run the example with it, install it in
+place of the Rust version:
.. code-block:: bash
- PYTHONPATH=examples/extension/my_ext_py python
examples/extension/run_example.py
+ uv pip install --no-deps ./examples/extension_guides/python
+ python examples/extension_guides/run_example.py
.. note::
@@ -57,14 +59,14 @@ real and imaginary parts, as in ``TensorType``:
.. group-tab:: Rust
- .. literalinclude:: ../../examples/extension/src/lib.rs
+ .. literalinclude:: ../../examples/extension_guides/rust/src/lib.rs
:language: rust
:start-after: // begin type
:end-before: // end type
.. group-tab:: Python
- .. literalinclude:: ../../examples/extension/my_ext_py/my_ext.py
+ .. literalinclude::
../../examples/extension_guides/python/my_ext/__init__.py
:language: python
:start-after: # begin type
:end-before: # end type
@@ -165,14 +167,14 @@ external state. The ``complex`` operator is registered as
follows:
.. group-tab:: Rust
- .. literalinclude:: ../../examples/extension/src/lib.rs
+ .. literalinclude:: ../../examples/extension_guides/rust/src/lib.rs
:language: rust
:start-after: // begin constructor
:end-before: // end constructor
.. group-tab:: Python
- .. literalinclude:: ../../examples/extension/my_ext_py/my_ext.py
+ .. literalinclude::
../../examples/extension_guides/python/my_ext/__init__.py
:language: python
:start-after: # begin constructor
:end-before: # end constructor
@@ -207,14 +209,14 @@ and computes the result type:
.. group-tab:: Rust
- .. literalinclude:: ../../examples/extension/src/lib.rs
+ .. literalinclude:: ../../examples/extension_guides/rust/src/lib.rs
:language: rust
:start-after: // begin inference
:end-before: // end inference
.. group-tab:: Python
- .. literalinclude:: ../../examples/extension/my_ext_py/my_ext.py
+ .. literalinclude::
../../examples/extension_guides/python/my_ext/__init__.py
:language: python
:pyobject: _infer_complex
@@ -280,7 +282,7 @@ The rewrites are:
.. group-tab:: Rust
- .. literalinclude:: ../../examples/extension/src/lib.rs
+ .. literalinclude:: ../../examples/extension_guides/rust/src/lib.rs
:language: rust
:start-after: // begin lowering
:end-before: // end lowering
@@ -297,7 +299,7 @@ The rewrites are:
.. group-tab:: Python
- .. literalinclude:: ../../examples/extension/my_ext_py/my_ext.py
+ .. literalinclude::
../../examples/extension_guides/python/my_ext/__init__.py
:language: python
:pyobject: _Lowering
@@ -322,14 +324,14 @@ The pass is built from an ordinary named function:
.. group-tab:: Rust
- .. literalinclude:: ../../examples/extension/src/lib.rs
+ .. literalinclude:: ../../examples/extension_guides/rust/src/lib.rs
:language: rust
:start-after: // begin function callback
:end-before: // end function callback
.. group-tab:: Python
- .. literalinclude:: ../../examples/extension/my_ext_py/my_ext.py
+ .. literalinclude::
../../examples/extension_guides/python/my_ext/__init__.py
:language: python
:pyobject: _lower_function
@@ -383,14 +385,14 @@ Finally, the function becomes a pass:
.. group-tab:: Rust
- .. literalinclude:: ../../examples/extension/src/lib.rs
+ .. literalinclude:: ../../examples/extension_guides/rust/src/lib.rs
:language: rust
:start-after: // begin pass
:end-before: // end pass
.. group-tab:: Python
- .. literalinclude:: ../../examples/extension/my_ext_py/my_ext.py
+ .. literalinclude::
../../examples/extension_guides/python/my_ext/__init__.py
:language: python
:pyobject: lower_complex
diff --git a/docs/advanced/extending_ir_in_rust.rst
b/docs/advanced/extending_ir_in_rust.rst
index 6615b20fd..c11fc56fa 100644
--- a/docs/advanced/extending_ir_in_rust.rst
+++ b/docs/advanced/extending_ir_in_rust.rst
@@ -20,8 +20,8 @@ Getting Started with Rust Extensions
.. note::
- All the code in this tutorial is in the repository, as the Python package
- `examples/extension
<https://github.com/apache/tvm-tirx/tree/main/examples/extension>`_.
+ All the code in this tutorial is in the repository, as the Python packages
in
+ `examples/extension_guides
<https://github.com/apache/tvm-tirx/tree/main/examples/extension_guides>`_.
TIRx is designed around the idea that "everything is a plugin": compiler
features can be provided by separately built libraries, using the same IR and
@@ -42,7 +42,7 @@ tensors element by element. ``A`` and ``B`` hold the real and
imaginary parts
of the first tensor, ``C`` and ``D`` those of the second. We will compile and
run it at the end of the tutorial.
-.. literalinclude:: ../../examples/extension/run_example.py
+.. literalinclude:: ../../examples/extension_guides/run_example.py
:language: python
:start-after: # begin kernel
:end-before: # end kernel
@@ -108,20 +108,22 @@ other package. The example package, ``my-ext``, has this
layout:
.. code-block:: text
- examples/extension/
- ├── Cargo.toml # the Rust library, which depends on tvm-tirx
- ├── src/lib.rs # the type, operators, and pass
- ├── CMakeLists.txt # builds the library with cargo and installs it into
my_ext
- ├── pyproject.toml # the Python package, built with scikit-build-core
- ├── python/my_ext/ # loads the library and registers a compilation
pipeline
- └── run_example.py # compiles and runs a function that uses the extension
+ examples/extension_guides/
+ ├── rust/ # the Rust version of my-ext
+ │ ├── Cargo.toml # the Rust library, which depends on tvm-tirx
+ │ ├── src/lib.rs # the type, operators, and pass
+ │ ├── CMakeLists.txt # builds the library with cargo and installs it
into my_ext
+ │ ├── pyproject.toml # the Python package, built with scikit-build-core
+ │ └── my_ext/ # loads the library and registers a compilation
pipeline
+ ├── python/ # the Python version of my-ext, with the same
my_ext package
+ └── run_example.py # compiles and runs a function that uses the
extension
The Rust library is an ordinary crate that builds a shared library. It depends
on the `tvm-tirx <https://crates.io/crates/tvm-tirx>`_ crate, which re-exports
TVM FFI as ``tvm_tirx::tvm_ffi``, and also lists ``tvm-ffi`` itself, because
the code that ``#[dispatch]`` generates refers to the crate by that name:
-.. literalinclude:: ../../examples/extension/Cargo.toml
+.. literalinclude:: ../../examples/extension_guides/rust/Cargo.toml
:language: toml
:start-at: [lib]
@@ -131,7 +133,7 @@ install the extension from the repository root in the same
Python environment:
.. code-block:: bash
uv pip install scikit-build-core
- uv pip install --no-build-isolation --no-deps ./examples/extension
+ uv pip install --no-build-isolation --no-deps
./examples/extension_guides/rust
The example uses this checkout's Rust crate and pinned TVM FFI sources.
The package depends on ``apache-tvm-tirx`` at run time; disabling build
diff --git a/docs/advanced/using_ir_extensions.rst
b/docs/advanced/using_ir_extensions.rst
index 345768a5a..6a7b4ea26 100644
--- a/docs/advanced/using_ir_extensions.rst
+++ b/docs/advanced/using_ir_extensions.rst
@@ -30,10 +30,10 @@ static init blocks, which run as soon as the library loads.
To write them in
TVMScript, Python also needs a module that holds their builders. Definitions
under ``tirx.`` go to ``T``, but an extension namespace such as ``my_ext`` is
published only into a module that the extension declares. The package's
-`__init__.py
<https://github.com/apache/tvm-tirx/blob/main/examples/extension/python/my_ext/__init__.py>`_
+`__init__.py
<https://github.com/apache/tvm-tirx/blob/main/examples/extension_guides/rust/my_ext/__init__.py>`_
therefore declares itself as that module before it loads the library:
-.. literalinclude:: ../../examples/extension/python/my_ext/__init__.py
+.. literalinclude:: ../../examples/extension_guides/rust/my_ext/__init__.py
:language: python
:start-after: # begin load
:end-before: # end load
@@ -65,7 +65,7 @@ The registered type and operators let the IR express complex
values;
pipeline does not know about the pass, so the package registers a pipeline,
also named ``my_ext``, that runs it first:
-.. literalinclude:: ../../examples/extension/python/my_ext/__init__.py
+.. literalinclude:: ../../examples/extension_guides/rust/my_ext/__init__.py
:language: python
:start-after: # begin pipeline
:end-before: # end pipeline
@@ -75,11 +75,11 @@ also named ``my_ext``, that runs it first:
Compile the Function
--------------------
-`run_example.py
<https://github.com/apache/tvm-tirx/blob/main/examples/extension/run_example.py>`_
+`run_example.py
<https://github.com/apache/tvm-tirx/blob/main/examples/extension_guides/run_example.py>`_
imports ``my_ext``, which loads the library and registers the pipeline, and
compiles the function from :doc:`extending_ir_in_rust` with it:
-.. literalinclude:: ../../examples/extension/run_example.py
+.. literalinclude:: ../../examples/extension_guides/run_example.py
:language: python
:start-after: # begin compile
:end-before: # end compile
@@ -95,7 +95,7 @@ the backend. To check that its rewrite also computes the
right numbers, the
script runs the compiled function on random ``float32`` inputs and compares
both outputs with NumPy's complex multiplication:
-.. literalinclude:: ../../examples/extension/run_example.py
+.. literalinclude:: ../../examples/extension_guides/run_example.py
:language: python
:start-after: # begin comparison
:end-before: # end comparison
@@ -104,7 +104,7 @@ With the package installed as in
:doc:`extending_ir_in_rust`, run the script:
.. code-block:: bash
- python examples/extension/run_example.py
+ python examples/extension_guides/run_example.py
A successful run prints:
diff --git a/examples/extension/README.md b/examples/extension_guides/README.md
similarity index 55%
rename from examples/extension/README.md
rename to examples/extension_guides/README.md
index 8c26307bd..2a6e596db 100644
--- a/examples/extension/README.md
+++ b/examples/extension_guides/README.md
@@ -15,19 +15,25 @@
<!--- specific language governing permissions and limitations -->
<!--- under the License. -->
-# TIRx Extension Example
-
-A Python package, `my-ext`, that extends TIRx from Rust: a complex-number type,
-operators that build and multiply complex values, and a pass that lowers them
to
-scalar arithmetic. Importing `my_ext` loads the Rust library, provides the
-`my_ext.*` builders for TVMScript, and registers the `my_ext` compilation
-pipeline. The tutorial in `docs/advanced/` walks through the code.
-
-- `src/lib.rs`: the type, operators, and pass, built by cargo as a shared
library
-- `CMakeLists.txt`, `pyproject.toml`: build the library and install it into
the package
-- `python/my_ext/`: loads the library and registers the pipeline
+# TIRx Extension Guide Examples
+
+The `my_ext` package that the extension guides in `docs/advanced/` walk
through:
+a complex-number type, operators that build and multiply complex values, and a
+pass that lowers them to scalar arithmetic. Importing `my_ext` registers them,
+provides the `my_ext.*` builders for TVMScript, and registers the `my_ext`
+compilation pipeline.
+
+The extension is written twice, as two Python projects that install the same
+distribution, `my-ext`, and the same package, `my_ext`:
+
+- `rust/`: the extension in Rust
+ - `src/lib.rs`: the type, operators, and pass, built by cargo as a shared
library
+ - `CMakeLists.txt`, `pyproject.toml`: build the library and install it into
the package
+ - `my_ext/`: loads the library and registers the pipeline
+- `python/`: the same extension in Python, a prototype that needs no build
+ - `pyproject.toml`: the pure Python project
+ - `my_ext/`: the type, operators, pass, and pipeline
- `run_example.py`: compiles a function that uses the extension and checks it
against NumPy
-- `my_ext_py/my_ext.py`: the same extension written in Python, a prototype
that needs no build
## Install and Run
@@ -36,8 +42,8 @@ run from the repository root in the same Python environment:
```bash
uv pip install scikit-build-core
-uv pip install --no-build-isolation --no-deps ./examples/extension
-python examples/extension/run_example.py
+uv pip install --no-build-isolation --no-deps ./examples/extension_guides/rust
+python examples/extension_guides/run_example.py
```
The build uses this checkout's `tvm-tirx` crate and pinned TVM FFI sources.
@@ -45,8 +51,10 @@ Disabling build isolation keeps the native libraries from
the same checkout
available to cargo, instead of installing a released package in a temporary
build environment.
-To run the example with the Python prototype instead, without building
anything:
+To run the example with the Python version instead, install it in place of the
+Rust one:
```bash
-PYTHONPATH=examples/extension/my_ext_py python
examples/extension/run_example.py
+uv pip install --no-deps ./examples/extension_guides/python
+python examples/extension_guides/run_example.py
```
diff --git a/examples/extension/my_ext_py/my_ext.py
b/examples/extension_guides/python/my_ext/__init__.py
similarity index 98%
rename from examples/extension/my_ext_py/my_ext.py
rename to examples/extension_guides/python/my_ext/__init__.py
index 8574d9ea2..898e3920f 100644
--- a/examples/extension/my_ext_py/my_ext.py
+++ b/examples/extension_guides/python/my_ext/__init__.py
@@ -16,7 +16,7 @@
# under the License.
"""A Python prototype of my_ext: the complex type, its operators, and the pass
that lowers them.
-It defines the same names as the Rust package in ``python/my_ext``, so
``run_example.py`` runs
+It defines the same names as the Rust package in ``rust/my_ext``, so
``run_example.py`` runs
with either; a process loads one of them.
"""
diff --git a/examples/extension/rustfmt.toml
b/examples/extension_guides/python/pyproject.toml
similarity index 70%
copy from examples/extension/rustfmt.toml
copy to examples/extension_guides/python/pyproject.toml
index f5a5255a8..4fc87f52b 100644
--- a/examples/extension/rustfmt.toml
+++ b/examples/extension_guides/python/pyproject.toml
@@ -15,5 +15,16 @@
# specific language governing permissions and limitations
# under the License.
-# Keep calls and method chains on one line up to the 100-column limit.
-use_small_heuristics = "Max"
+[project]
+name = "my-ext"
+version = "0.1.0"
+description = "A TIRx extension written in Python: a complex type, its
operators, and a lowering pass"
+requires-python = ">=3.10"
+dependencies = ["apache-tvm-tirx"]
+
+[build-system]
+requires = ["setuptools>=64"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools]
+packages = ["my_ext"]
diff --git a/examples/extension/run_example.py
b/examples/extension_guides/run_example.py
similarity index 100%
rename from examples/extension/run_example.py
rename to examples/extension_guides/run_example.py
diff --git a/examples/extension/.gitignore
b/examples/extension_guides/rust/.gitignore
similarity index 100%
rename from examples/extension/.gitignore
rename to examples/extension_guides/rust/.gitignore
diff --git a/examples/extension/CMakeLists.txt
b/examples/extension_guides/rust/CMakeLists.txt
similarity index 100%
rename from examples/extension/CMakeLists.txt
rename to examples/extension_guides/rust/CMakeLists.txt
diff --git a/examples/extension/Cargo.toml
b/examples/extension_guides/rust/Cargo.toml
similarity index 92%
rename from examples/extension/Cargo.toml
rename to examples/extension_guides/rust/Cargo.toml
index a22b57344..a224f8446 100644
--- a/examples/extension/Cargo.toml
+++ b/examples/extension_guides/rust/Cargo.toml
@@ -28,6 +28,6 @@ crate-type = ["cdylib"]
[dependencies]
# This example builds against this checkout. A standalone extension depends on
# the published releases instead, e.g. tvm-tirx = "<version>".
-tvm-tirx = { path = "../../rust" }
+tvm-tirx = { path = "../../../rust" }
# Procedural macros resolve the FFI crate directly; use the same pinned
sources.
-tvm-ffi = { path = "../../3rdparty/tvm-ffi/rust/tvm-ffi" }
+tvm-ffi = { path = "../../../3rdparty/tvm-ffi/rust/tvm-ffi" }
diff --git a/examples/extension/python/my_ext/__init__.py
b/examples/extension_guides/rust/my_ext/__init__.py
similarity index 100%
rename from examples/extension/python/my_ext/__init__.py
rename to examples/extension_guides/rust/my_ext/__init__.py
diff --git a/examples/extension/pyproject.toml
b/examples/extension_guides/rust/pyproject.toml
similarity index 97%
rename from examples/extension/pyproject.toml
rename to examples/extension_guides/rust/pyproject.toml
index 36a9f3e26..9efc41eec 100644
--- a/examples/extension/pyproject.toml
+++ b/examples/extension_guides/rust/pyproject.toml
@@ -29,5 +29,5 @@ build-backend = "scikit_build_core.build"
[tool.scikit-build]
wheel.py-api = "py3"
-wheel.packages = ["python/my_ext"]
+wheel.packages = ["my_ext"]
cmake.build-type = "Release"
diff --git a/examples/extension/rustfmt.toml
b/examples/extension_guides/rust/rustfmt.toml
similarity index 100%
rename from examples/extension/rustfmt.toml
rename to examples/extension_guides/rust/rustfmt.toml
diff --git a/examples/extension/src/lib.rs
b/examples/extension_guides/rust/src/lib.rs
similarity index 100%
rename from examples/extension/src/lib.rs
rename to examples/extension_guides/rust/src/lib.rs
diff --git a/tests/python/ir/test_op_rust.py b/tests/python/ir/test_op_rust.py
index 5b60ad6a3..0d129139e 100644
--- a/tests/python/ir/test_op_rust.py
+++ b/tests/python/ir/test_op_rust.py
@@ -37,7 +37,7 @@ import pytest
import tvm_tirx
RUST_DIR = Path(__file__).resolve().parents[3] / "rust"
-EXTENSION_DIR = RUST_DIR.parent / "examples" / "extension"
+EXTENSION_DIR = RUST_DIR.parent / "examples" / "extension_guides"
def find_cargo():
@@ -91,11 +91,11 @@ def define_ir_library():
@pytest.fixture(scope="module")
def extension_package(tmp_path_factory):
- """The my_ext package of examples/extension, built from its declared
source dependency and
- laid out as its wheel installs it."""
- debug_dir = cargo_build(EXTENSION_DIR / "Cargo.toml")
+ """The my_ext package of examples/extension_guides/rust, built from its
declared source
+ dependency and laid out as its wheel installs it."""
+ debug_dir = cargo_build(EXTENSION_DIR / "rust" / "Cargo.toml")
root = tmp_path_factory.mktemp("extension")
- package = shutil.copytree(EXTENSION_DIR / "python" / "my_ext", root /
"my_ext")
+ package = shutil.copytree(EXTENSION_DIR / "rust" / "my_ext", root /
"my_ext")
shutil.copy(debug_dir / library_name("my_ext"), package)
return root
@@ -169,7 +169,7 @@ ROUND_TRIP = """
# Checks of the extension beyond those of its example script: the type,
printing and parsing,
# invalid calls, chained products, float64 parts, and what the lowering
rejects.
EXTENSION_CHECKS = """
- # Run after examples/extension/run_example.py, whose names it uses.
+ # Run after examples/extension_guides/run_example.py, whose names it uses.
import tvm_ffi
from tvm_tirx import ir, tirx
@@ -337,7 +337,7 @@ def test_extension_example(extension_package):
def test_extension_prototype():
"""The Python prototype of the extension runs the same example and
checks."""
script = EXTENSION_DIR / "run_example.py"
- prototype = EXTENSION_DIR / "my_ext_py"
+ prototype = EXTENSION_DIR / "python"
run_python(script.read_text(), EXTENSION_CHECKS, PROTOTYPE_CHECKS,
python_path=[prototype])
