This is an automated email from the ASF dual-hosted git repository.
tqchen pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm-tirx.git
The following commit(s) were added to refs/heads/main by this push:
new bc7565391 [DOCS] Restructure and polish programming guides (#124)
bc7565391 is described below
commit bc7565391e4bd622e037ee30a0d7d64359eea783
Author: Tianqi Chen <[email protected]>
AuthorDate: Thu Oct 29 12:35:14 2026 -0400
[DOCS] Restructure and polish programming guides (#124)
Restructure the CUDA/PTX programming guides into a single "Programming
Guides" section, and polish the overview, Control Flow, and
Metaprogramming pages.
- **Layout:**
- All guides move to `docs/programming_guides/`, one level deep. Tile
operations are the one exception and keep their own `tile/` folder.
- Sidebar order: Programming Overview, Language Basics, Control Flow,
Types, Tensors, and Memory, Metaprogramming, Calling CUDA and PTX, Data
Layout, Tile Operations, Compiling and Inspecting, Profiling with IKET.
- The CUDA backend source guide and PTX TableGen move to the developer
manual as `dev/cuda/` ("CUDA Backend Development").
- **Programming Overview (new):** design philosophy and programming
model, adapted from the Apache TVM TIRx overview. It is reframed around
direct, hardware-native primitives, with tile operations and layouts as
an optional convenience layer. It also covers the authoring-layers table
and the lowering diagram.
- **Control Flow:** a new page with `if`, `for` (`range`, `T.unroll`,
`T.grid`), `while`, `break`/`continue`, and early thread exit
(`T.gpu_thread_return()`). The examples are real kernels: ReLU, row sum,
transpose, and sorted search.
- **Metaprogramming:** a new page with kernels parameterized by captured
values, `@T.inline` fragments, and `@T.jit` specialization.
- **Examples:** the code for Control Flow and Metaprogramming lives in
`examples/programming_guides/` and is included with `literalinclude`.
- **Other changes across the guides:**
- Calling CUDA and PTX now covers the synchronization and warp
intrinsics.
- Page titles and section headings use Title Case.
- Pages open with "This guide ..." instead of "In this chapter".
- CUDA launches use `grid=T.ceildiv(n, 256), block=256`.
- Cross-reference text matches the new titles. This includes the API
reference pages and two docstrings in `backend/cuda/tile/patterns`.
---
docs/compiler/symbolic_integer_analysis.rst | 13 +-
docs/cuda/language_basics.rst | 44 -------
.../cuda/index.rst} | 24 ++--
docs/{ => dev}/cuda/ptx_tablegen.rst | 13 +-
docs/get_started/quick_start.rst | 2 +-
docs/index.rst | 23 ++--
docs/ir/printing_and_parsing.rst | 2 +-
.../language => programming_guides}/compiling.rst | 11 +-
docs/programming_guides/control_flow.rst | 130 +++++++++++++++++++
.../cuda_ptx_interfaces.rst} | 28 ++--
.../data_layout.rst} | 26 ++--
.../iket_profiling.rst | 22 ++--
.../images/iket_affine_b200.png | Bin
docs/programming_guides/index.rst | 144 +++++++++++++++++++++
.../language_basics.rst} | 144 ++-------------------
docs/programming_guides/metaprogramming.rst | 115 ++++++++++++++++
.../tile}/arithmetic.rst | 14 +-
.../tile}/async_copies.rst | 10 +-
.../tile}/blackwell_tensor_cores.rst | 18 +--
.../tile}/data_movement.rst | 12 +-
.../tile/index.rst} | 28 ++--
.../tile}/tile_patterns.rst | 18 +--
.../tile_ops => programming_guides/tile}/tma.rst | 19 ++-
.../tile}/warp_matrix.rst | 14 +-
.../types_tensors_memory.rst} | 43 +++---
docs/reference/python/backend.cuda.iket.rst | 2 +-
docs/reference/python/backend.cuda.ptx.rst | 4 +-
docs/reference/python/backend.cuda.script.tile.rst | 2 +-
.../python/backend.cuda.tile.instructions.rst | 32 ++---
docs/reference/python/backend.cuda.tile.rst | 2 +-
examples/programming_guides/control_flow.py | 126 ++++++++++++++++++
examples/programming_guides/metaprogramming.py | 100 ++++++++++++++
.../backend/cuda/tile/patterns/__init__.py | 3 +-
.../tvm_tirx/backend/cuda/tile/patterns/ewise.py | 3 +-
34 files changed, 827 insertions(+), 364 deletions(-)
diff --git a/docs/compiler/symbolic_integer_analysis.rst
b/docs/compiler/symbolic_integer_analysis.rst
index 8d65c956f..010669eb0 100644
--- a/docs/compiler/symbolic_integer_analysis.rst
+++ b/docs/compiler/symbolic_integer_analysis.rst
@@ -18,13 +18,12 @@
Symbolic Integer Analysis
=========================
-In this chapter, we will introduce symbolic integer analysis for tensor
-shapes, loop bounds, and element indices. A pass that combines four scalar
-accesses into one vector access needs to preserve their positions and
-establish that the whole group is within the tensor. The length may only
-be known at runtime, so these decisions require reasoning about symbolic
-expressions. We will start from a loop's known range, then examine grouped
-indices and a neighboring-element access.
+This guide introduces symbolic integer analysis for tensor shapes, loop bounds,
+and element indices. A pass that combines four scalar accesses into one vector
+access needs to preserve their positions and establish that the whole group is
+within the tensor. The length may only be known at runtime, so these decisions
+require reasoning about symbolic expressions. We will start from a loop's known
+range, then examine grouped indices and a neighboring-element access.
The Analyzer
------------
diff --git a/docs/cuda/language_basics.rst b/docs/cuda/language_basics.rst
deleted file mode 100644
index 131e0b77c..000000000
--- a/docs/cuda/language_basics.rst
+++ /dev/null
@@ -1,44 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-.. Adapted from Apache TVM:
https://tvm.apache.org/docs/_sources/tirx/native_basics.rst
-
-CUDA Language Basics
-====================
-
-In this chapter, we will introduce the CUDA interfaces for writing and running
-TIRx kernels. We will explore how to express computations with explicit control
-over GPU threads and memory, then compile the kernels and inspect their IR and
-generated CUDA code.
-
-* :doc:`language/functions` covers function parameters, symbolic shapes,
- launch configuration, control flow, specialization, and reusable fragments.
-* :doc:`language/tensors` covers element types, vector accesses, shared and
- per-thread storage, mutable scalars, and tensor views.
-* :doc:`language/threads_sync` covers three ways to use CUDA functionality:
- backend intrinsics, PTX instructions, and custom CUDA C++ device functions.
-* :doc:`language/compiling` explains compiler configuration and how to
- inspect the IR and generated CUDA.
-
-.. toctree::
- :hidden:
- :maxdepth: 1
-
- language/functions
- language/tensors
- language/threads_sync
- language/compiling
diff --git a/docs/cuda/backend_implementation.rst b/docs/dev/cuda/index.rst
similarity index 92%
rename from docs/cuda/backend_implementation.rst
rename to docs/dev/cuda/index.rst
index 6a326e0f4..b45518bd9 100644
--- a/docs/cuda/backend_implementation.rst
+++ b/docs/dev/cuda/index.rst
@@ -15,13 +15,13 @@
specific language governing permissions and limitations
under the License.
-CUDA Backend Source Guide
-==========================
+CUDA Backend Development
+========================
-In this chapter, we will find the source files involved in extending or
-debugging the CUDA backend. We will follow ``T.cuda.thread_idx("x")`` from
-the :doc:`GPU add-one example </get_started/quick_start>` to its IR operator
-and CUDA output, then locate the compilation and runtime entry points.
+This guide shows how to find the source files involved in extending or
+debugging the CUDA backend. We will follow ``T.cuda.thread_idx("x")`` from the
+:doc:`GPU add-one example </get_started/quick_start>` to its IR operator and
+CUDA output, then locate the compilation and runtime entry points.
The source tree
---------------
@@ -79,7 +79,7 @@ source generator looks them up through
``tirx.intrinsics.cuda.get_codegen``,
registered in
`python/tvm_tirx/backend/cuda/codegen/registry.py
<https://github.com/apache/tvm-tirx/blob/main/python/tvm_tirx/backend/cuda/codegen/registry.py>`__.
-The :doc:`PTX Tablegen guide <ptx_tablegen>` follows that path for
+The :doc:`PTX TableGen guide <ptx_tablegen>` follows that path for
``add.rn.f32``.
Compilation and launch
@@ -100,8 +100,8 @@ compiles it with NVCC or NVRTC. The runtime implementation
in
`src/backend/cuda/runtime/cuda_module.cc
<https://github.com/apache/tvm-tirx/blob/main/src/backend/cuda/runtime/cuda_module.cc>`__
loads the device module and launches kernels with their grid, block, and
-shared-memory settings. The :doc:`compiling and inspecting guide
-<language/compiling>` shows how to inspect the generated source.
+shared-memory settings. The :doc:`Compiling and Inspecting guide
+</programming_guides/compiling>` shows how to inspect the generated source.
Where to make a change
----------------------
@@ -129,3 +129,9 @@ The following locations provide starting points for common
changes:
The :doc:`CUDA backend API </reference/python/backend.cuda>` documents
the public interfaces in these modules.
+
+.. toctree::
+ :hidden:
+ :maxdepth: 1
+
+ ptx_tablegen
diff --git a/docs/cuda/ptx_tablegen.rst b/docs/dev/cuda/ptx_tablegen.rst
similarity index 94%
rename from docs/cuda/ptx_tablegen.rst
rename to docs/dev/cuda/ptx_tablegen.rst
index 4e7cbf82c..132f272b8 100644
--- a/docs/cuda/ptx_tablegen.rst
+++ b/docs/dev/cuda/ptx_tablegen.rst
@@ -15,15 +15,14 @@
specific language governing permissions and limitations
under the License.
-PTX Tablegen
+PTX TableGen
============
-In this chapter, we will follow ``add.rn.f32`` from a TVMScript call to
-generated inline PTX. TIRx uses table-driven generation: one instruction
-description defines its operands and modifiers, and the backend uses that
-description to check calls and generate CUDA helpers. We will inspect
-this description and its generated code, then see how to add support for
-another instruction form.
+This guide follows ``add.rn.f32`` from a TVMScript call to generated inline
+PTX. TIRx uses table-driven generation: one instruction description defines its
+operands and modifiers, and the backend uses that description to check calls
+and generate CUDA helpers. We will inspect this description and its generated
+code, then see how to add support for another instruction form.
Operands and Modifiers
-----------------------
diff --git a/docs/get_started/quick_start.rst b/docs/get_started/quick_start.rst
index a7490f386..48cfed722 100644
--- a/docs/get_started/quick_start.rst
+++ b/docs/get_started/quick_start.rst
@@ -304,7 +304,7 @@ Further Reading
---------------
- :doc:`/ir/overview`: the IR objects in depth and how they fit together.
-- :doc:`TIRx CUDA Programming Guide </cuda/language_basics>`: writing CUDA
+- :doc:`Programming Guides </programming_guides/index>`: writing CUDA
kernels with TIRx.
- :doc:`Python API </reference/python/index>`: the full Python API reference.
- :doc:`/dev/source_build`: building TIRx from source.
diff --git a/docs/index.rst b/docs/index.rst
index ad34a7160..9a018804e 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -56,22 +56,26 @@ Table of Contents
.. toctree::
:maxdepth: 1
- :caption: Compiler Internals
+ :caption: Programming Guides
:titlesonly:
- compiler/symbolic_integer_analysis
+ programming_guides/index
+ programming_guides/language_basics
+ programming_guides/control_flow
+ programming_guides/types_tensors_memory
+ programming_guides/metaprogramming
+ programming_guides/cuda_ptx_interfaces
+ programming_guides/data_layout
+ programming_guides/tile/index
+ programming_guides/compiling
+ programming_guides/iket_profiling
.. toctree::
:maxdepth: 1
- :caption: CUDA/PTX Programming Guides
+ :caption: Compiler Internals
:titlesonly:
- cuda/language_basics
- cuda/layouts
- cuda/tile_operations
- cuda/iket_profiling
- cuda/backend_implementation
- cuda/ptx_tablegen
+ compiler/symbolic_integer_analysis
.. toctree::
:maxdepth: 1
@@ -87,5 +91,6 @@ Table of Contents
dev/source_build
dev/doc_build
+ dev/cuda/index
dev/ci_cd
dev/release_process
diff --git a/docs/ir/printing_and_parsing.rst b/docs/ir/printing_and_parsing.rst
index 408809c54..9440e97eb 100644
--- a/docs/ir/printing_and_parsing.rst
+++ b/docs/ir/printing_and_parsing.rst
@@ -370,7 +370,7 @@ An explicit ``with`` block shows the same region through
indentation:
@T.function
def add_one(A: T.Tensor((n,), T.float32), B: T.Tensor((n,), T.float32)):
- with T.device_entry(launch=T.cuda.LaunchConfig(grid=((n + 255) //
256,), block=(256,))):
+ with T.device_entry(launch=T.cuda.LaunchConfig(grid=T.ceildiv(n, 256),
block=256)):
i = T.cuda.block_idx("x") * 256 + T.cuda.thread_idx("x")
if i < n:
B[i] = A[i] + 1
diff --git a/docs/cuda/language/compiling.rst
b/docs/programming_guides/compiling.rst
similarity index 93%
rename from docs/cuda/language/compiling.rst
rename to docs/programming_guides/compiling.rst
index 5312ae840..006087b86 100644
--- a/docs/cuda/language/compiling.rst
+++ b/docs/programming_guides/compiling.rst
@@ -17,12 +17,13 @@
.. Adapted from Apache TVM:
https://tvm.apache.org/docs/_sources/tirx/native_basics/cuda/compiling.rst
-Compiling and inspecting
+Compiling and Inspecting
========================
-In this chapter, we will compile a CUDA function with a chosen target and
+This guide shows how to compile a CUDA function with a chosen target and
compiler options, then inspect its IR and generated code. The examples reuse
-``add_one`` from the GPU program in :doc:`Quick Start
</get_started/quick_start>`.
+``add_one`` from the GPU program in :doc:`Quick Start
+</get_started/quick_start>`.
Compiling for CUDA
------------------
@@ -55,7 +56,7 @@ input and output tensors just as in Quick Start. The
:class:`BackendConfig API <tvm_tirx.backend.cuda.BackendConfig>` describes
other compiler settings.
-Overriding options for a kernel
+Overriding Options for a Kernel
-------------------------------
A kernel can override the target's settings through its ``device_entry``.
@@ -65,7 +66,7 @@ its ``T.device_entry`` statement with:
.. code-block:: python
T.device_entry(
- launch=T.cuda.LaunchConfig(grid=((n + 255) // 256,), block=(256,)),
+ launch=T.cuda.LaunchConfig(grid=T.ceildiv(n, 256), block=256),
backend_config={"cuda": {
"nvrtc": ["--use_fast_math", "--generate-line-info"],
}},
diff --git a/docs/programming_guides/control_flow.rst
b/docs/programming_guides/control_flow.rst
new file mode 100644
index 000000000..a23f66174
--- /dev/null
+++ b/docs/programming_guides/control_flow.rst
@@ -0,0 +1,130 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Control Flow
+============
+
+.. note::
+
+ All the code in this guide is in
+ `examples/programming_guides/control_flow.py
+
<https://github.com/apache/tvm-tirx/blob/main/examples/programming_guides/control_flow.py>`_.
+
+This guide shows how to write conditions and loops in TIRx kernels. TIRx uses
+Python's own control flow syntax: ``if``, ``elif``, ``else``, ``for``,
+``while``, ``break``, and ``continue`` all mean what they mean in Python, and
+each thread executes them independently with its own values.
+
+The examples use the following imports and a symbolic length ``n``:
+
+.. literalinclude:: ../../examples/programming_guides/control_flow.py
+ :language: python
+ :start-after: # [imports.begin]
+ :end-before: # [imports.end]
+
+``if`` Statements
+-----------------
+
+An ``if`` statement runs its body only on the threads whose condition holds.
+The following kernel implements ReLU, which keeps positive values and replaces
+the rest with zero:
+
+.. literalinclude:: ../../examples/programming_guides/control_flow.py
+ :language: python
+ :start-after: # [if.begin]
+ :end-before: # [if.end]
+
+The outer ``if i < n`` is the usual bounds check: the last block may have more
+threads than elements left, and the check keeps those threads from reading or
+writing past the end of the tensors. Inside it, ``if`` and ``else`` choose
+between two stores for each element, and ``elif`` can add more branches, just
+as in Python. A branch-free ``T.max(A[i], 0.0)`` would also compute ReLU; the
+``if`` form shows how threads can take different paths.
+
+``for`` Loops
+-------------
+
+A ``for`` loop over ``range(...)`` runs its body once per iteration, in order,
+within each thread. ``T.unroll(...)`` iterates in the same way but asks the
+compiler to expand the loop into one copy of its body per iteration. The
+following kernel uses both to sum each row of an ``n x 64`` matrix, one row
+per thread:
+
+.. literalinclude:: ../../examples/programming_guides/control_flow.py
+ :language: python
+ :start-after: # [for.begin]
+ :end-before: # [for.end]
+
+The outer ``range(16)`` loop stays a loop in the generated code, while the
+inner ``T.unroll(4)`` loop becomes four consecutive additions. Unrolling suits
+short loops with a fixed trip count, where it removes the loop overhead and
+lets the compiler schedule the statements together. ``total`` is a mutable
+local scalar, declared with a type annotation so that it keeps its value
+across iterations.
+
+For nested loops over several dimensions, ``T.grid`` iterates over all index
+combinations, like nested ``range`` loops. The following kernel transposes a
+64 x 64 matrix, with each thread moving a 4 x 4 tile:
+
+.. literalinclude:: ../../examples/programming_guides/control_flow.py
+ :language: python
+ :start-after: # [grid.begin]
+ :end-before: # [grid.end]
+
+``for r, c in T.grid(4, 4)`` visits the 16 pairs ``(r, c)`` row by row, which
+is the same as ``for r in range(4)`` with ``for c in range(4)`` nested inside.
+
+``while`` Loops
+---------------
+
+A ``while`` loop repeats its body as long as its condition holds, which suits
+loops whose trip count depends on the data. The following kernel finds, for
+each query, the position where it would be inserted into a sorted tensor of 64
+values, using binary search:
+
+.. literalinclude:: ../../examples/programming_guides/control_flow.py
+ :language: python
+ :start-after: # [while.begin]
+ :end-before: # [while.end]
+
+Each thread narrows its own range ``[lo, hi)`` until it is empty, so different
+threads may run different numbers of iterations. ``lo`` and ``hi`` are mutable
+local scalars, so they keep their updates from one iteration to the next.
+
+``break`` and ``continue``
+--------------------------
+
+``break`` exits the innermost loop, and ``continue`` skips to its next
+iteration, just as in Python. Both work in ``for`` and ``while`` loops, and
+each thread decides on its own when to break or continue.
+
+Early Thread Exit
+-----------------
+
+``T.gpu_thread_return()`` ends the current thread early. A common use is the
+bounds check, written as an early exit instead of wrapping the rest of the
+kernel in an ``if``:
+
+.. literalinclude:: ../../examples/programming_guides/control_flow.py
+ :language: python
+ :start-after: # [return.begin]
+ :end-before: # [return.end]
+
+Threads past the end of the tensors exit at ``T.gpu_thread_return()``, and the
+remaining threads continue with the store. A plain Python ``return`` is
+different: it returns from the whole function and requires a value, so it
+cannot end a single GPU thread.
diff --git a/docs/cuda/language/threads_sync.rst
b/docs/programming_guides/cuda_ptx_interfaces.rst
similarity index 86%
rename from docs/cuda/language/threads_sync.rst
rename to docs/programming_guides/cuda_ptx_interfaces.rst
index be0ae7aa2..cbbfbc93b 100644
--- a/docs/cuda/language/threads_sync.rst
+++ b/docs/programming_guides/cuda_ptx_interfaces.rst
@@ -17,16 +17,16 @@
.. Adapted from Apache TVM:
https://tvm.apache.org/docs/_sources/tirx/native_basics/cuda/threads_sync.rst
-CUDA and PTX interfaces
-=======================
+Calling CUDA and PTX
+====================
-In this chapter, we will use three interfaces for CUDA programming in
-TVMScript. Backend intrinsics under ``T.cuda`` provide common operations,
-such as synchronization and warp shuffles. ``T.ptx`` lets us specify
-individual PTX instructions. ``T.cuda.func_call`` embeds CUDA C++ source
-so a kernel can call a custom ``__device__`` function.
+This guide shows how to use three interfaces for CUDA programming in TVMScript.
+Backend intrinsics under ``T.cuda`` provide common operations, such as
+synchronization and warp shuffles. ``T.ptx`` lets us specify individual PTX
+instructions. ``T.cuda.func_call`` embeds CUDA C++ source so a kernel can call
+a custom ``__device__`` function.
-Calling backend intrinsics
+Calling Backend Intrinsics
--------------------------
The backend provides helpers for thread synchronization, data movement,
@@ -77,12 +77,12 @@ The device helper calls CUDA's ``__shfl_xor_sync``:
v_ptr[0] = v_ptr[0] + __shfl_xor_sync(0xFFFFFFFF, v_ptr[0], i_ptr[0], 32);
-The :doc:`TMA guide </cuda/tile_ops/tma>` shows a complete barrier
+The :doc:`TMA Transfers guide </programming_guides/tile/tma>` shows a complete
barrier
initialization, arrival, and phase wait around an asynchronous transfer.
.. _cuda-direct-ptx:
-Calling PTX instructions
+Calling PTX Instructions
------------------------
``T.ptx`` selects individual PTX instructions. PTX names its destination
@@ -116,11 +116,11 @@ as in ``T.ptx.ld.global_.b32``; PTX's ``::`` becomes
``__``, as in
spelling, which is convenient for longer instruction names.
The :doc:`PTX reference </reference/python/backend.cuda.ptx>` covers
-predication and special operands. The :doc:`PTX Tablegen guide
-</cuda/ptx_tablegen>` explains how instruction definitions generate these
+predication and special operands. The :doc:`PTX TableGen guide
+</dev/cuda/ptx_tablegen>` explains how instruction definitions generate these
interfaces and their CUDA helpers.
-Calling CUDA C++ device functions
+Calling CUDA C++ Device Functions
---------------------------------
A kernel can also reuse a CUDA C++ device function by including its source
@@ -137,7 +137,7 @@ otherwise:
@T.function
def relu_cuda(A: T.Tensor((256,), "float32"), B: T.Tensor((256,),
"float32")):
- T.device_entry(launch=T.cuda.LaunchConfig(grid=(1,), block=(256,)))
+ T.device_entry(launch=T.cuda.LaunchConfig(grid=1, block=256))
tx = T.cuda.thread_idx("x")
B[tx] = T.cuda.func_call("my_relu", A[tx], SRC, ty="float32")
diff --git a/docs/cuda/layouts.rst b/docs/programming_guides/data_layout.rst
similarity index 92%
rename from docs/cuda/layouts.rst
rename to docs/programming_guides/data_layout.rst
index dbff65d6f..33118e362 100644
--- a/docs/cuda/layouts.rst
+++ b/docs/programming_guides/data_layout.rst
@@ -17,20 +17,20 @@
.. Adapted from Apache TVM:
https://tvm.apache.org/docs/_sources/tirx/layout.rst
-Axis Layouts for CUDA
-=====================
+Data Layout
+===========
-In this chapter, we will use axis layouts to place tensor elements across
-CUDA threads and memory. A layout maps a logical index to physical
-coordinates: for a local tensor, these can identify the lane that owns
-a value and its storage position within that thread.
+This guide shows how to use axis layouts to place tensor elements across CUDA
+threads and memory. A layout maps a logical index to physical coordinates: for
+a local tensor, these can identify the lane that owns a value and its storage
+position within that thread.
For the mapping model and its hardware motivation, see `Data Layout and
Its Notation
<https://mlc.ai/modern-gpu-programming-for-mlsys/chapter_data_layout/index.html>`_
in Modern GPU Programming for MLSys.
-Mapping elements to lanes
+Mapping Elements to Lanes
-------------------------
A ``TileLayout`` maps logical coordinates to named physical axes.
@@ -66,7 +66,7 @@ The `interactive layout demo
<https://mlc.ai/modern-gpu-programming-for-mlsys/_static/tirx-layout-demo/index.html>`_
lets you select logical elements and inspect their physical coordinates.
-Attaching a layout to a tensor
+Attaching a Layout to a Tensor
------------------------------
The ``layout`` argument attaches this placement to a tensor. The following
@@ -93,7 +93,7 @@ exposes those four local positions as ``lane_values``,
without allocating
new storage. Each thread reads row ``lane`` from ``A``, adds one through
this local view, and writes the values to the same row of ``B``.
-Replication and offsets
+Replication and Offsets
-----------------------
A layout can also describe copies of each logical value or shift its
@@ -117,7 +117,7 @@ For the full shard, replica, and offset model, see
The :class:`TileLayout API <tvm_tirx.tirx.TileLayout>` describes the
constructor and layout operations.
-Swizzled shared memory
+Swizzled Shared Memory
----------------------
Shared-memory access patterns can concentrate on the same memory banks.
@@ -156,7 +156,7 @@ section explains bank conflicts and the XOR mapping with
interactive
figures. The :class:`ComposeLayout API <tvm_tirx.tirx.ComposeLayout>` covers
its parameters.
-Instruction-specific layouts
+Instruction-Specific Layouts
----------------------------
Tensor Core instructions require particular operand layouts. The CUDA
@@ -166,8 +166,8 @@ backend provides :func:`mma_shared_layout
<tvm_tirx.backend.cuda.layout.tmem_mma_operand_layout>` for TMEM operands.
The :func:`tmem_datapath_layout API
<tvm_tirx.backend.cuda.layout.tmem_datapath_layout>` lists the datapaths,
-shapes, and sub-slab options. The :doc:`tile operation guides
-</cuda/tile_operations>` show how to prepare these operands and synchronize
+shapes, and sub-slab options. The :doc:`Tile Operations guides
+</programming_guides/tile/index>` show how to prepare these operands and
synchronize
their use.
For the hardware data paths behind these choices, see
diff --git a/docs/cuda/iket_profiling.rst
b/docs/programming_guides/iket_profiling.rst
similarity index 89%
rename from docs/cuda/iket_profiling.rst
rename to docs/programming_guides/iket_profiling.rst
index 5c11874b5..ebd1d1163 100644
--- a/docs/cuda/iket_profiling.rst
+++ b/docs/programming_guides/iket_profiling.rst
@@ -15,19 +15,19 @@
specific language governing permissions and limitations
under the License.
-IKET Profiling
-==============
+Profiling with IKET
+===================
-In this chapter, we will use NVIDIA IKET to trace the execution of a
-small kernel that computes ``B[i] = 2 * A[i] + 1``. We will place a named
-range around the computation and a checkpoint after the input read, then
-capture a timeline and inspect the recorded events.
+This guide shows how to use NVIDIA IKET to trace the execution of a small
+kernel that computes ``B[i] = 2 * A[i] + 1``. We will place a named range
+around the computation and a checkpoint after the input read, then capture a
+timeline and inspect the recorded events.
The example requires a CUDA-enabled TIRx build, an SM90-or-newer GPU,
NVRTC through ``cuda-bindings``, and the ``run-iket`` executable from
``nvidia-cutlass-dsl-libs-base``.
-Annotate a computation
+Annotate a Computation
----------------------
Save the following kernel and the host code below in
@@ -44,7 +44,7 @@ Save the following kernel and the host code below in
def profiled_affine(
A: T.Tensor((32,), T.float32), B: T.Tensor((32,), T.float32)
):
- with T.device_entry(launch=T.cuda.LaunchConfig(grid=(1,), block=(32,))):
+ with T.device_entry(launch=T.cuda.LaunchConfig(grid=1, block=32)):
profiler = IketProfiler()
tid = T.cuda.thread_idx("x")
profiler.range_push("affine")
@@ -59,7 +59,7 @@ places a checkpoint after the tensor read. These events will
appear in
the captured trace. They do not replace synchronization needed by the
kernel's data dependencies.
-Capture the kernel
+Capture the Kernel
------------------
The host workload compiles the instrumented kernel, runs it, and checks
@@ -92,7 +92,7 @@ After the capture completes, the parent process receives the
result and
prints the artifact paths. Use a fresh output directory for each capture,
or pass ``clobber=True`` to replace an earlier capture.
-Read the trace
+Read the Trace
--------------
``result.perfetto`` identifies the timeline file, ``result.html`` the
@@ -118,5 +118,5 @@ separately when assessing its overall performance.
The :doc:`IKET API </reference/python/backend.cuda.iket>` covers token-based
ranges, command-based capture with ``iket.profile``, and capture options.
-The :doc:`CUDA backend source guide <backend_implementation>` explains where
+The :doc:`CUDA Backend Development </dev/cuda/index>` explains where
the profiling annotations are lowered in the compilation pipeline.
diff --git a/docs/cuda/images/iket_affine_b200.png
b/docs/programming_guides/images/iket_affine_b200.png
similarity index 100%
rename from docs/cuda/images/iket_affine_b200.png
rename to docs/programming_guides/images/iket_affine_b200.png
diff --git a/docs/programming_guides/index.rst
b/docs/programming_guides/index.rst
new file mode 100644
index 000000000..5ffa3a647
--- /dev/null
+++ b/docs/programming_guides/index.rst
@@ -0,0 +1,144 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. Adapted from Apache TVM:
https://tvm.apache.org/docs/_sources/tirx/overview.rst
+
+Programming Overview
+====================
+
+This page provides a high-level overview of TIRx programming: the design
+philosophy behind it, the programming model, and the guides in this section.
+
+Design Philosophy
+-----------------
+
+Kernel DSLs are most effective when they choose the right boundary between the
+programmer and the machine. For mature kernels on mature hardware, that
+boundary can be high-level: the compiler hides thread assignment, memory
+movement, layout details, and instruction selection behind compact tensor or
+tile abstractions, and this works well for established kernel patterns.
+
+At the frontier, the same boundary is under more pressure. New instructions,
+memory spaces, cooperation patterns, and kernel algorithms often appear before
+a compiler has enough built-in machinery to automate them well. At the same
+time, kernel programming is shifting toward agents, which work best with
+direct, predictable behavior: code that maps closely to what the hardware
+does, rather than to decisions a compiler makes behind the scenes.
+
+TIRx chooses a lower and more explicit boundary. It is a generic language built
+for extension, and its core primitives stay direct and close to the native
+level of the hardware. The parts of a kernel that frequently require expert
+control, such as pipeline structure, synchronization, role assignment, memory
+placement, and backend intrinsics, are written directly in hardware-native
+source code.
+
+On top of these primitives, TIRx provides a predictable convenience layer for
+the recurring tile-level structure through three constructs: execution scope,
+tensor layout, and tile operation dispatch. The layer is optional, and any part
+of a kernel can drop down to the native primitives.
+
+The result is a language that can grow with the hardware. A new feature can
+first be used directly as a backend intrinsic, and later become a reusable tile
+operation once the pattern stabilizes. This is the core design philosophy
+behind TIRx: keep the foundation small and explicit, and let developers,
+agents, and the backend library extend it as new accelerator generations
+arrive.
+
+The Programming Model
+---------------------
+
+A TIRx program reads as a structured native kernel: loops, branches, tensors,
+synchronization, pipeline state, backend intrinsics, and hardware roles are
+written directly. Tile operations appear exactly where a repeated
+hardware-level operation should become reusable and dispatchable.
+
+Authoring Layers and IR
+~~~~~~~~~~~~~~~~~~~~~~~
+
+TIRx programs use one TVMScript namespace alias:
+
+.. code-block:: python
+
+ from tvm_tirx.script import tirx as T
+
+The namespace exposes three authoring layers. They share the same TIRx
+program, but enter the compiler at different levels:
+
+.. list-table::
+ :header-rows: 1
+ :widths: 25 30 45
+
+ * - Authoring layer
+ - Syntax
+ - Representation and lowering
+ * - Core language
+ - ``T.*``
+ - Creates TIRx statements, expressions, tensors, and control flow.
+ * - Backend operations
+ - ``T.cuda.*`` and ``T.ptx.*``
+ - Creates backend calls directly; these calls bypass tile operation
+ dispatch.
+ * - Tile operations
+ - ``T.cuda.tile.*``
+ - Creates a tile call that is replaced by a target-specific
+ implementation during tile operation dispatch.
+
+``tvm_tirx.script.tirx`` is the construction syntax for the TIRx IR, not a
+separate IR. Tile operation dispatch and layout and scope lowering replace
+high-level TIRx nodes with lower-level TIRx statements and calls:
+
+.. code-block:: text
+
+ T.* ───────────────────────────▶ TIRx statements and expressions ──┐
+ T.cuda.* / T.ptx.* ────────────▶ backend calls ────────────────────┤
+ T.cuda.tile.* ─▶ tile call ─▶ tile operation dispatch ─────────────┤
+ ▼
+ layout and scope lowering ─▶ codegen
+
+The model rests on three constructs:
+
+- **Execution scope** describes which threads take part in an operation and at
+ what level, such as a single thread, a warp, or a whole thread block.
+ Ordinary control flow selects the hardware roles that enter a region.
+- **Tensor layout** describes how a logical tensor maps to physical storage:
+ which memory it lives in and how its elements are spread across threads and
+ per-thread storage. Tile operation dispatch reads these declarations to
+ choose an implementation.
+- **Tile operation dispatch** selects an implementation for each tile
+ operation according to the operation, the execution scope, the operand
+ layouts, and the target, then generates the loops and addressing that apply
+ it across the whole tile.
+
+Across a full kernel, orchestration stays in ordinary source code, and the
+recurring hardware operations appear as tile operations.
+
+Guides in This Section
+----------------------
+
+- :doc:`language_basics` covers defining and launching kernels.
+- :doc:`control_flow` covers conditions, loops, and early thread exit.
+- :doc:`types_tensors_memory` covers element types, tensors, and memory
+ spaces.
+- :doc:`metaprogramming` covers kernels parameterized by captured values,
+ reusable code fragments, and compile-time specialization.
+- :doc:`cuda_ptx_interfaces` covers calling backend intrinsics, such as
+ synchronization and warp shuffles, PTX instructions, and CUDA C++ device
+ functions directly.
+- :doc:`data_layout` covers how tensor elements map to threads and memory.
+- :doc:`tile/index` covers tile operations and the instruction guides.
+- :doc:`compiling` covers compiler options and inspecting generated code.
+- :doc:`iket_profiling` covers profiling kernels with IKET.
diff --git a/docs/cuda/language/functions.rst
b/docs/programming_guides/language_basics.rst
similarity index 56%
rename from docs/cuda/language/functions.rst
rename to docs/programming_guides/language_basics.rst
index 435d5a8b7..87a03ddfa 100644
--- a/docs/cuda/language/functions.rst
+++ b/docs/programming_guides/language_basics.rst
@@ -20,15 +20,14 @@
https://tvm.apache.org/docs/_sources/tirx/native_basics/cuda/control_flow.rst
https://tvm.apache.org/docs/_sources/tirx/native_basics/cuda/parser_utils.rst
-Defining and launching kernels
-==============================
+Language Basics
+===============
-In this chapter, we will define CUDA functions, configure their launches,
-and assign work to threads. We will use the GPU add-one program from
-:doc:`Quick Start </get_started/quick_start>`, then extend it with a runtime
-parameter, a loop, and compile-time specialization.
+This guide shows how to define CUDA functions, configure their launches, and
+assign work to threads. We will use the GPU add-one program from :doc:`Quick
+Start </get_started/quick_start>`, then extend it with a runtime parameter.
-A CUDA function
+A CUDA Function
---------------
The ``@T.function`` decorator constructs a ``tirx.Function`` from a
@@ -44,7 +43,7 @@ symbolic length ``n``:
@T.function
def add_one(A: T.Tensor((n,), T.float32), B: T.Tensor((n,), T.float32)):
- T.device_entry(launch=T.cuda.LaunchConfig(grid=((n + 255) // 256,),
block=(256,)))
+ T.device_entry(launch=T.cuda.LaunchConfig(grid=T.ceildiv(n, 256),
block=256))
i = T.cuda.block_idx("x") * 256 + T.cuda.thread_idx("x")
if i < n:
B[i] = A[i] + 1
@@ -88,7 +87,7 @@ The ``increment: T.float32`` parameter controls that amount:
B: T.Tensor((n,), T.float32),
increment: T.float32,
):
- T.device_entry(launch=T.cuda.LaunchConfig(grid=((n + 255) // 256,),
block=(256,)))
+ T.device_entry(launch=T.cuda.LaunchConfig(grid=T.ceildiv(n, 256),
block=256))
i = T.cuda.block_idx("x") * 256 + T.cuda.thread_idx("x")
if i < n:
B[i] = A[i] + increment
@@ -111,7 +110,7 @@ from the tensor shapes, so the call needs no separate
length argument.
.. _cuda-symbolic-shapes:
-Symbolic shapes
+Symbolic Shapes
---------------
A kernel may need to handle tensors of different sizes. A symbolic shape
@@ -142,7 +141,7 @@ the launch's block count and the ``i < n`` bounds check in
``add_one``.
The caller passes only ``a`` and ``b``; the compiled function derives
``n`` from their shapes without recompilation.
-Launch configuration
+Launch Configuration
--------------------
Each call to ``add_one`` needs enough GPU threads for its current length
@@ -156,13 +155,13 @@ statements in ``add_one`` are:
.. code-block:: python
- T.device_entry(launch=T.cuda.LaunchConfig(grid=((n + 255) // 256,),
block=(256,)))
+ T.device_entry(launch=T.cuda.LaunchConfig(grid=T.ceildiv(n, 256),
block=256))
i = T.cuda.block_idx("x") * 256 + T.cuda.thread_idx("x")
if i < n:
B[i] = A[i] + 1
-``block=(256,)`` gives each block 256 threads. The grid rounds ``n / 256``
-up to provide at least ``n`` threads. For an input of 300 elements, this
+``block=256`` gives each block 256 threads. ``T.ceildiv(n, 256)`` rounds
+``n / 256`` up to provide at least ``n`` threads. For an input of 300
elements, this
launches two blocks, or 512 threads in total.
``T.cuda.block_idx("x")`` gives the block index.
@@ -183,120 +182,3 @@ bounds. The :doc:`CUDA launch API
</reference/python/backend.cuda.launch>`
covers these options; the
:doc:`CUDA script API </reference/python/backend.cuda.script>` lists the
coordinate helpers for blocks, threads, warps, and lanes.
-
-.. _cuda-kernel-control-flow:
-
-Conditions and loops
---------------------
-
-A thread can process several elements with a loop. This version assigns
-four consecutive elements to each thread, using 64 threads per block:
-
-.. code-block:: python
-
- @T.function
- def add_one_four(A: T.Tensor((n,), T.float32), B: T.Tensor((n,),
T.float32)):
- T.device_entry(launch=T.cuda.LaunchConfig(grid=((n + 255) // 256,),
block=(64,)))
- base = T.cuda.block_idx("x") * 256 + T.cuda.thread_idx("x") * 4
- for offset in T.unroll(4):
- i = base + offset
- if i < n:
- B[i] = A[i] + 1
-
-The launch creates 64 threads per block, and the loop assigns four
-consecutive elements to each thread, covering up to ``64 * 4 = 256``
-elements. A regular ``range(4)`` would process the four offsets
-sequentially within each thread. The example uses ``T.unroll(4)`` to ask
-the compiler to expand the loop into four copies of its body, one for
-each offset.
-
-The ``if i < n`` condition checks each element individually. A thread
-whose four-element group crosses the end of the tensor therefore writes
-only the valid positions.
-
-The :doc:`TIRx script API </reference/python/tirx.script>` also describes
-loop forms for vectorized operations (``T.vectorized``) and
-multidimensional iteration (``T.grid``).
-
-Compile-time specialization
----------------------------
-
-A symbolic dimension varies at runtime. When a size should be fixed before
-compilation, ``@T.jit`` lets us declare it as ``T.constexpr`` and set it
-through ``.specialize(...)``. Parsing happens at specialization time, and
-the result is an ordinary ``tirx.Function``.
-
-The following definition specializes the add-one function for a fixed
-length. Put ``from __future__ import annotations`` at the beginning of the
-Python file so the annotation can refer to ``N`` before specialization:
-
-.. code-block:: python
-
- from __future__ import annotations
- from tvm_tirx.script import tirx as T
-
- @T.jit
- def add_one_fixed(
- A: T.Tensor((N,), T.float32),
- B: T.Tensor((N,), T.float32),
- *,
- N: T.constexpr,
- ):
- T.device_entry(launch=T.cuda.LaunchConfig(grid=((N + 255) // 256,),
block=(256,)))
- i = T.cuda.block_idx("x") * 256 + T.cuda.thread_idx("x")
- if i < N:
- B[i] = A[i] + 1
-
- kernel = add_one_fixed.specialize(N=256)
-
-``kernel`` has length 256 fixed in its parameter types and body. It can be
-compiled with ``tvm_tirx.compile`` and called with its two tensor arguments;
-``N`` was fixed during specialization. In contrast, ``@T.function`` parses
-immediately, and the symbolic ``n`` in ``add_one`` remains a runtime
dimension. The
-:doc:`JIT API </reference/python/tirx.script.jit>` covers optional
-parameters and specialization rules.
-
-.. _cuda-inline-functions:
-
-Reusable code fragments
------------------------
-
-A kernel can use small functions to reuse common pieces of code. For
-example, the bounds check and store in ``add_one_four`` handle one element:
-
-.. code-block:: python
-
- if i < n:
- B[i] = A[i] + 1
-
-We can put these two statements in a function marked with ``@T.inline``:
-
-.. code-block:: python
-
- @T.inline
- def add_one_if_valid(A, B, i, n):
- if i < n:
- B[i] = A[i] + 1
-
-Each loop iteration in ``add_one_four`` can then call this function to
-process its element:
-
-.. code-block:: python
-
- for offset in T.unroll(4):
- i = base + offset
- add_one_if_valid(A, B, i, n)
-
-During parsing, TVMScript expands ``add_one_if_valid(...)`` into its two
-statements. The loop above is therefore equivalent to:
-
-.. code-block:: python
-
- for offset in T.unroll(4):
- i = base + offset
- if i < n:
- B[i] = A[i] + 1
-
-Other kernels can call ``add_one_if_valid`` in the same way to reuse the
-update. The GPU executes the expanded checks and stores directly. The
-:func:`inline API <tvm_tirx.tirx.script.inline>` describes its options.
diff --git a/docs/programming_guides/metaprogramming.rst
b/docs/programming_guides/metaprogramming.rst
new file mode 100644
index 000000000..748a38fd9
--- /dev/null
+++ b/docs/programming_guides/metaprogramming.rst
@@ -0,0 +1,115 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Metaprogramming
+===============
+
+.. note::
+
+ All the code in this guide is in
+ `examples/programming_guides/metaprogramming.py
+
<https://github.com/apache/tvm-tirx/blob/main/examples/programming_guides/metaprogramming.py>`_.
+
+This guide shows how to use Python to generate, reuse, and specialize TIRx
+kernels. TIRx functions are written in Python, so ordinary Python code runs
+before compilation and decides what each kernel contains: which dtypes and
+sizes it uses, which helper code it includes, and which values are fixed.
+
+The examples use the following imports:
+
+.. literalinclude:: ../../examples/programming_guides/metaprogramming.py
+ :language: python
+ :start-after: # [imports.begin]
+ :end-at: from tvm_tirx.script import tirx as T
+
+Parameterizing Kernels with Captured Values
+-------------------------------------------
+
+A Python function can build kernels. The inner ``@T.function`` can read any
+value from the enclosing Python scope, such as a shape variable, a dtype, a
+constant, or a launch size. Each call to the outer function produces a new
+kernel with those values filled in:
+
+.. literalinclude:: ../../examples/programming_guides/metaprogramming.py
+ :language: python
+ :start-after: # [generator.begin]
+ :end-before: # [generator.end]
+
+The parser reads ``n``, ``dtype``, ``value``, and ``block`` from the enclosing
+scope when it parses ``add``. In ``add_two_f16``, the tensors have element type
+``T.float16``, the constant becomes ``T.float16(2.0)``, and each block has 128
+threads. ``n`` stays a symbolic dimension, so the kernel still accepts tensors
+of any length. The script prints:
+
+.. code-block:: text
+
+ tensor([ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11.], device='cuda:0',
+ dtype=torch.float16)
+
+Calling ``make_add(T.float32, 1)`` instead produces a separate float32 kernel
+from the same definition. The parser needs the function's source code, so
+define kernel generators in a Python file rather than in an interactive
+session that has no source to read.
+
+.. _cuda-inline-functions:
+
+Reusable Code Fragments
+-----------------------
+
+Kernels often repeat the same few statements. A function marked with
+``@T.inline`` holds such a fragment, and a kernel calls it like a regular
+function. In the following kernel, each thread handles two neighboring
+elements and calls ``add_one_if_valid`` once for each:
+
+.. literalinclude:: ../../examples/programming_guides/metaprogramming.py
+ :language: python
+ :start-after: # [inline.begin]
+ :end-before: # [inline.end]
+
+During parsing, TVMScript expands each ``add_one_if_valid(...)`` call into the
+helper's statements, with the arguments substituted. The body of
+``add_one_pairs`` is therefore equivalent to:
+
+.. code-block:: python
+
+ i = (T.cuda.block_idx("x") * 256 + T.cuda.thread_idx("x")) * 2
+ if i < n:
+ B[i] = A[i] + 1
+ if i + 1 < n:
+ B[i + 1] = A[i + 1] + 1
+
+No function call remains in the compiled kernel, and other kernels can call
+``add_one_if_valid`` in the same way.
+
+Compile-Time Specialization
+---------------------------
+
+A symbolic dimension such as ``n`` varies at runtime. When a size should
+instead be fixed before compilation, ``@T.jit`` lets us declare it as a
+``T.constexpr`` parameter and set it later through ``.specialize(...)``:
+
+.. literalinclude:: ../../examples/programming_guides/metaprogramming.py
+ :language: python
+ :start-after: # [jit.begin]
+ :end-before: # [jit.end]
+
+Unlike ``@T.function``, which parses its definition immediately, ``@T.jit``
+waits until ``.specialize`` supplies the constexpr values. ``add_one_256`` is
+an ordinary TIRx function with 256 fixed in its parameter types, launch
+configuration, and bounds check, and it compiles with ``tvm_tirx.compile`` like
+any other function. The annotations refer to ``N`` before it has a value, which
+is why the file starts with ``from __future__ import annotations``.
diff --git a/docs/cuda/tile_ops/arithmetic.rst
b/docs/programming_guides/tile/arithmetic.rst
similarity index 96%
rename from docs/cuda/tile_ops/arithmetic.rst
rename to docs/programming_guides/tile/arithmetic.rst
index 4e0532b6e..0c522c187 100644
--- a/docs/cuda/tile_ops/arithmetic.rst
+++ b/docs/programming_guides/tile/arithmetic.rst
@@ -28,13 +28,13 @@
Arithmetic and Type Conversion
==============================
-In this chapter, we will compute with the register values loaded by tile
+This guide shows how to compute with the register values loaded by tile
operations. ``apply`` selects a native arithmetic or conversion instruction;
-``ewise`` applies a scalar TIRx function to each value. We will use an
-affine transformation followed by ReLU, then convert float32 values to
-float16 and express a SiLU activation as a scalar function.
+``ewise`` applies a scalar TIRx function to each value. We will use an affine
+transformation followed by ReLU, then convert float32 values to float16 and
+express a SiLU activation as a scalar function.
-An affine transformation and ReLU
+An Affine Transformation and ReLU
---------------------------------
A common activation step scales each value, adds a bias, and clamps the
@@ -119,7 +119,7 @@ approximation, and subnormal handling. Packed forms such as
the :ref:`register instruction reference <cuda-tile-contract-registers>`
specifies those representations and the predication options.
-Numeric conversion
+Numeric Conversion
------------------
The same flow can produce float16 output. ``cvt.rn.f16.f32`` converts one
@@ -160,7 +160,7 @@ Packed conversions such as ``cvt.rn.f16x2.f32`` place two
converted values
in one uint32 carrier. Their operand order and supported inverse forms
are specified in the :ref:`conversion reference <cuda-tile-contract-cvt>`.
-Scalar function compositions
+Scalar Function Compositions
----------------------------
A scalar TIRx function can describe a longer computation. For example,
diff --git a/docs/cuda/tile_ops/async_copies.rst
b/docs/programming_guides/tile/async_copies.rst
similarity index 95%
rename from docs/cuda/tile_ops/async_copies.rst
rename to docs/programming_guides/tile/async_copies.rst
index 3bc5c8417..c34068a2e 100644
--- a/docs/cuda/tile_ops/async_copies.rst
+++ b/docs/programming_guides/tile/async_copies.rst
@@ -22,12 +22,12 @@
Asynchronous Copies
===================
-In this chapter, we will copy data asynchronously and wait before using
-it. A kernel can issue a transfer, perform independent work, then wait for
-completion. The copy instruction and its completion calls are separate
-parts of that sequence.
+This guide shows how to copy data asynchronously and wait before using it. A
+kernel can issue a transfer, perform independent work, then wait for
+completion. The copy instruction and its completion calls are separate parts of
+that sequence.
-A global-to-shared copy
+A Global-to-Shared Copy
-----------------------
``cp.async`` copies a short contiguous chunk directly from global to
diff --git a/docs/cuda/tile_ops/blackwell_tensor_cores.rst
b/docs/programming_guides/tile/blackwell_tensor_cores.rst
similarity index 95%
rename from docs/cuda/tile_ops/blackwell_tensor_cores.rst
rename to docs/programming_guides/tile/blackwell_tensor_cores.rst
index df2201202..1bf3f46c4 100644
--- a/docs/cuda/tile_ops/blackwell_tensor_cores.rst
+++ b/docs/programming_guides/tile/blackwell_tensor_cores.rst
@@ -15,16 +15,16 @@
specific language governing permissions and limitations
under the License.
-Blackwell Tensor Core Operations
-================================
+Blackwell Tensor Cores
+======================
-In this chapter, we will multiply float16 matrices with ``tcgen05.mma``
-and retrieve its float32 result from Tensor Memory (TMEM). The example
-targets ``sm_100a`` and requires a GPU with that instruction support,
-such as a B200. TMEM holds the accumulator while the Tensor Core reads
-the input tiles from shared memory.
+This guide shows how to multiply float16 matrices with ``tcgen05.mma`` and
+retrieve its float32 result from Tensor Memory (TMEM). The example targets
+``sm_100a`` and requires a GPU with that instruction support, such as a B200.
+TMEM holds the accumulator while the Tensor Core reads the input tiles from
+shared memory.
-The accumulator's lifetime
+The Accumulator's Lifetime
--------------------------
The example computes ``Out = A @ B.T`` with A of shape ``(128, 64)`` and
@@ -175,7 +175,7 @@ On a supported GPU, we can check the product as follows:
compiled(a, b, out)
torch.testing.assert_close(out, a.float() @ b.float().T, rtol=2e-4,
atol=2e-4)
-Other TMEM transfers
+Other TMEM Transfers
--------------------
The accumulator above needs no initial fill because the first MMA ignores
diff --git a/docs/cuda/tile_ops/data_movement.rst
b/docs/programming_guides/tile/data_movement.rst
similarity index 93%
rename from docs/cuda/tile_ops/data_movement.rst
rename to docs/programming_guides/tile/data_movement.rst
index 8d6d2b4c9..be04efe88 100644
--- a/docs/cuda/tile_ops/data_movement.rst
+++ b/docs/programming_guides/tile/data_movement.rst
@@ -22,10 +22,10 @@
Data Movement
=============
-In this chapter, we will move tensor data between global memory, shared
-memory, and thread-local registers. The instruction determines the source
-and destination spaces, while tensor views select the values to transfer.
-All three instruction families take the destination first:
+This guide shows how to move tensor data between global memory, shared memory,
+and thread-local registers. The instruction determines the source and
+destination spaces, while tensor views select the values to transfer. All three
+instruction families take the destination first:
.. list-table::
:header-rows: 1
@@ -44,7 +44,7 @@ All three instruction families take the destination first:
- ``destination, source``
- Register values within the same thread
-A shared-memory exchange
+A Shared-Memory Exchange
------------------------
A block can use shared memory to exchange data between threads. In this
@@ -100,7 +100,7 @@ The two global/shared transfers pass through the caller's
register tensor.
For a direct asynchronous global-to-shared transfer, see :doc:`async_copies`,
which includes the required completion calls.
-Register copies and initialization
+Register Copies and Initialization
----------------------------------
Inside this kernel, we can save loaded values before reusing ``registers``
diff --git a/docs/cuda/tile_operations.rst
b/docs/programming_guides/tile/index.rst
similarity index 73%
rename from docs/cuda/tile_operations.rst
rename to docs/programming_guides/tile/index.rst
index 60ae5d75e..f70ea0ef0 100644
--- a/docs/cuda/tile_operations.rst
+++ b/docs/programming_guides/tile/index.rst
@@ -17,21 +17,21 @@
.. Adapted from Apache TVM:
https://tvm.apache.org/docs/_sources/tirx/tile_ops.rst
-CUDA Tile Operations
-====================
+Tile Operations
+===============
-In this chapter, we will use CUDA tile operations to apply a computation
-to tensor regions. ``T.cuda.tile.apply`` repeats a specified PTX instruction,
-such as a vector load or matrix multiplication. ``T.cuda.tile.ewise``
-applies a scalar TIRx function to register elements. You choose the operand
-views, instruction variants, and synchronization required by the kernel.
+This guide shows how to use CUDA tile operations to apply a computation to
+tensor regions. ``T.cuda.tile.apply`` repeats a specified PTX instruction, such
+as a vector load or matrix multiplication. ``T.cuda.tile.ewise`` applies a
+scalar TIRx function to register elements. You choose the operand views,
+instruction variants, and synchronization required by the kernel.
Start with :doc:`tile_patterns` for a load, compute, and store example and
an explanation of how operand axes divide the work across threads. The
:doc:`instruction reference </reference/python/backend.cuda.tile.instructions>`
lists each family's parameters, operand layouts, and alignment requirements.
-CUDA instruction guides
+CUDA Instruction Guides
-----------------------
The guides below show how to use these interfaces for data movement,
@@ -42,8 +42,8 @@ examples include the completion protocol required by their
instruction family.
:maxdepth: 1
tile_patterns
- tile_ops/data_movement
- tile_ops/arithmetic
+ data_movement
+ arithmetic
Asynchronous Transfers
~~~~~~~~~~~~~~~~~~~~~~
@@ -51,8 +51,8 @@ Asynchronous Transfers
.. toctree::
:maxdepth: 1
- tile_ops/async_copies
- tile_ops/tma
+ async_copies
+ tma
Matrix Instructions
~~~~~~~~~~~~~~~~~~~
@@ -60,5 +60,5 @@ Matrix Instructions
.. toctree::
:maxdepth: 1
- tile_ops/warp_matrix
- tile_ops/blackwell_tensor_cores
+ warp_matrix
+ blackwell_tensor_cores
diff --git a/docs/cuda/tile_patterns.rst
b/docs/programming_guides/tile/tile_patterns.rst
similarity index 93%
rename from docs/cuda/tile_patterns.rst
rename to docs/programming_guides/tile/tile_patterns.rst
index fc9a5337f..008f6ad31 100644
--- a/docs/cuda/tile_patterns.rst
+++ b/docs/programming_guides/tile/tile_patterns.rst
@@ -17,7 +17,7 @@
.. Adapted from Apache TVM:
https://tvm.apache.org/docs/_sources/tirx/cuda_tile_patterns.rst
-Tile operands and execution
+Tile Operands and Execution
===========================
A CUDA tile operation repeats a chosen computation over a tensor region.
@@ -26,7 +26,7 @@ A CUDA tile operation repeats a chosen computation over a
tensor region.
operand views describe which values each thread handles and how many times
it repeats that computation.
-Reading an operand's shape
+Reading an Operand's Shape
--------------------------
Suppose each of 64 threads processes twelve float32 values. A
@@ -58,7 +58,7 @@ repetitions and the threads that execute them. The register
layout below
maps the first two coordinates to ``warpid`` and ``laneid``, leaving
``3 * 4 = 12`` local positions per thread.
-A load, compute, and store example
+A Load, Compute, and Store Example
----------------------------------
The kernel uses this layout to load the values, add one to each, and
@@ -106,7 +106,7 @@ We can compile and check the result:
compiled(a, b)
torch.testing.assert_close(b, a + 1)
-Execution scope and scalar functions
+Execution Scope and Scalar Functions
------------------------------------
The kernel's ``LaunchConfig`` launches 64 threads; ``scope="cta"``
@@ -120,7 +120,7 @@ the group shapes listed in the :ref:`coordinate reference
instruction atom. For the same register view, ``(2, 32)`` still selects a
thread, while ``(3, 4)`` now specifies twelve scalar invocations.
-Preparing operand views
+Preparing Operand Views
-----------------------
The load's source and destination expose the same group and repeat
@@ -133,8 +133,8 @@ An existing tensor may need a ``view`` or ``rearrange`` to
expose these axes,
or a ``sub`` to select a region. These operations reuse its storage. They
can change the coordinates used to access values, but a transfer between
threads or memory spaces still requires an explicit load, store, or other
-communication instruction. See :doc:`language/tensors` for tensor views and
-:doc:`layouts` for their physical placement.
+communication instruction. See :doc:`/programming_guides/types_tensors_memory`
for tensor views and
+:doc:`/programming_guides/data_layout` for their physical placement.
The dtype describes storage elements. In the example, one element is one
32-bit word. An instruction that packs two float16 values into one 32-bit
@@ -143,7 +143,7 @@ pack values automatically. The :ref:`instruction reference
<cuda-tile-instruction-contracts>` lists the required operand order,
representation, alignment, and optional parameters for each family.
-Synchronization belongs to the caller
+Synchronization Belongs to the Caller
-------------------------------------
The example uses thread-private registers, so no thread waits for another.
@@ -154,6 +154,6 @@ that synchronization.
Asynchronous instructions also require their own completion protocol.
``cp.async`` uses copy groups and waits; TMA loads use transaction barriers;
``tcgen05`` operations have their own commit and wait instructions. The
-corresponding :doc:`instruction guides <tile_operations>` show these calls
+corresponding :doc:`instruction guides <index>` show these calls
alongside the data movement. Selecting an execution scope does not perform
thread election or complete an asynchronous transfer.
diff --git a/docs/cuda/tile_ops/tma.rst b/docs/programming_guides/tile/tma.rst
similarity index 93%
rename from docs/cuda/tile_ops/tma.rst
rename to docs/programming_guides/tile/tma.rst
index 8d871562b..7e18e5afa 100644
--- a/docs/cuda/tile_ops/tma.rst
+++ b/docs/programming_guides/tile/tma.rst
@@ -21,18 +21,17 @@
TMA Transfers
=============
-In this chapter, we will use the Tensor Memory Accelerator (TMA) to move
-multidimensional tiles between global and shared memory. One thread issues
-the transfer using a TensorMap descriptor, which records the global
-shape, strides, element format, and tile configuration. TIRx prepares this
-descriptor in the host launcher from the tensor operand and explicit
-``box`` parameters.
+This guide shows how to use the Tensor Memory Accelerator (TMA) to move
+multidimensional tiles between global and shared memory. One thread issues the
+transfer using a TensorMap descriptor, which records the global shape, strides,
+element format, and tile configuration. TIRx prepares this descriptor in the
+host launcher from the tensor operand and explicit ``box`` parameters.
TMA requires a Hopper-or-later GPU. The `Async Data Movement: TMA
<https://mlc.ai/modern-gpu-programming-for-mlsys/chapter_tma/index.html>`_
chapter explains its hardware role and layout support in detail.
-A tile load, computation, and store
+A Tile Load, Computation, and Store
-----------------------------------
The following kernel loads an ``(8, 32)`` float32 tile, adds one to each
@@ -82,7 +81,7 @@ The shared tensor has the same tile shape.
``layout="default"`` gives it
contiguous row-major storage, and ``align=128`` aligns its base. The load
takes ``shared, A``; the store reverses the direction with ``B, shared``.
-Load completion
+Load Completion
---------------
The load reports completion through ``mbar``, an eight-byte barrier object
@@ -99,7 +98,7 @@ expected arrivals and transaction bytes have been satisfied.
The
<https://mlc.ai/modern-gpu-programming-for-mlsys/chapter_async_barriers/index.html>`_
explains phases and reuse across pipeline stages.
-Store completion
+Store Completion
----------------
After the updates, each thread executes the proxy fence to make its
@@ -113,7 +112,7 @@ A TMA store uses a bulk async group. Lane 0 commits its
store with
that wait finishes, allowing the shared buffer to be reused. These bulk
groups are distinct from the ordinary copy groups used by :doc:`async_copies`.
-Compile and run
+Compile and Run
---------------
The host launcher needs tensor-map encoding support, available with an
diff --git a/docs/cuda/tile_ops/warp_matrix.rst
b/docs/programming_guides/tile/warp_matrix.rst
similarity index 94%
rename from docs/cuda/tile_ops/warp_matrix.rst
rename to docs/programming_guides/tile/warp_matrix.rst
index 85775632a..df164ab6c 100644
--- a/docs/cuda/tile_ops/warp_matrix.rst
+++ b/docs/programming_guides/tile/warp_matrix.rst
@@ -18,13 +18,13 @@
Warp Matrix Instructions
========================
-In this chapter, we will multiply two float16 matrices with a warp of
-32 threads. ``ldmatrix`` loads matrix fragments from shared memory into
-registers, and ``mma.sync`` multiplies those fragments with float32
-accumulation. The example computes ``Out = A @ B.T`` for A of shape
-``(16, 16)`` and B of shape ``(8, 16)`` on an SM80-or-later GPU.
+This guide shows how to multiply two float16 matrices with a warp of 32
+threads. ``ldmatrix`` loads matrix fragments from shared memory into registers,
+and ``mma.sync`` multiplies those fragments with float32 accumulation. The
+example computes ``Out = A @ B.T`` for A of shape ``(16, 16)`` and B of shape
+``(8, 16)`` on an SM80-or-later GPU.
-From shared memory to a matrix product
+From Shared Memory to a Matrix Product
--------------------------------------
All 32 lanes cooperate on one ``16 x 8`` output tile. The data follows
@@ -158,7 +158,7 @@ product:
compiled(a, b, out)
torch.testing.assert_close(out, a.float() @ b.float().T, rtol=2e-4,
atol=2e-4)
-Other fragment shapes and transfers
+Other Fragment Shapes and Transfers
-----------------------------------
A longer K dimension needs multiple MMA issues. The tile interface
diff --git a/docs/cuda/language/tensors.rst
b/docs/programming_guides/types_tensors_memory.rst
similarity index 92%
rename from docs/cuda/language/tensors.rst
rename to docs/programming_guides/types_tensors_memory.rst
index 28ac93169..39753f6fb 100644
--- a/docs/cuda/language/tensors.rst
+++ b/docs/programming_guides/types_tensors_memory.rst
@@ -19,15 +19,15 @@
https://tvm.apache.org/docs/_sources/tirx/native_basics/cuda/data_types.rst
https://tvm.apache.org/docs/_sources/tirx/native_basics/cuda/tensors.rst
-Types, tensors and memory
-=========================
+Types, Tensors, and Memory
+==========================
-In this chapter, we will look at how CUDA kernels represent values and manage
-their storage. A tensor's dtype describes its elements, and its scope selects
-the memory space. We will extend the add-one computation with vector accesses,
-shared storage, and thread-private values, then create views of existing data.
+This guide looks at how CUDA kernels represent values and manage their storage.
+A tensor's dtype describes its elements, and its scope selects the memory
+space. We will extend the add-one computation with vector accesses, shared
+storage, and thread-private values, then create views of existing data.
-Element types and vector accesses
+Element Types and Vector Accesses
---------------------------------
Tensor elements and intermediate values can be integers, floating-point
@@ -60,8 +60,7 @@ CUDA C++ names this type ``float4``: each ``float`` is 32
bits, and ``4``
is the number of components.
We can extend the add-one kernel with vector accesses so each thread
-reads and updates four elements as a group. The thread assignment matches
-:ref:`add_one_four in Conditions and loops <cuda-kernel-control-flow>`:
+reads and updates four elements as a group:
.. code-block:: python
@@ -91,7 +90,7 @@ elements. The ``align=16`` annotations declare
16-byte-aligned pointers;
each four-float group also starts at a 16-byte boundary, allowing aligned
vector loads and stores.
-Declaring tensors
+Declaring Tensors
-----------------
A parameter such as ``A: T.Tensor((128,), "float32")`` describes a tensor
@@ -137,13 +136,13 @@ to the same storage, so a write through ``shared_view``
is also visible through
The tensor's ``layout`` determines how logical indices map to storage;
``align`` specifies data-pointer alignment in bytes. The
-:doc:`Tensor Layout </cuda/layouts>` guide covers explicit layouts, and the
+:doc:`Data Layout </programming_guides/data_layout>` guide covers explicit
layouts, and the
:class:`TensorType API <tvm_tirx.tirx.TensorType>` lists the full parameters.
-Shared memory
+Shared Memory
-------------
-Static shared memory
+Static Shared Memory
~~~~~~~~~~~~~~~~~~~~
``T.alloc_shared`` creates a shared tensor with a size known at compile
@@ -167,7 +166,7 @@ reached it, making those writes visible before the
neighboring reads.
For example, thread 0 writes ``A[0]`` but reads the value that thread 1
stored from ``A[1]``.
-Dynamic shared memory
+Dynamic Shared Memory
~~~~~~~~~~~~~~~~~~~~~
With dynamic shared memory, the kernel launch specifies how many bytes to
@@ -207,7 +206,7 @@ region. ``pool.commit()`` should follow all allocations. The
:class:`SMEMPool API <tvm_tirx.backend.cuda.lang.SMEMPool>` covers alignment
and storage reuse.
-Per-thread local storage
+Per-Thread Local Storage
------------------------
``T.alloc_local`` creates storage private to each thread. For example,
@@ -230,7 +229,7 @@ Every thread has its own ``values`` array. The CUDA
compiler may place its
elements in registers when the accesses permit; local scope alone does not
guarantee register storage.
-Mutable scalars and ``let``
+Mutable Scalars and ``let``
~~~~~~~~~~~~~~~~~~~~~~~~~~~
In TVMScript, an ordinary assignment such as ``base = tx * 4`` creates an
@@ -278,7 +277,7 @@ has its own accumulator. The
``local_scalar``, ``shared_scalar``, and ``alloc_scalar`` for explicit
scalar allocation.
-Tensor views and pointers
+Tensor Views and Pointers
-------------------------
Tensor variables provide methods for accessing addresses and creating
@@ -333,7 +332,7 @@ the coordinates; the tensor's layout determines their
physical placement.
``sub`` selects a region to pass to an operation. For example,
``matrix.sub[i, :]`` selects its four-element row ``i``. A tile load can
-then read that region into registers, as shown in
:doc:`/cuda/tile_ops/data_movement`.
+then read that region into registers, as shown in
:doc:`/programming_guides/tile/data_movement`.
A dtype view changes the storage units used to access the bytes. Inside
a device region, eight float16 elements can be viewed as four uint32 words:
@@ -346,20 +345,20 @@ a device region, eight float16 elements can be viewed as
four uint32 words:
``packed_words`` has shape ``(4,)``. Each word contains the bits of two
adjacent float16 elements. This is the representation used by instructions
that accept packed half values; a numeric conversion uses ``cvt`` instead,
-as shown in :doc:`/cuda/tile_ops/arithmetic`.
+as shown in :doc:`/programming_guides/tile/arithmetic`.
All these views and regions reuse the original storage. The
:class:`TensorType API <tvm_tirx.tirx.TensorType>` covers the available
-methods and options, while :doc:`/cuda/layouts` explains explicit storage
+methods and options, while :doc:`/programming_guides/data_layout` explains
explicit storage
mappings, including layouts distributed across threads.
-Tensor memory
+Tensor Memory
-------------
Blackwell tensor memory uses ``scope="tmem"`` and must be reserved and
released explicitly. ``T.cuda.decl_tmem`` declares a tensor at an allocated
tensor-memory address; access uses ``tcgen05`` instructions. The
-:doc:`Blackwell Tensor Core guide </cuda/tile_ops/blackwell_tensor_cores>`
shows the
+:doc:`Blackwell Tensor Cores guide
</programming_guides/tile/blackwell_tensor_cores>` shows the
allocation, synchronization, transfer, and release steps together. The
:func:`decl_tmem API <tvm_tirx.backend.cuda.script.decl_tmem>` describes its
arguments.
diff --git a/docs/reference/python/backend.cuda.iket.rst
b/docs/reference/python/backend.cuda.iket.rst
index c1285c60a..7bbc226fe 100644
--- a/docs/reference/python/backend.cuda.iket.rst
+++ b/docs/reference/python/backend.cuda.iket.rst
@@ -79,5 +79,5 @@ The function signatures list timeout and capture-limit
options.
``IketProfileResult.trace_json``, ``perfetto``, and ``html`` return the
single artifact of each requested format. For captures producing multiple
artifacts, use ``json_traces``, ``perfetto_traces``, and ``html_reports``
-instead. The :doc:`IKET Profiling guide </cuda/iket_profiling>` contains a
+instead. The :doc:`Profiling with IKET guide
</programming_guides/iket_profiling>` contains a
complete annotation and capture example.
diff --git a/docs/reference/python/backend.cuda.ptx.rst
b/docs/reference/python/backend.cuda.ptx.rst
index cd0f5aaec..410442bbb 100644
--- a/docs/reference/python/backend.cuda.ptx.rst
+++ b/docs/reference/python/backend.cuda.ptx.rst
@@ -106,5 +106,5 @@ available:
``gen_stubs`` generates signatures for editor tooling, and ``gen_coverage``
reports instruction families and variants. These files support inspection;
runtime code generation reads the instruction table directly. Each command's
-``--help`` describes its available options. The :doc:`PTX Tablegen guide
-</cuda/ptx_tablegen>` follows an instruction through the implementation.
+``--help`` describes its available options. The :doc:`PTX TableGen guide
+</dev/cuda/ptx_tablegen>` follows an instruction through the implementation.
diff --git a/docs/reference/python/backend.cuda.script.tile.rst
b/docs/reference/python/backend.cuda.script.tile.rst
index 6136b6367..aff22a1c7 100644
--- a/docs/reference/python/backend.cuda.script.tile.rst
+++ b/docs/reference/python/backend.cuda.script.tile.rst
@@ -25,7 +25,7 @@ tvm_tirx.backend.cuda.script.tile
.. currentmodule:: tvm_tirx.backend.cuda.script.tile
These constructors build tile operations in TVMScript. The
-:doc:`operand guide </cuda/tile_patterns>` introduces the group, repeat,
+:doc:`Tile Operands and Execution guide
</programming_guides/tile/tile_patterns>` introduces the group, repeat,
and atom coordinates. The :doc:`instruction reference
<backend.cuda.tile.instructions>` specifies the supported families and
their parameters, operand layouts, alignment, and completion requirements.
diff --git a/docs/reference/python/backend.cuda.tile.instructions.rst
b/docs/reference/python/backend.cuda.tile.instructions.rst
index 61a928c63..987132e9a 100644
--- a/docs/reference/python/backend.cuda.tile.instructions.rst
+++ b/docs/reference/python/backend.cuda.tile.instructions.rst
@@ -24,8 +24,8 @@ CUDA tile instruction reference
This reference specifies the operands, parameters, storage requirements, and
lowering behavior of CUDA tile instructions. For a worked introduction to
-operand views, see :doc:`/cuda/tile_patterns`. The :doc:`instruction guides
-</cuda/tile_operations>` provide complete kernels and completion protocols.
+operand views, see :doc:`/programming_guides/tile/tile_patterns`. The
:doc:`instruction guides
+</programming_guides/tile/index>` provide complete kernels and completion
protocols.
.. contents:: Instruction families
:local:
@@ -120,7 +120,7 @@ Register instructions
- Meaning
* - ``scope``
- static str; default "thread"
- - Execution scope; group axes follow :doc:`/cuda/tile_patterns`. It does
not elect an issuing thread.
+ - Execution scope; group axes follow
:doc:`/programming_guides/tile/tile_patterns`. It does not elect an issuing
thread.
* - ``pred``
- runtime bool; default None
- Execute the instruction when true; None is unconditional.
@@ -167,7 +167,7 @@ Operand order: ``dst, src; or the explicit
packing/unpacking register groups``.
``mov`` copies register bits or assembles/disassembles a native register
group. It
performs no numeric conversion and cannot change which thread owns a value. Use
-:ref:`numeric conversion <cuda-tile-contract-cvt>` for conversion and
:doc:`/cuda/tile_ops/data_movement` for memory.
+:ref:`numeric conversion <cuda-tile-contract-cvt>` for conversion and
:doc:`/programming_guides/tile/data_movement` for memory.
The scalar form ends both tensor operands in ``(1,)``. All earlier axes after
the
execution-group prefix are repeats. A scalar source expression can broadcast,
for
@@ -407,7 +407,7 @@ registers``. For example::
Both calls transfer four 32-bit words per repeat. Global-to-shared copying
through
registers consists of these two explicit calls and caller-allocated register
storage. To
-issue a direct asynchronous transfer, use :doc:`/cuda/tile_ops/async_copies`.
+issue a direct asynchronous transfer, use
:doc:`/programming_guides/tile/async_copies`.
Contract and instruction selection
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -446,7 +446,7 @@ word order. Emit one selected ld/st, optionally with
cache_policy and pred. Repe
unroll; group axes bind hardware IDs. Ordinary ld/st performs no asynchronous
completion
protocol.
-For a complete kernel, see :doc:`/cuda/tile_ops/data_movement`.
+For a complete kernel, see :doc:`/programming_guides/tile/data_movement`.
.. _cuda-tile-contract-ldmatrix-stmatrix:
@@ -581,7 +581,7 @@ An insufficiently aligned row, wrong register owner,
overlapping loaded register
partially active warp is rejected. Gaps **between** rows or register slots are
legal;
gaps **within** a shared 16-byte row are not.
-For a complete kernel, see :doc:`/cuda/tile_ops/warp_matrix`.
+For a complete kernel, see :doc:`/programming_guides/tile/warp_matrix`.
.. _cuda-tile-contract-cp-async:
@@ -716,7 +716,7 @@ shared-to-global request, a noncontiguous chunk, an
unaligned base, or a missing
axis fails its contract; lowering never substitutes a synchronous copy or a
narrower
instruction.
-For a complete kernel, see :doc:`/cuda/tile_ops/async_copies`.
+For a complete kernel, see :doc:`/programming_guides/tile/async_copies`.
.. _cuda-tile-contract-cp-async-bulk:
@@ -768,7 +768,7 @@ This pattern issues
``cp.async.bulk.shared::cluster.shared::cta.mbarrier::complete_tx::bytes``. It
copies
one contiguous shared-memory chunk to another CTA in the launch cluster and
completes
transaction bytes on a barrier in the destination CTA. For global tensor maps,
use
-:doc:`/cuda/tile_ops/tma` instead.
+:doc:`/programming_guides/tile/tma` instead.
The pointer contract
^^^^^^^^^^^^^^^^^^^^
@@ -1066,7 +1066,7 @@ sixteen-byte shared block. The remaining eight bytes are
undefined padding; barr
accounting counts payload bytes, not this padding. Swizzle 0/3 and gather4 use
this
explicit packed image.
-For a complete kernel, see :doc:`/cuda/tile_ops/tma`.
+For a complete kernel, see :doc:`/programming_guides/tile/tma`.
.. _cuda-tile-contract-mma-sync:
@@ -1184,11 +1184,11 @@ values between lanes; an incompatible physical owner
requires explicit preparati
the intended rounding order.
The tile call inserts no shared loads, ``ldmatrix``, communication, or
barrier. Use
-:doc:`/cuda/tile_ops/warp_matrix` to prepare fragments when that native
shared/register image
+:doc:`/programming_guides/tile/warp_matrix` to prepare fragments when that
native shared/register image
matches your operands. Sparse and other MMA shapes require their own explicit
pattern;
this page does not claim those forms.
-For a complete kernel, see :doc:`/cuda/tile_ops/warp_matrix`.
+For a complete kernel, see :doc:`/programming_guides/tile/warp_matrix`.
.. _cuda-tile-contract-tcgen05-ld-st:
@@ -1332,7 +1332,7 @@ complete a preceding MMA. Allocation, deallocation,
fences and synchronization r
explicit.
For TMEM allocation, MMA completion, and result readback in one kernel, see
-:doc:`/cuda/tile_ops/blackwell_tensor_cores`.
+:doc:`/programming_guides/tile/blackwell_tensor_cores`.
.. _cuda-tile-contract-tcgen05-cp:
@@ -1466,7 +1466,7 @@ chooses this qualifier.
Allocation, shared proxy fencing, election, commit, wait and deallocation are
caller
code. ``pred`` gates the cp but does not adjust a surrounding completion
protocol.
-The :doc:`Blackwell Tensor Core guide </cuda/tile_ops/blackwell_tensor_cores>`
+The :doc:`Blackwell Tensor Cores guide
</programming_guides/tile/blackwell_tensor_cores>`
compares this completion protocol with TMEM loads and stores.
.. _cuda-tile-contract-tcgen05-mma:
@@ -1653,7 +1653,7 @@ input kind/scale-vector mode.
tile does not initialize barriers, elect a thread, perform TMA, prepare scale
factors,
commit or wait. Keep the shared/TMEM allocations alive until their consumers
complete.
-For a complete kernel, see :doc:`/cuda/tile_ops/blackwell_tensor_cores`.
+For a complete kernel, see
:doc:`/programming_guides/tile/blackwell_tensor_cores`.
.. _cuda-tile-contract-ewise:
@@ -1727,7 +1727,7 @@ corresponding destination slot. There is one scalar
function evaluation per elem
a guaranteed single PTX instruction. Numeric conversion, packed arithmetic and
load/store staging must be written explicitly by the caller.
-For a complete kernel, see :doc:`/cuda/tile_ops/arithmetic`.
+For a complete kernel, see :doc:`/programming_guides/tile/arithmetic`.
.. _cuda-matrix-descriptor-extraction:
diff --git a/docs/reference/python/backend.cuda.tile.rst
b/docs/reference/python/backend.cuda.tile.rst
index f267e994f..3e39ce47e 100644
--- a/docs/reference/python/backend.cuda.tile.rst
+++ b/docs/reference/python/backend.cuda.tile.rst
@@ -24,7 +24,7 @@ tvm_tirx.backend.cuda.tile
.. currentmodule:: tvm_tirx.backend.cuda.tile
The CUDA tile backend lowers explicit instructions over tensor regions.
-See :doc:`/cuda/tile_patterns` for an introduction and the
+See :doc:`/programming_guides/tile/tile_patterns` for an introduction and the
:doc:`instruction reference <backend.cuda.tile.instructions>` for
parameters, operand coordinates, and storage requirements. The public
constructors are :doc:`apply and ewise <backend.cuda.script.tile>`.
diff --git a/examples/programming_guides/control_flow.py
b/examples/programming_guides/control_flow.py
new file mode 100644
index 000000000..c37e0582c
--- /dev/null
+++ b/examples/programming_guides/control_flow.py
@@ -0,0 +1,126 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Conditions and loops in CUDA kernels."""
+
+# [imports.begin]
+import torch
+
+import tvm_tirx
+from tvm_tirx.script import tirx as T
+
+n = T.dynamic("n")
+# [imports.end]
+
+
+# [if.begin]
[email protected]
+def relu(A: T.Tensor((n,), T.float32), B: T.Tensor((n,), T.float32)):
+ T.device_entry(launch=T.cuda.LaunchConfig(grid=T.ceildiv(n, 256),
block=256))
+ i = T.cuda.block_idx("x") * 256 + T.cuda.thread_idx("x")
+ if i < n:
+ if A[i] > 0.0:
+ B[i] = A[i]
+ else:
+ B[i] = 0.0
+ # [if.end]
+
+
+a = torch.randn(1000, device="cuda")
+b = torch.empty_like(a)
+tvm_tirx.compile(relu, target="cuda")(a, b)
+torch.testing.assert_close(b, torch.relu(a))
+
+
+# [for.begin]
[email protected]
+def row_sum(A: T.Tensor((n, 64), T.float32), B: T.Tensor((n,), T.float32)):
+ T.device_entry(launch=T.cuda.LaunchConfig(grid=T.ceildiv(n, 128),
block=128))
+ row = T.cuda.block_idx("x") * 128 + T.cuda.thread_idx("x")
+ if row < n:
+ total: T.float32 = 0.0
+ for j in range(16):
+ for k in T.unroll(4):
+ total += A[row, j * 4 + k]
+ B[row] = total
+ # [for.end]
+
+
+a = torch.randn(300, 64, device="cuda")
+b = torch.empty(300, device="cuda")
+tvm_tirx.compile(row_sum, target="cuda")(a, b)
+torch.testing.assert_close(b, a.sum(dim=1))
+
+
+# [grid.begin]
[email protected]
+def transpose(A: T.Tensor((64, 64), T.float32), B: T.Tensor((64, 64),
T.float32)):
+ T.device_entry(launch=T.cuda.LaunchConfig(grid=16, block=16))
+ row0 = T.cuda.block_idx("x") * 4
+ col0 = T.cuda.thread_idx("x") * 4
+ for r, c in T.grid(4, 4):
+ B[col0 + c, row0 + r] = A[row0 + r, col0 + c]
+ # [grid.end]
+
+
+a = torch.randn(64, 64, device="cuda")
+b = torch.empty_like(a)
+tvm_tirx.compile(transpose, target="cuda")(a, b)
+torch.testing.assert_close(b, a.T)
+
+
+# [while.begin]
[email protected]
+def search_sorted(
+ S: T.Tensor((64,), T.float32), Q: T.Tensor((n,), T.float32), B:
T.Tensor((n,), T.int32)
+):
+ T.device_entry(launch=T.cuda.LaunchConfig(grid=T.ceildiv(n, 256),
block=256))
+ i = T.cuda.block_idx("x") * 256 + T.cuda.thread_idx("x")
+ if i < n:
+ lo: T.int32 = 0
+ hi: T.int32 = 64
+ while lo < hi:
+ mid = (lo + hi) // 2
+ if S[mid] < Q[i]:
+ lo = mid + 1
+ else:
+ hi = mid
+ B[i] = lo
+ # [while.end]
+
+
+s = torch.sort(torch.randn(64, device="cuda")).values
+q = torch.randn(1000, device="cuda")
+b = torch.empty(1000, dtype=torch.int32, device="cuda")
+tvm_tirx.compile(search_sorted, target="cuda")(s, q, b)
+torch.testing.assert_close(b, torch.searchsorted(s, q).to(torch.int32))
+
+
+# [return.begin]
[email protected]
+def scale(A: T.Tensor((n,), T.float32), B: T.Tensor((n,), T.float32)):
+ T.device_entry(launch=T.cuda.LaunchConfig(grid=T.ceildiv(n, 256),
block=256))
+ i = T.cuda.block_idx("x") * 256 + T.cuda.thread_idx("x")
+ if i >= n:
+ T.gpu_thread_return()
+ B[i] = A[i] * 2.0
+ # [return.end]
+
+
+a = torch.randn(1000, device="cuda")
+b = torch.empty_like(a)
+tvm_tirx.compile(scale, target="cuda")(a, b)
+torch.testing.assert_close(b, a * 2)
diff --git a/examples/programming_guides/metaprogramming.py
b/examples/programming_guides/metaprogramming.py
new file mode 100644
index 000000000..962115476
--- /dev/null
+++ b/examples/programming_guides/metaprogramming.py
@@ -0,0 +1,100 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Generate, reuse, and specialize CUDA kernels with Python."""
+
+# T.jit binds constexpr parameters such as N before the annotations are parsed.
+# ruff: noqa: F821
+
+# [imports.begin]
+from __future__ import annotations
+
+import torch
+
+import tvm_tirx
+from tvm_tirx.script import tirx as T
+
+# [imports.end]
+
+
+# [generator.begin]
+def make_add(dtype, value, block=256):
+ n = T.dynamic("n")
+
+ @T.function
+ def add(A: T.Tensor((n,), dtype), B: T.Tensor((n,), dtype)):
+ T.device_entry(launch=T.cuda.LaunchConfig(grid=T.ceildiv(n, block),
block=block))
+ i = T.cuda.block_idx("x") * block + T.cuda.thread_idx("x")
+ if i < n:
+ B[i] = A[i] + value
+
+ return add
+
+
+add_two_f16 = make_add(T.float16, 2, block=128)
+add_two_f16_compiled = tvm_tirx.compile(add_two_f16, target="cuda")
+a = torch.arange(10, dtype=torch.float16, device="cuda")
+b = torch.empty_like(a)
+add_two_f16_compiled(a, b)
+print(b)
+# [generator.end]
+
+
+# [inline.begin]
[email protected]
+def add_one_if_valid(A, B, i, n):
+ if i < n:
+ B[i] = A[i] + 1
+
+
+n = T.dynamic("n")
+
+
[email protected]
+def add_one_pairs(A: T.Tensor((n,), T.float32), B: T.Tensor((n,), T.float32)):
+ T.device_entry(launch=T.cuda.LaunchConfig(grid=T.ceildiv(n, 512),
block=256))
+ i = (T.cuda.block_idx("x") * 256 + T.cuda.thread_idx("x")) * 2
+ add_one_if_valid(A, B, i, n)
+ add_one_if_valid(A, B, i + 1, n)
+ # [inline.end]
+
+
+a = torch.arange(1001, dtype=torch.float32, device="cuda")
+b = torch.empty_like(a)
+tvm_tirx.compile(add_one_pairs, target="cuda")(a, b)
+torch.testing.assert_close(b, a + 1)
+
+
+# [jit.begin]
[email protected]
+def add_one_fixed(
+ A: T.Tensor((N,), T.float32),
+ B: T.Tensor((N,), T.float32),
+ *,
+ N: T.constexpr,
+):
+ T.device_entry(launch=T.cuda.LaunchConfig(grid=T.ceildiv(N, 256),
block=256))
+ i = T.cuda.block_idx("x") * 256 + T.cuda.thread_idx("x")
+ if i < N:
+ B[i] = A[i] + 1
+
+
+add_one_256 = add_one_fixed.specialize(N=256)
+# [jit.end]
+a = torch.arange(256, dtype=torch.float32, device="cuda")
+b = torch.empty_like(a)
+tvm_tirx.compile(add_one_256, target="cuda")(a, b)
+torch.testing.assert_close(b, a + 1)
diff --git a/python/tvm_tirx/backend/cuda/tile/patterns/__init__.py
b/python/tvm_tirx/backend/cuda/tile/patterns/__init__.py
index 2c9a632c4..dd76c0956 100644
--- a/python/tvm_tirx/backend/cuda/tile/patterns/__init__.py
+++ b/python/tvm_tirx/backend/cuda/tile/patterns/__init__.py
@@ -106,7 +106,8 @@ def apply(instruction, *operands, scope="thread",
**parameters):
Notes
-----
- The :doc:`operand guide </cuda/tile_patterns>` explains the coordinate
+ The :doc:`Tile Operands and Execution guide
+ </programming_guides/tile/tile_patterns>` explains the coordinate
conventions with a complete load, compute, and store example.
"""
if isinstance(instruction, StringImm):
diff --git a/python/tvm_tirx/backend/cuda/tile/patterns/ewise.py
b/python/tvm_tirx/backend/cuda/tile/patterns/ewise.py
index df0661e23..57f575433 100644
--- a/python/tvm_tirx/backend/cuda/tile/patterns/ewise.py
+++ b/python/tvm_tirx/backend/cuda/tile/patterns/ewise.py
@@ -147,7 +147,8 @@ def ewise(function, dst, *inputs, scope="thread", **kwargs):
Unlike ``apply``, ``ewise`` has no trailing instruction atom: every
coordinate
after the group prefix selects one scalar invocation. See the
:ref:`scalar function contract <cuda-tile-contract-ewise>` for
restrictions and
- the :doc:`operand guide </cuda/tile_patterns>` for an example.
+ the :doc:`Tile Operands and Execution guide
+ </programming_guides/tile/tile_patterns>` for an example.
"""
if len(inputs) == 1 and isinstance(inputs[0], Tuple | tuple | list):
values = inputs[0]