This is an automated email from the ASF dual-hosted git repository.

tqchen pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm-tirx.git


The following commit(s) were added to refs/heads/main by this push:
     new f43b3030d [FEAT] Support PTX ISA 9.5 and the remaining sm_107 PTX 
instructions (#241)
f43b3030d is described below

commit f43b3030d33196d3a5f21e56b37549a6d3a5c70b
Author: Hongyi Jin <[email protected]>
AuthorDate: Thu Nov 12 09:52:32 2026 -0500

    [FEAT] Support PTX ISA 9.5 and the remaining sm_107 PTX instructions (#241)
    
    This PR extends the `T.ptx` table to PTX ISA 9.5 (CUDA 13.5) and fills
    the sm_107 (Rubin) gaps that remained after the PTX 9.4 update.
    
    ## PTX ISA 9.5 additions (`_PTX_95_ENTRIES`)
    - `fabric.submit.commit_group` and `fabric.wait.sync_restrict::read N`
    (sm_100+). The old `fabric.wait.sync_restrict::reads` stays registered
    and is noted as deprecated.
    - `cp.async.bulk.wait_group.sync_restrict::proxy::async N` (sm_100+).
    - `fence.alias.sync_restrict::read.release.sys` (sm_90+).
    - `sin`/`cos` `.2pi`, which takes its input in cycles. It is an optional
    slot on the existing entries, and a check enforces the ISA rule that
    `.2pi` requires `.ftz`.
    - `tensormap.replace` `.alloc_class` (the `global_address` form and the
    field-less form) and `tensormap.query.alloc_class_large` (sm_90a and the
    sm_100f/110f/120f families).
    - `%locality_domain_id` / `%locality_domain_supplemental_sm`.
    `T.cuda.mov_sreg` already reads them; this adds tests.
    
    ## sm_107 gaps and fixes
    - The packed-integer `add/sub{.sat}.{u8x4,s8x4}`,
    `add.sat.{u32,u16x2,s16x2}`, `neg.s8x4` and `min/max{.relu}` on
    `.u8x4`/`.s8x4` were excluded as "sm_120f-only". The ISA gives them to
    sm_107f and sm_120f, and ptxas agrees. They are now sm_107f sibling
    entries, so the existing families keep their floors.
    - `fabric.try_{get,put,red}.async.tensor` (PTX 9.4, sm_107f; 25 entries
    covering TMA's override shapes) and `fabric.try_atom` (PTX 9.4,
    sm_100+).
    - `tcgen05.shift` (sm_100a/103a/107a/110a).
    - **Fix:** the PTX 9.3 `fabric.try_*` entries stored the completion
    mechanism as one modifier token containing dots. `T.ptx["..."]` and
    attribute chains split on dots, so these instructions could not be
    called from a program at all. The mechanism is now split into separate
    slots. Rendered PTX and helper names are byte-identical. A new
    table-wide test rejects dotted tokens.
    - The module header now states the table is certified against CUDA 13.5.
    
    ## Validation
    - **ptxas:** arch floors were measured with CUDA 13.5 ptxas and recorded
    in `MEASURED` notes. Where ptxas is more permissive than the ISA, the
    table follows the ISA:
    `cp.async.bulk.wait_group.sync_restrict::proxy::async` assembles at
    sm_90, and `fabric.try_atom` accepts `.min/.max.s32`.
    - **Full certification:** `PTX_CERT=1` certifies the whole table under
    CUDA 13.5, with no regressions in existing entries.
    - **On a VR200 (sm_107a)**, GPU tests check:
      - the 9.5 waits execute;
      - `sin`/`cos .2pi` meets the ISA error bounds;
    - the tensor-map allocation class seen through the query after each
    replace, including PTX 9.5's clarification that `global_address` without
    `.alloc_class` resets the class to Small;
      - the locality-domain registers;
    - lane-wise wrap-around, saturation and relu semantics of all 16
    packed-integer forms against numpy.
    - **Assembled only, not executed:** the fabric tensor ops and `try_atom`
    need host-created logical endpoints across NVLink peers, and
    `tcgen05.shift` would need a full tmem round trip.
    - The hard-coded variant count in `test_ptx_all_variants_render_unique`
    and the address-operand census in `test_ptx_addr.py` are updated. With
    the new entries removed, the census gives back the old numbers exactly.
    
    ## Notes
    - **Query `.shared::cta` not registered:** CUDA 13.5 ptxas rejects
    `tensormap.query.alloc_class_large.shared::cta` with a 32-bit address
    register ("Arguments mismatch for instruction 'mov'"), but accepts a
    64-bit register or a symbol. Our shared-address ABI uses 32-bit
    registers, so that form is left out and the table notes why. It looks
    like a ptxas bug.
    - **Not included:** `trace.mark` and `.metadata_section`. The directive
    needs module-level PTX emission, which deserves its own design.
    - **Running on sm_107:** torch (cu132) pulls in a pip NVRTC 13.2 that
    cannot target sm_107, and `cuda.pathfinder` prefers it. Running these
    tests on sm_107 needs the CUDA 13.5 NVRTC loaded first, e.g. via
    `LD_PRELOAD`. Separately,
    `test_sparse_decode_conversion_intrinsics_codegen` fails on CUDA 13.5 on
    `main` as well: ptxas requires `sm_100a/f` for the `cvt` forms it
    compiles at plain `sm_100`.
---
 python/tvm_tirx/backend/cuda/ptx/table.py     | 617 ++++++++++++++++++++--
 python/tvm_tirx/script/tirx.pyi               |  91 +++-
 tests/python/tirx/codegen/test_cuda.py        |  64 +++
 tests/python/tirx/codegen/test_ptx_addr.py    |   8 +-
 tests/python/tirx/codegen/test_ptx_dialect.py | 706 +++++++++++++++++++++++++-
 5 files changed, 1391 insertions(+), 95 deletions(-)

diff --git a/python/tvm_tirx/backend/cuda/ptx/table.py 
b/python/tvm_tirx/backend/cuda/ptx/table.py
index ac283dca1..e304e53fb 100644
--- a/python/tvm_tirx/backend/cuda/ptx/table.py
+++ b/python/tvm_tirx/backend/cuda/ptx/table.py
@@ -20,14 +20,19 @@ Pure data + pure functions: this module deliberately 
imports nothing from
 ``tvm`` so the thin generators (``gen_stubs``, ``gen_coverage``,
 ``gen_helpers``) can load it standalone.
 
-Section and table numbers cite PTX ISA 9.4, the version implemented by the
-CUDA 13.4 ptxas this table is certified against:
-``https://docs.nvidia.com/cuda/developer-preview/13.4/parallel-thread-execution/index.html``.
-Every ``MEASURED`` note records CUDA 13.4 ptxas behaviour.  ``_PTX_94_ENTRIES``
-groups the instructions and qualifiers PTX ISA 9.4 introduced (the SM107
-family additions) ahead of the long-standing families in ``_ENTRIES``; it is a
-grouping by ISA release, not a second document version -- every entry cites
-the same document.
+The table is certified against the CUDA 13.5 ptxas, which implements PTX ISA
+9.5: ``https://docs.nvidia.com/cuda/parallel-thread-execution/index.html``.
+Section and table numbers cite PTX ISA 9.4
+(``https://docs.nvidia.com/cuda/developer-preview/13.4/parallel-thread-execution/index.html``)
+except in ``_PTX_95_ENTRIES``, which cites 9.5; 9.5 inserts a few sections
+(e.g. 9.7.10.30 tensormap.query.alloc_class_large), shifting the numbers after
+them by one.  A ``MEASURED`` note records the ptxas behaviour of the toolkit it
+was written against -- CUDA 13.4 unless it says CUDA 13.5 -- and the whole
+table, those notes included, re-certifies under CUDA 13.5 (``PTX_CERT=1``).
+``_PTX_94_ENTRIES`` and ``_PTX_95_ENTRIES`` group the instructions and
+qualifiers PTX ISA 9.4 (the SM107 family additions) and 9.5 introduced ahead of
+the long-standing families in ``_ENTRIES``; they are groupings by ISA release,
+not separate document versions to certify against.
 
 The converged :class:`InstructionEntry` design:
 
@@ -1179,20 +1184,20 @@ def _check_int_addsub(m, mnemonic):
         sub.type1        d, a, b;   .type1 = {.u16, .u32, .u64, .s16, .s64}
         sub{.sat}.type2  d, a, b;   .type2 = {.s32, .u8x4, .s8x4}
 
-    Of the saturating forms only `.s32` is registered. `add.sat` on
-    .u32/.u16x2/.s16x2 arrived in PTX ISA 9.2 as "sm_120f or higher in the same
-    family". The remaining types in these entries belong to syntax lines that
-    never carry `.sat`: .u16/.u64/.s16/.s64 for add, and
-    .u16/.u32/.u64/.s16/.s64 for sub. The `.u8x4`/`.s8x4` types are excluded
-    outright because they are sm_120f-only (see `_MINMAX_INT_PLAIN` below).
-    `add.sat.s32` and `sub.sat.s32` are the PTX 1.0 forms.
+    Of the saturating forms only `.s32` is in these entries: `add.sat.s32` and
+    `sub.sat.s32` are the PTX 1.0 forms. `add.sat` on .u32/.u16x2/.s16x2 and
+    the `.u8x4`/`.s8x4` types arrived in PTX ISA 9.2 for "sm_107f or higher in
+    the same family, sm_120f or higher in the same family", and are the sm_107f
+    `add_int_sm107f`/`sub_int_sm107f` entries. The remaining types here belong
+    to syntax lines that never carry `.sat`: .u16/.u64/.s16/.s64 for add, and
+    .u16/.u32/.u64/.s16/.s64 for sub.
     """
     if not m["sat"] or m["type"] == "s32":
         return None
     if mnemonic == "add" and m["type"] in ("u32", "u16x2", "s16x2"):
         return (
-            f"add.sat.{m['type']} is an sm_120f-only form (PTX ISA 9.2); "
-            ".sat is registered on .s32 only"
+            f"add.sat.{m['type']} is an sm_107f/sm_120f form (PTX ISA 9.2), "
+            "registered as the add_int_sm107f entry"
         )
     return f".sat is not on the {mnemonic}.{m['type']} syntax line"
 
@@ -1207,6 +1212,24 @@ def _check_int_sub(m):
     return _check_int_addsub(m, "sub")
 
 
+def _check_int_add_sm107f(m):
+    """The add lines PTX gives to sm_107f/sm_120f (9.7.1.1 Target ISA Notes).
+
+    `add{.sat}.{u8x4,s8x4}` and `add.sat.{u32,u16x2,s16x2}`; the non-saturating
+    .u32/.u16x2/.s16x2 forms are the sm_90 `add_int` entry's.
+    """
+    if m["type"] in ("u32", "u16x2", "s16x2") and not m["sat"]:
+        return f"add.{m['type']} without .sat is the add_int entry (sm_90 
floor)"
+    return None
+
+
+def _check_minmax_int8x4(m):
+    """`.relu` is on the signed line only: `min{.relu}.s8x4`, `min.u8x4`."""
+    if m["relu"] and m["type"] != "s8x4":
+        return ".relu applies only to the signed .s8x4 line"
+    return None
+
+
 def _check_int_mad(m):
     """`.sat` on the multiply-add lines: `.hi` mode, `.s32` type, nothing else.
 
@@ -1221,9 +1244,9 @@ def _check_int_mad(m):
 
 # The .type tokens of the integer min/max lines (PTX ISA 9.7.1.13/9.7.1.14).
 # `.relu` is only on the signed line (type2); the unsigned/other line takes 
none.
-# NOT REGISTERED: `.u8x4` and `{.relu}.s8x4`, which the ISA supports only on
-# "sm_120f or higher in the same family" -- outside the architectures this
-# dialect certifies, and ptxas rejects them at sm_100.
+# `.u8x4` and `{.relu}.s8x4` are not here: the ISA gives them only to sm_107f
+# and sm_120f (and ptxas rejects them at sm_100), so they are the sm_107f
+# `{min,max}_int8x4` sibling entries.
 _MINMAX_INT_PLAIN = ("u16", "u32", "u64", "u16x2", "s16", "s64")
 _MINMAX_INT_RELU = ("s16x2", "s32")
 # The floating-point and half-precision lines (9.7.3.11/9.7.3.12, 
9.7.4.7/9.7.4.8).
@@ -3235,6 +3258,78 @@ def _tma_global_dim_stride_operands(bracket, size_type):
     )
 
 
+# Attribute-override shapes of a fabric tensor handle, as for TMA's
+# `.override_attribute` (PTX ISA "Overriding tensor property value"): no
+# override; `.override::global_dim` on 1D tensors (a singleton size vector); or
+# `.override::global_dim_stride` on 2D-5D tensors (sizes, lower strides, upper
+# stride). The size vector binds .b8 or .b16 registers, one entry each.
+_FABRIC_TENSOR_OVERRIDES = (
+    ("", None, ("1d", "2d", "3d", "4d", "5d"), None),
+    ("_global_dim_b8", "override::global_dim", ("1d",), "tma_size_b8"),
+    ("_global_dim_b16", "override::global_dim", ("1d",), "b16"),
+    (
+        "_global_dim_stride_b8",
+        "override::global_dim_stride",
+        ("2d", "3d", "4d", "5d"),
+        "tma_size_b8",
+    ),
+    ("_global_dim_stride_b16", "override::global_dim_stride", ("2d", "3d", 
"4d", "5d"), "b16"),
+)
+
+
+def _fabric_tensor_handle_operands(bracket, handle, override, size_type):
+    """`[tensorMap, leId, off{, counterOff}{, attributeOverrideInfo}, 
tensorCoords]`.
+
+    The TMA override operands with the fabric handle (logical-endpoint id, byte
+    offset, and the counter offset of counted completion) in place of TMA's
+    `global_address`.
+    """
+    if override is None:
+        attributes = ()
+        coords = OperandSlot("coords", dtype="s32", lanes=_tma_coords_lanes, 
bracket=bracket)
+    elif override == "override::global_dim":
+        attributes = (
+            OperandSlot("tensor_size", dtype=size_type, lanes=1, vector=True, 
bracket=bracket),
+        )
+        coords = OperandSlot("coords", dtype="s32", lanes=1, vector=True, 
bracket=bracket)
+    else:
+        attributes = (
+            OperandSlot("tensor_size", dtype=size_type, lanes=_tma_dim_lanes, 
bracket=bracket),
+            OperandSlot(
+                "lower_stride", dtype="b32", lanes=_tma_lower_stride_lanes, 
bracket=bracket
+            ),
+            OperandSlot("upper_stride", dtype="b16", bracket=bracket),
+        )
+        coords = OperandSlot("coords", dtype="s32", lanes=_tma_coords_lanes, 
bracket=bracket)
+    return (
+        OperandSlot("tmap", kind="addr", space="global", bracket=bracket),
+        *(OperandSlot(name, dtype=dtype, bracket=bracket) for name, dtype in 
handle),
+        *attributes,
+        coords,
+    )
+
+
+_FABRIC_ATOM_TYPES = {
+    "and": ("b32", "b64"),
+    "xor": ("b32", "b64"),
+    "or": ("b32", "b64"),
+    "exch": ("b32", "b64", "b128"),
+    "cas": ("b32", "b64", "b128"),
+    "min": ("u32", "u64", "f16x2", "bf16x2"),
+    "max": ("u32", "u64", "f16x2", "bf16x2"),
+    "add": ("u32", "u64", "f16x2", "bf16x2", "f32", "f64"),
+}
+
+
+def _check_fabric_atom(m):
+    """Exact fabric.try_atom operation/type grid (PTX ISA 9.4, 9.7.11.5.10)."""
+    op, ty = m["redop"], m["type"]
+    if ty not in _FABRIC_ATOM_TYPES[op]:
+        allowed = " / ".join("." + item for item in _FABRIC_ATOM_TYPES[op])
+        return f".{op} fabric atomic takes {allowed}"
+    return None
+
+
 _TCGEN05_COLLECTOR_A = (
     "collector::a::fill",
     "collector::a::use",
@@ -3271,6 +3366,13 @@ def _check_set_packed(m):
     return None
 
 
+def _check_sin_cos_2pi(m):
+    """`.2pi` requires `.ftz` (PTX ISA 9.7.3.18/19, PTX ISA 9.5)."""
+    if m["unit"] and not m["ftz"]:
+        return ".ftz is required when .2pi is specified"
+    return None
+
+
 _PTX_94_ENTRIES = [
     # PTX ISA 9.4, 9.7.5.1/2 -- SM107 mixed packed add/sub.  Keep the
     # three type positions independent: the syntax defines one up-conversion
@@ -5724,17 +5826,311 @@ _PTX_94_ENTRIES = [
             OperandSlot("p", kind="addr", allow_imm_offset=True),
         ),
     ),
+    # fabric.try_{get,put,red}.async.tensor, per PTX ISA 9.4 9.7.11.5.7-9:
+    # the TMA-shaped fabric operations, which address a tensor through a
+    # tensor map plus a fabric handle `[tensorMap, leId, off{, counterOff}
+    # {, attributeOverrideInfo}, tensorCoords]`. `.override::global_address`
+    # is mandatory (the handle replaces the tensor map's global address);
+    # `.override_attribute` follows TMA's shapes (`_FABRIC_TENSOR_OVERRIDES`).
+    # try_get ends in `.b128` like try_put, while try_red ends in its `.redOp`
+    # alone: "the type of each data element ... is specified by the tensorMap".
+    # "Requires sm_107f or higher in the same family." MEASURED on CUDA 13.5:
+    # every shape assembles at sm_107f and sm_107a and is refused at sm_100a,
+    # sm_103a and sm_120f; `.override::global_dim` with `.2d` is refused
+    # ("Illegal modifier"), so is a type on try_red and a form without
+    # `.override::global_address`. ptxas also takes `.L2::cache_hint` after
+    # `.relaxed.sys`, as the ISA's own Examples write it; the entries spell the
+    # syntax line's order.
+    *[
+        InstructionEntry(
+            name=f"fabric_try_get_tensor{suffix}",
+            mnemonic="fabric",
+            slots=(
+                ModifierSlot("action", ("try_get",)),
+                ModifierSlot("api", ("async",)),
+                ModifierSlot("unit", ("tensor",)),
+                ModifierSlot("dim", dims),
+                ModifierSlot("dst", ("shared::cta",)),
+                ModifierSlot("load_mode", ("tile",), optional=True),
+                ModifierSlot("completion", ("mbarrier::complete_tx::bytes",)),
+                ModifierSlot("report", ("mbarrier::report::fabric",)),
+                ModifierSlot("cache", ("L2::cache_hint",), optional=True),
+                ModifierSlot("override_address", 
("override::global_address",)),
+                *((ModifierSlot("override_attribute", (override,)),) if 
override else ()),
+                ModifierSlot("sem", ("relaxed",)),
+                ModifierSlot("scope", ("sys",)),
+                ModifierSlot("type", ("b128",)),
+            ),
+            cert_arch="sm_107f",
+            operands=(
+                OperandSlot("dst_mem", kind="addr", allow_imm_offset=True, 
space="shared::cta"),
+                *_fabric_tensor_handle_operands(
+                    "src", (("src_le_id", "u32"), ("src_data_off", "u64")), 
override, size_type
+                ),
+                OperandSlot("mbar", kind="addr", allow_imm_offset=True, 
space="shared::cta"),
+                OperandSlot("cache_policy", dtype="u64", 
lanes=_tma_cache_lanes, vector=False),
+            ),
+        )
+        for suffix, override, dims, size_type in _FABRIC_TENSOR_OVERRIDES
+    ],
+    *[
+        InstructionEntry(
+            name=f"fabric_try_{kind}_tensor{'_counted' if counted else 
''}{suffix}",
+            mnemonic="fabric",
+            slots=(
+                ModifierSlot("action", (f"try_{kind}",)),
+                ModifierSlot("api", ("async",)),
+                ModifierSlot("unit", ("tensor",)),
+                ModifierSlot("multimem", ("multimem",), optional=True),
+                ModifierSlot("dim", dims),
+                ModifierSlot("src", ("shared::cta",)),
+                ModifierSlot("load_mode", ("tile",), optional=True),
+                ModifierSlot("completion", ("mbarrier::complete_tx::16B",)),
+                ModifierSlot("report", ("mbarrier::report::fabric",)),
+                *((ModifierSlot("counted", ("counted::bytes",)),) if counted 
else ()),
+                ModifierSlot("cache", ("L2::cache_hint",), optional=True),
+                ModifierSlot("override_address", 
("override::global_address",)),
+                *((ModifierSlot("override_attribute", (override,)),) if 
override else ()),
+                ModifierSlot("sem", ("relaxed",)),
+                ModifierSlot("scope", ("sys",)),
+                (
+                    ModifierSlot("type", ("b128",))
+                    if kind == "put"
+                    else ModifierSlot("redop", ("and", "xor", "or", "add", 
"min", "max"))
+                ),
+            ),
+            cert_arch="sm_107f",
+            operands=(
+                *_fabric_tensor_handle_operands(
+                    "dst",
+                    (
+                        ("dst_le_id", "u32"),
+                        ("dst_data_off", "u64"),
+                        *((("dst_counter_off", "u64"),) if counted else ()),
+                    ),
+                    override,
+                    size_type,
+                ),
+                OperandSlot("src_mem", kind="addr", allow_imm_offset=True, 
space="shared::cta"),
+                OperandSlot("size", dtype="u32"),
+                OperandSlot("mbar", kind="addr", allow_imm_offset=True, 
space="shared::cta"),
+                OperandSlot("cache_policy", dtype="u64", 
lanes=_tma_cache_lanes, vector=False),
+            ),
+        )
+        for kind in ("put", "red")
+        for counted in (False, True)
+        for suffix, override, dims, size_type in _FABRIC_TENSOR_OVERRIDES
+    ],
+    # fabric.try_atom, per PTX ISA 9.4 9.7.11.5.10: an atomic on a fabric
+    # handle that leaves the previous value in shared memory. Four syntax
+    # lines that differ only in their operation/type pair, so one entry whose
+    # `_check_fabric_atom` grid is the ISA's `.redOp*`/`.type*` sets. "Requires
+    # sm_100 or higher." MEASURED on CUDA 13.5: every listed pair assembles at
+    # sm_100 (and sm_100a, sm_103a, sm_107f), sm_90a refuses the instruction,
+    # and pairs outside the grid are refused ("Incorrect type '.s32' for
+    # operation '.add'") -- except `.min/.max.s32`, which ptxas accepts though
+    # the ISA does not list it; the table follows the ISA. ptxas also takes
+    # `.L2::cache_hint` before the completion mechanism, as the ISA's third
+    # Example writes it; the entry spells the syntax line's order.
+    InstructionEntry(
+        name="fabric_try_atom",
+        mnemonic="fabric",
+        slots=(
+            ModifierSlot("action", ("try_atom",)),
+            ModifierSlot("api", ("async",)),
+            ModifierSlot("src", ("shared::cta",)),
+            ModifierSlot("completion", ("mbarrier::complete_tx::16B",)),
+            ModifierSlot("report", ("mbarrier::report::fabric",)),
+            ModifierSlot("cache", ("L2::cache_hint",), optional=True),
+            ModifierSlot("sem", ("relaxed",)),
+            ModifierSlot("scope", ("sys",)),
+            ModifierSlot("redop", tuple(_FABRIC_ATOM_TYPES)),
+            ModifierSlot(
+                "type", ("b32", "b64", "b128", "u32", "u64", "f16x2", 
"bf16x2", "f32", "f64")
+            ),
+        ),
+        check=_check_fabric_atom,
+        cert_arch="sm_100",
+        operands=(
+            OperandSlot("dst_le_id", dtype="u32", bracket="dst"),
+            OperandSlot("dst_data_off", dtype="u64", bracket="dst"),
+            OperandSlot("smem_dst", kind="addr", allow_imm_offset=True, 
space="shared::cta"),
+            OperandSlot("smem_src", kind="addr", allow_imm_offset=True, 
space="shared::cta"),
+            OperandSlot("mbar", kind="addr", allow_imm_offset=True, 
space="shared::cta"),
+            OperandSlot("cache_policy", dtype="u64", lanes=_tma_cache_lanes, 
vector=False),
+        ),
+    ),
+]
+
+# PTX ISA 9.5 additions, grouped by ISA release like `_PTX_94_ENTRIES`.  Every
+# MEASURED note in this group records CUDA 13.5 ptxas behaviour, and ptxas
+# refuses these spellings under `.version 9.4` ("Feature ... requires PTX ISA
+# .version 9.5 or later").
+_PTX_95_ENTRIES = [
+    # fabric.submit.commit_group / fabric.wait.sync_restrict::read N, per PTX
+    # ISA 9.7.11.5.5 and 9.7.11.5.6 (both "Requires sm_100 or higher").
+    #
+    # A fabric submit group tracks the submit of every prior fabric operation
+    # of the issuing thread; `fabric.wait.sync_restrict::read N` waits until at
+    # most N of the most recent groups are pending, after which the
+    # .shared::cta data those groups read may be overwritten -- the same
+    # commit/wait shape as cp.async.bulk.  They are entries of their own rather
+    # than extra choices on the PTX 9.3 `fabric_submit`/`fabric_wait`: they are
+    # separate syntax lines (the wait gains an operand), and the 9.3 entries
+    # keep their own sm_103a certification manifest.
+    #
+    # N is an integer constant with no ISA-declared domain, kept OPEN like
+    # cp.async.bulk.wait_group's.  MEASURED: N = 0 and N = 255 assemble at
+    # sm_100, sm_100a, sm_103a and sm_107f; sm_90 and sm_90a refuse the
+    # instruction.
+    InstructionEntry(  # fabric.submit.commit_group;
+        name="fabric_submit_commit_group",
+        mnemonic="fabric",
+        slots=(
+            ModifierSlot("action", ("submit",)),
+            ModifierSlot("group", ("commit_group",)),
+        ),
+        cert_arch="sm_100",
+        orders_memory=True,
+        operands=(),
+    ),
+    InstructionEntry(  # fabric.wait.sync_restrict::read N;
+        name="fabric_wait_read",
+        mnemonic="fabric",
+        slots=(
+            ModifierSlot("action", ("wait",)),
+            ModifierSlot("waitop", ("sync_restrict::read",)),
+        ),
+        cert_arch="sm_100",
+        orders_memory=True,
+        operands=(OperandSlot("group", kind="imm"),),
+    ),
+    # cp.async.bulk.wait_group.sync_restrict::proxy::async N, per PTX ISA
+    # 9.7.10.28.6.2: the qualifier "only orders async-proxy global and shared
+    # memory accesses before subsequent async-proxy accesses; not before
+    # subsequent generic-proxy accesses".  It is its own syntax line, without
+    # `{.read}`: MEASURED, ptxas answers "Invalid combination of
+    # '.sync_restrict::proxy::async' and '.read'" at every architecture.
+    #
+    # The ISA requires sm_100 or higher.  MEASURED: ptxas also assembles it at
+    # sm_90 and sm_90a; the table follows the ISA where ptxas is the more
+    # permissive of the two, so this entry certifies at sm_100.
+    InstructionEntry(  # cp.async.bulk.wait_group.sync_restrict::proxy::async 
N;
+        name="cp_async_bulk_wait_group_proxy_async",
+        mnemonic="cp",
+        slots=(
+            ModifierSlot("api", ("async",)),
+            ModifierSlot("kind", ("bulk",)),
+            ModifierSlot("action", ("wait_group",)),
+            ModifierSlot("sync_restrict", ("sync_restrict::proxy::async",)),
+        ),
+        cert_arch="sm_100",
+        orders_memory=True,
+        operands=(OperandSlot("group", kind="imm"),),
+    ),
+    # fence.alias.sync_restrict::read.release.sys, per PTX ISA 9.7.15.4: the
+    # generic-proxy to generic-proxy alias fence ("requires sm_90 or higher").
+    # A fixed token sequence.  MEASURED: ptxas refuses any other scope
+    # ("Modifier '.sync_restrict::read' cannot be combined with modifier
+    # '.gpu'").
+    InstructionEntry(  # fence.alias.sync_restrict::read.release.sys;
+        name="fence_alias_sync_restrict_read",
+        mnemonic="fence",
+        slots=(
+            ModifierSlot("proxykind", ("alias",)),
+            ModifierSlot("sync_restrict", ("sync_restrict::read",)),
+            ModifierSlot("sem", ("release",)),
+            ModifierSlot("scope", ("sys",)),
+        ),
+        cert_arch="sm_90",
+        orders_memory=True,
+        operands=(),
+    ),
+    # tensormap.replace `{.alloc_class}` and tensormap.query.alloc_class_large,
+    # per PTX ISA 9.7.10.29/30: the Tensor Allocation Size of a tensor-map
+    # object, small or large.  Two new tensormap.replace syntax lines --
+    #   tensormap.replace.mode.field4{.alloc_class}{.ss}.b1024.type [addr], 
new_val;
+    #   tensormap.replace.mode.alloc_class{.ss}.b1024               [addr];
+    # -- where the field-less line takes only .alloc_small/.alloc_large and
+    # `.alloc_same` ("unchanged") exists only beside `.global_address`.  The
+    # global_address line without `.alloc_class` stays the pre-9.5
+    # `tensormap_replace_address` entry; PTX ISA 9.5 clarifies that it now sets
+    # the allocation size to Small.
+    #
+    # "Qualifier .alloc_class is supported on sm_90a, and on following
+    # family-specific architectures: sm_100f / sm_110f / sm_120f or higher in
+    # the same family", and the query has the same list.  MEASURED: all three
+    # assemble at sm_90a, sm_100a, sm_100f, sm_103a, sm_107a, sm_107f and
+    # sm_120f, and are refused at sm_90 and sm_100; `.alloc_same` without a
+    # field answers "Modifier '.alloc_same' requires modifier
+    # '.global_address'", and the other fields refuse `.alloc_*`.
+    #
+    # NOT REGISTERED: the query's `.const` and `.param{::entry}` state spaces.
+    # Neither address can cross the helper-function ABI (the reason `ld` leaves
+    # out `.param`); a `__grid_constant__` kernel-parameter tensor map is
+    # reached through generic addressing, the no-`.ss` spelling registered 
here.
+    # Nor the query's `.shared::cta`: MEASURED on CUDA 13.5 (sm_90a, sm_100a,
+    # sm_107a), ptxas refuses it with a 32-bit address register -- "Arguments
+    # mismatch for instruction 'mov'" -- while accepting a 64-bit register or a
+    # variable name, and the shared-address ABI here is the 32-bit window
+    # offset (the field-less and global_address replaces take it fine). So a
+    # tensor map in shared memory has no query helper yet; the replaces do.
+    InstructionEntry(  # 
tensormap.replace.tile.global_address.alloc_class{.ss}.b1024.b64
+        name="tensormap_replace_address_alloc_class",
+        mnemonic="tensormap.replace",
+        slots=(
+            ModifierSlot("mode", ("tile",)),
+            ModifierSlot("field", ("global_address",)),
+            ModifierSlot("alloc_class", ("alloc_small", "alloc_large", 
"alloc_same")),
+            ModifierSlot("space", ("global", "shared::cta"), optional=True),
+            ModifierSlot("width", ("b1024",)),
+            ModifierSlot("type", ("b64",)),
+        ),
+        cert_arch="sm_90a",
+        operands=(
+            OperandSlot("addr", kind="addr", allow_imm_offset=True),
+            OperandSlot("new_val", dtype="b64"),
+        ),
+    ),
+    InstructionEntry(  # tensormap.replace.tile.alloc_class{.ss}.b1024 [addr];
+        name="tensormap_replace_alloc_class",
+        mnemonic="tensormap.replace",
+        slots=(
+            ModifierSlot("mode", ("tile",)),
+            ModifierSlot("alloc_class", ("alloc_small", "alloc_large")),
+            ModifierSlot("space", ("global", "shared::cta"), optional=True),
+            ModifierSlot("width", ("b1024",)),
+        ),
+        cert_arch="sm_90a",
+        operands=(OperandSlot("addr", kind="addr", allow_imm_offset=True),),
+    ),
+    InstructionEntry(  # tensormap.query.alloc_class_large{.ss}.b1024 p, 
[addr];
+        name="tensormap_query",
+        mnemonic="tensormap.query",
+        slots=(
+            ModifierSlot("query", ("alloc_class_large",)),
+            ModifierSlot("space", ("global",), optional=True),
+            ModifierSlot("width", ("b1024",)),
+        ),
+        cert_arch="sm_90a",
+        operands=(
+            OperandSlot("p", rw="w", dtype="pred"),
+            OperandSlot("addr", kind="addr", allow_imm_offset=True),
+        ),
+    ),
 ]
 
 _ENTRIES = [
     *_PTX_94_ENTRIES,
+    *_PTX_95_ENTRIES,
     # ------------------------------------------------------------------
     # PTX ISA 9.7.1 — Integer Arithmetic Instructions
     # ------------------------------------------------------------------
-    # The existing public families cover this section with one exception,
-    # stated at the entry it belongs to: the packed `.u8x4`/`.s8x4` types
-    # (and the saturating forms that arrived with them), which the ISA gives
-    # only to "sm_120f or higher in the same family".  The distinct `clmad`
+    # The existing public families cover this section. The packed
+    # `.u8x4`/`.s8x4` types (and the saturating forms that arrived with them),
+    # which the ISA gives only to "sm_107f or higher in the same family, 
sm_120f
+    # or higher in the same family", are sm_107f sibling entries beside the
+    # families they extend, so those families keep their floors.  The distinct 
`clmad`
     # family (9.7.1.5, introduced in PTX ISA 9.3) is registered below.
     #
     # Several mnemonics have a floating-point line further down the table. The
@@ -5747,9 +6143,9 @@ _ENTRIES = [
     # (single/double), 9.7.4.7, 9.7.4.8 (half).
     # Each mnemonic gets two entries because the ISA gives it two operand
     # shapes: the usual `d, a, b` and the three-source `d, a, b, c` line.
-    # NOT REGISTERED: only the `.u8x4` and `{.relu}.s8x4` tokens of the integer
-    # lines (reason at the `_MINMAX_INT_PLAIN` note above: sm_120f-only). Every
-    # other syntax line and type token of both families is here.
+    # The `.u8x4` and `{.relu}.s8x4` tokens of the integer lines are the 
sm_107f
+    # `{min,max}_int8x4` entries that follow; every syntax line and type token
+    # of both families is registered.
     *[
         InstructionEntry(
             name=name,
@@ -5773,6 +6169,29 @@ _ENTRIES = [
         )
         for name in ("max", "min")
     ],
+    # `min/max.u8x4` and `min/max{.relu}.s8x4` (9.7.1.13/9.7.1.14), given only
+    # to "sm_107f or higher in the same family, sm_120f or higher in the same
+    # family": a sibling entry certified at sm_107f. MEASURED on CUDA 13.5:
+    # assembles at sm_107a, sm_107f and sm_120f, refused at sm_90, sm_100a and
+    # sm_103a; `.relu.u8x4` is refused everywhere.
+    *[
+        InstructionEntry(
+            name=f"{name}_int8x4",
+            mnemonic=name,
+            slots=(
+                ModifierSlot("relu", ("relu",), optional=True),
+                ModifierSlot("type", ("u8x4", "s8x4")),
+            ),
+            check=_check_minmax_int8x4,
+            cert_arch="sm_107f",
+            operands=(
+                OperandSlot("d", rw="w"),
+                OperandSlot("a"),
+                OperandSlot("b"),
+            ),
+        )
+        for name in ("max", "min")
+    ],
     *[
         InstructionEntry(
             name=f"{name}3",
@@ -5826,9 +6245,9 @@ _ENTRIES = [
     #   sub{.sat}.type2  d, a, b;   .type2 = {.s32, .u8x4, .s8x4}
     # The two lines differ only in whether `.sat` is offered, so one entry with
     # an optional slot spells both and `_check_int_addsub` draws the boundary.
-    # NOT REGISTERED: the `.u8x4`/`.s8x4` types on either mnemonic, and `.sat`
-    # on .u32/.u16x2/.s16x2 -- all sm_120f-only (PTX ISA 9.2), the same
-    # exclusion the min/max entry above makes for the same reason.
+    # The `.u8x4`/`.s8x4` types on either mnemonic, and `.sat` on
+    # .u32/.u16x2/.s16x2, are the sm_107f `add_int_sm107f`/`sub_int_sm107f`
+    # entries after these two.
     InstructionEntry(
         name="add_int",
         mnemonic="add",
@@ -5860,6 +6279,43 @@ _ENTRIES = [
             OperandSlot("b"),
         ),
     ),
+    # The integer add/sub forms the ISA gives only to "sm_107f or higher in the
+    # same family, sm_120f or higher in the same family" (9.7.1.1/9.7.1.2 
Target
+    # ISA Notes, PTX ISA 9.2): the packed `.u8x4`/`.s8x4` types with or without
+    # `.sat`, and `.sat` on .u32/.u16x2/.s16x2. Sibling entries certified at
+    # sm_107f, so `add_int`/`sub_int` keep their floors. MEASURED on CUDA 13.5:
+    # every one of these assembles at sm_107a, sm_107f and sm_120f and is 
refused
+    # at sm_90, sm_100a and sm_103a; `sub.sat.u32` is refused everywhere, as
+    # the syntax (`sub{.sat}.type2`, `.type2 = {.s32, .u8x4, .s8x4}`) says.
+    InstructionEntry(
+        name="add_int_sm107f",
+        mnemonic="add",
+        slots=(
+            ModifierSlot("sat", ("sat",), optional=True),
+            ModifierSlot("type", ("u32", "u16x2", "u8x4", "s16x2", "s8x4")),
+        ),
+        check=_check_int_add_sm107f,
+        cert_arch="sm_107f",
+        operands=(
+            OperandSlot("d", rw="w"),
+            OperandSlot("a"),
+            OperandSlot("b"),
+        ),
+    ),
+    InstructionEntry(
+        name="sub_int_sm107f",
+        mnemonic="sub",
+        slots=(
+            ModifierSlot("sat", ("sat",), optional=True),
+            ModifierSlot("type", ("u8x4", "s8x4")),
+        ),
+        cert_arch="sm_107f",
+        operands=(
+            OperandSlot("d", rw="w"),
+            OperandSlot("a"),
+            OperandSlot("b"),
+        ),
+    ),
     # mul / mad per PTX ISA 9.7.1.3, 9.7.1.4:
     #   mul.mode.type  d, a, b;      mad.mode.type   d, a, b, c;
     #   .mode = {.hi, .lo, .wide}    mad.hi.sat.s32  d, a, b, c;
@@ -6002,7 +6458,7 @@ _ENTRIES = [
     # abs / neg per PTX ISA 9.7.1.11, 9.7.1.12 -- signed integers only. Their
     # floating-point lines (9.7.3.9, 9.7.3.10) are the `abs_f` and `neg`
     # entries below; which of each pair carries a suffix is just which arrived
-    # second. NOT REGISTERED: `neg.s8x4`, sm_120f-only as above.
+    # second. `neg.s8x4` is the sm_107f `neg_int_sm107f` entry after them.
     InstructionEntry(
         name="abs",
         slots=(ModifierSlot("type", ("s16", "s32", "s64")),),
@@ -6020,6 +6476,20 @@ _ENTRIES = [
             OperandSlot("a"),
         ),
     ),
+    # `neg.s8x4` (9.7.1.12), "sm_107f or higher in the same family, sm_120f or
+    # higher in the same family" -- the sm_107f sibling of `neg_int`. MEASURED
+    # on CUDA 13.5 as for the packed add/sub above; `neg.u8x4` and `abs.s8x4`
+    # are not in the syntax and ptxas refuses them.
+    InstructionEntry(
+        name="neg_int_sm107f",
+        mnemonic="neg",
+        slots=(ModifierSlot("type", ("s8x4",)),),
+        cert_arch="sm_107f",
+        operands=(
+            OperandSlot("d", rw="w"),
+            OperandSlot("a"),
+        ),
+    ),
     # popc / clz per PTX ISA 9.7.1.15, 9.7.1.16: `popc.type d, a;`. Both count
     # bits of a `.b32`/`.b64` source into a `.u32` destination whatever the
     # source width -- "destination d has type .u32" -- so d is typed outright
@@ -6340,20 +6810,36 @@ _ENTRIES = [
     # half-precision forms (9.7.4.10) are the `ex2_half` entry below: they
     # cannot share this optional `.ftz` slot, because theirs is mandatory on
     # the bf16 line and unspelled on the f16 one.
+    #
+    # PTX ISA 9.5 adds `{.2pi}` to sin and cos only -- 
`op.approx{.2pi}{.ftz}.f32`,
+    # reading the input in cycles (1.0 is one full turn) -- and "the .ftz
+    # qualifier is required when .2pi is specified", which `_check_sin_cos_2pi`
+    # states.  It is a qualifier on the same syntax line, so it is a slot here
+    # rather than an entry in `_PTX_95_ENTRIES`, and the helper signature of
+    # `T.ptx.sin`/`T.ptx.cos` stays the one shape it was.  `2pi` is not a 
Python
+    # identifier, so programs spell it through the string form
+    # (`T.ptx["sin.approx.2pi.ftz.f32"]`), as they do the TMA `1d`..`5d` dims.
+    # "Supported on all target architectures."  MEASURED on CUDA 13.5: 
assembles
+    # at sm_75, sm_90, sm_100 and sm_107a; without `.ftz` ptxas answers ".ftz
+    # modifier required for instruction 'sin' with modifier '.2pi'"; lg2 and 
ex2
+    # refuse `.2pi` ("Illegal modifier"); `.version 9.4` refuses it.
     *[
         InstructionEntry(
             name=name,
             slots=(
                 ModifierSlot("mode", ("approx",)),
+                *((ModifierSlot("unit", ("2pi",), optional=True),) if trig 
else ()),
                 ModifierSlot("ftz", ("ftz",), optional=True),
                 ModifierSlot("type", ("f32",)),
             ),
+            check=_check_sin_cos_2pi if trig else None,
             operands=(
                 OperandSlot("d", rw="w"),
                 OperandSlot("value"),
             ),
         )
         for name in ("ex2", "sin", "cos", "lg2")
+        for trig in (name in ("sin", "cos"),)
     ],
     # tanh per PTX ISA 9.7.3.22: `tanh.approx.f32 d, a;` -- alone among the
     # approximations in spelling no `.ftz`, so it gets no such slot. Requires
@@ -7479,7 +7965,8 @@ _ENTRIES = [
     # enforces by name.
     # NOT REGISTERED: `.swizzle_atomicity` (sm_100a and later only, while the
     # rest of this instruction is sm_90a), and the `.elemtype` values 13-15 and
-    # `.swizzle_mode` value 4, which need newer architectures still.
+    # `.swizzle_mode` value 4, which need newer architectures still. PTX ISA
+    # 9.5's `.alloc_class` lines and tensormap.query are in `_PTX_95_ENTRIES`.
     *[
         InstructionEntry(
             name=f"tensormap_replace_{group}",
@@ -9029,6 +9516,11 @@ _ENTRIES = [
     # PTX ISA 9.7.11 — Fabric Instructions (introduced in PTX ISA 9.3)
     # ------------------------------------------------------------------
     # A fabric handle is the composite [logical-endpoint id, byte offset].
+    # The completion mechanism `.mbarrier::complete_tx::{bytes,16B}
+    # .mbarrier::report::fabric{.counted::bytes}` is two or three dotted
+    # qualifiers, so it is that many slots: a single slot holding the dotted
+    # text renders the same PTX but cannot be spelled by `T.ptx[...]` or an
+    # attribute chain, which split on the dots.
     # Counted completion adds a third counter-offset member and is therefore a
     # distinct operand shape. Likewise, cp_mask has a trailing byte-mask
     # operand and is incompatible with counted::bytes (ISA 9.7.11.5.2).
@@ -9039,7 +9531,8 @@ _ENTRIES = [
             ModifierSlot("action", ("try_get",)),
             ModifierSlot("api", ("async",)),
             ModifierSlot("dst", ("shared::cta",)),
-            ModifierSlot("completion", 
("mbarrier::complete_tx::bytes.mbarrier::report::fabric",)),
+            ModifierSlot("completion", ("mbarrier::complete_tx::bytes",)),
+            ModifierSlot("report", ("mbarrier::report::fabric",)),
             ModifierSlot("sem", ("relaxed",)),
             ModifierSlot("scope", ("sys",)),
             ModifierSlot("type", ("b128",)),
@@ -9062,13 +9555,9 @@ _ENTRIES = [
                 ModifierSlot("api", ("async",)),
                 ModifierSlot("multimem", ("multimem",), optional=True),
                 ModifierSlot("src", ("shared::cta",)),
-                ModifierSlot(
-                    "completion",
-                    (
-                        "mbarrier::complete_tx::16B.mbarrier::report::fabric"
-                        + (".counted::bytes" if counted else ""),
-                    ),
-                ),
+                ModifierSlot("completion", ("mbarrier::complete_tx::16B",)),
+                ModifierSlot("report", ("mbarrier::report::fabric",)),
+                *((ModifierSlot("counted", ("counted::bytes",)),) if counted 
else ()),
                 *((ModifierSlot("cp_mask", ("cp_mask",)),) if cp_mask else ()),
                 ModifierSlot("sem", ("relaxed",)),
                 ModifierSlot("scope", ("sys",)),
@@ -9100,13 +9589,9 @@ _ENTRIES = [
                 ModifierSlot("api", ("async",)),
                 ModifierSlot("multimem", ("multimem",), optional=True),
                 ModifierSlot("src", ("shared::cta",)),
-                ModifierSlot(
-                    "completion",
-                    (
-                        "mbarrier::complete_tx::16B.mbarrier::report::fabric"
-                        + (".counted::bytes" if counted else ""),
-                    ),
-                ),
+                ModifierSlot("completion", ("mbarrier::complete_tx::16B",)),
+                ModifierSlot("report", ("mbarrier::report::fabric",)),
+                *((ModifierSlot("counted", ("counted::bytes",)),) if counted 
else ()),
                 ModifierSlot("sem", ("relaxed",)),
                 ModifierSlot("scope", ("sys",)),
                 ModifierSlot("redop", tuple(_FABRIC_RED_TYPES)),
@@ -9139,7 +9624,8 @@ _ENTRIES = [
             ModifierSlot("api", ("async",)),
             ModifierSlot("multimem", ("multimem",)),
             ModifierSlot("dst", ("shared::cta",)),
-            ModifierSlot("completion", 
("mbarrier::complete_tx::bytes.mbarrier::report::fabric",)),
+            ModifierSlot("completion", ("mbarrier::complete_tx::bytes",)),
+            ModifierSlot("report", ("mbarrier::report::fabric",)),
             ModifierSlot("sem", ("relaxed",)),
             ModifierSlot("scope", ("sys",)),
             ModifierSlot("redop", tuple(_FABRIC_PULLRED_TYPES)),
@@ -9168,6 +9654,10 @@ _ENTRIES = [
         orders_memory=True,
         operands=(),
     ),
+    # `fabric.wait.sync_restrict::reads` is deprecated by PTX ISA 9.5 in
+    # favour of `fabric.wait.sync_restrict::read N` with
+    # `fabric.submit.commit_group` (both in `_PTX_95_ENTRIES`), and "has no
+    # effect" (9.7.11.5.6). Kept so existing programs keep their spelling.
     InstructionEntry(
         name="fabric_wait",
         mnemonic="fabric",
@@ -9486,9 +9976,10 @@ _ENTRIES = [
     # alone is not enough. Each syntax line whose token sequence differs is its
     # own entry, all sharing the `fence` mnemonic, so the surface stays
     # `T.ptx.fence...` and the chain narrows on the tokens themselves.
-    # NOT REGISTERED: the `.sync_restrict` lines (fixed token sequences with no
-    # users yet), and the deprecated `membar` spellings, which the ISA itself
-    # marks as the old style for `fence`.
+    # NOT REGISTERED: the pre-9.5 `.sync_restrict` lines (fixed token
+    # sequences with no users yet), and the deprecated `membar` spellings,
+    # which the ISA itself marks as the old style for `fence`. PTX ISA 9.5's
+    # `fence.alias.sync_restrict::read.release.sys` is in `_PTX_95_ENTRIES`.
     InstructionEntry(  # fence{.sem}.scope;
         name="fence",
         slots=(
@@ -11211,10 +11702,26 @@ _ENTRIES = [
     # Every one of these carries `orders_memory=True`: the legacy helpers all
     # had `::: "memory"`, and the waits/fences name no address at all.
     #
-    # NOT REGISTERED:
-    # - `tcgen05.shift.cta_group.down [taddr]`: no call site. (The tmem address
-    #   space its bracketed operand needs now exists -- 
`tcgen05.ld`/`.st`/`.cp`
-    #   all use it -- so this one is registrable the day something calls it.)
+    #
+    # tcgen05.shift per PTX ISA 9.7.18.9.3: asynchronously shift the rows of a
+    # tmem matrix down by one, completed through tcgen05.commit like
+    # tcgen05.mma/.cp. Supported on sm_100a, sm_103a, sm_107a and sm_110a only
+    # -- no family-specific (`f`) target -- so it certifies at sm_100a.
+    # MEASURED on CUDA 13.5: assembles at sm_100a, sm_103a, sm_107a and sm_110a
+    # and is refused at sm_100f, sm_107f and sm_120a. ptxas also takes the
+    # `.down.cta_group::N` order of the ISA's own Examples; the entry spells 
the
+    # syntax line's `.cta_group.down`.
+    InstructionEntry(  # tcgen05.shift.cta_group.down [taddr];
+        name="tcgen05_shift",
+        mnemonic="tcgen05",
+        slots=(
+            ModifierSlot("action", ("shift",)),
+            ModifierSlot("cta_group", ("cta_group::1", "cta_group::2")),
+            ModifierSlot("direction", ("down",)),
+        ),
+        cert_arch="sm_100a",
+        operands=(OperandSlot("taddr", kind="addr", space="tmem"),),
+    ),
     InstructionEntry(  # 
tcgen05.alloc.cta_group.sync.aligned{.shared::cta}.b32 [dst], nCols;
         name="tcgen05_alloc",
         mnemonic="tcgen05",
diff --git a/python/tvm_tirx/script/tirx.pyi b/python/tvm_tirx/script/tirx.pyi
index 8c0d91434..1106ae0fe 100644
--- a/python/tvm_tirx/script/tirx.pyi
+++ b/python/tvm_tirx/script/tirx.pyi
@@ -52,7 +52,7 @@ class _Chain_activemask:
     ) -> None: ...
 
 class _Chain_add:
-    """`add` — 6 entries sharing this mnemonic; PTX puts their difference in 
the operand list,
+    """`add` — 7 entries sharing this mnemonic; PTX puts their difference in 
the operand list,
     so the call selects one. Shapes: (d, a, c); (d, a, b)
     """
 
@@ -72,11 +72,13 @@ class _Chain_add:
     s16x2: _Chain_add
     s32: _Chain_add
     s64: _Chain_add
+    s8x4: _Chain_add
     sat: _Chain_add
     u16: _Chain_add
     u16x2: _Chain_add
     u32: _Chain_add
     u64: _Chain_add
+    u8x4: _Chain_add
     def __call__(self, *args: Any, pred: Any = None, preserve_dst: bool = 
False) -> None: ...
 
 class _Chain_and:
@@ -373,7 +375,9 @@ class _Chain_copysign:
     ) -> None: ...
 
 class _Chain_cos:
-    """`cos` — mode∈{approx}; ftz∈{ftz} (opt); type∈{f32}"""
+    """`cos` — mode∈{approx}; unit∈{2pi} (opt); ftz∈{ftz} (opt); type∈{f32} — 
`.2pi` requires
+    `.ftz` (PTX ISA 9.7.3.18/19, PTX ISA 9.5).
+    """
 
     approx: _Chain_cos
     f32: _Chain_cos
@@ -388,7 +392,7 @@ class _Chain_cos:
     ) -> None: ...
 
 class _Chain_cp:
-    """`cp` — 77 entries sharing this mnemonic; PTX puts their difference in 
the operand list,
+    """`cp` — 78 entries sharing this mnemonic; PTX puts their difference in 
the operand list,
     so the call selects one. Shapes: (*__operands); (src_mem, size); (tmap, 
global_address,
     tensor_size, coords); (group); (); (dst_mem, src_mem, size, mbar); (addr)
     """
@@ -458,6 +462,7 @@ class _Chain_cp:
     shared: _Chain_cp
     shared__cluster: _Chain_cp
     shared__cta: _Chain_cp
+    sync_restrict__proxy__async: _Chain_cp
     sys: _Chain_cp
     tensor: _Chain_cp
     tile: _Chain_cp
@@ -669,12 +674,14 @@ class _Chain_ex2:
     def __call__(self, *args: Any, pred: Any = None, preserve_dst: bool = 
False) -> None: ...
 
 class _Chain_fabric:
-    """`fabric` — 9 entries sharing this mnemonic; PTX puts their difference 
in the operand
-    list, so the call selects one. Shapes: (dst_mem, src_le_id, src_data_off, 
size, mbar);
-    (dst_le_id, dst_data_off, src_mem, size, mbar); (dst_le_id, dst_data_off, 
src_mem, size,
-    mbar, byte_mask); (dst_le_id, dst_data_off, dst_counter_off, src_mem, 
size, mbar); ()
+    """`fabric` — 37 entries sharing this mnemonic; PTX puts their difference 
in the operand
+    list, so the call selects one. Shapes: (*__operands); (); (group); 
(dst_mem, src_le_id,
+    src_data_off, size, mbar); (dst_le_id, dst_data_off, src_mem, size, mbar); 
(dst_le_id,
+    dst_data_off, src_mem, size, mbar, byte_mask); (dst_le_id, dst_data_off,
+    dst_counter_off, src_mem, size, mbar)
     """
 
+    L2__cache_hint: _Chain_fabric
     add: _Chain_fabric
     and_: _Chain_fabric
     async_: _Chain_fabric
@@ -682,23 +689,39 @@ class _Chain_fabric:
     b32: _Chain_fabric
     b64: _Chain_fabric
     bf16: _Chain_fabric
+    bf16x2: _Chain_fabric
+    cas: _Chain_fabric
+    commit_group: _Chain_fabric
+    counted__bytes: _Chain_fabric
     cp_mask: _Chain_fabric
+    exch: _Chain_fabric
     f16: _Chain_fabric
+    f16x2: _Chain_fabric
     f32: _Chain_fabric
     f64: _Chain_fabric
     max: _Chain_fabric
+    mbarrier__complete_tx__16B: _Chain_fabric
+    mbarrier__complete_tx__bytes: _Chain_fabric
+    mbarrier__report__fabric: _Chain_fabric
     min: _Chain_fabric
     multimem: _Chain_fabric
     op_restrict__fetching: _Chain_fabric
     or_: _Chain_fabric
+    override__global_address: _Chain_fabric
+    override__global_dim: _Chain_fabric
+    override__global_dim_stride: _Chain_fabric
     relaxed: _Chain_fabric
     s32: _Chain_fabric
     s64: _Chain_fabric
     shared__cta: _Chain_fabric
     submit: _Chain_fabric
     sync: _Chain_fabric
+    sync_restrict__read: _Chain_fabric
     sync_restrict__reads: _Chain_fabric
     sys: _Chain_fabric
+    tensor: _Chain_fabric
+    tile: _Chain_fabric
+    try_atom: _Chain_fabric
     try_get: _Chain_fabric
     try_pullred: _Chain_fabric
     try_put: _Chain_fabric
@@ -710,7 +733,7 @@ class _Chain_fabric:
     def __call__(self, *args: Any, pred: Any = None) -> None: ...
 
 class _Chain_fence:
-    """`fence` — 6 entries sharing this mnemonic; PTX puts their difference in 
the operand
+    """`fence` — 7 entries sharing this mnemonic; PTX puts their difference in 
the operand
     list, so the call selects one. Shapes: (); (addr)
     """
 
@@ -731,6 +754,7 @@ class _Chain_fence:
     sc: _Chain_fence
     shared__cluster: _Chain_fence
     shared__cta: _Chain_fence
+    sync_restrict__read: _Chain_fence
     sys: _Chain_fence
     tensormap__generic: _Chain_fence
     def __call__(self, *args: Any, pred: Any = None) -> None: ...
@@ -1015,7 +1039,7 @@ class _Chain_match:
     def __call__(self, *args: Any, pred: Any = None, preserve_dst: bool = 
False) -> None: ...
 
 class _Chain_max:
-    """`max` — 2 entries sharing this mnemonic; PTX puts their difference in 
the operand list,
+    """`max` — 3 entries sharing this mnemonic; PTX puts their difference in 
the operand list,
     so the call selects one. Shapes: (d, a, b); (d, a, b, c)
     """
 
@@ -1033,10 +1057,12 @@ class _Chain_max:
     s16x2: _Chain_max
     s32: _Chain_max
     s64: _Chain_max
+    s8x4: _Chain_max
     u16: _Chain_max
     u16x2: _Chain_max
     u32: _Chain_max
     u64: _Chain_max
+    u8x4: _Chain_max
     xorsign: _Chain_max
     def __call__(self, *args: Any, pred: Any = None, preserve_dst: bool = 
False) -> None: ...
 
@@ -1080,7 +1106,7 @@ class _Chain_mbarrier:
     def __call__(self, *args: Any, pred: Any = None, preserve_dst: bool = 
False) -> None: ...
 
 class _Chain_min:
-    """`min` — 2 entries sharing this mnemonic; PTX puts their difference in 
the operand list,
+    """`min` — 3 entries sharing this mnemonic; PTX puts their difference in 
the operand list,
     so the call selects one. Shapes: (d, a, b); (d, a, b, c)
     """
 
@@ -1098,10 +1124,12 @@ class _Chain_min:
     s16x2: _Chain_min
     s32: _Chain_min
     s64: _Chain_min
+    s8x4: _Chain_min
     u16: _Chain_min
     u16x2: _Chain_min
     u32: _Chain_min
     u64: _Chain_min
+    u8x4: _Chain_min
     xorsign: _Chain_min
     def __call__(self, *args: Any, pred: Any = None, preserve_dst: bool = 
False) -> None: ...
 
@@ -1417,7 +1445,7 @@ class _Chain_multimem_st_async:
     def __call__(self, addr: Any, b: Any, *args: Any, pred: Any = None) -> 
None: ...
 
 class _Chain_neg:
-    """`neg` — 3 entries sharing this mnemonic; PTX puts their difference in 
the operand list,
+    """`neg` — 4 entries sharing this mnemonic; PTX puts their difference in 
the operand list,
     so the call selects one. Shapes: (d, a)
     """
 
@@ -1431,6 +1459,7 @@ class _Chain_neg:
     s16: _Chain_neg
     s32: _Chain_neg
     s64: _Chain_neg
+    s8x4: _Chain_neg
     def __call__(self, *args: Any, pred: Any = None, preserve_dst: bool = 
False) -> None: ...
 
 class _Chain_not:
@@ -1896,7 +1925,9 @@ class _Chain_shr:
     ) -> None: ...
 
 class _Chain_sin:
-    """`sin` — mode∈{approx}; ftz∈{ftz} (opt); type∈{f32}"""
+    """`sin` — mode∈{approx}; unit∈{2pi} (opt); ftz∈{ftz} (opt); type∈{f32} — 
`.2pi` requires
+    `.ftz` (PTX ISA 9.7.3.18/19, PTX ISA 9.5).
+    """
 
     approx: _Chain_sin
     f32: _Chain_sin
@@ -2124,7 +2155,7 @@ class _Chain_stmatrix:
     def __call__(self, *args: Any, pred: Any = None) -> None: ...
 
 class _Chain_sub:
-    """`sub` — 6 entries sharing this mnemonic; PTX puts their difference in 
the operand list,
+    """`sub` — 7 entries sharing this mnemonic; PTX puts their difference in 
the operand list,
     so the call selects one. Shapes: (d, a, c); (d, a, b)
     """
 
@@ -2143,10 +2174,12 @@ class _Chain_sub:
     s16: _Chain_sub
     s32: _Chain_sub
     s64: _Chain_sub
+    s8x4: _Chain_sub
     sat: _Chain_sub
     u16: _Chain_sub
     u32: _Chain_sub
     u64: _Chain_sub
+    u8x4: _Chain_sub
     def __call__(self, *args: Any, pred: Any = None, preserve_dst: bool = 
False) -> None: ...
 
 class _Chain_szext:
@@ -2180,7 +2213,7 @@ class _Chain_tanh:
     def __call__(self, *args: Any, pred: Any = None, preserve_dst: bool = 
False) -> None: ...
 
 class _Chain_tcgen05:
-    """`tcgen05` — 59 entries sharing this mnemonic; PTX puts their difference 
in the operand
+    """`tcgen05` — 60 entries sharing this mnemonic; PTX puts their difference 
in the operand
     list, so the call selects one. Shapes: (dst, ncols); (taddr, ncols); 
(mbar, cta_mask);
     (mbar); (*__operands); (d_tmem, a_desc, b_desc, idesc, enable_input_d); 
(d_tmem, a_desc,
     b_desc, idesc, enable_input_d, zero_col_mask); (d_tmem, a_tmem, b_desc, 
idesc,
@@ -2194,8 +2227,8 @@ class _Chain_tcgen05:
     (d_tmem, a_desc, b_desc, idesc, sfa_tmem, sfb_tmem, enable_input_d); 
(d_tmem, a_tmem,
     b_desc, idesc, sfa_tmem, sfb_tmem, enable_input_d); (d_tmem, a_desc, 
b_desc,
     sp_meta_tmem, idesc, sfa_tmem, sfb_tmem, enable_input_d); (d_tmem, a_tmem, 
b_desc,
-    sp_meta_tmem, idesc, sfa_tmem, sfb_tmem, enable_input_d); (); (taddr, 
s_desc); (mbar,
-    mask)
+    sp_meta_tmem, idesc, sfa_tmem, sfb_tmem, enable_input_d); (taddr); (); 
(taddr, s_desc);
+    (mbar, mask)
     """
 
     NaN: _Chain_tcgen05
@@ -2242,6 +2275,7 @@ class _Chain_tcgen05:
     cta_group__2: _Chain_tcgen05
     dealloc: _Chain_tcgen05
     decompress__lut__b: _Chain_tcgen05
+    down: _Chain_tcgen05
     exclusive: _Chain_tcgen05
     f32: _Chain_tcgen05
     fence__after_thread_sync: _Chain_tcgen05
@@ -2271,6 +2305,7 @@ class _Chain_tcgen05:
     scale_vec__4X: _Chain_tcgen05
     shared__cluster: _Chain_tcgen05
     shared__cta: _Chain_tcgen05
+    shift: _Chain_tcgen05
     sp: _Chain_tcgen05
     sp__2__colon__4: _Chain_tcgen05
     spcompress: _Chain_tcgen05
@@ -2312,11 +2347,30 @@ class _Chain_tensormap_cp_fenceproxy:
     tensormap__generic: _Chain_tensormap_cp_fenceproxy
     def __call__(self, dst_mem: Any, src_mem: Any, *args: Any, pred: Any = 
None) -> None: ...
 
+class _Chain_tensormap_query:
+    """`tensormap_query` — query∈{alloc_class_large}; space∈{global} (opt); 
width∈{b1024}"""
+
+    alloc_class_large: _Chain_tensormap_query
+    b1024: _Chain_tensormap_query
+    global_: _Chain_tensormap_query
+    def __call__(
+        self,
+        p: Any,
+        addr: Any,
+        *args: Any,
+        pred: Any = None,
+        preserve_dst: bool = False,
+    ) -> None: ...
+
 class _Chain_tensormap_replace:
-    """`tensormap_replace` — 9 entries sharing this mnemonic; PTX puts their 
difference in the
-    operand list, so the call selects one. Shapes: (addr, new_val); (addr, 
ord, new_val)
+    """`tensormap_replace` — 11 entries sharing this mnemonic; PTX puts their 
difference in the
+    operand list, so the call selects one. Shapes: (addr, new_val); (addr); 
(addr, ord,
+    new_val)
     """
 
+    alloc_large: _Chain_tensormap_replace
+    alloc_same: _Chain_tensormap_replace
+    alloc_small: _Chain_tensormap_replace
     b1024: _Chain_tensormap_replace
     b32: _Chain_tensormap_replace
     b64: _Chain_tensormap_replace
@@ -2623,6 +2677,7 @@ class _PTX:
     tanh: _Chain_tanh
     tcgen05: _Chain_tcgen05
     tensormap_cp_fenceproxy: _Chain_tensormap_cp_fenceproxy
+    tensormap_query: _Chain_tensormap_query
     tensormap_replace: _Chain_tensormap_replace
     testp: _Chain_testp
     vote_sync: _Chain_vote_sync
diff --git a/tests/python/tirx/codegen/test_cuda.py 
b/tests/python/tirx/codegen/test_cuda.py
index c7c9900bc..bbf487576 100644
--- a/tests/python/tirx/codegen/test_cuda.py
+++ b/tests/python/tirx/codegen/test_cuda.py
@@ -1282,5 +1282,69 @@ def test_uint32_loop_var_runs_correctly():
     tvm_tirx.testing.run_with_gpu_lock(run_and_check)
 
 
+def _locality_domain_kernel():
+    @T.function
+    def kernel(ID: T.Tensor((128, 32), "int32"), SUP: T.Tensor((128, 32), 
"int32")):
+        T.device_entry(launch=T.cuda.LaunchConfig(grid=128, block=32))
+        bx = T.cuda.block_idx("x")
+        tx = T.cuda.thread_idx("x")
+        ID[bx, tx] = T.cuda.mov_sreg(32, "locality_domain_id")
+        SUP[bx, tx] = T.cuda.mov_sreg(32, "locality_domain_supplemental_sm")
+
+    return kernel
+
+
[email protected](
+    not env.has_nvcc_version(13, 5), reason="locality-domain special registers 
are PTX 9.5"
+)
+def test_mov_sreg_locality_domain_codegen():
+    """PTX ISA 9.5's %locality_domain_* registers (10.36/10.37) read through 
mov_sreg."""
+    src, _ = _get_source(_locality_domain_kernel())
+    assert 'asm volatile("mov.u32 %0, %locality_domain_id;" : "=r"(x));' in src
+    assert 'asm volatile("mov.u32 %0, %locality_domain_supplemental_sm;" : 
"=r"(x));' in src
+
+
[email protected]
[email protected](
+    not env.has_nvcc_version(13, 5), reason="locality-domain special registers 
are PTX 9.5"
+)
[email protected](
+    not env.has_cuda() or env.cuda_arch() is None or int(env.cuda_arch()[3:6]) 
< 100,
+    reason="needs an sm_100+ CUDA GPU",
+)
+def test_mov_sreg_locality_domain_gpu():
+    """The domain id is in [0, LOCALITY_DOMAIN_COUNT) and both registers are 
CTA-uniform."""
+    target = tvm_tirx.target.Target({"kind": "cuda", "arch": env.cuda_arch()})
+    mod = tvm_tirx.compile(
+        tvm_tirx.IRModule({"main": _locality_domain_kernel()}), target=target, 
pipeline="tirx"
+    )
+    domain_count = None
+    try:
+        from cuda.bindings import driver
+
+        attr = 
driver.CUdevice_attribute.CU_DEVICE_ATTRIBUTE_LOCALITY_DOMAIN_COUNT
+        driver.cuInit(0)
+        status, count = driver.cuDeviceGetAttribute(attr, 0)
+        if status == driver.CUresult.CUDA_SUCCESS:
+            domain_count = count
+    except (ImportError, AttributeError):
+        pass
+
+    def run_and_check():
+        dev = tvm_tirx.cuda(0)
+        ids = tvm_tirx.runtime.tensor(np.full((128, 32), -1, np.int32), 
device=dev)
+        sup = tvm_tirx.runtime.tensor(np.full((128, 32), -1, np.int32), 
device=dev)
+        mod(ids, sup)
+        ids, sup = ids.numpy(), sup.numpy()
+        # "For the lifetime of a cluster [resp. CTA], all threads ... read the 
same value."
+        assert (ids == ids[:, :1]).all() and (sup == sup[:, :1]).all()
+        assert ids.min() >= 0
+        if domain_count is not None:
+            assert ids.max() < domain_count
+        assert set(np.unique(sup)) <= {0, 1}
+
+    tvm_tirx.testing.run_with_gpu_lock(run_and_check)
+
+
 if __name__ == "__main__":
     tvm_tirx.testing.main()
diff --git a/tests/python/tirx/codegen/test_ptx_addr.py 
b/tests/python/tirx/codegen/test_ptx_addr.py
index 595bf72d8..fb8f29908 100644
--- a/tests/python/tirx/codegen/test_ptx_addr.py
+++ b/tests/python/tirx/codegen/test_ptx_addr.py
@@ -53,10 +53,10 @@ def test_ptx_addr_registration_and_table_capabilities():
     assert "tirx.ptx.addr" in CODEGEN_REGISTRY
 
     addresses = [slot for entry in TABLE.values() for slot in entry.operands 
if slot.kind == "addr"]
-    assert len(addresses) == 417
-    assert sum(slot.allow_imm_offset for slot in addresses) == 253
-    assert sum(slot.bracket is not None and not slot.allow_imm_offset for slot 
in addresses) == 60
-    assert sum(slot.space == "tmem" and not slot.allow_imm_offset for slot in 
addresses) == 104
+    assert len(addresses) == 499
+    assert sum(slot.allow_imm_offset for slot in addresses) == 309
+    assert sum(slot.bracket is not None and not slot.allow_imm_offset for slot 
in addresses) == 85
+    assert sum(slot.space == "tmem" and not slot.allow_imm_offset for slot in 
addresses) == 105
 
 
 def test_ptx_addr_table_validation_rejects_wrong_operand_classes():
diff --git a/tests/python/tirx/codegen/test_ptx_dialect.py 
b/tests/python/tirx/codegen/test_ptx_dialect.py
index a25cad006..ca6694fb9 100644
--- a/tests/python/tirx/codegen/test_ptx_dialect.py
+++ b/tests/python/tirx/codegen/test_ptx_dialect.py
@@ -437,7 +437,9 @@ def test_ptx_93_sm103a_exact_renderings():
                 action="try_put",
                 api="async",
                 src="shared::cta",
-                
completion=("mbarrier::complete_tx::16B.mbarrier::report::fabric.counted::bytes"),
+                completion="mbarrier::complete_tx::16B",
+                report="mbarrier::report::fabric",
+                counted="counted::bytes",
                 sem="relaxed",
                 scope="sys",
                 type="b128",
@@ -1227,16 +1229,15 @@ def test_ptx_integer_arithmetic_dispatch():
             T.device_entry()
             T.ptx.mad.lo.sat.s32(out[0], T.int32(2), T.int32(3), T.int32(4))
 
-    # The unavailable add.sat forms added in PTX 9.2 are distinct from the
-    # integer lines on which `.sat` has never been legal. Keep both reasons
-    # visible so an invalid spelling is not misreported as an architecture
-    # gate. In particular, sub.sat.u32 is not the counterpart of add.sat.u32.
-    with pytest.raises(ValueError, match="sm_120f-only"):
-
-        @T.function
-        def add_sat_sm120(out: T.Tensor((1,), "uint32")):
-            T.device_entry()
-            T.ptx.add.sat.u32(out[0], T.uint32(2), T.uint32(3))
+    # The add.sat forms PTX 9.2 gave to sm_107f/sm_120f resolve to their 
sm_107f
+    # sibling entry. They are distinct from the integer lines on which `.sat` 
has
+    # never been legal, whose reason stays visible below so an invalid spelling
+    # is not misreported as an architecture gate. In particular, sub.sat.u32 is
+    # not the counterpart of add.sat.u32.
+    @T.function
+    def add_sat_sm107f(out: T.Tensor((1,), "uint32")):
+        T.device_entry()
+        T.ptx.add.sat.u32(out[0], T.uint32(2), T.uint32(3))
 
     with pytest.raises(ValueError, match=r"not on the add\.u64"):
 
@@ -3059,6 +3060,669 @@ def test_ptx_94_sm107_arch_floors_and_delta():
     assert {TABLE[name].cert_arch for name in noftz_siblings} == {"sm_90"}
 
 
+_PTX_95_SYNC_FLOORS = {
+    "fabric_submit_commit_group": "sm_100",
+    "fabric_wait_read": "sm_100",
+    "cp_async_bulk_wait_group_proxy_async": "sm_100",
+    "fence_alias_sync_restrict_read": "sm_90",
+}
+
+
+def test_ptx_95_sync_entries_floors_and_delta():
+    """PTX 9.5 commit/wait/fence lines live in the 9.5 delta at their ISA 
floor."""
+    from tvm_tirx.backend.cuda.ptx.table import _PTX_95_ENTRIES, TABLE
+
+    assert {name: TABLE[name].cert_arch for name in _PTX_95_SYNC_FLOORS} == 
_PTX_95_SYNC_FLOORS
+    assert set(_PTX_95_SYNC_FLOORS) <= {entry.name for entry in 
_PTX_95_ENTRIES}
+    # `.read` and `.sync_restrict::proxy::async` are separate syntax lines.
+    proxy = TABLE["cp_async_bulk_wait_group_proxy_async"]
+    assert "read" not in {slot.name for slot in proxy.slots}
+
+
+def _ptx_95_sync_kernel():
+    @T.function
+    def kernel(A: T.Tensor((32,), "uint32")):
+        T.device_entry(launch=T.cuda.LaunchConfig(grid=1, block=32))
+        tx = T.cuda.thread_idx("x")
+        T.ptx.fabric.submit.commit_group()
+        T.ptx.fabric.wait.sync_restrict__read(0)
+        T.ptx["fabric.wait.sync_restrict::read"](3)
+        T.ptx.fabric.wait.sync_restrict__reads()
+        T.ptx.cp.async_.bulk.wait_group.sync_restrict__proxy__async(2)
+        T.ptx.cp.async_.bulk.wait_group(1)
+        T.ptx.cp.async_.bulk.wait_group.read(0)
+        T.ptx.fence.alias.sync_restrict__read.release.sys()
+        A[tx] = tx + T.uint32(7)
+
+    return kernel
+
+
[email protected](
+    not env.has_nvcc_version(13, 5),
+    reason="fabric commit groups, proxy-restricted bulk waits and the alias 
fence are PTX 9.5",
+)
+@requires_nvcc
+def test_ptx_95_sync_codegen():
+    """The PTX 9.5 lines render verbatim beside their pre-9.5 siblings and 
assemble."""
+    src = _cuda_source(
+        _ptx_95_sync_kernel(), tvm_tirx.target.Target({"kind": "cuda", "arch": 
"sm_100a"})
+    )
+    for text in (
+        "fabric.submit.commit_group;",
+        "fabric.wait.sync_restrict::read 0;",
+        "fabric.wait.sync_restrict::read 3;",
+        "fabric.wait.sync_restrict::reads;",
+        "cp.async.bulk.wait_group.sync_restrict::proxy::async 2;",
+        "cp.async.bulk.wait_group 1;",
+        "cp.async.bulk.wait_group.read 0;",
+        "fence.alias.sync_restrict::read.release.sys;",
+    ):
+        assert text in src, text
+    # Every one of them orders memory without naming an address.
+    assert src.count('"fabric.submit.commit_group;" :  :  : "memory"') == 1
+    _assert_ptxas_ok(src, arch="sm_100a")
+
+
[email protected]
[email protected](
+    not env.has_nvcc_version(13, 5),
+    reason="fabric commit groups, proxy-restricted bulk waits and the alias 
fence are PTX 9.5",
+)
[email protected](
+    not env.has_cuda() or env.cuda_arch() is None or int(env.cuda_arch()[3:6]) 
< 100,
+    reason="needs an sm_100+ CUDA GPU",
+)
+def test_ptx_95_sync_gpu_executes():
+    """With nothing pending, every PTX 9.5 wait returns at once and the kernel 
completes."""
+    mod = tvm_tirx.compile(
+        tvm_tirx.IRModule({"main": _ptx_95_sync_kernel()}), target=TARGET, 
pipeline="tirx"
+    )
+
+    def run_and_check():
+        dev = tvm_tirx.cuda(0)
+        a = tvm_tirx.runtime.tensor(np.zeros(32, dtype=np.uint32), device=dev)
+        mod(a)
+        np.testing.assert_array_equal(a.numpy(), np.arange(32, 
dtype=np.uint32) + 7)
+
+    tvm_tirx.testing.run_with_gpu_lock(run_and_check)
+
+
+def test_ptx_95_sin_cos_2pi_variants():
+    """sin/cos gain PTX 9.5's `.2pi`, legal only together with `.ftz`; lg2/ex2 
do not."""
+    from tvm_tirx.backend.cuda.ptx.render import render_variant
+    from tvm_tirx.backend.cuda.ptx.table import TABLE, tokens_for, variants
+
+    for name in ("sin", "cos"):
+        entry = TABLE[name]
+        assert entry.cert_arch is None  # "Supported on all target 
architectures."
+        assert entry.check({"mode": "approx", "unit": "2pi", "ftz": "", 
"type": "f32"})
+        assert len(variants(entry)) == 3  # {plain, .ftz, .2pi.ftz}
+        opcode = render_variant(
+            entry, tokens_for(entry, mode="approx", unit="2pi", ftz="ftz", 
type="f32")
+        )[0]
+        assert opcode == f"{name}.approx.2pi.ftz.f32"
+    for name in ("lg2", "ex2"):
+        assert "unit" not in {slot.name for slot in TABLE[name].slots}
+
+
+def _ptx_95_sin_cos_2pi_kernel():
+    @T.function
+    def kernel(
+        A: T.Tensor((256,), "float32"),
+        S: T.Tensor((256,), "float32"),
+        C: T.Tensor((256,), "float32"),
+    ):
+        T.device_entry(launch=T.cuda.LaunchConfig(grid=1, block=256))
+        tx = T.cuda.thread_idx("x")
+        s = T.local_scalar("float32")
+        c = T.local_scalar("float32")
+        T.ptx["sin.approx.2pi.ftz.f32"](s, A[tx])
+        T.ptx["cos.approx.2pi.ftz.f32"](c, A[tx])
+        S[tx] = s
+        C[tx] = c
+
+    return kernel
+
+
[email protected](not env.has_nvcc_version(13, 5), reason="sin/cos .2pi is a 
PTX 9.5 form")
+@requires_nvcc
+def test_ptx_95_sin_cos_2pi_codegen():
+    src = _cuda_source(_ptx_95_sin_cos_2pi_kernel())
+    assert "sin.approx.2pi.ftz.f32 %0, %1;" in src
+    assert "cos.approx.2pi.ftz.f32 %0, %1;" in src
+    _assert_ptxas_ok(src)
+
+
[email protected]
[email protected](not env.has_nvcc_version(13, 5), reason="sin/cos .2pi is a 
PTX 9.5 form")
[email protected](not env.has_cuda(), reason="CUDA GPU not available")
+def test_ptx_95_sin_cos_2pi_gpu_accuracy():
+    """`.2pi` reads its input in cycles; check the ISA's stated absolute error 
bounds.
+
+    9.7.3.18/19: for cycles as input the maximum absolute error is 2^-20 (sin) 
and
+    2^-19.9 (cos) over [-1, 1], and 2^-14.2 for both over [-50, 50].
+    """
+    mod = tvm_tirx.compile(
+        tvm_tirx.IRModule({"main": _ptx_95_sin_cos_2pi_kernel()}), 
target=TARGET, pipeline="tirx"
+    )
+    rng = np.random.default_rng(0)
+    cases = (
+        (np.float32(1.0), 2.0**-20, 2.0**-19.9),
+        (np.float32(50.0), 2.0**-14.2, 2.0**-14.2),
+    )
+
+    def run_and_check():
+        dev = tvm_tirx.cuda(0)
+        for bound, sin_tol, cos_tol in cases:
+            x = rng.uniform(-bound, bound, 256).astype(np.float32)
+            # Exact turns and quarter turns, where the result is exactly 0 or 
+-1.
+            x[:9] = np.array([0, 0.25, 0.5, 0.75, 1, -0.25, -0.5, -0.75, -1], 
dtype=np.float32)
+            a = tvm_tirx.runtime.tensor(x, device=dev)
+            s = tvm_tirx.runtime.tensor(np.zeros(256, dtype=np.float32), 
device=dev)
+            c = tvm_tirx.runtime.tensor(np.zeros(256, dtype=np.float32), 
device=dev)
+            mod(a, s, c)
+            turns = 2.0 * np.pi * x.astype(np.float64)
+            assert np.max(np.abs(s.numpy() - np.sin(turns))) <= sin_tol
+            assert np.max(np.abs(c.numpy() - np.cos(turns))) <= cos_tol
+
+    tvm_tirx.testing.run_with_gpu_lock(run_and_check)
+
+
+_PTX_95_TENSORMAP_ENTRIES = (
+    "tensormap_replace_address_alloc_class",
+    "tensormap_replace_alloc_class",
+    "tensormap_query",
+)
+
+
+def test_ptx_95_tensormap_alloc_class_entries():
+    """The 9.5 allocation-class lines certify at sm_90a; `.alloc_same` needs a 
field."""
+    from tvm_tirx.backend.cuda.ptx.table import _PTX_95_ENTRIES, TABLE
+
+    delta = {entry.name for entry in _PTX_95_ENTRIES}
+    for name in _PTX_95_TENSORMAP_ENTRIES:
+        assert name in delta and TABLE[name].cert_arch == "sm_90a"
+
+    def alloc_choices(name):
+        (slot,) = [s for s in TABLE[name].slots if s.name == "alloc_class"]
+        return slot.choices
+
+    assert alloc_choices("tensormap_replace_address_alloc_class") == (
+        "alloc_small",
+        "alloc_large",
+        "alloc_same",
+    )
+    assert alloc_choices("tensormap_replace_alloc_class") == ("alloc_small", 
"alloc_large")
+
+
+def _ptx_95_tensormap_alloc_class_kernel():
+    @T.function
+    def kernel(TM: T.Tensor((32,), "uint32"), OUT: T.Tensor((6,), "uint32")):
+        T.device_entry(launch=T.cuda.LaunchConfig(grid=1, block=1))
+        p = T.local_scalar("uint32")
+        smem = T.alloc_tensor(ty_args=[T.Tensor((32,), "uint32", 
scope="shared")])
+        tmap = TM.ptr_to([0])
+        query = T.ptx["tensormap_query.alloc_class_large.global.b1024"]
+        # Field-less lines.
+        T.ptx["tensormap_replace.tile.alloc_large.global.b1024"](tmap)
+        query(p, tmap)
+        OUT[0] = p
+        T.ptx["tensormap_replace.tile.alloc_small.global.b1024"](tmap)
+        query(p, tmap)
+        OUT[1] = p
+        # `.global_address` with an explicit class.
+        
T.ptx["tensormap_replace.tile.global_address.alloc_large.global.b1024.b64"](
+            tmap, T.uint64(0x10000)
+        )
+        query(p, tmap)
+        OUT[2] = p
+        # PTX 9.5 clarification: `.global_address` without `.alloc_class` sets 
Small.
+        T.ptx["tensormap_replace.tile.global_address.global.b1024.b64"](tmap, 
T.uint64(0x20000))
+        query(p, tmap)
+        OUT[3] = p
+        # `.alloc_same` keeps whatever class the object already has.
+        T.ptx["tensormap_replace.tile.alloc_large.global.b1024"](tmap)
+        
T.ptx["tensormap_replace.tile.global_address.alloc_same.global.b1024.b64"](
+            tmap, T.uint64(0x30000)
+        )
+        query(p, tmap)
+        OUT[4] = p
+        # Generic addressing reads the same object.
+        T.ptx["tensormap_query.alloc_class_large.b1024"](p, tmap)
+        OUT[5] = p
+        # The shared::cta replaces assemble and run; the query has no 
shared::cta
+        # helper (see the table), so their effect is not observed here.
+        
T.ptx["tensormap_replace.tile.alloc_small.shared::cta.b1024"](smem.ptr_to([0]))
+        
T.ptx["tensormap_replace.tile.global_address.alloc_large.shared::cta.b1024.b64"](
+            smem.ptr_to([0]), T.uint64(0x40000)
+        )
+
+    return kernel
+
+
[email protected](
+    not env.has_nvcc_version(13, 5), reason="tensor-map allocation classes are 
PTX 9.5"
+)
+@requires_nvcc
+def test_ptx_95_tensormap_alloc_class_codegen():
+    src = _cuda_source(
+        _ptx_95_tensormap_alloc_class_kernel(),
+        tvm_tirx.target.Target({"kind": "cuda", "arch": "sm_90a"}),
+    )
+    for text in (
+        "tensormap.replace.tile.alloc_large.global.b1024 [%0];",
+        "tensormap.replace.tile.alloc_small.shared::cta.b1024 [%0];",
+        
"tensormap.replace.tile.global_address.alloc_large.shared::cta.b1024.b64 [%0], 
%1;",
+        "tensormap.replace.tile.global_address.alloc_large.global.b1024.b64 
[%0], %1;",
+        "tensormap.replace.tile.global_address.alloc_same.global.b1024.b64 
[%0], %1;",
+        "tensormap.query.alloc_class_large.global.b1024 pd0, [%1];",
+        "tensormap.query.alloc_class_large.b1024 pd0, [%1];",
+    ):
+        assert text in src, text
+    _assert_ptxas_ok(src, arch="sm_90a")
+
+
[email protected]
[email protected](
+    not env.has_nvcc_version(13, 5), reason="tensor-map allocation classes are 
PTX 9.5"
+)
[email protected](
+    not env.has_cuda() or env.cuda_arch() is None or int(env.cuda_arch()[3:6]) 
< 90,
+    reason="needs an sm_90+ CUDA GPU",
+)
+def test_ptx_95_tensormap_alloc_class_gpu():
+    """Each replace leaves the class the ISA says, as observed by the query."""
+    mod = tvm_tirx.compile(
+        tvm_tirx.IRModule({"main": _ptx_95_tensormap_alloc_class_kernel()}),
+        target=TARGET,
+        pipeline="tirx",
+    )
+
+    def run_and_check():
+        dev = tvm_tirx.cuda(0)
+        tm = tvm_tirx.runtime.tensor(np.zeros(32, dtype=np.uint32), device=dev)
+        out = tvm_tirx.runtime.tensor(np.full(6, 7, dtype=np.uint32), 
device=dev)
+        mod(tm, out)
+        np.testing.assert_array_equal(out.numpy(), [1, 0, 1, 0, 1, 1])
+
+    tvm_tirx.testing.run_with_gpu_lock(run_and_check)
+
+
+_SM107F_PACKED_INT_OPCODES = {
+    "add_int_sm107f": {
+        "add.u8x4",
+        "add.s8x4",
+        "add.sat.u8x4",
+        "add.sat.s8x4",
+        "add.sat.u32",
+        "add.sat.u16x2",
+        "add.sat.s16x2",
+    },
+    "sub_int_sm107f": {"sub.u8x4", "sub.s8x4", "sub.sat.u8x4", "sub.sat.s8x4"},
+    "neg_int_sm107f": {"neg.s8x4"},
+    "min_int8x4": {"min.u8x4", "min.s8x4", "min.relu.s8x4"},
+    "max_int8x4": {"max.u8x4", "max.s8x4", "max.relu.s8x4"},
+}
+
+
+def test_ptx_sm107f_packed_int_entries():
+    """The sm_107f/sm_120f packed-integer lines are sm_107f siblings, exactly 
these opcodes."""
+    from tvm_tirx.backend.cuda.ptx.render import render_variant
+    from tvm_tirx.backend.cuda.ptx.table import TABLE, variants
+
+    for name, expected in _SM107F_PACKED_INT_OPCODES.items():
+        entry = TABLE[name]
+        assert entry.cert_arch == "sm_107f", name
+        assert {render_variant(entry, tokens)[0] for tokens in 
variants(entry)} == expected
+    # The families they extend keep their pre-sm_107 floors.
+    assert TABLE["add_int"].cert_arch == "sm_90"
+    assert TABLE["sub_int"].cert_arch is None and TABLE["neg_int"].cert_arch 
is None
+
+
+def _sm107f_packed_int_kernel():
+    @T.function
+    def kernel(
+        A: T.Tensor((256,), "uint32"),
+        B: T.Tensor((256,), "uint32"),
+        D: T.Tensor((16, 256), "uint32"),
+    ):
+        T.device_entry(launch=T.cuda.LaunchConfig(grid=1, block=256))
+        tx = T.cuda.thread_idx("x")
+        a = A[tx]
+        b = B[tx]
+        d = T.local_scalar("uint32")
+        T.ptx.add.u8x4(d, a, b)
+        D[0, tx] = d
+        T.ptx.add.sat.u8x4(d, a, b)
+        D[1, tx] = d
+        T.ptx.add.s8x4(d, a, b)
+        D[2, tx] = d
+        T.ptx.add.sat.s8x4(d, a, b)
+        D[3, tx] = d
+        T.ptx.sub.u8x4(d, a, b)
+        D[4, tx] = d
+        T.ptx.sub.sat.u8x4(d, a, b)
+        D[5, tx] = d
+        T.ptx.sub.sat.s8x4(d, a, b)
+        D[6, tx] = d
+        T.ptx.neg.s8x4(d, a)
+        D[7, tx] = d
+        T.ptx.min.u8x4(d, a, b)
+        D[8, tx] = d
+        T.ptx.max.s8x4(d, a, b)
+        D[9, tx] = d
+        T.ptx.min.relu.s8x4(d, a, b)
+        D[10, tx] = d
+        T.ptx.max.relu.s8x4(d, a, b)
+        D[11, tx] = d
+        T.ptx.add.sat.u32(d, a, b)
+        D[12, tx] = d
+        T.ptx.add.sat.u16x2(d, a, b)
+        D[13, tx] = d
+        T.ptx.add.sat.s16x2(d, a, b)
+        D[14, tx] = d
+        T.ptx.add.u32(d, a, b)  # the sm_90 `add_int` line, still selected
+        D[15, tx] = d
+
+    return kernel
+
+
[email protected](not env.has_nvcc_version(13, 4), reason="need nvcc >= 13.4 
for sm_107f")
+@requires_nvcc
+def test_ptx_sm107f_packed_int_codegen():
+    src = _cuda_source(
+        _sm107f_packed_int_kernel(), tvm_tirx.target.Target({"kind": "cuda", 
"arch": "sm_107f"})
+    )
+    for opcode in (
+        "add.u8x4",
+        "add.sat.u8x4",
+        "add.s8x4",
+        "add.sat.s8x4",
+        "sub.u8x4",
+        "sub.sat.u8x4",
+        "sub.sat.s8x4",
+        "neg.s8x4",
+        "min.u8x4",
+        "max.s8x4",
+        "min.relu.s8x4",
+        "max.relu.s8x4",
+        "add.sat.u32",
+        "add.sat.u16x2",
+        "add.sat.s16x2",
+        "add.u32",
+    ):
+        assert f'"{opcode} %0, %1' in src, opcode
+    _assert_ptxas_ok(src, arch="sm_107f")
+
+
+def _lanes(x, dtype, n):
+    return x.view(dtype).reshape(-1, n).astype(np.int64)
+
+
+def _pack(lanes, dtype):
+    info = np.iinfo(dtype)
+    wrapped = (lanes - info.min) % (info.max - info.min + 1) + info.min
+    return wrapped.astype(dtype).reshape(-1).view(np.uint32)
+
+
[email protected]
[email protected](not env.has_nvcc_version(13, 4), reason="need nvcc >= 13.4 
for sm_107f")
[email protected](
+    env.cuda_arch() not in ("sm_107a", "sm_120a", "sm_121a"),
+    reason="needs an sm_107/sm_12x GPU (the packed int lines are 
sm_107f/sm_120f-only)",
+)
+def test_ptx_sm107f_packed_int_gpu():
+    """Lane-wise wrap-around and saturation semantics against numpy."""
+    mod = tvm_tirx.compile(
+        tvm_tirx.IRModule({"main": _sm107f_packed_int_kernel()}), 
target=TARGET, pipeline="tirx"
+    )
+    rng = np.random.default_rng(0)
+    a = rng.integers(0, 2**32, 256, dtype=np.uint64).astype(np.uint32)
+    b = rng.integers(0, 2**32, 256, dtype=np.uint64).astype(np.uint32)
+    # Edge lanes: overflow/underflow in every lane, and the most negative 
value.
+    a[:4] = [0xFFFFFFFF, 0x80808080, 0x7F7F7F7F, 0x00000000]
+    b[:4] = [0x01010101, 0x80808080, 0x01010101, 0xFFFFFFFF]
+
+    def clamp(x, dtype):
+        info = np.iinfo(dtype)
+        return np.clip(x, info.min, info.max)
+
+    u8a, u8b = _lanes(a, np.uint8, 4), _lanes(b, np.uint8, 4)
+    s8a, s8b = _lanes(a, np.int8, 4), _lanes(b, np.int8, 4)
+    u16a, u16b = _lanes(a, np.uint16, 2), _lanes(b, np.uint16, 2)
+    s16a, s16b = _lanes(a, np.int16, 2), _lanes(b, np.int16, 2)
+    a64, b64 = a.astype(np.int64), b.astype(np.int64)
+    expected = [
+        _pack(u8a + u8b, np.uint8),
+        _pack(clamp(u8a + u8b, np.uint8), np.uint8),
+        _pack(s8a + s8b, np.int8),
+        _pack(clamp(s8a + s8b, np.int8), np.int8),
+        _pack(u8a - u8b, np.uint8),
+        _pack(clamp(u8a - u8b, np.uint8), np.uint8),
+        _pack(clamp(s8a - s8b, np.int8), np.int8),
+        _pack(-s8a, np.int8),
+        _pack(np.minimum(u8a, u8b), np.uint8),
+        _pack(np.maximum(s8a, s8b), np.int8),
+        _pack(np.maximum(np.minimum(s8a, s8b), 0), np.int8),
+        _pack(np.maximum(np.maximum(s8a, s8b), 0), np.int8),
+        np.minimum(a64 + b64, 2**32 - 1).astype(np.uint32),
+        _pack(clamp(u16a + u16b, np.uint16), np.uint16),
+        _pack(clamp(s16a + s16b, np.int16), np.int16),
+        ((a64 + b64) % 2**32).astype(np.uint32),
+    ]
+
+    def run_and_check():
+        dev = tvm_tirx.cuda(0)
+        d = tvm_tirx.runtime.tensor(np.zeros((16, 256), dtype=np.uint32), 
device=dev)
+        mod(tvm_tirx.runtime.tensor(a, device=dev), tvm_tirx.runtime.tensor(b, 
device=dev), d)
+        got = d.numpy()
+        for row, want in enumerate(expected):
+            np.testing.assert_array_equal(got[row], want, err_msg=f"row {row}")
+
+    tvm_tirx.testing.run_with_gpu_lock(run_and_check)
+
+
+@requires_nvcc
[email protected](not env.has_nvcc_version(13, 4), reason="need nvcc >= 13.4 
for sm_107a")
+def test_ptx_tcgen05_shift_codegen():
+    """tcgen05.shift takes a bracketed tmem address and exists only on `a` 
targets."""
+    from tvm_tirx.backend.cuda.ptx.render import render_variant
+    from tvm_tirx.backend.cuda.ptx.table import TABLE, variants
+
+    entry = TABLE["tcgen05_shift"]
+    assert entry.cert_arch == "sm_100a"
+    assert {render_variant(entry, tokens)[0] for tokens in variants(entry)} == 
{
+        "tcgen05.shift.cta_group::1.down",
+        "tcgen05.shift.cta_group::2.down",
+    }
+
+    @T.function
+    def kernel(A: T.Tensor((32,), "uint32")):
+        T.device_entry(launch=T.cuda.LaunchConfig(grid=1, block=32))
+        tx = T.cuda.thread_idx("x")
+        if tx == 0:
+            T.ptx.tcgen05.shift.cta_group__1.down(A[0])
+        A[tx] = A[tx]
+
+    for arch in ("sm_100a", "sm_107a"):
+        src = _cuda_source(kernel, tvm_tirx.target.Target({"kind": "cuda", 
"arch": arch}))
+        assert "tcgen05.shift.cta_group::1.down [%0];" in src
+        _assert_ptxas_ok(src, arch=arch)
+
+
+def test_ptx_every_modifier_token_is_spellable():
+    """No modifier choice may contain a dot: `T.ptx[...]` and attribute chains 
split on it."""
+    from tvm_tirx.backend.cuda.ptx.table import TABLE
+
+    dotted = sorted(
+        (entry.name, slot.name, choice)
+        for entry in TABLE.values()
+        for slot in entry.slots
+        for choice in slot.choices
+        if "." in choice
+    )
+    assert not dotted, dotted
+
+
[email protected](not env.has_nvcc_version(13, 3), reason="fabric operations 
are PTX 9.3")
+@requires_nvcc
+def test_ptx_fabric_called_through_string_form():
+    """The PTX 9.3 fabric operations are callable by their exact PTX text."""
+
+    @T.function
+    def kernel(A: T.Tensor((64,), "uint32"), P: T.Tensor((8,), "uint64")):
+        T.device_entry(launch=T.cuda.LaunchConfig(grid=1, block=32))
+        tx = T.cuda.thread_idx("x")
+        smem = T.alloc_tensor(ty_args=[T.Tensor((64,), "uint32", 
scope="shared")])
+        mbar = T.alloc_tensor(ty_args=[T.Tensor((2,), "uint64", 
scope="shared")])
+        if tx == 0:
+            T.ptx[
+                "fabric.try_get.async.shared::cta"
+                
".mbarrier::complete_tx::bytes.mbarrier::report::fabric.relaxed.sys.b128"
+            ](smem.ptr_to([0]), A[0], P[1], T.uint32(16), mbar.ptr_to([0]))
+            T.ptx[
+                "fabric.try_put.async.shared::cta.mbarrier::complete_tx::16B"
+                ".mbarrier::report::fabric.counted::bytes.relaxed.sys.b128"
+            ](A[0], P[1], P[2], smem.ptr_to([0]), T.uint32(16), 
mbar.ptr_to([0]))
+            T.ptx.fabric.submit()
+        A[tx] = A[tx]
+
+    src = _cuda_source(kernel, tvm_tirx.target.Target({"kind": "cuda", "arch": 
"sm_103a"}))
+    assert (
+        
"fabric.try_get.async.shared::cta.mbarrier::complete_tx::bytes.mbarrier::report::fabric"
+        ".relaxed.sys.b128 [%0], [%1, %2], %3, [%4];" in src
+    )
+    assert ".counted::bytes.relaxed.sys.b128 [%0, %1, %2], [%3], %4, [%5];" in 
src
+    _assert_ptxas_ok(src, arch="sm_103a")
+
+
+def test_ptx_fabric_tensor_and_atom_entries():
+    """PTX 9.4's fabric tensor operations and try_atom, in the 9.4 delta at 
their floors."""
+    from tvm_tirx.backend.cuda.ptx.render import render_variant
+    from tvm_tirx.backend.cuda.ptx.table import _PTX_94_ENTRIES, TABLE, 
tokens_for, variants
+
+    delta = {entry.name for entry in _PTX_94_ENTRIES}
+    tensor = {name for name in TABLE if name.startswith("fabric_try_") and 
"_tensor" in name}
+    assert len(tensor) == 25  # get + put/red x {plain, counted}, x 5 override 
shapes
+    assert tensor <= delta and {TABLE[name].cert_arch for name in tensor} == 
{"sm_107f"}
+    assert "fabric_try_atom" in delta and TABLE["fabric_try_atom"].cert_arch 
== "sm_100"
+
+    atom = TABLE["fabric_try_atom"]
+    pairs = {
+        tuple(render_variant(atom, tokens)[0].rsplit(".", 2)[-2:]) for tokens 
in variants(atom)
+    }
+    assert {("add", "f16x2"), ("cas", "b128")} <= pairs
+    # Off the ISA grid; ptxas accepts `.min.s32`, but the table follows the 
ISA.
+    assert not {("min", "f32"), ("add", "b32"), ("min", "s32")} & pairs
+    assert len(pairs) == 26  # 3x2 bitwise + 2x3 exch/cas + 2x4 min/max + 6 add
+
+    red = TABLE["fabric_try_red_tensor"]
+    opcode = render_variant(
+        red,
+        tokens_for(
+            red,
+            action="try_red",
+            api="async",
+            unit="tensor",
+            dim="2d",
+            src="shared::cta",
+            load_mode="tile",
+            completion="mbarrier::complete_tx::16B",
+            report="mbarrier::report::fabric",
+            override_address="override::global_address",
+            sem="relaxed",
+            scope="sys",
+            redop="max",
+        ),
+    )[0]
+    assert opcode.endswith(".override::global_address.relaxed.sys.max")  # no 
element type
+
+
[email protected](not env.has_nvcc_version(13, 4), reason="fabric tensor 
operations are PTX 9.4")
+@requires_nvcc
+def test_ptx_fabric_tensor_codegen():
+    """Each override shape renders its handle bracket and assembles at 
sm_107f."""
+
+    @T.function
+    def kernel(A: T.Tensor((64,), "uint32"), P: T.Tensor((8,), "uint64")):
+        T.device_entry(launch=T.cuda.LaunchConfig(grid=1, block=32))
+        tx = T.cuda.thread_idx("x")
+        smem = T.alloc_tensor(ty_args=[T.Tensor((64,), "uint32", 
scope="shared")])
+        mbar = T.alloc_tensor(ty_args=[T.Tensor((2,), "uint64", 
scope="shared")])
+        if tx == 0:
+            tmap = P.ptr_to([0])
+            le = A[0]
+            off = P[1]
+            c0 = T.int32(A[1])
+            c1 = T.int32(A[2])
+            T.ptx[
+                "fabric.try_get.async.tensor.2d.shared::cta.tile"
+                ".mbarrier::complete_tx::bytes.mbarrier::report::fabric"
+                ".override::global_address.relaxed.sys.b128"
+            ](smem.ptr_to([0]), tmap, le, off, c0, c1, mbar.ptr_to([0]))
+            T.ptx[
+                "fabric.try_get.async.tensor.1d.shared::cta.tile"
+                
".mbarrier::complete_tx::bytes.mbarrier::report::fabric.L2::cache_hint"
+                
".override::global_address.override::global_dim.relaxed.sys.b128"
+            ](smem.ptr_to([0]), tmap, le, off, T.uint16(64), c0, 
mbar.ptr_to([0]), P[2])
+            T.ptx[
+                "fabric.try_put.async.tensor.multimem.3d.shared::cta.tile"
+                
".mbarrier::complete_tx::16B.mbarrier::report::fabric.counted::bytes"
+                
".override::global_address.override::global_dim_stride.relaxed.sys.b128"
+            ](
+                tmap, le, off, P[3],
+                T.uint16(8), T.uint16(8), T.uint16(8),
+                T.uint32(256), T.uint32(2048),
+                T.uint16(0),
+                c0, c1, c0,
+                smem.ptr_to([0]), T.uint32(256), mbar.ptr_to([0]),
+            )  # fmt: skip
+            T.ptx[
+                "fabric.try_red.async.tensor.2d.shared::cta.tile"
+                ".mbarrier::complete_tx::16B.mbarrier::report::fabric"
+                ".override::global_address.relaxed.sys.add"
+            ](tmap, le, off, c0, c1, smem.ptr_to([0]), T.uint32(256), 
mbar.ptr_to([0]))
+        A[tx] = A[tx]
+
+    src = _cuda_source(kernel, tvm_tirx.target.Target({"kind": "cuda", "arch": 
"sm_107f"}))
+    for text in (
+        ".override::global_address.relaxed.sys.b128 [%0], [%1, %2, %3, {%4, 
%5}], [%6];",
+        ".override::global_dim.relaxed.sys.b128 [%0], [%1, %2, %3, {%4}, 
{%5}], [%6], %7;",
+        
".counted::bytes.override::global_address.override::global_dim_stride.relaxed.sys.b128
 "
+        "[%0, %1, %2, %3, {%4, %5, %6}, {%7, %8}, %9, {%10, %11, %12}], [%13], 
%14, [%15];",
+        ".override::global_address.relaxed.sys.add [%0, %1, %2, {%3, %4}], 
[%5], %6, [%7];",
+    ):
+        assert text in src, text
+    _assert_ptxas_ok(src, arch="sm_107f")
+
+
+@requires_nvcc
[email protected](not env.has_nvcc_version(13, 4), reason="fabric.try_atom 
is PTX 9.4")
+def test_ptx_fabric_try_atom_codegen():
+    @T.function
+    def kernel(A: T.Tensor((64,), "uint32"), P: T.Tensor((8,), "uint64")):
+        T.device_entry(launch=T.cuda.LaunchConfig(grid=1, block=32))
+        tx = T.cuda.thread_idx("x")
+        smem = T.alloc_tensor(ty_args=[T.Tensor((64,), "uint32", 
scope="shared")])
+        mbar = T.alloc_tensor(ty_args=[T.Tensor((2,), "uint64", 
scope="shared")])
+        if tx == 0:
+            T.ptx[
+                "fabric.try_atom.async.shared::cta"
+                
".mbarrier::complete_tx::16B.mbarrier::report::fabric.relaxed.sys.add.bf16x2"
+            ](A[0], P[1], smem.ptr_to([0]), smem.ptr_to([16]), 
mbar.ptr_to([0]))
+            T.ptx[
+                "fabric.try_atom.async.shared::cta"
+                
".mbarrier::complete_tx::16B.mbarrier::report::fabric.L2::cache_hint"
+                ".relaxed.sys.cas.b128"
+            ](A[0], P[1], smem.ptr_to([0]), smem.ptr_to([16]), 
mbar.ptr_to([0]), P[2])
+        A[tx] = A[tx]
+
+    src = _cuda_source(kernel, tvm_tirx.target.Target({"kind": "cuda", "arch": 
"sm_100a"}))
+    assert ".relaxed.sys.add.bf16x2 [%0, %1], [%2], [%3], [%4];" in src
+    assert ".L2::cache_hint.relaxed.sys.cas.b128 [%0, %1], [%2], [%3], [%4], 
%5;" in src
+    _assert_ptxas_ok(src, arch="sm_100a")
+
+
 def test_ptx_tcgen05_mapa_address_rendering():
     """Optional state spaces select generic versus shared address carriers."""
     from tvm_tirx.backend.cuda.ptx.render import render_variant
@@ -4159,7 +4823,7 @@ def test_ptx_all_variants_render_unique():
             _, helper, _ = render_variant(entry, *args, 
addr_offsets=addr_offsets)
             assert helper not in names, f"address-offset helper name 
collision: {helper}"
             names.add(helper)
-    assert total == 762178  # update when the table grows or a ptxas gap 
narrows it
+    assert total == 791291  # update when the table grows or a ptxas gap 
narrows it
 
 
 def test_ptx_no_instruction_registered_twice():
@@ -4373,14 +5037,20 @@ _PTX_93_SM103A_PHASE_ENTRIES = frozenset(
 
 def _ptx_93_sm103a_manifest():
     """Yield every rendering introduced or extended by PTX 9.3 for SM103a."""
-    from tvm_tirx.backend.cuda.ptx.table import _PTX_94_ENTRIES, TABLE, mods, 
renderings
+    from tvm_tirx.backend.cuda.ptx.table import (
+        _PTX_94_ENTRIES,
+        _PTX_95_ENTRIES,
+        TABLE,
+        mods,
+        renderings,
+    )
 
-    ptx94 = {entry.name for entry in _PTX_94_ENTRIES}
+    later = {entry.name for entry in (*_PTX_94_ENTRIES, *_PTX_95_ENTRIES)}
     for name, entry in TABLE.items():
-        if name in ptx94:
-            # The 9.4 delta (sm_107f floors, including the cp.async.bulk
-            # `*_report` entries) is certified at its own arch by
-            # test_ptx_all_helpers_certify, not at sm_103a here.
+        if name in later:
+            # The 9.4 and 9.5 deltas (sm_107f floors, the cp.async.bulk
+            # `*_report` entries, fabric submit groups) are certified at their
+            # own arch by test_ptx_all_helpers_certify, not at sm_103a here.
             continue
         for rendering in renderings(entry):
             tokens = rendering[0]

Reply via email to