tqchen commented on code in PR #394: URL: https://github.com/apache/tvm-ffi/pull/394#discussion_r2678758130
########## docs/concepts/func_module.rst: ########## @@ -0,0 +1,584 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +Function and Module +=================== + +TVM-FFI provides a unified and ABI-stable calling convention that enables +cross-language function calls between C++, Python, Rust and other languages. +Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be: + +- **Invoked with type-erased arguments** using a stable C ABI +- **Registered globally** and accessed by name across languages +- **Exported as C symbols** for efficient codegen and linking +- **Wrapped in modules** for dynamic loading and namespace isolation + +This tutorial covers everything you need to know about defining, registering, and calling +TVM-FFI functions, as well as working with modules. + +Glossary +-------- + +TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType` + A stable C calling convention where every function is represented by a single signature, + which enables type-erased, cross-language function calls. + This calling convention is used across all TVM-FFI function calls at the ABI boundary. + See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction. + +TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function` + A reference-counted function object and its managed reference, which wraps any callable, + including language-agnostic functions and lambdas (C++, Python, Rust, etc.), + member functions, external C symbols, and other callable objects, + all sharing the same calling convention. + +TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module` + A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows), + or statically linked to the current executable. + +Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func` + A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects + and their metadata (name, docs, signatures, etc.) for cross-language access. + Functions in the registry are called **global functions**. + +Common Usage +------------ + +TVM-FFI C Symbols +~~~~~~~~~~~~~~~~~ + +**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export +a function as a C symbol that follows the TVM-FFI ABI: + +.. code-block:: cpp + + static int AddTwo(int x) { return x + 2; } + + TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo) + +This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library, +and later loaded and called via :py:func:`tvm_ffi.load_module`: + +.. code-block:: python + + import tvm_ffi + + mod = tvm_ffi.load_module("path/to/library.so") + result = mod.add_two(40) # -> 42 + +**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol` +is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`. +See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow. + + +Global Functions +~~~~~~~~~~~~~~~~ + +**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to +register a function: + +.. code-block:: cpp + + #include <tvm/ffi/tvm_ffi.h> + + static int AddOne(int x) { return x + 1; } + + TVM_FFI_STATIC_INIT_BLOCK() { + namespace refl = tvm::ffi::reflection; + refl::GlobalDef() + .def("my_ext.add_one", AddOne, "Add one to the input"); + } + +:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens +during library initialization. The registered function is then accessible from +Python under the name ``my_ext.add_one``. + +In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function: + +.. code-block:: python + + import tvm_ffi + + @tvm_ffi.register_global_func("my_ext.add_one") + def add_one(x: int) -> int: + return x + 1 + +**Retrieve a global function**. After registration, functions are accessible by name. + +In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function: + +.. code-block:: python + + import tvm_ffi + + # Get a function from the global registry + add_one = tvm_ffi.get_global_func("my_ext.add_one") + result = add_one(41) # -> 42 + +In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function: + +.. code-block:: cpp + + ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one"); + int result = func(41) // -> 42 + + +Create Functions +~~~~~~~~~~~~~~~~ + +**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped` +or :cpp:class:`tvm::ffi::TypedFunction`'s constructor. + +.. code-block:: cpp + + // Create type-erased function: add_type_erased + ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) { + return x + y; + }); + + // Create a typed function: add_typed + ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) { + return x + y; + }; + + // Converts typed function to type-erased function + ffi::Function generic = add; + +**From Python**. An arbitrary Python :py:class:`Callable <collections.abc.Callable>` can be automatically converted +to a :py:class:`tvm_ffi.Function` at ABI boundary. The example below shows that in function ``my_ext.bind``: + +- Its input ``func`` is automatically converted to :py:class:`tvm_ffi.Function`; +- Its output, a lambda function is automatically converted to :py:class:`tvm_ffi.Function`. + +.. code-block:: python + + import tvm_ffi + + @tvm_ffi.register_global_func("my_ext.bind") + def bind(func, x): + assert isinstance(func, tvm_ffi.Function) + return lambda *args: func(x, *args) # converted to `tvm_ffi.Function` + + def add_x_y(x, y): + return x + y + + func_bind = tvm_ffi.get_global_func("my_ext.bind") + add_y = func_bind(add_x_y, 1) # bind x = 1 + assert isinstance(add_y, tvm_ffi.Function) + print(add_y(2)) # -> 3 + + +:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`: + +.. code-block:: python + + import tvm_ffi + + def add(x, y): + return x + y + + func_add = tvm_ffi.convert(add) + print(func_add(1, 2)) + + +C ABI Calling Convention +------------------------ + +.. important:: + + This section focuses on in-depth of TVM-FFI's stable C ABI definition, + describing it in pure C for precision and clarity. + + Compiler code generation may follow similar patterns to call functions with TVM-FFI ABI, + or to generate functions that conform to TVM-FFI ABI. + +All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature, +which provides a stable C ABI for cross-language calls. The C calling convention is defined as: + +.. code-block:: cpp + + int tvm_ffi_c_abi( + void* handle, // Resource handle + const TVMFFIAny* args, // Inputs (non-owning) + int num_args, // Number of arguments + TVMFFIAny* result // Output (owning) + ); + +.. note:: + As part of the contract, the return value ``result`` should be zero-initialized by the caller. + +The returning integer is **error code**, which indicates + +- ``0``: Success +- ``-1``: Error occurred, retrieve via :cpp:func:`TVMFFIErrorMoveFromRaised` +- ``-2``: Frontend error (e.g., Python exception) + + +Input and Output Arguments +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The input and output arguments are passed as :cpp:type:`TVMFFIAny` pointers. + +- Input arguments are an array of ``num_args`` non-owning :cpp:type:`tvm::ffi::AnyView`; +- Output argument is an owning :cpp:type:`tvm::ffi::Any`, which the caller has to guarantee to be zero-initialized. + +See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`. + +Retrieve Errors in C +~~~~~~~~~~~~~~~~~~~~ + +When a TVM-FFI function returns a non-zero code, it indicates an error occurred, +and the error object is stored in the TLS slot. This section shows how to retrieve +the error object and print the error message and backtrace. + +.. hint:: + Compiler code generation may use similar patterns to call any function with TVM-FFI ABI. + + +**Print an Error**. The example code below shows how to print an error message and backtrace. + +.. code-block:: cpp + + #include <tvm/ffi/c_api.h> + + static void PrintError(TVMFFIObject* err) { + TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject)); + fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data); + if (cell->backtrace.size) { + fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data); + } + } + +The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure, +which contains the error kind, message, and backtrace, which can be accessed +by skipping the :cpp:type:`TVMFFIObject` header with pointer arithmetics. + +.. note:: + + An :cpp:class:`~tvm::ffi::Error` object is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload + as defined below: + + .. code-block:: cpp + + typedef struct { + TVMFFIByteArray kind; // Error type (e.g., "ValueError") + TVMFFIByteArray message; // Error message + TVMFFIByteArray backtrace; // Stack trace (most-recent call first) + void (*update_backtrace)(...); // Hook to append/replace backtrace + } TVMFFIErrorCell; + + +**Retrieve error object**. When error code is ``-1``, the error object is stored in the TLS slot, +and you can retrieve it with :cpp:func:`TVMFFIErrorMoveFromRaised`. + +.. code-block:: cpp + + void HandleReturnCode(int rc) { + TVMFFIObject* err = NULL; + if (rc == 0) { + // Success + } + else if (rc == -1) { + // Move the raised error from TLS (clears TLS slot) + TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object + if (err != NULL) { + PrintError(err); // print the error + TVMFFIObjectDecRef(err); // Release the error object + } + } + else if (rc == -2) { + // Frontend (e.g., Python) already has an exception set. + // Do not fetch from TLS; consult the frontend's error mechanism. + } + } + +This method transfers ownership of the error object to the caller and clears the TLS slot, +and therefore you must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done +to avoid memory leaks. + +Raise Errors in C +~~~~~~~~~~~~~~~~~ + +As part of TVM-FFI's calling convention, returning ``-1`` indicates an error occurred, +and the error object is stored in the TLS slot. The error object may contain abitrary +user-defined information, for example, error message, backtrace, or Python frame-local variables. + +.. hint:: + Compiler code generation may use similar patterns to raise errors in generated code. + +The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`: + +.. code-block:: cpp + + #include <tvm/ffi/c_api.h> + + int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args, + int32_t num_args, TVMFFIAny* result) { + // Validate inputs + if (num_args < 2) { + TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments"); + return -1; + } + // ... kernel implementation ... + return 0; + } + +Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` can also be used to concatenate +multiple parts of the error message into a single string. + +**Propagating errors**. For chains of generated calls, simply propagate return codes - TLS carries +the error details: + +.. code-block:: cpp + + int outer_function(...) { + int err_code = 0; + + err_code = inner_function(...); + if (err_code != 0) goto RAII; // Propagate error; TLS has the details + + RAII: + // clean up owned resources + return err_code; + } + +Handling Non-C++ Errors +~~~~~~~~~~~~~~~~~~~~~~~ + +TVM-FFI preserves frontend errors (e.g., Python exceptions) through a dedicated ``-2`` return path. Review Comment: This is not accurate. For common path, TVM-FFI catches translate python errors to ffi::Error with stacktrace. This allows python stack trace and c++ stack trace to be preserved across multiple cross function calls. For rare cases where the python error is triggered by signal handlersg, such error cannot be caught by the c++ frontend, in such case, we raise an ErrorAlreadySet and translate to -2 in the case. see also https://github.com/apache/tvm-ffi/blob/main/include/tvm/ffi/error.h#L73 The code there is the only use-case of -2 return values atm ########## docs/concepts/func_module.rst: ########## @@ -0,0 +1,584 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +Function and Module +=================== + +TVM-FFI provides a unified and ABI-stable calling convention that enables +cross-language function calls between C++, Python, Rust and other languages. +Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be: + +- **Invoked with type-erased arguments** using a stable C ABI +- **Registered globally** and accessed by name across languages +- **Exported as C symbols** for efficient codegen and linking +- **Wrapped in modules** for dynamic loading and namespace isolation + +This tutorial covers everything you need to know about defining, registering, and calling +TVM-FFI functions, as well as working with modules. + +Glossary +-------- + +TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType` + A stable C calling convention where every function is represented by a single signature, + which enables type-erased, cross-language function calls. + This calling convention is used across all TVM-FFI function calls at the ABI boundary. + See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction. + +TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function` + A reference-counted function object and its managed reference, which wraps any callable, + including language-agnostic functions and lambdas (C++, Python, Rust, etc.), + member functions, external C symbols, and other callable objects, + all sharing the same calling convention. + +TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module` + A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows), + or statically linked to the current executable. + +Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func` + A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects + and their metadata (name, docs, signatures, etc.) for cross-language access. + Functions in the registry are called **global functions**. + +Common Usage +------------ + +TVM-FFI C Symbols +~~~~~~~~~~~~~~~~~ + +**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export +a function as a C symbol that follows the TVM-FFI ABI: + +.. code-block:: cpp + + static int AddTwo(int x) { return x + 2; } + + TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo) + +This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library, +and later loaded and called via :py:func:`tvm_ffi.load_module`: + +.. code-block:: python + + import tvm_ffi + + mod = tvm_ffi.load_module("path/to/library.so") + result = mod.add_two(40) # -> 42 + +**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol` +is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`. +See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow. + + +Global Functions +~~~~~~~~~~~~~~~~ + +**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to +register a function: + +.. code-block:: cpp + + #include <tvm/ffi/tvm_ffi.h> + + static int AddOne(int x) { return x + 1; } + + TVM_FFI_STATIC_INIT_BLOCK() { + namespace refl = tvm::ffi::reflection; + refl::GlobalDef() + .def("my_ext.add_one", AddOne, "Add one to the input"); + } + +:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens +during library initialization. The registered function is then accessible from +Python under the name ``my_ext.add_one``. + +In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function: + +.. code-block:: python + + import tvm_ffi + + @tvm_ffi.register_global_func("my_ext.add_one") + def add_one(x: int) -> int: + return x + 1 + +**Retrieve a global function**. After registration, functions are accessible by name. + +In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function: + +.. code-block:: python + + import tvm_ffi + + # Get a function from the global registry + add_one = tvm_ffi.get_global_func("my_ext.add_one") + result = add_one(41) # -> 42 + +In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function: + +.. code-block:: cpp + + ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one"); + int result = func(41) // -> 42 + + +Create Functions +~~~~~~~~~~~~~~~~ + +**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped` +or :cpp:class:`tvm::ffi::TypedFunction`'s constructor. + +.. code-block:: cpp + + // Create type-erased function: add_type_erased + ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) { + return x + y; + }); + + // Create a typed function: add_typed + ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) { + return x + y; + }; + + // Converts typed function to type-erased function + ffi::Function generic = add; + +**From Python**. An arbitrary Python :py:class:`Callable <collections.abc.Callable>` can be automatically converted +to a :py:class:`tvm_ffi.Function` at ABI boundary. The example below shows that in function ``my_ext.bind``: + +- Its input ``func`` is automatically converted to :py:class:`tvm_ffi.Function`; +- Its output, a lambda function is automatically converted to :py:class:`tvm_ffi.Function`. + +.. code-block:: python + + import tvm_ffi + + @tvm_ffi.register_global_func("my_ext.bind") + def bind(func, x): + assert isinstance(func, tvm_ffi.Function) + return lambda *args: func(x, *args) # converted to `tvm_ffi.Function` + + def add_x_y(x, y): + return x + y + + func_bind = tvm_ffi.get_global_func("my_ext.bind") + add_y = func_bind(add_x_y, 1) # bind x = 1 + assert isinstance(add_y, tvm_ffi.Function) + print(add_y(2)) # -> 3 + + +:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`: + +.. code-block:: python + + import tvm_ffi + + def add(x, y): + return x + y + + func_add = tvm_ffi.convert(add) + print(func_add(1, 2)) + + +C ABI Calling Convention +------------------------ + +.. important:: + + This section focuses on in-depth of TVM-FFI's stable C ABI definition, + describing it in pure C for precision and clarity. + + Compiler code generation may follow similar patterns to call functions with TVM-FFI ABI, + or to generate functions that conform to TVM-FFI ABI. + +All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature, +which provides a stable C ABI for cross-language calls. The C calling convention is defined as: + +.. code-block:: cpp + + int tvm_ffi_c_abi( + void* handle, // Resource handle + const TVMFFIAny* args, // Inputs (non-owning) + int num_args, // Number of arguments + TVMFFIAny* result // Output (owning) + ); + +.. note:: + As part of the contract, the return value ``result`` should be zero-initialized by the caller. + +The returning integer is **error code**, which indicates + +- ``0``: Success +- ``-1``: Error occurred, retrieve via :cpp:func:`TVMFFIErrorMoveFromRaised` +- ``-2``: Frontend error (e.g., Python exception) + + +Input and Output Arguments +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The input and output arguments are passed as :cpp:type:`TVMFFIAny` pointers. + +- Input arguments are an array of ``num_args`` non-owning :cpp:type:`tvm::ffi::AnyView`; +- Output argument is an owning :cpp:type:`tvm::ffi::Any`, which the caller has to guarantee to be zero-initialized. + +See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`. + +Retrieve Errors in C +~~~~~~~~~~~~~~~~~~~~ + +When a TVM-FFI function returns a non-zero code, it indicates an error occurred, +and the error object is stored in the TLS slot. This section shows how to retrieve +the error object and print the error message and backtrace. + +.. hint:: + Compiler code generation may use similar patterns to call any function with TVM-FFI ABI. + + +**Print an Error**. The example code below shows how to print an error message and backtrace. + +.. code-block:: cpp + + #include <tvm/ffi/c_api.h> + + static void PrintError(TVMFFIObject* err) { + TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject)); + fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data); + if (cell->backtrace.size) { + fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data); + } + } + +The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure, +which contains the error kind, message, and backtrace, which can be accessed +by skipping the :cpp:type:`TVMFFIObject` header with pointer arithmetics. + +.. note:: + + An :cpp:class:`~tvm::ffi::Error` object is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload + as defined below: + + .. code-block:: cpp + + typedef struct { + TVMFFIByteArray kind; // Error type (e.g., "ValueError") + TVMFFIByteArray message; // Error message + TVMFFIByteArray backtrace; // Stack trace (most-recent call first) + void (*update_backtrace)(...); // Hook to append/replace backtrace + } TVMFFIErrorCell; + + +**Retrieve error object**. When error code is ``-1``, the error object is stored in the TLS slot, +and you can retrieve it with :cpp:func:`TVMFFIErrorMoveFromRaised`. + +.. code-block:: cpp + + void HandleReturnCode(int rc) { + TVMFFIObject* err = NULL; + if (rc == 0) { + // Success + } + else if (rc == -1) { + // Move the raised error from TLS (clears TLS slot) + TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object + if (err != NULL) { + PrintError(err); // print the error + TVMFFIObjectDecRef(err); // Release the error object + } + } + else if (rc == -2) { + // Frontend (e.g., Python) already has an exception set. + // Do not fetch from TLS; consult the frontend's error mechanism. + } + } + +This method transfers ownership of the error object to the caller and clears the TLS slot, +and therefore you must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done +to avoid memory leaks. + +Raise Errors in C +~~~~~~~~~~~~~~~~~ + +As part of TVM-FFI's calling convention, returning ``-1`` indicates an error occurred, +and the error object is stored in the TLS slot. The error object may contain abitrary +user-defined information, for example, error message, backtrace, or Python frame-local variables. + +.. hint:: + Compiler code generation may use similar patterns to raise errors in generated code. + +The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`: + +.. code-block:: cpp + + #include <tvm/ffi/c_api.h> + + int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args, + int32_t num_args, TVMFFIAny* result) { + // Validate inputs + if (num_args < 2) { + TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments"); + return -1; + } + // ... kernel implementation ... + return 0; + } + +Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` can also be used to concatenate +multiple parts of the error message into a single string. + +**Propagating errors**. For chains of generated calls, simply propagate return codes - TLS carries +the error details: + +.. code-block:: cpp + + int outer_function(...) { + int err_code = 0; + + err_code = inner_function(...); + if (err_code != 0) goto RAII; // Propagate error; TLS has the details + + RAII: + // clean up owned resources + return err_code; + } + +Handling Non-C++ Errors +~~~~~~~~~~~~~~~~~~~~~~~ + +TVM-FFI preserves frontend errors (e.g., Python exceptions) through a dedicated ``-2`` return path. + +When a Python exception occurs during a TVM-FFI call: + +- The exception remains a native Python exception with its original traceback +- The C++ side returns ``-2`` to indicate "frontend error already set" +- Python retains the real exception object with actual frame locals + +This path preserves Python stack variables because the exception is never serialized into +a TVM-FFI Error object. + +Function +-------- + +Layout and ABI +~~~~~~~~~~~~~~ + +:cpp:class:`tvm::ffi::FunctionObj` stores two call pointers in :cpp:class:`TVMFFIFunctionCell`: + +.. code-block:: cpp + + typedef struct { + TVMFFISafeCallType safe_call; + void* cpp_call; + } TVMFFIFunctionCell; + +``safe_call`` is used for cross-ABI function calling, which intercepts and stores exceptions in TLS, +while ``cpp_call`` is used inside the same DSO, and exceptions are thrown directly for performance. + +.. important:: + + Do not directly use ``safe_call`` or ``cpp_call`` to call functions. Instead, a C API :cpp:func:`TVMFFIFunctionCall` + is provided to handle error propagation and resource management. + +**Conversion with Any**. :py:class:`tvm_ffi.Function` is a TVM-FFI object, and thus conversion follows +exactly the same rule as a TVM-FFI object does. See :ref:`Object Conversion with Any <object-conversion-with-any>` for more details. + + +Throw and Catch Errors +~~~~~~~~~~~~~~~~~~~~~~ + +TVM-FFI gracefully handles exceptions across language boundaries without having to manually +set error codes. + +**Python**. In Python, raise its language-native :py:class:`Exception <Exception>` and its +derived classes, and TVM-FFI will catch it at ABI boundary, and convert it to a +:cpp:class:`tvm::ffi::Error` object. If C++ code is calling into Python, the Python exception +will be propagated back and re-raised as a :cpp:class:`tvm::ffi::Error` object to C++ side, +and the C++ code can handle it appropriately. + +**C++**. In C++, use :cpp:class:`tvm::ffi::Error` or the :c:macro:`TVM_FFI_THROW` macro: + +.. code-block:: cpp + + #include <tvm/ffi/error.h> + + void ThrowError(int x) { + if (x < 0) { + TVM_FFI_THROW(ValueError) << "x must be non-negative, got " << x; + } + } + +The example above shows how to throw an error in C++. :c:macro:`TVM_FFI_THROW` captures +the current file name, line number, stacktrace, and error message, and then constructs a +:cpp:class:`tvm::ffi::Error` object. At ABI boundary, this :cpp:class:`tvm::ffi::Error` +object is stored in the TLS slot, and the function returns ``-1`` as error code in the C ABI +:cpp:type:`TVMFFISafeCallType` signature. + +.. hint:: + A detailed implementation of such graceful handling behavior can be found Review Comment: blank line needed in rst ########## docs/concepts/func_module.rst: ########## @@ -0,0 +1,584 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +Function and Module +=================== + +TVM-FFI provides a unified and ABI-stable calling convention that enables +cross-language function calls between C++, Python, Rust and other languages. +Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be: + +- **Invoked with type-erased arguments** using a stable C ABI +- **Registered globally** and accessed by name across languages +- **Exported as C symbols** for efficient codegen and linking +- **Wrapped in modules** for dynamic loading and namespace isolation + +This tutorial covers everything you need to know about defining, registering, and calling +TVM-FFI functions, as well as working with modules. + +Glossary +-------- + +TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType` + A stable C calling convention where every function is represented by a single signature, + which enables type-erased, cross-language function calls. + This calling convention is used across all TVM-FFI function calls at the ABI boundary. + See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction. + +TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function` + A reference-counted function object and its managed reference, which wraps any callable, + including language-agnostic functions and lambdas (C++, Python, Rust, etc.), + member functions, external C symbols, and other callable objects, + all sharing the same calling convention. + +TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module` + A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows), + or statically linked to the current executable. + +Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func` + A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects + and their metadata (name, docs, signatures, etc.) for cross-language access. + Functions in the registry are called **global functions**. + +Common Usage +------------ + +TVM-FFI C Symbols +~~~~~~~~~~~~~~~~~ + +**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export +a function as a C symbol that follows the TVM-FFI ABI: + +.. code-block:: cpp + + static int AddTwo(int x) { return x + 2; } + + TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo) + +This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library, +and later loaded and called via :py:func:`tvm_ffi.load_module`: + +.. code-block:: python + + import tvm_ffi + + mod = tvm_ffi.load_module("path/to/library.so") + result = mod.add_two(40) # -> 42 + +**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol` +is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`. +See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow. + + +Global Functions +~~~~~~~~~~~~~~~~ + +**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to +register a function: + +.. code-block:: cpp + + #include <tvm/ffi/tvm_ffi.h> + + static int AddOne(int x) { return x + 1; } + + TVM_FFI_STATIC_INIT_BLOCK() { + namespace refl = tvm::ffi::reflection; + refl::GlobalDef() + .def("my_ext.add_one", AddOne, "Add one to the input"); + } + +:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens +during library initialization. The registered function is then accessible from +Python under the name ``my_ext.add_one``. + +In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function: + +.. code-block:: python + + import tvm_ffi + + @tvm_ffi.register_global_func("my_ext.add_one") + def add_one(x: int) -> int: + return x + 1 + +**Retrieve a global function**. After registration, functions are accessible by name. + +In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function: + +.. code-block:: python + + import tvm_ffi + + # Get a function from the global registry + add_one = tvm_ffi.get_global_func("my_ext.add_one") + result = add_one(41) # -> 42 + +In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function: + +.. code-block:: cpp + + ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one"); + int result = func(41) // -> 42 + + +Create Functions +~~~~~~~~~~~~~~~~ + +**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped` +or :cpp:class:`tvm::ffi::TypedFunction`'s constructor. + +.. code-block:: cpp + + // Create type-erased function: add_type_erased + ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) { + return x + y; + }); + + // Create a typed function: add_typed + ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) { + return x + y; + }; + + // Converts typed function to type-erased function + ffi::Function generic = add; + +**From Python**. An arbitrary Python :py:class:`Callable <collections.abc.Callable>` can be automatically converted +to a :py:class:`tvm_ffi.Function` at ABI boundary. The example below shows that in function ``my_ext.bind``: + +- Its input ``func`` is automatically converted to :py:class:`tvm_ffi.Function`; +- Its output, a lambda function is automatically converted to :py:class:`tvm_ffi.Function`. + +.. code-block:: python + + import tvm_ffi + + @tvm_ffi.register_global_func("my_ext.bind") + def bind(func, x): + assert isinstance(func, tvm_ffi.Function) + return lambda *args: func(x, *args) # converted to `tvm_ffi.Function` + + def add_x_y(x, y): + return x + y + + func_bind = tvm_ffi.get_global_func("my_ext.bind") + add_y = func_bind(add_x_y, 1) # bind x = 1 + assert isinstance(add_y, tvm_ffi.Function) + print(add_y(2)) # -> 3 + + +:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`: + +.. code-block:: python + + import tvm_ffi + + def add(x, y): + return x + y + + func_add = tvm_ffi.convert(add) + print(func_add(1, 2)) + + +C ABI Calling Convention +------------------------ + +.. important:: + + This section focuses on in-depth of TVM-FFI's stable C ABI definition, + describing it in pure C for precision and clarity. + + Compiler code generation may follow similar patterns to call functions with TVM-FFI ABI, + or to generate functions that conform to TVM-FFI ABI. + +All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature, +which provides a stable C ABI for cross-language calls. The C calling convention is defined as: + +.. code-block:: cpp + + int tvm_ffi_c_abi( + void* handle, // Resource handle + const TVMFFIAny* args, // Inputs (non-owning) + int num_args, // Number of arguments + TVMFFIAny* result // Output (owning) + ); + +.. note:: + As part of the contract, the return value ``result`` should be zero-initialized by the caller. + +The returning integer is **error code**, which indicates + +- ``0``: Success +- ``-1``: Error occurred, retrieve via :cpp:func:`TVMFFIErrorMoveFromRaised` +- ``-2``: Frontend error (e.g., Python exception) + + +Input and Output Arguments +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The input and output arguments are passed as :cpp:type:`TVMFFIAny` pointers. + +- Input arguments are an array of ``num_args`` non-owning :cpp:type:`tvm::ffi::AnyView`; +- Output argument is an owning :cpp:type:`tvm::ffi::Any`, which the caller has to guarantee to be zero-initialized. + +See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`. + +Retrieve Errors in C +~~~~~~~~~~~~~~~~~~~~ + +When a TVM-FFI function returns a non-zero code, it indicates an error occurred, +and the error object is stored in the TLS slot. This section shows how to retrieve +the error object and print the error message and backtrace. + +.. hint:: + Compiler code generation may use similar patterns to call any function with TVM-FFI ABI. + + +**Print an Error**. The example code below shows how to print an error message and backtrace. + +.. code-block:: cpp + + #include <tvm/ffi/c_api.h> + + static void PrintError(TVMFFIObject* err) { + TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject)); + fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data); + if (cell->backtrace.size) { + fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data); + } + } + +The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure, +which contains the error kind, message, and backtrace, which can be accessed +by skipping the :cpp:type:`TVMFFIObject` header with pointer arithmetics. + +.. note:: + + An :cpp:class:`~tvm::ffi::Error` object is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload + as defined below: + + .. code-block:: cpp + + typedef struct { + TVMFFIByteArray kind; // Error type (e.g., "ValueError") + TVMFFIByteArray message; // Error message + TVMFFIByteArray backtrace; // Stack trace (most-recent call first) + void (*update_backtrace)(...); // Hook to append/replace backtrace + } TVMFFIErrorCell; + + +**Retrieve error object**. When error code is ``-1``, the error object is stored in the TLS slot, +and you can retrieve it with :cpp:func:`TVMFFIErrorMoveFromRaised`. + +.. code-block:: cpp + + void HandleReturnCode(int rc) { + TVMFFIObject* err = NULL; + if (rc == 0) { + // Success + } + else if (rc == -1) { + // Move the raised error from TLS (clears TLS slot) + TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object + if (err != NULL) { + PrintError(err); // print the error + TVMFFIObjectDecRef(err); // Release the error object + } + } + else if (rc == -2) { + // Frontend (e.g., Python) already has an exception set. + // Do not fetch from TLS; consult the frontend's error mechanism. + } + } + +This method transfers ownership of the error object to the caller and clears the TLS slot, +and therefore you must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done +to avoid memory leaks. + +Raise Errors in C +~~~~~~~~~~~~~~~~~ + +As part of TVM-FFI's calling convention, returning ``-1`` indicates an error occurred, +and the error object is stored in the TLS slot. The error object may contain abitrary +user-defined information, for example, error message, backtrace, or Python frame-local variables. + +.. hint:: + Compiler code generation may use similar patterns to raise errors in generated code. + +The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`: + +.. code-block:: cpp + + #include <tvm/ffi/c_api.h> + + int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args, + int32_t num_args, TVMFFIAny* result) { + // Validate inputs + if (num_args < 2) { + TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments"); + return -1; + } + // ... kernel implementation ... + return 0; + } + +Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` can also be used to concatenate +multiple parts of the error message into a single string. + +**Propagating errors**. For chains of generated calls, simply propagate return codes - TLS carries +the error details: + +.. code-block:: cpp + + int outer_function(...) { + int err_code = 0; + + err_code = inner_function(...); + if (err_code != 0) goto RAII; // Propagate error; TLS has the details + + RAII: + // clean up owned resources + return err_code; + } + +Handling Non-C++ Errors +~~~~~~~~~~~~~~~~~~~~~~~ + +TVM-FFI preserves frontend errors (e.g., Python exceptions) through a dedicated ``-2`` return path. + +When a Python exception occurs during a TVM-FFI call: + +- The exception remains a native Python exception with its original traceback +- The C++ side returns ``-2`` to indicate "frontend error already set" +- Python retains the real exception object with actual frame locals + +This path preserves Python stack variables because the exception is never serialized into +a TVM-FFI Error object. + +Function +-------- + +Layout and ABI +~~~~~~~~~~~~~~ + +:cpp:class:`tvm::ffi::FunctionObj` stores two call pointers in :cpp:class:`TVMFFIFunctionCell`: + +.. code-block:: cpp + + typedef struct { + TVMFFISafeCallType safe_call; + void* cpp_call; + } TVMFFIFunctionCell; + +``safe_call`` is used for cross-ABI function calling, which intercepts and stores exceptions in TLS, +while ``cpp_call`` is used inside the same DSO, and exceptions are thrown directly for performance. + +.. important:: + + Do not directly use ``safe_call`` or ``cpp_call`` to call functions. Instead, a C API :cpp:func:`TVMFFIFunctionCall` + is provided to handle error propagation and resource management. + +**Conversion with Any**. :py:class:`tvm_ffi.Function` is a TVM-FFI object, and thus conversion follows +exactly the same rule as a TVM-FFI object does. See :ref:`Object Conversion with Any <object-conversion-with-any>` for more details. + + +Throw and Catch Errors +~~~~~~~~~~~~~~~~~~~~~~ + +TVM-FFI gracefully handles exceptions across language boundaries without having to manually +set error codes. + +**Python**. In Python, raise its language-native :py:class:`Exception <Exception>` and its +derived classes, and TVM-FFI will catch it at ABI boundary, and convert it to a +:cpp:class:`tvm::ffi::Error` object. If C++ code is calling into Python, the Python exception +will be propagated back and re-raised as a :cpp:class:`tvm::ffi::Error` object to C++ side, +and the C++ code can handle it appropriately. + +**C++**. In C++, use :cpp:class:`tvm::ffi::Error` or the :c:macro:`TVM_FFI_THROW` macro: + +.. code-block:: cpp + + #include <tvm/ffi/error.h> + + void ThrowError(int x) { + if (x < 0) { + TVM_FFI_THROW(ValueError) << "x must be non-negative, got " << x; + } + } + +The example above shows how to throw an error in C++. :c:macro:`TVM_FFI_THROW` captures +the current file name, line number, stacktrace, and error message, and then constructs a +:cpp:class:`tvm::ffi::Error` object. At ABI boundary, this :cpp:class:`tvm::ffi::Error` +object is stored in the TLS slot, and the function returns ``-1`` as error code in the C ABI +:cpp:type:`TVMFFISafeCallType` signature. + +.. hint:: + A detailed implementation of such graceful handling behavior can be found + in :c:macro:`TVM_FFI_SAFE_CALL_BEGIN` / :c:macro:`TVM_FFI_SAFE_CALL_END` macros. + + +C Registry APIs +~~~~~~~~~~~~~~~ + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - C API + - Description + * - :cpp:func:`TVMFFIFunctionGetGlobal` + - Get function by name. It returns an owning handle to the function. + * - :cpp:func:`TVMFFIFunctionSetGlobal` + - Register a function to the global registry. + * - :cpp:func:`TVMFFIFunctionCall` + - Call a function with variadic arguments. + +It is common need for a compiler developer to look up global functions in generated code. To do so, +use :cpp:func:`TVMFFIFunctionGetGlobal` to get a function by name, and then call it with :cpp:func:`TVMFFIFunctionCall`. +The example below shows how to lookup and call a global function in C code. + +.. code-block:: cpp + + int LookupAndCall(const char* global_function_name, const TVMFFIAny* args, int num_args, TVMFFIAny* result) { + TVMFFIObject* func = NULL; + int err_code; + if ((err_code = TVMFFIFunctionGetGlobal(global_function_name, &func)) != 0) + goto RAII; + if ((err_code = TVMFFIFunctionCall(func, args, num_args, result)) != 0) + goto RAII; + + RAII: // clean up owned resources + if (func != NULL) TVMFFIObjectDecRef(func); + return err_code; + } + +Modules +------- + +A :py:class:`tvm_ffi.Module` is a namespace for a collection of functions that can be loaded +from a shared library or bundled with the current executable. Modules provide namespace isolation +and dynamic loading capabilities for TVM-FFI functions. + +Shared Library +~~~~~~~~~~~~~~ + +Shared library modules are loaded dynamically at runtime via ``dlopen`` (Linux, macOS) or +``LoadLibraryW`` (Windows). This is the most common way to distribute and load compiled functions. + +**Export functions from C++**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export +a function as a C symbol that follows the TVM-FFI ABI: + +.. code-block:: cpp + + #include <tvm/ffi/tvm_ffi.h> + + static int AddTwo(int x) { return x + 2; } + + // Exports as symbol `__tvm_ffi_add_two` + TVM_FFI_DLL_EXPORT_TYPED_FUNC(add_two, AddTwo); + +**Load and call from Python**. Use :py:func:`tvm_ffi.load_module` to load the shared library: + +.. code-block:: python + + import tvm_ffi + + # Load the shared library + mod = tvm_ffi.load_module("path/to/library.so") + + # Access functions by name + result = mod.add_two(40) # -> 42 + + # Alternative: explicit function retrieval + func = mod.get_function("add_two") + result = func(40) # -> 42 + +**Build and load from source**. For rapid prototyping, :py:func:`tvm_ffi.cpp.load` compiles +C++/CUDA source files and loads them as a module in one step: + +.. code-block:: python + + import tvm_ffi.cpp + + # Compile and load in one step + mod = tvm_ffi.cpp.load( + name="my_ops", + cpp_files="my_ops.cpp", + ) + result = mod.add_two(40) + +Essentially, :py:func:`tvm_ffi.cpp.load` is a shortcut that combines JIT-building the library, +and then loading the library into a :py:class:`tvm_ffi.Module`. + +System Library +~~~~~~~~~~~~~~ + +System library modules contain symbols that are statically linked to the current executable. +This is useful for AOT (ahead-of-time) compilation where functions are bundled with the binary. Review Comment: This technique is useful when we would like to preserve module dynamic module loading behavior but do not have dlopen available in the system in a static bundle. Instead, we register the symbols in the system so they can be looked up via tvm_ffi.system_lib ########## docs/concepts/func_module.rst: ########## @@ -0,0 +1,584 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +Function and Module +=================== + +TVM-FFI provides a unified and ABI-stable calling convention that enables +cross-language function calls between C++, Python, Rust and other languages. +Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be: + +- **Invoked with type-erased arguments** using a stable C ABI +- **Registered globally** and accessed by name across languages +- **Exported as C symbols** for efficient codegen and linking +- **Wrapped in modules** for dynamic loading and namespace isolation + +This tutorial covers everything you need to know about defining, registering, and calling +TVM-FFI functions, as well as working with modules. + +Glossary +-------- + +TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType` + A stable C calling convention where every function is represented by a single signature, + which enables type-erased, cross-language function calls. + This calling convention is used across all TVM-FFI function calls at the ABI boundary. + See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction. + +TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function` + A reference-counted function object and its managed reference, which wraps any callable, + including language-agnostic functions and lambdas (C++, Python, Rust, etc.), + member functions, external C symbols, and other callable objects, + all sharing the same calling convention. + +TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module` + A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows), + or statically linked to the current executable. + +Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func` + A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects + and their metadata (name, docs, signatures, etc.) for cross-language access. + Functions in the registry are called **global functions**. + +Common Usage +------------ + +TVM-FFI C Symbols +~~~~~~~~~~~~~~~~~ + +**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export +a function as a C symbol that follows the TVM-FFI ABI: + +.. code-block:: cpp + + static int AddTwo(int x) { return x + 2; } + + TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo) + +This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library, +and later loaded and called via :py:func:`tvm_ffi.load_module`: + +.. code-block:: python + + import tvm_ffi + + mod = tvm_ffi.load_module("path/to/library.so") + result = mod.add_two(40) # -> 42 + +**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol` +is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`. +See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow. + + +Global Functions +~~~~~~~~~~~~~~~~ + +**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to +register a function: + +.. code-block:: cpp + + #include <tvm/ffi/tvm_ffi.h> + + static int AddOne(int x) { return x + 1; } + + TVM_FFI_STATIC_INIT_BLOCK() { + namespace refl = tvm::ffi::reflection; + refl::GlobalDef() + .def("my_ext.add_one", AddOne, "Add one to the input"); + } + +:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens +during library initialization. The registered function is then accessible from +Python under the name ``my_ext.add_one``. + +In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function: + +.. code-block:: python + + import tvm_ffi + + @tvm_ffi.register_global_func("my_ext.add_one") + def add_one(x: int) -> int: + return x + 1 + +**Retrieve a global function**. After registration, functions are accessible by name. + +In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function: + +.. code-block:: python + + import tvm_ffi + + # Get a function from the global registry + add_one = tvm_ffi.get_global_func("my_ext.add_one") + result = add_one(41) # -> 42 + +In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function: + +.. code-block:: cpp + + ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one"); + int result = func(41) // -> 42 + + +Create Functions +~~~~~~~~~~~~~~~~ + +**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped` +or :cpp:class:`tvm::ffi::TypedFunction`'s constructor. + +.. code-block:: cpp + + // Create type-erased function: add_type_erased + ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) { + return x + y; + }); + + // Create a typed function: add_typed + ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) { + return x + y; + }; + + // Converts typed function to type-erased function + ffi::Function generic = add; + +**From Python**. An arbitrary Python :py:class:`Callable <collections.abc.Callable>` can be automatically converted +to a :py:class:`tvm_ffi.Function` at ABI boundary. The example below shows that in function ``my_ext.bind``: + +- Its input ``func`` is automatically converted to :py:class:`tvm_ffi.Function`; +- Its output, a lambda function is automatically converted to :py:class:`tvm_ffi.Function`. + +.. code-block:: python + + import tvm_ffi + + @tvm_ffi.register_global_func("my_ext.bind") + def bind(func, x): + assert isinstance(func, tvm_ffi.Function) + return lambda *args: func(x, *args) # converted to `tvm_ffi.Function` + + def add_x_y(x, y): + return x + y + + func_bind = tvm_ffi.get_global_func("my_ext.bind") + add_y = func_bind(add_x_y, 1) # bind x = 1 + assert isinstance(add_y, tvm_ffi.Function) + print(add_y(2)) # -> 3 + + +:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`: + +.. code-block:: python + + import tvm_ffi + + def add(x, y): + return x + y + + func_add = tvm_ffi.convert(add) + print(func_add(1, 2)) + + +C ABI Calling Convention +------------------------ + +.. important:: + + This section focuses on in-depth of TVM-FFI's stable C ABI definition, + describing it in pure C for precision and clarity. + + Compiler code generation may follow similar patterns to call functions with TVM-FFI ABI, + or to generate functions that conform to TVM-FFI ABI. + +All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature, +which provides a stable C ABI for cross-language calls. The C calling convention is defined as: + +.. code-block:: cpp + + int tvm_ffi_c_abi( + void* handle, // Resource handle + const TVMFFIAny* args, // Inputs (non-owning) + int num_args, // Number of arguments + TVMFFIAny* result // Output (owning) + ); + +.. note:: + As part of the contract, the return value ``result`` should be zero-initialized by the caller. + +The returning integer is **error code**, which indicates + +- ``0``: Success +- ``-1``: Error occurred, retrieve via :cpp:func:`TVMFFIErrorMoveFromRaised` +- ``-2``: Frontend error (e.g., Python exception) + + +Input and Output Arguments +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The input and output arguments are passed as :cpp:type:`TVMFFIAny` pointers. + +- Input arguments are an array of ``num_args`` non-owning :cpp:type:`tvm::ffi::AnyView`; +- Output argument is an owning :cpp:type:`tvm::ffi::Any`, which the caller has to guarantee to be zero-initialized. + +See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`. + +Retrieve Errors in C +~~~~~~~~~~~~~~~~~~~~ + +When a TVM-FFI function returns a non-zero code, it indicates an error occurred, +and the error object is stored in the TLS slot. This section shows how to retrieve +the error object and print the error message and backtrace. + +.. hint:: + Compiler code generation may use similar patterns to call any function with TVM-FFI ABI. + + +**Print an Error**. The example code below shows how to print an error message and backtrace. + +.. code-block:: cpp + + #include <tvm/ffi/c_api.h> + + static void PrintError(TVMFFIObject* err) { + TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject)); + fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data); + if (cell->backtrace.size) { + fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data); + } + } + +The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure, +which contains the error kind, message, and backtrace, which can be accessed +by skipping the :cpp:type:`TVMFFIObject` header with pointer arithmetics. + +.. note:: + + An :cpp:class:`~tvm::ffi::Error` object is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload + as defined below: + + .. code-block:: cpp + + typedef struct { + TVMFFIByteArray kind; // Error type (e.g., "ValueError") + TVMFFIByteArray message; // Error message + TVMFFIByteArray backtrace; // Stack trace (most-recent call first) + void (*update_backtrace)(...); // Hook to append/replace backtrace + } TVMFFIErrorCell; + + +**Retrieve error object**. When error code is ``-1``, the error object is stored in the TLS slot, +and you can retrieve it with :cpp:func:`TVMFFIErrorMoveFromRaised`. + +.. code-block:: cpp + + void HandleReturnCode(int rc) { + TVMFFIObject* err = NULL; + if (rc == 0) { + // Success + } + else if (rc == -1) { + // Move the raised error from TLS (clears TLS slot) + TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object + if (err != NULL) { + PrintError(err); // print the error + TVMFFIObjectDecRef(err); // Release the error object + } + } + else if (rc == -2) { + // Frontend (e.g., Python) already has an exception set. + // Do not fetch from TLS; consult the frontend's error mechanism. + } + } + +This method transfers ownership of the error object to the caller and clears the TLS slot, +and therefore you must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done +to avoid memory leaks. + +Raise Errors in C +~~~~~~~~~~~~~~~~~ + +As part of TVM-FFI's calling convention, returning ``-1`` indicates an error occurred, +and the error object is stored in the TLS slot. The error object may contain abitrary +user-defined information, for example, error message, backtrace, or Python frame-local variables. + +.. hint:: + Compiler code generation may use similar patterns to raise errors in generated code. + +The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`: + +.. code-block:: cpp + + #include <tvm/ffi/c_api.h> + + int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args, + int32_t num_args, TVMFFIAny* result) { + // Validate inputs + if (num_args < 2) { + TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments"); + return -1; + } + // ... kernel implementation ... + return 0; + } + +Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` can also be used to concatenate +multiple parts of the error message into a single string. + +**Propagating errors**. For chains of generated calls, simply propagate return codes - TLS carries +the error details: + +.. code-block:: cpp + + int outer_function(...) { + int err_code = 0; + + err_code = inner_function(...); + if (err_code != 0) goto RAII; // Propagate error; TLS has the details + + RAII: + // clean up owned resources + return err_code; + } + +Handling Non-C++ Errors +~~~~~~~~~~~~~~~~~~~~~~~ + +TVM-FFI preserves frontend errors (e.g., Python exceptions) through a dedicated ``-2`` return path. + +When a Python exception occurs during a TVM-FFI call: + +- The exception remains a native Python exception with its original traceback +- The C++ side returns ``-2`` to indicate "frontend error already set" +- Python retains the real exception object with actual frame locals + +This path preserves Python stack variables because the exception is never serialized into +a TVM-FFI Error object. + +Function +-------- + +Layout and ABI +~~~~~~~~~~~~~~ + +:cpp:class:`tvm::ffi::FunctionObj` stores two call pointers in :cpp:class:`TVMFFIFunctionCell`: + +.. code-block:: cpp + + typedef struct { + TVMFFISafeCallType safe_call; + void* cpp_call; + } TVMFFIFunctionCell; + +``safe_call`` is used for cross-ABI function calling, which intercepts and stores exceptions in TLS, +while ``cpp_call`` is used inside the same DSO, and exceptions are thrown directly for performance. + +.. important:: + + Do not directly use ``safe_call`` or ``cpp_call`` to call functions. Instead, a C API :cpp:func:`TVMFFIFunctionCall` Review Comment: It is OK to use `safe_call` directly in low-level abi manipulations, `TVMFFIFunctionCall` was mainly meant as a way to expose the call in cases where function ptr manipulation is not accessible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
