This is an automated email from the ASF dual-hosted git repository.
tqchen pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm-ffi.git
The following commit(s) were added to refs/heads/main by this push:
new a472db8d [FFI] Refcount ordering, Array copy-on-write mutators, Rust
API surface cleanup (#943)
a472db8d is described below
commit a472db8d80b57c6d6be34965e84133f7ce450c57
Author: Tianqi Chen <[email protected]>
AuthorDate: Fri Nov 6 10:17:06 2026 -0500
[FFI] Refcount ordering, Array copy-on-write mutators, Rust API surface
cleanup (#943)
The strong-count decrement in `dec_ref` used `Relaxed`, so the Acquire
fence before the deleter had no Release to pair with: writes made by
another owner before it dropped its reference were not guaranteed to be
visible to the thread running the destructor. Use `Release`, matching
C++ `Object::DecRef` and `std::sync::Arc`.
In-place mutation (error context, backtrace append, `Array::push`,
structural mutate) was gated on a strong count of 1, which ignores weak
references: one could be upgraded concurrently and observe the write.
Gate it on a new crate-internal `is_unique`, which requires one strong
and no weak reference, as C++ `Object::unique()`, and loads with
`Acquire` to pair with the releasing decrements, as `Arc::get_mut`.
`strong_count` stays a `Relaxed` diagnostic.
C++ `Object::unique()` loaded the count `Relaxed` (MSVC: a volatile
read, not an acquire on arm64), yet copy-on-write paths write in place
after it. Load with acquire (`ReadAcquire64` on MSVC), pairing with
`DecRef`'s release.
Document on `Function::from_extern_c` that `safe_call`, `deleter` and
the handle state must be safe to use from any thread.
Implement `std::error::Error::source()` for `Error` from its cause
chain, so generic error tooling (anyhow, eyre, `Box<dyn Error>`) sees
the cause.
Make the documented Rust surface match the supported API before it is
locked, without changing what is reachable:
- hide the `macros` module and its `paste` re-export (the exported
macros live at the crate root);
- document the items a hand-written reference type relies on:
`type_traits::ContainerElement`, `object::is_instance_of` and
`ObjectArc::is_null`; add `unsafe_impl_ref_downcast!`, which implements
the in-place reference downcasts (`array[i]`, `downcast_ref`) for such a
type, so their traits and helpers stay hidden;
- state what `impl_try_from_any!`, `impl_arg_into_ref!` and
`impl_into_arg_holder_default!` enable, and give `cached_global_func!`
an example;
- add a casts section to `ObjectRefCore`: `downcast_ref` borrows
strictly, `downcast` takes the value and hands it back on a mismatch,
and `TryFrom` returns a `TypeError`;
- make `StructuralVarRemap` crate-private (engine internal).
Declare `rust-version = "1.95"` on the three crates, the oldest
toolchain that builds them (`core::hint::cold_path`), as TIRx does.
Rust `Array` gains public `push`, `insert`, `pop`, `truncate` and `set`,
all `&mut self`, with C++ `Array`'s copy-on-write contract stated once
on
the type: copied when shared, edited in place when this handle is the
only reference. A private `make_mut` (as C++ `Array::CopyOnWrite` and
`Rc::make_mut`) holds the copy logic previously inline in `push`. Index
errors match `List`: `insert`, `set` and `pop` return an `IndexError`.
Structural mutate has one default path: `StructuralMutatorObj`'s
`StructuralMutatorBase::default_mutate_raw` reaches the variable remap
through the installed vtable, as every other native prefix does, so a
base that reuses it is correct by construction. This removes the
`ORDINARY_ENGINE` fast path and the same-named inherent
`default_mutate_raw`; performance is confirmed separately.
`Error` updates in place go through one private `make_mut` (copy when
shared), and `Array::make_mut` keeps the reallocation in a helper.
Structural mutate errors are labeled "mutate" (the map adapter keeps
"map"), and the duplicate type-key lookup is removed. A test checks the
Rust `VisitErrorContextObj` binding against the field offsets C++
registers.
C++ `UpdateVisitErrorContext` skips the append when the context's last
frame is already `node` (callback, policy and default descent can report
the same frame), before copying the error, as Rust does.
---
include/tvm/ffi/extra/visit_error_context.h | 8 ++
include/tvm/ffi/object.h | 9 +-
rust/tvm-ffi-macros/Cargo.toml | 2 +
rust/tvm-ffi-sys/Cargo.toml | 2 +
rust/tvm-ffi/Cargo.toml | 2 +
rust/tvm-ffi/src/collections/array.rs | 208 +++++++++++++++++++++------
rust/tvm-ffi/src/error.rs | 88 ++++++------
rust/tvm-ffi/src/extra/structural_common.rs | 64 ++++++---
rust/tvm-ffi/src/extra/structural_map.rs | 8 +-
rust/tvm-ffi/src/extra/structural_mutate.rs | 184 ++++++------------------
rust/tvm-ffi/src/extra/structural_visit.rs | 7 +-
rust/tvm-ffi/src/function.rs | 10 +-
rust/tvm-ffi/src/lib.rs | 3 +-
rust/tvm-ffi/src/macros.rs | 164 ++++++++++++++++++---
rust/tvm-ffi/src/object.rs | 91 ++++++------
rust/tvm-ffi/src/type_traits.rs | 14 +-
rust/tvm-ffi/tests/test_array.rs | 27 ++++
rust/tvm-ffi/tests/test_error.rs | 1 +
rust/tvm-ffi/tests/test_structural_mutate.rs | 12 +-
19 files changed, 551 insertions(+), 353 deletions(-)
diff --git a/include/tvm/ffi/extra/visit_error_context.h
b/include/tvm/ffi/extra/visit_error_context.h
index 4d5bde91..27903443 100644
--- a/include/tvm/ffi/extra/visit_error_context.h
+++ b/include/tvm/ffi/extra/visit_error_context.h
@@ -290,6 +290,14 @@ inline VisitErrorContext MakeVisitErrorContext(const
ObjectRef& node) {
*/
TVM_FFI_COLD_CODE
inline void UpdateVisitErrorContext(Error* err, const ObjectRef& node) { //
NOLINT(*)
+ // Callback, policy and default descent can report the same frame.
+ const Object* current = static_cast<const Object*>(
+
static_cast<ErrorObj*>(detail::ObjectUnsafe::RawObjectPtrFromObjectRef(*err))->extra_context);
+ if (current != nullptr && current->IsInstance<VisitErrorContextObj>()) {
+ const Array<ObjectRef>& pattern =
+ static_cast<const
VisitErrorContextObj*>(current)->reverse_visit_pattern;
+ if (!pattern.empty() && pattern.back().same_as(node)) return;
+ }
// Copy-on-write at each level: the Error is replaced by a shallow copy when
shared, and its
// VisitErrorContext likewise. Each level is borrowed through its parent's
field, so an Error
// owned only by the unwinding frame is updated in place.
diff --git a/include/tvm/ffi/object.h b/include/tvm/ffi/object.h
index 7144b432..7cc3fa5e 100644
--- a/include/tvm/ffi/object.h
+++ b/include/tvm/ffi/object.h
@@ -187,12 +187,13 @@ class Object {
* copy-on-write in multi-threaded setting.
*/
bool unique() const {
+ // Acquire pairs with the releasing decrement: other owners' accesses
precede an in-place write.
#ifdef _MSC_VER
- return (reinterpret_cast<const volatile uint64_t*>(
- &header_.combined_ref_count))[0] == // NOLINT(*)
- kCombinedRefCountBothOne;
+ return static_cast<uint64_t>(
+ ReadAcquire64(reinterpret_cast<const volatile LONG64*>( //
NOLINT(*)
+ &header_.combined_ref_count))) == kCombinedRefCountBothOne;
#else
- return __atomic_load_n(&(header_.combined_ref_count), __ATOMIC_RELAXED) ==
+ return __atomic_load_n(&(header_.combined_ref_count), __ATOMIC_ACQUIRE) ==
kCombinedRefCountBothOne;
#endif
}
diff --git a/rust/tvm-ffi-macros/Cargo.toml b/rust/tvm-ffi-macros/Cargo.toml
index 6c646904..259ba5f3 100644
--- a/rust/tvm-ffi-macros/Cargo.toml
+++ b/rust/tvm-ffi-macros/Cargo.toml
@@ -22,6 +22,8 @@ description = "Procedural macro crate for tvm-ffi"
# Development placeholder; publish_crates.yml sets the release version from
the Git tag.
version = "0.1.0-alpha.0"
edition = "2021"
+# The oldest toolchain that builds the crates (`core::hint::cold_path`).
+rust-version = "1.95"
license = "Apache-2.0"
diff --git a/rust/tvm-ffi-sys/Cargo.toml b/rust/tvm-ffi-sys/Cargo.toml
index 04ad1744..1ae83a30 100644
--- a/rust/tvm-ffi-sys/Cargo.toml
+++ b/rust/tvm-ffi-sys/Cargo.toml
@@ -22,6 +22,8 @@ description = "Low-level sys crate for tvm-ffi"
# Development placeholder; publish_crates.yml sets the release version from
the Git tag.
version = "0.1.0-alpha.0"
edition = "2021"
+# The oldest toolchain that builds the crates (`core::hint::cold_path`).
+rust-version = "1.95"
license = "Apache-2.0"
diff --git a/rust/tvm-ffi/Cargo.toml b/rust/tvm-ffi/Cargo.toml
index 97616acb..e83b3fd6 100644
--- a/rust/tvm-ffi/Cargo.toml
+++ b/rust/tvm-ffi/Cargo.toml
@@ -22,6 +22,8 @@ description = "tvm-ffi rust support"
# Development placeholder; publish_crates.yml sets the release version from
the Git tag.
version = "0.1.0-alpha.0"
edition = "2021"
+# The oldest toolchain that builds the crates (`core::hint::cold_path`).
+rust-version = "1.95"
license = "Apache-2.0"
diff --git a/rust/tvm-ffi/src/collections/array.rs
b/rust/tvm-ffi/src/collections/array.rs
index 930458d6..7f824182 100644
--- a/rust/tvm-ffi/src/collections/array.rs
+++ b/rust/tvm-ffi/src/collections/array.rs
@@ -52,6 +52,13 @@ unsafe impl ObjectCoreWithExtraItems for ArrayObj {
}
}
+/// Move `value` into the uninitialized `cell`.
+unsafe fn write_cell<T: ContainerElement>(cell: *mut TVMFFIAny, value: T) {
+ let mut raw = TVMFFIAny::new();
+ T::container_move_to_any(value, &mut raw);
+ std::ptr::write(cell, raw);
+}
+
// Runs only for a Rust-allocated array; one allocated in C++ is released by
its
// own deleter.
impl Drop for ArrayObj {
@@ -64,26 +71,39 @@ impl Drop for ArrayObj {
}
}
-/// An immutable, reference-counted sequence of `T`, shared with C++ and
-/// Python as `ffi.Array`.
+/// A reference-counted sequence of `T` with value semantics, shared with C++
+/// and Python as `ffi.Array`.
///
-/// Cloning an `Array` is cheap: it shares the same elements. Use [`List`]
-/// for a sequence that changes in place.
+/// Cloning an `Array` is cheap: it shares the same elements. The mutators
+/// ([`push`], [`insert`], [`pop`], [`truncate`], [`set`]) are copy-on-write:
+/// they copy the elements when the array is shared, and write in place when
+/// this handle is the only reference, so other handles never see the change.
+/// Use [`List`] for a sequence whose changes all handles see.
///
/// [`List`]: crate::List
+/// [`push`]: Array::push
+/// [`insert`]: Array::insert
+/// [`pop`]: Array::pop
+/// [`truncate`]: Array::truncate
+/// [`set`]: Array::set
///
/// # Examples
///
/// ```
/// use tvm_ffi::Array;
///
-/// let array = Array::from_vec(vec![1_i64, 2, 3]);
+/// let mut array = Array::from_vec(vec![1_i64, 2, 3]);
/// assert_eq!(array.len(), 3);
/// assert_eq!(array.get(1)?, 2);
/// assert!(array.get(3).is_err());
///
+/// let snapshot = array.clone();
+/// array.push(4);
+/// assert_eq!(array.len(), 4);
+/// assert_eq!(snapshot.len(), 3);
+///
/// let doubled: Array<i64> = array.iter().map(|x| x * 2).collect();
-/// assert_eq!(doubled.iter().copied().collect::<Vec<_>>(), vec![2, 4, 6]);
+/// assert_eq!(doubled.iter().copied().collect::<Vec<_>>(), vec![2, 4, 6, 8]);
/// # Ok::<(), tvm_ffi::Error>(())
/// ```
#[repr(C)]
@@ -93,7 +113,8 @@ pub struct Array<T> {
_marker: PhantomData<T>,
}
-// SAFETY: an `Array` is immutable, so sharing it is as safe as sharing its
elements.
+// SAFETY: a shared `Array` is never mutated (mutators copy unless this handle
+// is unique), so sharing it is as safe as sharing its elements.
unsafe impl<T: Send + Sync> Send for Array<T> {}
unsafe impl<T: Send + Sync> Sync for Array<T> {}
@@ -226,48 +247,149 @@ impl<T: ContainerElement + Clone> Array<T> {
unsafe { Self::from_data(arc) }
}
- /// Append `value`, copy-on-write like C++ `Array::push_back`: in place
when
- /// this handle is the only reference and the buffer has room, otherwise
into
- /// a new buffer that grows geometrically when full.
- pub(crate) fn push(&mut self, value: T) {
+ /// Append `value` to the end.
+ ///
+ /// See also [`List::push`](crate::List::push).
+ pub fn push(&mut self, value: T) {
let size = self.len();
- let capacity = self.data.capacity as usize;
- let unique = ObjectArc::strong_count(&self.data) == 1;
- if !unique || size == capacity {
- let capacity = if size == capacity {
- (capacity * 2).max(size + 1)
+ let container = self.make_mut(size + 1);
+ // SAFETY: `make_mut` left room for one more cell past `size`.
+ unsafe { write_cell((container.data as *mut TVMFFIAny).add(size),
value) };
+ container.size += 1;
+ }
+
+ /// Insert `value` at `index`, shifting the elements after it.
+ ///
+ /// # Errors
+ ///
+ /// Returns an `IndexError` if `index > len`.
+ ///
+ /// See also [`List::insert`](crate::List::insert).
+ pub fn insert(&mut self, index: usize, value: T) -> Result<(),
crate::Error> {
+ let size = self.len();
+ if index > size {
+ crate::bail!(crate::error::INDEX_ERROR, "Array insert index out of
bound");
+ }
+ let container = self.make_mut(size + 1);
+ // SAFETY: `make_mut` left room for one more cell; the tail moves up
by one.
+ unsafe {
+ let cell = (container.data as *mut TVMFFIAny).add(index);
+ std::ptr::copy(cell, cell.add(1), size - index);
+ write_cell(cell, value);
+ }
+ container.size += 1;
+ Ok(())
+ }
+
+ /// Remove and return the last element.
+ ///
+ /// # Errors
+ ///
+ /// Returns an `IndexError` if the array is empty.
+ ///
+ /// See also [`List::pop`](crate::List::pop).
+ pub fn pop(&mut self) -> Result<T, crate::Error> {
+ let size = self.len();
+ if size == 0 {
+ crate::bail!(crate::error::INDEX_ERROR, "Array pop on an empty
array");
+ }
+ let container = self.make_mut(size);
+ container.size -= 1;
+ // SAFETY: the last cell is initialized, holds a `T`, and is no longer
+ // counted by `size`, so it is moved out exactly once.
+ unsafe {
+ let mut cell = std::ptr::read((container.data as *const
TVMFFIAny).add(size - 1));
+ debug_assert!(T::container_check_any_strict(&cell));
+ Ok(T::container_move_from_any_after_check(&mut cell))
+ }
+ }
+
+ /// Shorten the array to `len` elements, dropping the rest; a no-op when it
+ /// is already that short.
+ ///
+ /// See also [`List::truncate`](crate::List::truncate).
+ pub fn truncate(&mut self, len: usize) {
+ let size = self.len();
+ if len >= size {
+ return;
+ }
+ let container = self.make_mut(size);
+ // Shrink `size` first, so a panicking element drop leaks rather than
+ // double-frees the remaining cells.
+ container.size = len as i64;
+ let cells = container.data as *const TVMFFIAny;
+ for i in len..size {
+ // SAFETY: cells `len..size` are initialized and no longer counted.
+ drop(unsafe { Any::from_raw_ffi_any(*cells.add(i)) });
+ }
+ }
+
+ /// Replace the element at `index`.
+ ///
+ /// # Errors
+ ///
+ /// Returns an `IndexError` if `index >= len`.
+ ///
+ /// See also [`List::set`](crate::List::set).
+ pub fn set(&mut self, index: usize, value: T) -> Result<(), crate::Error> {
+ let size = self.len();
+ if index >= size {
+ crate::bail!(crate::error::INDEX_ERROR, "Array set index out of
bound");
+ }
+ let container = self.make_mut(size);
+ // SAFETY: the cell at `index` is initialized; its old value is dropped
+ // after the new one is written.
+ unsafe {
+ let cell = (container.data as *mut TVMFFIAny).add(index);
+ let old = Any::from_raw_ffi_any(std::ptr::read(cell));
+ write_cell(cell, value);
+ drop(old);
+ }
+ Ok(())
+ }
+
+ /// Make this handle the only reference to its cells, with room for at
least
+ /// `capacity` of them, and return the container to edit in place.
+ fn make_mut(&mut self, capacity: usize) -> &mut ArrayObj {
+ // As C++ `Array::CopyOnWrite(reserve)`.
+ let unique = ObjectArc::is_unique(&self.data);
+ let current = self.data.capacity as usize;
+ if !unique || capacity > current {
+ let capacity = if capacity > current {
+ (current * 2).max(capacity)
} else {
- capacity
+ current
};
- let mut grown = Self::new_with_capacity(Vec::new(), capacity);
- let source = self.as_container().data as *const TVMFFIAny;
- // SAFETY: `grown` is fresh with room for `size` cells. A unique
source
- // moves its cells and forgets them; a shared one keeps its cells,
so each
- // copy takes its own reference.
- unsafe {
- let target = ObjectArc::as_mut_ptr(&mut grown.data);
- let cells = (*target).data as *mut TVMFFIAny;
- if unique {
- std::ptr::copy_nonoverlapping(source, cells, size);
- (*ObjectArc::as_mut_ptr(&mut self.data)).size = 0;
- } else {
- for i in 0..size {
- let copy =
Any::from(AnyView::from_raw_ffi_any(*source.add(i)));
- std::ptr::write(cells.add(i),
Any::into_raw_ffi_any(copy));
- }
- }
- (*target).size = size as i64;
- }
- *self = grown;
+ self.copy_to_new_buffer(capacity, unique);
}
- // SAFETY: this handle is now the only reference, with room for one
more cell.
+ // SAFETY: this handle is now the only reference, so no one else
observes
+ // the container while it is mutably borrowed.
+ unsafe { &mut *ObjectArc::as_mut_ptr(&mut self.data) }
+ }
+
+ /// Replace `self` with a unique array of `capacity` cells holding the same
+ /// elements: moved out of a `unique` source, otherwise each re-referenced.
+ fn copy_to_new_buffer(&mut self, capacity: usize, unique: bool) {
+ let size = self.len();
+ let mut grown = Self::new_with_capacity(Vec::new(), capacity);
+ let source = self.as_container().data as *const TVMFFIAny;
+ // SAFETY: `grown` is fresh with room for `size` cells. A unique source
+ // forgets the cells it moved; a shared one keeps its own references.
unsafe {
- let container = ObjectArc::as_mut_ptr(&mut self.data);
- let mut cell = TVMFFIAny::new();
- T::container_move_to_any(value, &mut cell);
- std::ptr::write(((*container).data as *mut TVMFFIAny).add(size),
cell);
- (*container).size += 1;
+ let target = ObjectArc::as_mut_ptr(&mut grown.data);
+ let cells = (*target).data as *mut TVMFFIAny;
+ if unique {
+ std::ptr::copy_nonoverlapping(source, cells, size);
+ (*ObjectArc::as_mut_ptr(&mut self.data)).size = 0;
+ } else {
+ for i in 0..size {
+ let copy =
Any::from(AnyView::from_raw_ffi_any(*source.add(i)));
+ std::ptr::write(cells.add(i), Any::into_raw_ffi_any(copy));
+ }
+ }
+ (*target).size = size as i64;
}
+ *self = grown;
}
/// Return the number of elements.
diff --git a/rust/tvm-ffi/src/error.rs b/rust/tvm-ffi/src/error.rs
index 1cdf41be..5dd42ef8 100644
--- a/rust/tvm-ffi/src/error.rs
+++ b/rust/tvm-ffi/src/error.rs
@@ -108,8 +108,7 @@ pub struct ErrorObj {
/// assert_eq!(error.message(), "not a digit: 'x'");
/// ```
///
-/// An `Error` is `Send + Sync`, so a `Result` can cross threads; its attached
-/// context is treated as immutable diagnostic data.
+/// An `Error` is `Send + Sync`, so a `Result` can cross threads.
#[repr(C)]
#[derive(Clone, ObjectRef)]
#[custom_debug]
@@ -117,8 +116,7 @@ pub struct Error {
data: ObjectArc<ErrorObj>,
}
-// SAFETY: an `Error` is immutable once built; `extra_context` is diagnostic
data.
-// Deliberately thread-safe so a `Result` can cross threads.
+// SAFETY: a shared Error is never mutated: updates copy unless this handle is
unique.
unsafe impl Send for Error {}
unsafe impl Sync for Error {}
@@ -179,13 +177,13 @@ impl Error {
///
/// * `kind` - The kind of the error, such as [`VALUE_ERROR`].
/// * `message` - The message of the error.
- /// * `traceback` - The backtrace of the error, which may be empty.
+ /// * `backtrace` - The backtrace of the error, which may be empty.
///
/// # Returns
///
/// The error.
- pub fn new(kind: ErrorKind<'_>, message: &str, traceback: &str) -> Self {
- Self::new_with_cause_and_extra_context(kind, message, traceback, None,
None)
+ pub fn new(kind: ErrorKind<'_>, message: &str, backtrace: &str) -> Self {
+ Self::new_with_cause_and_extra_context(kind, message, backtrace, None,
None)
}
/// Create an error retaining its cause and any application-specific
context.
@@ -194,7 +192,7 @@ impl Error {
///
/// * `kind` - The kind of the error.
/// * `message` - The message of the error.
- /// * `traceback` - The backtrace of the error, which may be empty.
+ /// * `backtrace` - The backtrace of the error, which may be empty.
/// * `cause_chain` - The error that caused this one, if any.
/// * `extra_context` - An object of application-specific context, if any.
///
@@ -204,19 +202,19 @@ impl Error {
pub fn new_with_cause_and_extra_context(
kind: ErrorKind<'_>,
message: &str,
- traceback: &str,
+ backtrace: &str,
cause_chain: Option<&Error>,
extra_context: Option<&object::ObjectRef>,
) -> Self {
unsafe {
let kind_data = TVMFFIByteArray::from_str(kind.as_str());
let message_data = TVMFFIByteArray::from_str(message);
- let traceback_data = TVMFFIByteArray::from_str(traceback);
+ let backtrace_data = TVMFFIByteArray::from_str(backtrace);
let mut error_handle: TVMFFIObjectHandle = std::ptr::null_mut();
let ret = TVMFFIErrorCreateWithCauseAndExtraContext(
&kind_data,
&message_data,
- &traceback_data,
+ &backtrace_data,
cause_chain.map_or(std::ptr::null_mut(), |cause| {
ObjectArc::as_ptr(&cause.data) as TVMFFIObjectHandle
}),
@@ -323,10 +321,21 @@ impl Error {
Some(unsafe { &*(handle as *const _ as *const object::ObjectRef) })
}
- /// The `extra_context` slot, to update in place. A shared error is first
- /// replaced by a shallow copy, so the update reaches no other holder.
+ /// The `extra_context` slot, to update in place.
pub(crate) fn extra_context_mut(&mut self) -> &mut
Option<object::ObjectRef> {
- if ObjectArc::strong_count(&self.data) != 1 {
+ let error = self.make_mut();
+ // SAFETY: this handle is the only reference, and the slot owns a
nullable
+ // object handle, which is exactly an `Option<ObjectRef>`.
+ unsafe {
+ &mut *std::ptr::addr_of_mut!((*error).cell.extra_context)
+ .cast::<Option<object::ObjectRef>>()
+ }
+ }
+
+ /// Make this handle the only reference to its error, copying a shared
one, as
+ /// `Arc::make_mut`.
+ fn make_mut(&mut self) -> *mut ErrorObj {
+ if !ObjectArc::is_unique(&self.data) {
*self = Self::new_with_cause_and_extra_context(
self.kind(),
self.message(),
@@ -335,13 +344,7 @@ impl Error {
self.extra_context(),
);
}
- // SAFETY: this handle is the only reference, and the slot owns a
nullable
- // object handle, which is exactly an `Option<ObjectRef>`.
- unsafe {
- let error = ObjectArc::as_mut_ptr(&mut self.data);
- &mut *std::ptr::addr_of_mut!((*error).cell.extra_context)
- .cast::<Option<object::ObjectRef>>()
- }
+ ObjectArc::as_mut_ptr(&mut self.data)
}
/// Get the traceback of the error in the order of most recent call last,
as
@@ -375,33 +378,18 @@ impl Error {
/// # Returns
///
/// The error with the appended backtrace.
- pub fn with_appended_backtrace(this: Self, backtrace: &str) -> Self {
- if ObjectArc::strong_count(&this.data) == 1 {
- // this is the only reference to the error
- // we can safely mutate the error
- unsafe {
- let backtrace_data = TVMFFIByteArray::from_str(backtrace);
- (this.data.cell.update_backtrace)(
- ObjectArc::as_ptr(&this.data) as *mut ErrorObj as *mut
c_void,
- &backtrace_data,
- kTVMFFIBacktraceUpdateModeAppend as i32,
- );
- this
- }
- } else {
- // we need to create a new error because there is more than one
unique reference
- // to the error
- let mut new_backtrace = String::new();
- new_backtrace.push_str(this.backtrace());
- new_backtrace.push_str(backtrace);
- Self::new_with_cause_and_extra_context(
- this.kind(),
- this.message(),
- &new_backtrace,
- this.cause_chain(),
- this.extra_context(),
- )
+ pub fn with_appended_backtrace(mut this: Self, backtrace: &str) -> Self {
+ let error = this.make_mut();
+ // SAFETY: `make_mut` made this handle the only reference to `error`.
+ unsafe {
+ let backtrace_data = TVMFFIByteArray::from_str(backtrace);
+ ((*error).cell.update_backtrace)(
+ error.cast::<c_void>(),
+ &backtrace_data,
+ kTVMFFIBacktraceUpdateModeAppend as i32,
+ );
}
+ this
}
}
@@ -423,4 +411,8 @@ impl std::fmt::Debug for Error {
}
}
-impl std::error::Error for Error {}
+impl std::error::Error for Error {
+ fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
+ self.cause_chain().map(|cause| cause as _)
+ }
+}
diff --git a/rust/tvm-ffi/src/extra/structural_common.rs
b/rust/tvm-ffi/src/extra/structural_common.rs
index 0c1ed75f..8c931e92 100644
--- a/rust/tvm-ffi/src/extra/structural_common.rs
+++ b/rust/tvm-ffi/src/extra/structural_common.rs
@@ -22,9 +22,7 @@ use crate::collections::array::Array;
use crate::derive::Object;
use crate::error::{Error, Result, RUNTIME_ERROR};
use crate::object::{self, Object, ObjectArc, ObjectRef, ObjectRefCore};
-use crate::tvm_ffi_sys::{
- TVMFFIAny, TVMFFIAnyDataUnion, TVMFFIGetTypeInfo, TVMFFIObject,
TVMFFITypeIndex,
-};
+use crate::tvm_ffi_sys::{TVMFFIAny, TVMFFIAnyDataUnion, TVMFFIObject,
TVMFFITypeIndex};
//
-----------------------------------------------------------------------------
// Shared outlined errors
@@ -80,24 +78,6 @@ pub(super) unsafe fn structural_error_from_raw(
}
}
-/// Return the registered type key for `type_index`.
-///
-/// When the runtime returns no type information, return `<type_index N>` for
diagnostics.
-///
-/// # Arguments
-///
-/// * `type_index` - Runtime type index whose registered type key is requested.
-pub(crate) fn type_key_of(type_index: i32) -> String {
- unsafe {
- let info = TVMFFIGetTypeInfo(type_index);
- if info.is_null() {
- format!("<type_index {type_index}>")
- } else {
- (*info).type_key.as_str().to_string()
- }
- }
-}
-
// Perf note: cold marks these error-composition paths as uncommon; inlining
// within error paths is compiler-selected. Use explicit inline(never) at
boundaries
// where scratch must stay outside recursive callers.
@@ -168,7 +148,7 @@ pub(crate) fn with_visit_error_context(mut error: Error,
raw: TVMFFIAny) -> Erro
.cast::<VisitErrorContextObj>(),
)
};
- if ObjectArc::strong_count(&context) != 1 {
+ if !ObjectArc::is_unique(&context) {
context = ObjectArc::new(VisitErrorContextObj {
base: Object::new(),
reverse_visit_pattern:
context.reverse_visit_pattern.clone(),
@@ -284,3 +264,43 @@ pub(crate) unsafe fn raw_ffi_any_into_result(raw:
TVMFFIAny) -> Result<Any> {
raw.data_union,
))
}
+
+#[cfg(test)]
+mod tests {
+ use super::VisitErrorContextObj;
+ use crate::object::ObjectCore;
+
+ // The binding must match the layout C++ registers for
`ffi.VisitErrorContext`.
+ #[test]
+ fn visit_error_context_layout_matches_registered_fields() {
+ let expected = [
+ (
+ "reverse_visit_pattern",
+ std::mem::offset_of!(VisitErrorContextObj,
reverse_visit_pattern),
+ ),
+ (
+ "prev_error_context",
+ std::mem::offset_of!(VisitErrorContextObj, prev_error_context),
+ ),
+ ];
+ // SAFETY: the type is registered by the runtime; its field table
lives as long.
+ let fields = unsafe {
+ let info =
crate::tvm_ffi_sys::TVMFFIGetTypeInfo(VisitErrorContextObj::type_index());
+ assert!(!info.is_null());
+ std::slice::from_raw_parts((*info).fields, (*info).num_fields as
usize)
+ };
+ assert_eq!(fields.len(), expected.len());
+ for (name, offset) in expected {
+ let field = fields
+ .iter()
+ .find(|field| field.name.as_str() == name)
+ .unwrap_or_else(|| panic!("`{name}` is not a registered
field"));
+ assert_eq!(field.offset as usize, offset, "offset of `{name}`");
+ assert_eq!(
+ field.size as usize,
+ std::mem::size_of::<usize>(),
+ "size of `{name}`"
+ );
+ }
+ }
+}
diff --git a/rust/tvm-ffi/src/extra/structural_map.rs
b/rust/tvm-ffi/src/extra/structural_map.rs
index 81313a9e..cf25e3f6 100644
--- a/rust/tvm-ffi/src/extra/structural_map.rs
+++ b/rust/tvm-ffi/src/extra/structural_map.rs
@@ -124,7 +124,7 @@ where
return mutator.default_mutate(self, value).map(Into::into);
};
let mapped: Any = callback_result
- .map_err(|error| with_mutation_error_context(error, raw))?
+ .map_err(|error| with_mutation_error_context(error, raw,
"map"))?
.into();
if is_unchanged(&mapped) || raw_ffi_any_same_as(raw,
*mapped.as_raw_ffi_any()) {
// Drop temporary ownership before default descent checks
uniqueness.
@@ -150,9 +150,9 @@ where
};
match self.dispatch.dispatch_tuple((current, kind)) {
Some(Ok(result)) if result.is_unchanged() => Ok(mapped),
- Some(result) => result
- .map(Any::from)
- .map_err(|error| with_mutation_error_context(error,
*current.as_raw_ffi_any())),
+ Some(result) => result.map(Any::from).map_err(|error| {
+ with_mutation_error_context(error,
*current.as_raw_ffi_any(), "map")
+ }),
None => Ok(mapped),
}
}
diff --git a/rust/tvm-ffi/src/extra/structural_mutate.rs
b/rust/tvm-ffi/src/extra/structural_mutate.rs
index 224dfa8c..75aeec40 100644
--- a/rust/tvm-ffi/src/extra/structural_mutate.rs
+++ b/rust/tvm-ffi/src/extra/structural_mutate.rs
@@ -75,7 +75,7 @@ use crate::tvm_ffi_sys::{TVMFFIObjectHandle,
TVMFFISEqHashKind};
use super::structural_common::{
missing_field_getter_error, raw_ffi_any_into_result, raw_ffi_any_same_as,
- result_into_raw_ffi_any, runtime_error, type_key_of,
with_structural_error_context,
+ result_into_raw_ffi_any, runtime_error, with_structural_error_context,
with_structural_field_error_context, with_visit_error_context,
};
use super::structural_visit::{
@@ -193,10 +193,13 @@ unsafe impl StructuralMutatorBase for
StructuralMutatorObj {
#[inline(always)]
unsafe fn default_mutate_raw(runtime: *mut Self, value: MutationView<'_>)
-> TVMFFIAny {
- // This implementation is entered only by the ordinary Rust engine, so
- // reflection can access its remap directly. Prefix-only callers use
- // `default_mutate_raw`, which dispatches remaps through the native
vtable.
- Self::default_mutate_impl::<true>(runtime, value)
+ // Prefix-only: remaps go through the vtable, so any base may reuse
this.
+ let raw = value.value;
+ // Registered hooks own their remapping policy; reflection handles its
own.
+ if let Some(attr) = Self::structural_mutate_hook(raw, value.mode) {
+ return Self::call_structural_mutate_hook_raw(runtime, raw, attr);
+ }
+ Self::mutate_without_hook_raw(runtime, raw.type_index,
raw.small_str_len, raw.data_union)
}
}
@@ -231,19 +234,18 @@ impl Default for StructuralMutatorObj {
/// Each pointer key owns one strong reference to its original object,
preventing
/// address reuse until removal. Values are owning replacements, including
/// [`Unchanged`] markers. Replacing a value does not acquire another key
reference;
-/// lookup clones the replacement, and clearing or dropping releases every
owner.
+/// lookup clones the replacement, and dropping releases every owner.
// Perf note: Store only the replacement Any; raw identity keys avoid a second
// Any payload, but still own one retained reference per distinct key.
Replacing
-// a value does not retain the key again; clear/Drop release each key exactly
once.
-#[doc(hidden)]
+// a value does not retain the key again; Drop releases each key exactly once.
#[derive(Default)]
-pub struct StructuralVarRemap {
+pub(crate) struct StructuralVarRemap {
entries: HashMap<NonNull<TVMFFIObject>, Any>,
}
impl StructuralVarRemap {
/// Look up an identity replacement previously stored for `var`.
- pub fn get(&self, var: AnyView<'_>) -> Result<Option<Any>> {
+ pub(crate) fn get(&self, var: AnyView<'_>) -> Result<Option<Any>> {
let key = object_identity_key(*var.as_raw_ffi_any())?;
Ok(self.entries.get(&key).cloned())
}
@@ -252,7 +254,7 @@ impl StructuralVarRemap {
///
/// A new key acquires one strong reference; replacing its value preserves
/// that reference. The replacement is cloned before modifying the table.
- pub fn set(&mut self, var: AnyView<'_>, mutated_value: &Any) -> Result<()>
{
+ pub(crate) fn set(&mut self, var: AnyView<'_>, mutated_value: &Any) ->
Result<()> {
let key = object_identity_key(*var.as_raw_ffi_any())?;
let value = mutated_value.clone();
match self.entries.entry(key) {
@@ -268,27 +270,8 @@ impl StructuralVarRemap {
}
Ok(())
}
-
- /// Remove every substitution, releasing each key and replacement owner.
- pub fn clear(&mut self) {
- let empty = HashMap::with_hasher(self.entries.hasher().clone());
- let mut entries = std::mem::replace(&mut self.entries, empty);
- // Release all key references before dropping values. Object deleters
- // use the non-unwinding C ABI, so this pass cannot leave keys
unreleased.
- // Subsequent cleanup only drops pointer bits, never dereferences them.
- for key in entries.keys() {
- unsafe { object::unsafe_::dec_ref(key.as_ptr()) };
- }
- entries.clear();
- // Reuse capacity without replacing any entries created during cleanup.
- if self.entries.is_empty() {
- self.entries = entries;
- }
- }
}
-// Perf note: Terminal destruction needs no replacement table or reusable
capacity.
-// Keep it separate from clear(), which leaves a valid remap ready for later
use.
impl Drop for StructuralVarRemap {
fn drop(&mut self) {
// Release keys first: their C ABI deleters cannot unwind. Field Drop
@@ -968,7 +951,7 @@ fn missing_shallow_copy_error(type_index: i32, attribute:
&str) -> Error {
TYPE_ERROR,
&format!(
"type `{}` cannot use reflected structural mutation because it
does not define `{attribute}`",
- type_key_of(type_index)
+ crate::object::type_key_of_index(type_index)
),
"",
)
@@ -1008,7 +991,7 @@ fn missing_field_setter_error(name: &str) -> Error {
#[inline(never)]
fn invalid_field_offset_error(name: &str) -> Error {
runtime_error(&format!(
- "native structural map: reflected field `{name}` has an invalid offset"
+ "native structural mutate: reflected field `{name}` has an invalid
offset"
))
}
@@ -1036,7 +1019,7 @@ fn var_remap_key_error() -> Error {
#[inline(never)]
fn unregistered_type_error(type_index: i32) -> Error {
runtime_error(&format!(
- "native structural map: unregistered type index {type_index}"
+ "native structural mutate: unregistered type index {type_index}"
))
}
@@ -1048,14 +1031,17 @@ fn wrong_mutation_dispatcher() -> ! {
}
#[cold]
-pub(super) fn with_mutation_error_context(error: Error, raw: TVMFFIAny) ->
Error {
+pub(super) fn with_mutation_error_context(error: Error, raw: TVMFFIAny,
operation: &str) -> Error {
if raw.type_index < TVMFFITypeIndex::kTVMFFIStaticObjectBegin as i32 {
error
} else {
with_structural_error_context(
with_visit_error_context(error, raw),
- "map",
- &format!("object `{}`", type_key_of(raw.type_index)),
+ operation,
+ &format!(
+ "object `{}`",
+ crate::object::type_key_of_index(raw.type_index)
+ ),
)
}
}
@@ -1148,7 +1134,7 @@ fn object_identity_key(raw: TVMFFIAny) ->
Result<NonNull<TVMFFIObject>> {
let pointer = unsafe { raw.data_union.v_obj };
NonNull::new(pointer).ok_or_else(|| {
core::hint::cold_path();
- runtime_error("native structural map: identity object has a null
pointer")
+ runtime_error("native structural mutate: identity object has a null
pointer")
})
}
@@ -1168,7 +1154,7 @@ fn object_is_unique(raw: TVMFFIAny) -> bool {
return false;
}
let pointer = unsafe { raw.data_union.v_obj };
- !pointer.is_null() && unsafe { object::unsafe_::strong_count(pointer) == 1
}
+ !pointer.is_null() && unsafe { object::unsafe_::is_unique(pointer) }
}
/// Acquire an owning value from a valid borrowed carrier.
@@ -1187,7 +1173,7 @@ fn raw_ffi_any_to_owned_any(raw: TVMFFIAny) ->
Result<Any> {
let object = unsafe { raw.data_union.v_obj };
if object.is_null() {
return Err(runtime_error(
- "native structural map: object-backed value has a null
pointer",
+ "native structural mutate: object-backed value has a null
pointer",
));
}
unsafe { object::unsafe_::inc_ref(object) };
@@ -1334,7 +1320,7 @@ impl<Base: StructuralMutatorBase>
StructuralMutateEngine<Base> {
Some(result) => {
let result = result.map(Any::from);
result_into_raw_ffi_any(if CALLBACK_CONTEXT {
- result.map_err(|error| with_mutation_error_context(error,
raw))
+ result.map_err(|error| with_mutation_error_context(error,
raw, "mutate"))
} else {
result
})
@@ -1532,7 +1518,7 @@ impl StructuralMutatorObj {
};
result_into_raw_ffi_any(
raw_ffi_any_into_result(result)
- .map_err(|error| with_mutation_error_context(error, raw)),
+ .map_err(|error| with_mutation_error_context(error, raw,
"mutate")),
)
}
@@ -1563,7 +1549,7 @@ impl StructuralMutatorObj {
let output_object = unsafe { output_raw.data_union.v_obj.cast::<u8>()
};
if output_object.is_null() {
return Err(runtime_error(
- "native structural map: shallow copy has a null object
pointer",
+ "native structural mutate: shallow copy has a null object
pointer",
));
}
@@ -1615,7 +1601,7 @@ impl StructuralMutatorObj {
) -> Result<()> {
let Some(getter) = field.getter else {
return Err(missing_field_getter_error(
- "structural map",
+ "structural mutate",
field.name.as_str(),
));
};
@@ -1633,7 +1619,7 @@ impl StructuralMutatorObj {
if getter(source_address, Any::as_raw_ffi_any_mut(&mut child)) != 0 {
return Err(with_structural_field_error_context(
Error::from_raised(),
- "map",
+ "mutate",
field.name.as_str(),
));
}
@@ -1644,13 +1630,15 @@ impl StructuralMutatorObj {
let mapped = Self::with_def_region_kind(runtime, child_region, |_| {
Self::call_mutator(runtime, child_raw, InplaceMode::Disallow)
})
- .map_err(|error| with_structural_field_error_context(error, "map",
field.name.as_str()))?;
+ .map_err(|error| {
+ with_structural_field_error_context(error, "mutate",
field.name.as_str())
+ })?;
if is_unchanged(&mapped) || raw_ffi_any_same_as(child_raw,
*mapped.as_raw_ffi_any()) {
return Ok(());
}
call_field_setter(field, source_address,
mapped.as_raw_ffi_any()).map_err(|error| {
- with_structural_field_error_context(error, "map",
field.name.as_str())
+ with_structural_field_error_context(error, "mutate",
field.name.as_str())
})?;
*field_changed = true;
Ok(())
@@ -1777,7 +1765,9 @@ impl StructuralMutatorObj {
Err(invalid_mutation_hook_error())
};
// Preserve successful Error-valued returns until the public
conversion.
- result_into_raw_ffi_any(result.map_err(|error|
with_mutation_error_context(error, raw)))
+ result_into_raw_ffi_any(
+ result.map_err(|error| with_mutation_error_context(error, raw,
"mutate")),
+ )
}
/// Borrow a live runtime mutator as an object-valued ABI argument.
@@ -1829,55 +1819,6 @@ impl StructuralMutatorObj {
}
}
- /// Default descent through native hooks or reflected fields, preserving
the
- /// capability's reuse permission. Return one owning
replacement/Unchanged/Error.
- ///
- /// # Safety
- /// `runtime` must be a live initialized mutator prefix with a matching
installed
- /// vtable. Publish the current exclusive dispatcher reborrow first. The
runtime
- /// and input remain borrowed for this synchronous call; hooks obey
- /// [`StructuralMutatorBase`]'s non-escaping contract.
- // Operate in the raw path to optimize stack frame cost. Each returned raw
- // carrier transfers one owning result; fallible/destructor-bearing code
- // imports it into Any before performing work that may unwind.
- #[inline(always)]
- pub unsafe fn default_mutate_raw(runtime: *mut Self, value:
MutationView<'_>) -> TVMFFIAny {
- Self::default_mutate_impl::<false>(runtime, value)
- }
-
- // ORDINARY_ENGINE is true only for the StructuralMutatorBase
implementation
- // above, whose runtime has the exact ordinary Rust engine layout.
Arbitrary
- // native prefixes must keep using the installed remap callbacks.
- #[inline(always)]
- unsafe fn default_mutate_impl<const ORDINARY_ENGINE: bool>(
- runtime: *mut Self,
- value: MutationView<'_>,
- ) -> TVMFFIAny {
- let raw = value.value;
- let mode = value.mode;
- // Registered hooks own their remapping policy; reflection handles its
own.
- if let Some(attr) = Self::structural_mutate_hook(raw, mode) {
- return Self::call_structural_mutate_hook_raw(runtime, raw, attr);
- }
- unsafe {
- if ORDINARY_ENGINE {
- Self::ordinary_mutate_without_hook_raw(
- runtime,
- raw.type_index,
- raw.small_str_len,
- raw.data_union,
- )
- } else {
- Self::mutate_without_hook_raw(
- runtime,
- raw.type_index,
- raw.small_str_len,
- raw.data_union,
- )
- }
- }
- }
-
/// Snapshot the live `runtime` region before fallback work, reconstruct
borrowed
/// input from scalar fields, and transfer one owning
replacement/Unchanged/Error.
/// The private C-unwind ABI keeps AArch64 results in registers while
permitting
@@ -1896,28 +1837,6 @@ impl StructuralMutatorObj {
type_index: i32,
small_str_len: u32,
data_union: TVMFFIAnyDataUnion,
- ) -> TVMFFIAny {
- Self::mutate_without_hook_impl::<false>(runtime, type_index,
small_str_len, data_union)
- }
-
- // Keep the ordinary fallback in this crate, like the native-prefix
fallback;
- // it does not depend on the caller's dispatcher monomorphization.
- #[inline(never)]
- unsafe extern "C-unwind" fn ordinary_mutate_without_hook_raw(
- runtime: StructuralMutatorHandle,
- type_index: i32,
- small_str_len: u32,
- data_union: TVMFFIAnyDataUnion,
- ) -> TVMFFIAny {
- Self::mutate_without_hook_impl::<true>(runtime, type_index,
small_str_len, data_union)
- }
-
- #[inline(always)]
- unsafe fn mutate_without_hook_impl<const ORDINARY_ENGINE: bool>(
- runtime: StructuralMutatorHandle,
- type_index: i32,
- small_str_len: u32,
- data_union: TVMFFIAnyDataUnion,
) -> TVMFFIAny {
// Perf note: Read region state here, before fallback semantics need
it,
// rather than loading it on the native-hook path that does not use it.
@@ -1929,8 +1848,8 @@ impl StructuralMutatorObj {
data_union,
};
result_into_raw_ffi_any(
- Self::mutate_without_hook::<ORDINARY_ENGINE>(runtime, raw,
def_region_kind)
- .map_err(|error| with_mutation_error_context(error, raw)),
+ Self::mutate_without_hook(runtime, raw, def_region_kind)
+ .map_err(|error| with_mutation_error_context(error, raw,
"mutate")),
)
}
@@ -1968,7 +1887,7 @@ impl StructuralMutatorObj {
/// policy and owning reflected results. Internal callers guarantee valid
inputs;
/// normal Rust owners clean up partial work if reflection fails or
unwinds.
#[inline(always)]
- fn mutate_without_hook<const ORDINARY_ENGINE: bool>(
+ fn mutate_without_hook(
runtime: StructuralMutatorHandle,
raw: TVMFFIAny,
def_region_kind: DefRegionKind,
@@ -1988,18 +1907,9 @@ impl StructuralMutatorObj {
let is_free_var = kind ==
Some(TVMFFISEqHashKind::kTVMFFISEqHashKindFreeVar as i32);
let is_dag_node = kind ==
Some(TVMFFISEqHashKind::kTVMFFISEqHashKindDAGNode as i32);
if is_free_var || is_dag_node {
- let mutated = unsafe {
- if ORDINARY_ENGINE {
-
StructuralMutateEngine::<Self>::runtime_var_remap_get(runtime, raw)?
- } else {
- Self::var_remap_get(runtime, raw)?
- }
- };
- if let Some(mutated) = mutated {
- // The ABI uses None for a miss; an Unchanged marker is a
cached result.
- if mutated.type_index() != TVMFFITypeIndex::kTVMFFINone as i32
{
- return Ok(mutated);
- }
+ // The ABI uses None for a miss; an Unchanged marker is a cached
result.
+ if let Some(mutated) = unsafe { Self::var_remap_get(runtime, raw)?
} {
+ return Ok(mutated);
}
}
// A variable use with no binding retains its identity without
visiting fields.
@@ -2014,13 +1924,7 @@ impl StructuralMutatorObj {
{
// Keep markers for DAG nodes and pattern definitions. An
unchanged simple
// definition needs no binding: subsequent uses retain the
original on a miss.
- unsafe {
- if ORDINARY_ENGINE {
-
StructuralMutateEngine::<Self>::runtime_var_remap_set(runtime, raw, &result)?;
- } else {
- Self::var_remap_set(runtime, raw, &result)?;
- }
- }
+ unsafe { Self::var_remap_set(runtime, raw, &result)? };
}
Ok(result)
}
diff --git a/rust/tvm-ffi/src/extra/structural_visit.rs
b/rust/tvm-ffi/src/extra/structural_visit.rs
index a27c9158..7ca70d05 100644
--- a/rust/tvm-ffi/src/extra/structural_visit.rs
+++ b/rust/tvm-ffi/src/extra/structural_visit.rs
@@ -95,7 +95,7 @@ use crate::tvm_ffi_sys::{
};
use super::structural_common::{
- missing_field_getter_error, runtime_error, type_key_of,
with_structural_error_context,
+ missing_field_getter_error, runtime_error, with_structural_error_context,
with_structural_field_error_context, with_visit_error_context,
};
@@ -562,7 +562,10 @@ fn annotate_visit_error(
with_structural_error_context(
error,
"visit",
- &format!("object `{}`", type_key_of(value.type_index)),
+ &format!(
+ "object `{}`",
+ crate::object::type_key_of_index(value.type_index)
+ ),
)
}
diff --git a/rust/tvm-ffi/src/function.rs b/rust/tvm-ffi/src/function.rs
index a71b1981..520fe300 100644
--- a/rust/tvm-ffi/src/function.rs
+++ b/rust/tvm-ffi/src/function.rs
@@ -72,7 +72,8 @@ pub struct Function {
data: ObjectArc<FunctionObj>,
}
-// SAFETY: a `Function` is immutable, and every Rust callback it wraps is
`Send + Sync`.
+// SAFETY: a `Function` is immutable, every Rust callback it wraps is `Send +
Sync`,
+// and `from_extern_c` requires the same of foreign callbacks.
unsafe impl Send for Function {}
unsafe impl Sync for Function {}
@@ -467,11 +468,12 @@ impl Function {
Self::from_packed(closure)
}
+ /// Wrap a C callback as a Function.
+ ///
/// # Safety
///
- /// `handle` must be a valid pointer (or null) that is compatible with
- /// `safe_call` and `deleter`. The caller must ensure the handle outlives
- /// the returned `Function` (or that `deleter` properly frees it).
+ /// `handle` (may be null) is the state `safe_call` receives; `deleter`
frees it
+ /// when the Function drops. All three must be safe to use from any thread.
///
/// # Errors
///
diff --git a/rust/tvm-ffi/src/lib.rs b/rust/tvm-ffi/src/lib.rs
index f6163a62..76b19503 100644
--- a/rust/tvm-ffi/src/lib.rs
+++ b/rust/tvm-ffi/src/lib.rs
@@ -30,6 +30,7 @@ pub mod extra;
pub mod function;
#[doc(hidden)]
pub mod function_internal;
+#[doc(hidden)]
pub mod macros;
#[doc(hidden)]
pub mod match_any_internal;
@@ -59,8 +60,6 @@ pub use crate::error::{
pub use crate::extra::dispatch::{Dispatcher, IntoDispatcher};
pub use crate::extra::module::Module;
pub use crate::extra::structural_map::structural_map;
-#[doc(hidden)]
-pub use crate::extra::structural_mutate::StructuralVarRemap;
pub use crate::extra::structural_mutate::{
structural_mutate, InplaceMode, MutationView, StructuralMutateEngine,
StructuralMutatorBase,
StructuralMutatorObj, StructuralMutatorView,
diff --git a/rust/tvm-ffi/src/macros.rs b/rust/tvm-ffi/src/macros.rs
index 9e1d4839..47a43e81 100644
--- a/rust/tvm-ffi/src/macros.rs
+++ b/rust/tvm-ffi/src/macros.rs
@@ -16,7 +16,8 @@
* specific language governing permissions and limitations
* under the License.
*/
-// rexport paste under macro namespace so downstream do not need to specify dep
+// Re-export paste for the export macros, so downstream crates need not depend
on it.
+#[doc(hidden)]
pub use paste;
// ----------------------------------------------------------------------------
// Macros for error handling
@@ -41,12 +42,24 @@ macro_rules! function_name {
}};
}
-/// Resolves a registered global [`Function`](crate::function::Function) by
name
-/// and caches it for the lifetime of the process (lock-free after the first
-/// lookup). Each call site gets its own cache and the macro evaluates to a
-/// `&'static Function`. Panics — naming the function — if it is not
registered.
+/// Look up a registered global [`Function`](crate::Function) by name once, and
+/// keep it for the process: a `&'static Function`.
+///
+/// Each call site has its own cache, which later calls read without a lock.
+/// Use it for a function that must be registered, such as a built-in one; it
+/// panics, naming the function, if the function is not. See also
+/// [`Function::get_global`](crate::Function::get_global), which returns an
+/// error instead.
+///
+/// # Examples
+///
+/// ```
+/// use tvm_ffi::{cached_global_func, Array};
///
-/// Usage: `cached_global_func!("ffi.Map").call_packed(args)`
+/// let array = cached_global_func!("ffi.Array").call_tuple((1_i64, 2_i64))?;
+/// assert_eq!(Array::<i64>::try_from(array)?.len(), 2);
+/// # Ok::<(), tvm_ffi::Error>(())
+/// ```
#[macro_export]
macro_rules! cached_global_func {
($name:literal) => {{
@@ -172,6 +185,113 @@ macro_rules! unsafe_impl_object_upcast {
};
}
+/// Implement the in-place reference downcasts (`array[i]`,
`Any::downcast_ref`,
+/// [`ObjectRefCore::downcast_ref`](crate::ObjectRefCore::downcast_ref)) for
each
+/// listed reference type implemented by hand, as `#[derive(ObjectRef)]` does.
+///
+/// # Safety
+///
+/// Each type is `repr(transparent)` over one object pointer, and its
+/// [`AnyCompatible::check_any_strict`](crate::AnyCompatible::check_any_strict)
+/// accepts exactly the objects that are a valid value of it.
+///
+/// # Examples
+///
+/// ```
+/// use tvm_ffi::object::{ObjectArc, ObjectRef};
+/// use tvm_ffi::{def_object_ref, rs_object, unsafe_impl_ref_downcast, Object,
ObjectRefCore};
+/// # use tvm_ffi::{tvm_ffi_sys::TVMFFIAny, AnyCompatible};
+///
+/// #[rs_object("my_ext.MyPair", final)]
+/// pub struct MyPairObj {
+/// base: Object,
+/// pub first: i64,
+/// pub second: i64,
+/// }
+/// def_object_ref!(pub MyPair, ObjectRef, MyPairObj);
+///
+/// /// A `MyPair` whose `first` is positive.
+/// #[repr(transparent)]
+/// #[derive(Clone)]
+/// pub struct PositivePair(ObjectArc<MyPairObj>);
+/// # unsafe impl ObjectRefCore for PositivePair {
+/// # type ContainerType = MyPairObj;
+/// # fn data(this: &Self) -> &ObjectArc<MyPairObj> { &this.0 }
+/// # fn into_data(this: Self) -> ObjectArc<MyPairObj> { this.0 }
+/// # unsafe fn from_data(data: ObjectArc<MyPairObj>) -> Self { Self(data)
}
+/// # }
+/// # unsafe impl AnyCompatible for PositivePair {
+/// # fn type_str() -> String { "PositivePair".into() }
+/// # unsafe fn copy_to_any_view(src: &Self, data: &mut TVMFFIAny) {
+/// # MyPair::copy_to_any_view(std::mem::transmute::<&Self,
&MyPair>(src), data)
+/// # }
+/// # unsafe fn move_to_any(src: Self, data: &mut TVMFFIAny) {
+/// # MyPair::move_to_any(MyPair::from_data(src.0), data)
+/// # }
+/// # unsafe fn check_any_strict(data: &TVMFFIAny) -> bool {
+/// # MyPair::check_any_strict(data) &&
(*data.data_union.v_obj.cast::<MyPairObj>()).first > 0
+/// # }
+/// # unsafe fn copy_from_any_view_after_check(data: &TVMFFIAny) -> Self {
+/// #
Self(MyPair::into_data(MyPair::copy_from_any_view_after_check(data)))
+/// # }
+/// # unsafe fn move_from_any_after_check(data: &mut TVMFFIAny) -> Self {
+/// # Self(MyPair::into_data(MyPair::move_from_any_after_check(data)))
+/// # }
+/// # unsafe fn try_cast_from_any_view(data: &TVMFFIAny) -> Result<Self,
()> {
+/// # if Self::check_any_strict(data) {
Ok(Self::copy_from_any_view_after_check(data)) } else { Err(()) }
+/// # }
+/// # }
+/// // (`ObjectRefCore` and `AnyCompatible` for `PositivePair` are written by
hand.)
+/// unsafe_impl_ref_downcast!(PositivePair);
+///
+/// let pair: ObjectRef = MyPair::from(MyPairObj::new(Object::new(), 1,
2)).into();
+/// assert!(pair.downcast_ref::<PositivePair>().is_some());
+/// ```
+#[macro_export]
+macro_rules! unsafe_impl_ref_downcast {
+ ($($name:ty),+ $(,)?) => {
+ $(
+ // SAFETY: the macro declaration promises that `$name` is one
object
+ // pointer whose `check_any_strict` is its type check.
+ unsafe impl $crate::type_traits::RefFromAnyUnchecked for $name {
+ #[inline(always)]
+ unsafe fn ref_from_any_unchecked(
+ cell: &$crate::tvm_ffi_sys::TVMFFIAny,
+ ) -> &Self {
+ const {
+ assert!(
+ ::core::mem::size_of::<$name>()
+ == ::core::mem::size_of::<*mut
$crate::tvm_ffi_sys::TVMFFIObject>()
+ && ::core::mem::align_of::<$name>()
+ == ::core::mem::align_of::<*mut
$crate::tvm_ffi_sys::TVMFFIObject>(),
+ "a reference downcast target must be exactly one
object pointer"
+ )
+ };
+ unsafe { &*(::core::ptr::addr_of!(cell.data_union) as
*const Self) }
+ }
+ }
+
+ unsafe impl $crate::type_traits::DowncastRefFromAny for $name {
+ #[inline(always)]
+ unsafe fn downcast_ref_from_any(
+ cell: &$crate::tvm_ffi_sys::TVMFFIAny,
+ ) -> ::core::option::Option<&Self> {
+ unsafe {
$crate::type_traits::downcast_ref_from_any_in_place::<Self>(cell) }
+ }
+ }
+
+ unsafe impl $crate::type_traits::DowncastRefFromObject for $name {
+ #[inline(always)]
+ unsafe fn downcast_ref_from_object(
+ place: &*mut $crate::tvm_ffi_sys::TVMFFIObject,
+ ) -> ::core::option::Option<&Self> {
+ unsafe {
$crate::type_traits::downcast_ref_from_object_in_place::<Self>(place) }
+ }
+ }
+ )+
+ };
+}
+
/// Check the return code of the safe call
///
/// # Arguments
@@ -332,15 +452,15 @@ macro_rules! attach_context {
// Macros for any definitions
// ----------------------------------------------------------------------------
-// implements try from any for all integer types
/// Implement `TryFrom<AnyView>`, `TryFrom<Any>` and `TryFrom<&Any>` for each
listed
-/// [`AnyCompatible`](crate::type_traits::AnyCompatible) type, so it converts
out of
-/// FFI values.
-///
-/// `#[derive(ObjectRef)]` calls this. Call it yourself for a reference type
you
-/// implement by hand, such as a typed view over another reference, together
with
-/// [`impl_arg_into_ref!`](crate::impl_arg_into_ref) and
-/// [`impl_into_arg_holder_default!`](crate::impl_into_arg_holder_default).
+/// [`AnyCompatible`](crate::AnyCompatible) type, so it converts out of FFI
values
+/// by its `AnyCompatible` checks.
+///
+/// `#[derive(ObjectRef)]` calls it. A reference type implemented by hand, such
+/// as a typed view over another reference, calls it with
+/// [`impl_arg_into_ref!`](crate::impl_arg_into_ref),
+/// [`impl_into_arg_holder_default!`](crate::impl_into_arg_holder_default) and
+/// [`unsafe_impl_ref_downcast!`](crate::unsafe_impl_ref_downcast).
#[macro_export]
macro_rules! impl_try_from_any {
($($t:ty),* $(,)?) => {
@@ -417,11 +537,11 @@ macro_rules! impl_try_from_any_for_parametric {
};
}
-/// Implement `IntoArgHolder` for each listed type, so it can be passed to a
-/// [`Function`](crate::Function) call.
+/// Let each listed [`AnyCompatible`](crate::AnyCompatible) type, owned or
+/// borrowed, be an argument of a function from
+/// [`into_typed_fn!`](crate::into_typed_fn).
///
-/// `#[derive(ObjectRef)]` calls this. Call it yourself for a reference type
you
-/// implement by hand, such as a typed view over another reference, together
with
+/// A reference type implemented by hand calls it with
/// [`impl_try_from_any!`](crate::impl_try_from_any) and
/// [`impl_arg_into_ref!`](crate::impl_arg_into_ref).
#[macro_export]
@@ -444,11 +564,11 @@ macro_rules! impl_into_arg_holder_default {
};
}
-/// Implement `ArgIntoRef` for each listed type, so a typed function can take
it
-/// as an argument by reference.
+/// Let each listed [`AnyCompatible`](crate::AnyCompatible) type, owned or
+/// borrowed, be an element of the arguments of
+/// [`Function::call_tuple`](crate::Function::call_tuple).
///
-/// `#[derive(ObjectRef)]` calls this. Call it yourself for a reference type
you
-/// implement by hand, such as a typed view over another reference, together
with
+/// A reference type implemented by hand calls it with
/// [`impl_try_from_any!`](crate::impl_try_from_any) and
/// [`impl_into_arg_holder_default!`](crate::impl_into_arg_holder_default).
#[macro_export]
diff --git a/rust/tvm-ffi/src/object.rs b/rust/tvm-ffi/src/object.rs
index 36c0ff51..316b3243 100644
--- a/rust/tvm-ffi/src/object.rs
+++ b/rust/tvm-ffi/src/object.rs
@@ -154,19 +154,21 @@ pub unsafe trait ObjectCoreWithExtraItems: ObjectCore {
}
}
-/// A managed object reference: the category of types such as
[`ObjectRef`](struct@ObjectRef),
-/// `Array<T>` and every `#[derive(ObjectRef)]` type, with the operations every
-/// reference supports: identity ([`same_as`](Self::same_as)) and strict casts
-/// between reference types, which check the object's runtime type and never
-/// convert. [`downcast_ref`](Self::downcast_ref) borrows in place, and
-/// [`downcast`](Self::downcast) takes the reference, as std's
-/// `dyn Any::downcast_ref` and `Box<dyn Any>::downcast` do. Upcasts are
-/// `From`/`Into`, which `#[derive(ObjectRef)]` and
-/// [`unsafe_impl_object_upcast!`](crate::unsafe_impl_object_upcast) generate.
+/// A managed object reference, such as [`ObjectRef`](struct@ObjectRef),
`Array<T>`
+/// or a `#[derive(ObjectRef)]` type, with identity
([`same_as`](Self::same_as))
+/// and strict casts that check the runtime type and never convert; upcasts are
+/// `From`/`Into`.
///
-/// The derive implements it. `data`, `into_data` and `from_data` are the
-/// handle plumbing the FFI uses, called as associated functions; the trait is
-/// `unsafe` to implement because they move the reference's internals.
+/// # Casts
+///
+/// * [`downcast_ref`](Self::downcast_ref) borrows, strictly, as std's
+/// `dyn Any::downcast_ref` does.
+/// * [`downcast`](Self::downcast) takes the value, strictly, and hands it back
+/// on a mismatch, as `Box<dyn Any>::downcast` does; the counterpart of C++
+/// `as<T>()`.
+/// * `TryFrom` takes the value and returns a `TypeError`, for `?`. Between
+/// references it only checks; from an [`Any`](crate::Any) it converts by
+/// the FFI rules, such as an integer into a float.
///
/// # Safety
///
@@ -222,13 +224,9 @@ pub unsafe trait ObjectRefCore: Sized + Clone +
AnyCompatible {
}
}
- /// Take `self` as the reference type `B` when its object is one, without
- /// copying the object or changing its reference count.
- ///
- /// `B` is a subtype, or a parameterized container such as `Array<i64>`,
- /// whose elements are checked too. On a mismatch the reference is returned
- /// unchanged, so another type can be tried. For `?`, use the `TryFrom`
that
- /// each upcast provides, which returns a `TypeError` instead.
+ /// Take `self` as `B`, a subtype or a container such as `Array<i64>` whose
+ /// elements are checked too, when its object is one, without copying it.
+ /// On a mismatch the reference is returned unchanged.
///
/// ```
/// use tvm_ffi::{object::ObjectRef, Array, ObjectRefCore};
@@ -385,21 +383,17 @@ impl std::fmt::Debug for ObjectIdentity {
}
}
-/// Check whether a runtime type index refers to `Target` or one of its
-/// subtypes.
-///
-/// The subtype relation lives in the process-wide type table maintained by the
-/// tvm-ffi library: every registered type records its depth in the single
-/// inheritance tree together with the chain of its ancestors. The check is
-/// O(1) — if `target` really is an ancestor, it must appear in the candidate's
-/// ancestor array exactly at `target`'s depth.
+/// Whether the runtime type index of an object is `Target` or one of its
+/// subtypes, as C++ `Object::IsInstance<T>()` checks.
///
-/// This is a hidden support function for derive-generated object checks.
Object
-/// indices in the registered range must refer to entries in the runtime type
-/// table.
-#[doc(hidden)]
+/// Use it in the strict check of a reference type implemented by hand; the
+/// casts of [`ObjectRefCore`] call it already. An index in the object range
+/// must be registered.
#[inline(always)]
pub fn is_instance_of<Target: ObjectCore>(object_type_index: i32) -> bool {
+ // Every registered type records its depth in the single inheritance tree
+ // and its ancestors, so the check is O(1): an ancestor `Target` appears in
+ // the candidate's ancestor array exactly at `Target`'s depth.
let target_type_index = Target::type_index();
if object_type_index == target_type_index {
return true;
@@ -540,32 +534,23 @@ pub mod unsafe_ {
kTVMFFIObjectDeleterFlagBitMaskWeak,
};
- /// Increase the strong reference count of the object
- ///
- /// This function is same as TVMFFIObjectIncRef but implemented natively
in Rust
- ///
- /// # Arguments
- /// * `obj` - The object to increase the reference count
+ /// Increase the strong reference count of `handle`.
#[inline]
pub unsafe fn inc_ref(handle: *mut TVMFFIObject) {
let obj = &*handle;
obj.combined_ref_count.fetch_add(1, Ordering::Relaxed);
}
- /// Decrease the strong reference count of the object
- ///
- /// This function is same as TVMFFIObjectDecRef but implemented natively
in Rust
- ///
- /// # Arguments
- /// * `obj` - The object to decrease the reference count
+ /// Decrease the strong reference count of `handle`, deleting the object
when it was the last.
#[inline]
pub(crate) unsafe fn dec_ref(handle: *mut TVMFFIObject) {
+ // Release publishes this owner's writes; the Acquire fence before the
deleter pairs with it.
let old_combined_count = (*handle)
.combined_ref_count
- .fetch_sub(COMBINED_REF_COUNT_STRONG_ONE, Ordering::Relaxed);
+ .fetch_sub(COMBINED_REF_COUNT_STRONG_ONE, Ordering::Release);
if old_combined_count == COMBINED_REF_COUNT_BOTH_ONE {
+ fence(Ordering::Acquire);
if let Some(deleter) = (*handle).deleter {
- fence(Ordering::Acquire);
deleter(
handle.cast::<c_void>(),
kTVMFFIObjectDeleterFlagBitMaskBoth as i32,
@@ -604,6 +589,13 @@ pub mod unsafe_ {
(obj.combined_ref_count.load(Ordering::Relaxed) &
COMBINED_REF_COUNT_MASK_U32) as usize
}
+ /// Whether `handle` is the only reference, so it may be mutated in place
(C++ `unique()`).
+ #[inline]
+ pub(crate) unsafe fn is_unique(handle: *mut TVMFFIObject) -> bool {
+ // Acquire pairs with the releasing decrement: other owners' accesses
precede an in-place write.
+ (*handle).combined_ref_count.load(Ordering::Acquire) ==
COMBINED_REF_COUNT_BOTH_ONE
+ }
+
#[inline]
pub(crate) unsafe fn weak_count(handle: *mut TVMFFIObject) -> usize {
let obj = &*handle;
@@ -787,10 +779,11 @@ impl<T: ObjectCore> ObjectArc<T> {
/// Whether `this` holds a null pointer, as all-zero bytes do: no
/// `ObjectArc`, but the field of an object that a creator left undefined.
///
+ /// See also [`AnyCompatible::is_undefined`].
+ ///
/// # Safety
///
/// `this` points to an `ObjectArc<T>` or to all-zero bytes.
- #[doc(hidden)]
#[inline]
pub unsafe fn is_null(this: *const Self) -> bool {
(*std::ptr::addr_of!((*this).ptr).cast::<*const T>()).is_null()
@@ -873,6 +866,12 @@ impl<T: ObjectCore> ObjectArc<T> {
unsafe {
unsafe_::strong_count(this.ptr.as_ptr().cast::<TVMFFIObject>()) }
}
+ /// Whether this is the only reference, so it may be mutated in place.
+ #[inline]
+ pub(crate) fn is_unique(this: &Self) -> bool {
+ unsafe { unsafe_::is_unique(this.ptr.as_ptr().cast::<TVMFFIObject>()) }
+ }
+
/// Get the weak reference count of the ObjectArc
///
/// # Arguments
diff --git a/rust/tvm-ffi/src/type_traits.rs b/rust/tvm-ffi/src/type_traits.rs
index 9a541b19..cb849cfe 100644
--- a/rust/tvm-ffi/src/type_traits.rs
+++ b/rust/tvm-ffi/src/type_traits.rs
@@ -229,16 +229,12 @@ impl TypeSchema for AnyView<'_> {
}
}
-/// Marker for a value that can be stored in an FFI container.
+/// A value an FFI container such as [`Array`](crate::Array) or
+/// [`Map`](crate::Map) holds: every [`AnyCompatible`] type, and [`Any`].
///
-/// This is an implementation detail of [`crate::Array`] and [`crate::Map`].
-/// Users should implement [`AnyCompatible`]; the blanket implementation below
-/// then makes that type a container element automatically. [`Any`] is handled
-/// separately so heterogeneous containers such as `Array<Any>` also work.
-///
-/// The private supertrait seals this marker and owns all conversion
operations,
-/// keeping them out of the user-facing trait API.
-#[doc(hidden)]
+/// Use it as the bound of code generic over container elements, as
+/// `Array<T: ContainerElement>` is. The trait is sealed: implement
+/// [`AnyCompatible`] instead, and the type is an element.
pub trait ContainerElement: container_element_ops::Ops {}
impl<T: AnyCompatible> ContainerElement for T {}
diff --git a/rust/tvm-ffi/tests/test_array.rs b/rust/tvm-ffi/tests/test_array.rs
index 3b9c3d81..0288da8d 100644
--- a/rust/tvm-ffi/tests/test_array.rs
+++ b/rust/tvm-ffi/tests/test_array.rs
@@ -171,3 +171,30 @@ fn test_array_parametric_heterogeneity() {
&[1, 2, 3]
);
}
+
+#[test]
+fn test_array_copy_on_write() {
+ let ptr = |a: &Array<String>| ObjectArc::as_ptr(<Array<String> as
ObjectRefCore>::data(a));
+ let s = |x: &str| String::from(x);
+ let values = |a: &Array<String>| a.iter().map(|x|
x.as_str().to_owned()).collect::<Vec<_>>();
+
+ let mut array = Array::from_vec(vec![s("a"), s("b"), s("c")]);
+ let shared = array.clone();
+ array.set(0, s("x")).unwrap();
+ assert_ne!(ptr(&array), ptr(&shared));
+ assert_eq!(values(&shared), ["a", "b", "c"]);
+
+ let unique = ptr(&array);
+ array.set(1, s("y")).unwrap();
+ array.truncate(2);
+ assert_eq!(array.pop().unwrap().as_str(), "y");
+ assert_eq!(ptr(&array), unique);
+
+ array.push(s("z"));
+ array.insert(0, s("w")).unwrap();
+ assert_eq!(values(&array), ["w", "x", "z"]);
+
+ assert!(array.set(3, s("q")).is_err());
+ assert!(array.insert(4, s("q")).is_err());
+ assert!(Array::<String>::new().pop().is_err());
+}
diff --git a/rust/tvm-ffi/tests/test_error.rs b/rust/tvm-ffi/tests/test_error.rs
index 96e31fb7..aba8619d 100644
--- a/rust/tvm-ffi/tests/test_error.rs
+++ b/rust/tvm-ffi/tests/test_error.rs
@@ -73,6 +73,7 @@ fn test_error_with_context() {
let output = Error::with_appended_backtrace(error, " frame");
assert_eq!(output.backtrace(), "origin frame");
assert!(output.cause_chain().unwrap().same_as(&error0));
+ assert!(std::error::Error::source(&output).is_some());
assert!(output.extra_context().unwrap().same_as(&payload));
if let Some(retained) = retained {
assert_eq!(retained.backtrace(), "origin");
diff --git a/rust/tvm-ffi/tests/test_structural_mutate.rs
b/rust/tvm-ffi/tests/test_structural_mutate.rs
index bac6563c..914e0c24 100644
--- a/rust/tvm-ffi/tests/test_structural_mutate.rs
+++ b/rust/tvm-ffi/tests/test_structural_mutate.rs
@@ -27,8 +27,8 @@ use tvm_ffi::{
dispatch, structural_map, structural_mutate, structural_visit,
structural_walk, Any, AnyView,
Array, DefRegionKind, Dispatcher, Error, FieldGetter, Function,
InplaceMode, IntoDispatcher,
Map, MutationView, Object, ObjectArc, Result, String as FfiString,
StructuralMutatorView,
- StructuralVarRemap, StructuralVisitorView, TypeIndex, Unchanged,
UnchangedOr, VisitInterrupt,
- WalkOrder, WalkResult, RUNTIME_ERROR,
+ StructuralVisitorView, TypeIndex, Unchanged, UnchangedOr, VisitInterrupt,
WalkOrder,
+ WalkResult, RUNTIME_ERROR,
};
/// The value when it is stored as a `bool`, not converted from an integer.
@@ -55,7 +55,7 @@ impl<'value> Dispatcher<(AnyView<'value>, DefRegionKind),
Option<Result<Unchange
#[derive(Default)]
struct ManualIncrement {
- remap: StructuralVarRemap,
+ remapped: Option<Any>,
remap_probe: Option<Any>,
}
@@ -118,7 +118,7 @@ impl<'v, 'c, 'r>
})?;
assert_eq!(mutator.def_region_kind(), initial);
let replacement = mutator.var_remap_get(self,
AnyView::from(&key))?.unwrap();
- self.remap.set(AnyView::from(&key), &replacement)?;
+ self.remapped = Some(replacement.clone());
Ok(UnchangedOr::changed(replacement))
})()
} else if let Some(integer) =
value.as_any_view().downcast_ref::<i64>().copied() {
@@ -404,12 +404,10 @@ fn
user_driven_mutator_controls_default_recursion_and_in_place_opt_in() {
41
);
assert_eq!(
-
i64::try_from(dispatch.remap.get(AnyView::from(&key)).unwrap().unwrap()).unwrap(),
+ i64::try_from(dispatch.remapped.take().unwrap()).unwrap(),
41
);
}
- dispatch.remap.clear();
- assert!(dispatch.remap.get(AnyView::from(&key)).unwrap().is_none());
}
#[test]
