(iceberg-rust) branch main updated: feat: Add trait for ObjectCache and ObjectCacheProvider (#1222)

[liurenjie1024]

This is an automated email from the ASF dual-hosted git repository.

liurenjie1024 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/iceberg-rust.git


The following commit(s) were added to refs/heads/main by this push:
     new ca22eaaf feat: Add trait for ObjectCache and ObjectCacheProvider 
(#1222)
ca22eaaf is described below

commit ca22eaaf422b5d0e5a360e443bfda6b18301cb35
Author: Xuanwo <git...@xuanwo.io>
AuthorDate: Wed Apr 16 20:26:25 2025 +0800

    feat: Add trait for ObjectCache and ObjectCacheProvider (#1222)
    
    ## Which issue does this PR close?
    
    Part of https://github.com/apache/iceberg-rust/issues/1217
    
    I plan to introduce `ObjectCache` and `ObjectCachePorvide` to replace
    the existing `ObjectCache`. These cache traits will serve as the entry
    point for users to configure the cache we need to use in Iceberg. Users
    can easily implement a cache provider and share the same cache across
    multiple Iceberg tables.
    
    This design also move the entire cache building logic to users side so
    that iceberg don't need to specify anything hardcode.
    
    I also plan to move the existsing moka based cache to our future lite
    engine so users can still have a good enough default.
    
    ## What changes are included in this PR?
    
    - Add `ObjectCache` trait for get/set cache enties.
    - Add `ObjectCacheProvide` trait for users to implement their own cache
    logic.
    
    ## Are these changes tested?
    
    No need.
    
    ---------
    
    Signed-off-by: Xuanwo <git...@xuanwo.io>
---
 crates/iceberg/src/cache.rs | 63 +++++++++++++++++++++++++++++++++++++++++++++
 crates/iceberg/src/lib.rs   |  1 +
 2 files changed, 64 insertions(+)

diff --git a/crates/iceberg/src/cache.rs b/crates/iceberg/src/cache.rs
new file mode 100644
index 00000000..37fe0712
--- /dev/null
+++ b/crates/iceberg/src/cache.rs
@@ -0,0 +1,63 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Cache management for Iceberg.
+
+use std::sync::Arc;
+
+use crate::spec::{Manifest, ManifestList};
+
+/// A trait for caching in-memory objects of given type.
+///
+/// # Notes
+///
+/// ObjectCache will store deeply nested objects, such as `Manifest`,
+/// which contains `Schema`. Please ensure that the cache stores the
+/// object in memory as-is, without attempting to serialize it, as
+/// serialization could be extremely expensive.
+pub trait ObjectCache<K, V>: Send + Sync {
+    /// Gets an object from the cache by its key.
+    fn get(&self, key: &K) -> Option<V>;
+    /// Sets an object in the cache with the given key and value.
+    fn set(&self, key: K, value: V);
+}
+
+/// A trait for caching different in-memory objects used by iceberg.
+///
+/// # Notes
+///
+/// ObjectCache will store deeply nested objects, such as `Manifest`,
+/// which contains `Schema`. Please ensure that the cache stores the
+/// object in memory as-is, without attempting to serialize it, as
+/// serialization could be extremely expensive.
+pub trait ObjectCacheProvide: Send + Sync {
+    /// Gets a cache for manifests.
+    fn manifest_cache(&self) -> &dyn ObjectCache<String, Arc<Manifest>>;
+    /// Gets a cache for manifest lists.
+    fn manifest_list_cache(&self) -> &dyn ObjectCache<String, 
Arc<ManifestList>>;
+}
+
+/// CacheProvider is a type alias for a thread-safe reference-counted pointer 
to a CacheProvide trait object.
+pub type ObjectCacheProvider = Arc<dyn ObjectCacheProvide>;
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    struct _TestDynCompatibleForObjectCache(Arc<dyn ObjectCache<String, 
Arc<Manifest>>>);
+    struct _TestDynCompatibleForObjectCacheProvider(ObjectCacheProvider);
+}
diff --git a/crates/iceberg/src/lib.rs b/crates/iceberg/src/lib.rs
index 278fe18f..556ff3e0 100644
--- a/crates/iceberg/src/lib.rs
+++ b/crates/iceberg/src/lib.rs
@@ -70,6 +70,7 @@ pub use catalog::{
 pub mod table;
 
 mod avro;
+pub mod cache;
 pub mod io;
 pub mod spec;