istinnstudio opened a new issue, #1863: URL: https://github.com/apache/fury/issues/1863
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/fury/issues) and found no similar issues. ### Version 0.70.1 ### Component(s) Java ### Minimal reproduce step There is a class that implements Map interface. This class cannot be deserialized. I will provide full reproducible example.. The example contains 2 classes, the customHashMap class and a deep copy example I use. Uncomment the field to switch between a valid Map and the erroneous explicit CustomHashMap Object ``` import java.io.ByteArrayOutputStream; import java.nio.ByteBuffer; import org.apache.fury.Fury; import org.apache.fury.config.CompatibleMode; import static org.apache.fury.config.Language.JAVA; import org.apache.fury.logging.FuryLogger; import org.apache.fury.memory.MemoryBuffer; import org.apache.fury.memory.MemoryUtils; public class DeepCopyApacheFury { // Method to deep copy an object using Apache Fury with dynamic buffer allocation public static Object deepCopyFuryWithByteBufferExample(Object theObj) throws Exception { long startTime = System.nanoTime(); // Start time for performance tracking // Initialize Apache Fury serialization Fury fury = Fury.builder() .withLanguage(JAVA) // Use Java language configuration .withRefTracking(true) // Enable reference tracking (if needed) .requireClassRegistration(true) // Ensure all classes are registered .withCompatibleMode(CompatibleMode.SCHEMA_CONSISTENT) // .withMetaShare(false) // .withScopedMetaShare(false) // .withRefCopy(false) // .withAsyncCompilation(false) .build(); // Manually register classes to improve performance registerFuryLargeObjectExample(fury); System.out.println("Starting Fury serialization..."); // Use ByteArrayOutputStream for dynamic buffer allocation ByteArrayOutputStream byteArrayOutStream = new ByteArrayOutputStream(); Object newObj = null; try { // Serialize the object into the ByteArrayOutputStream fury.serialize(byteArrayOutStream, theObj); byte[] serializedData = byteArrayOutStream.toByteArray(); long serializationEndTime = System.nanoTime(); System.out.println("Serialization completed in " + (serializationEndTime - startTime) / 1_000_000_000.0 + " seconds"); // Calculate the size of the serialized data in MB double serializedSizeMB = serializedData.length / (1024.0 * 1024.0); System.out.println("Serialized object size: " + serializedSizeMB + " MB"); // Allocate a ByteBuffer of the exact size of the serialized data ByteBuffer byteBuffer = ByteBuffer.allocateDirect(serializedData.length); byteBuffer.put(serializedData); byteBuffer.flip(); // Switch to reading mode // Wrap the ByteBuffer in a MemoryBuffer for deserialization MemoryBuffer memoryBuffer = MemoryUtils.wrap(byteBuffer); System.out.println("memoryBuffer size = "+memoryBuffer.size()); // Deserialize the object from the ByteBuffer using Fury System.out.println("Starting Fury deserialization..."); newObj = fury.deserialize(memoryBuffer); long deserializationEndTime = System.nanoTime(); System.out.println("Deserialization completed in " + (deserializationEndTime - startTime) / 1_000_000_000.0 + " seconds"); } catch (Exception e) { // not in use as it breaks fury debugging log // getLogger(DeepCopyApacheFury.class.getName()).log(System.Logger.Level.DEBUG, "Exception in DeepCopyApacheFury", e); FuryLogger furyLogger = new FuryLogger(Object.class); String msg = "FURY LOGGER: Exception in DeepCopyApacheFury"; furyLogger.error(msg, e); // getLogger(DeepCopyApacheFury.class.getName()).log(System.Logger.Level.DEBUG, "Exception in DeepCopyApacheFury", e); } // Verify types for debugging System.out.println("------------------ VERIFY ------------------"); System.out.println("Original object type: " + theObj.getClass().getName()); if (newObj!=null) { System.out.println("Deserialized object type: " + newObj.getClass().getName()); } else{System.err.println("newObj IS NULL");} return newObj; } private static void registerFuryLargeObjectExample(Fury fury) { fury.register(LargeMapObject.class); fury.register(CustomHashMap.class); } // Main method demonstrating the deep copy with large object public static void main(String[] args) throws Exception { // Create a large object (e.g., Map with a large number of entries) LargeMapObject originalObject = new LargeMapObject(); originalObject.populateLargeMap(1_000_000); // Populate with 1,000,000 entries // Perform deep copy using Fury LargeMapObject copiedObject = (LargeMapObject) deepCopyFuryWithByteBuffer(originalObject); // Check if deep copy was successful (you can add further validation if needed) System.out.println("Original object size: " + originalObject.getMapSize()); if (copiedObject!=null) { System.out.println("Copied object size: " + copiedObject.getMapSize()); } else{System.err.println("COPY OF MAP IS NULL");} } // Example class to demonstrate serialization with large data static class LargeMapObject { // private Map<String, Integer> largeMap = new HashMap<>(); private CustomHashMap<String, Integer> largeMap = new CustomHashMap<>(); // Populate the map with a large number of entries public void populateLargeMap(int numEntries) { for (int i = 0; i < numEntries; i++) { largeMap.put("key" + i, i); } } // Get the size of the map (for validation) public int getMapSize() { return largeMap.size(); } @Override public String toString() { return "LargeObject{mapSize=" + largeMap.size() + '}'; }} } ``` ``` import java.io.Serializable; import java.util.Collection; import java.util.Collections; import java.util.HashMap; import java.util.HashSet; import java.util.Iterator; import java.util.Map; import java.util.Set; @SuppressWarnings({ "rawtypes", "unchecked" }) public class CustomHashMap<K, V> implements Map<K, V> ,Serializable , Cloneable { // Cloneable for deep clone method private Map<K, V> entryMap; // SET: Adds the specified element to this set if it is not already present. private Set<K> entrySet; public CustomHashMap() { super(); entryMap = new HashMap<K, V>(); // changed to linked entrySet = Collections.newSetFromMap(new HashMap<>()); } @Override public Set<Map.Entry<K, V>> entrySet() { return new HashSet<>(entryMap.entrySet()); } @Override public V put(K key, V value) { V oldValue = entryMap.get(key); insertionRule(key, value); return oldValue; } @Override public void putAll(Map<? extends K, ? extends V> t) { for (Iterator i = t.keySet().iterator(); i.hasNext();) { K key = (K) i.next(); insertionRule(key, t.get(key)); } } @Override public void clear() { entryMap.clear(); entrySet.clear(); } @Override public boolean containsKey(Object key) { return entryMap.containsKey(key); } @Override public boolean containsValue(Object value) { return entryMap.containsValue(value); } public Set entrySetOriginal() { return entryMap.entrySet(); } @Override public boolean equals(Object o) { return entryMap.equals(o); } @Override public V get(Object key) { return entryMap.get(key); } @Override public int hashCode() { return entryMap.hashCode(); } @Override public boolean isEmpty() { return entryMap.isEmpty(); } @Override public Set keySet() { return entrySet; } @Override public V remove(Object key) { entrySet.remove(key); return entryMap.remove(key); } @Override public int size() { return entryMap.size(); } @Override public Collection values() { return entryMap.values(); } // Method to return a Map from a CustomHashMap. It can return a map anyway but it is more obvious this way public Map<K, V> getMap() { return this; } public Map<K, V> getEntryMap() { return entryMap; } public Set<K> getEntrySet() { return entrySet; } @Override public Map<K, V> clone() { try { CustomHashMap<K, V> clonedMap = (CustomHashMap<K, V>) super.clone(); clonedMap.entryMap = new HashMap<>(entryMap); // Shallow copy the entryMap clonedMap.entrySet = new HashSet<>(entrySet); // Shallow copy the entrySet return clonedMap; } catch (CloneNotSupportedException e) { // This should never happen since CustomHashMap is Cloneable throw new InternalError(e); } } public synchronized boolean insertionRule(K key, V value) { // KEY as null and EMPTY String is not allowed. if (key == null || (key instanceof String && ((String) key).trim().equals("") ) ) { return false; } // If key already available then, we are not overriding its value. if (entrySet.contains(key)) { // Then override its value, but we are not allowing return false; } else { // Add the entry entrySet.add(key); entryMap.put(key, value); return true; } } } ``` ### What did you expect to see? a deserialized object ### What did you see instead? Exception in thread "main" java.lang.StringIndexOutOfBoundsException: Range [260, 2) out of bounds for length 262. This probably is for the example, could be different on other Map content, generally is "OutOfBoundsException" ### Anything Else? I have various error messages as this custom map class is being used on more complex classes, and I have a fair conclusion that this one creates several issues, but I am not entirely sure. This explicit customhashmap usage was done by mistake and stayed there for one small object even if it is a bit "heavier" comparing to the native Map as it is now an Object. Eventually it will be replaced by a true Map but that will be tricky as versioning will take place in various classes. FST serializer v2.57 works OK with it, so there should be a bug or a missing case in Fury... ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
