GitHub user StevenCurran edited a comment on the discussion: How does the zero copy works ?
hey @chaokunyang Sorry to bring this thread up again, but felt its applicable instead of creating a new thread. In your example for zero copy, could you elaborate a bit on what is happening here? https://fury.apache.org/docs/guide/java_object_graph_guide#zero-copy-serialization What are the BufferObjects used for, and what about the MemoryBuffers list used a couple of lines later? I am just trying to get an idea of the flow of data here, and where it is done off heap at each line. Could a comment on the lines after the objects are created in lst help? Also looking at the source code, toBuffer, and creating BufferObjects are on heap from MemoryBuffer.newHeapBuffer(this.totalBytes()), so could you provide a full offheap version of this? ```java public static void main(String[] args) { //3 objects allocated that we want to serialize off heap List<Object> list = Arrays.asList("str", new byte[1000], new int[100], new double[100]); //Off heap BufferObjects that are pre allocated, and can be reused as a serialization / deserialization memory location? Collection<BufferObject> bufferObjects = new ArrayList<>(); //Serialize the objects from list out of band (off heap) into each of the buffer objects? aka 0 allocation done and bufferObjects can be reused? This returns a byte[] though, so there is heap allocation happening? byte[] bytes = fury.serialize(list, e -> !bufferObjects.add(e)); //Then we take those objects and put them into a bytebuffer, also off heap? But return this list? Confused as to why List<MemoryBuffer> buffers = bufferObjects.stream() .map(BufferObject::toBuffer).collect(Collectors.toList()); //Move these back on heap when using the deserialize, by using the pre allocated buffers? System.out.println(fury.deserialize(bytes, buffers)); } } ``` I have tried to put in comments as to what I believe, but perhaps it is not so clear to me what is happening here. Any guidance would be geatly appreciated. Many thanks! GitHub link: https://github.com/apache/fury/discussions/1377#discussioncomment-10254568 ---- This is an automatically sent email for commits@fury.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@fury.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
