gianm commented on code in PR #12987:
URL: https://github.com/apache/druid/pull/12987#discussion_r958772489
##########
processing/src/main/java/org/apache/druid/frame/file/FrameFile.java:
##########
@@ -80,91 +79,173 @@ public class FrameFile implements Closeable
/**
* Delete the opened frame file when all references are closed.
*/
- DELETE_ON_CLOSE,
-
- /**
- * Map using ByteBuffer. Used only for testing.
- */
- BB_MEMORY_MAP,
-
- /**
- * Map using DataSketches Memory. Used only for testing.
- */
- DS_MEMORY_MAP
+ DELETE_ON_CLOSE
}
private final File file;
- private final Memory memory;
+ private final long fileLength;
+ private final Memory footerMemory; // Footer is everything from the final
MARKER_NO_MORE_FRAMES to EOF.
+ private final int maxMmapSize;
private final int numFrames;
private final int numPartitions;
private final ReferenceCountingCloseableObject<Closeable> referenceCounter;
private final Closeable referenceReleaser;
+ /**
+ * Mapped memory, starting from {@link #bufferOffset} in {@link #file}, up
to max of {@link #maxMmapSize}. Acts as
+ * a window on the underlying file. Remapped using {@link
#remapBuffer(long)}, freed using {@link #releaseBuffer()}.
Review Comment:
My thinking is we don't need to block this patch on it, but yes, we should
definitely add performance tests / benchmarks for the frame stuff generally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
