[GitHub] [incubator-hudi] nbalajee commented on issue #1077: [HUDI-335] : Improvements to DiskbasedMap
nbalajee commented on issue #1077: [HUDI-335] : Improvements to DiskbasedMap URL: https://github.com/apache/incubator-hudi/pull/1077#issuecomment-569528285 > @nbalajee But this seems simple enough that we can do ourselves right.. I am bit hesitant to reuse code as-is from Cassandra.. There is ongoing maintenance overhead to consider @vinothchandar - Yes. Let me update the diff with a custom implementation that I had for POC. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] nbalajee commented on issue #1077: [HUDI-335] : Improvements to DiskbasedMap
nbalajee commented on issue #1077: [HUDI-335] : Improvements to DiskbasedMap URL: https://github.com/apache/incubator-hudi/pull/1077#issuecomment-568491020 > We already have a `BufferedFSInputStream` in the codebase. Can we see if this can be reused instead? We anyway just read bytes.. > > ``` > SerializationUtils > .deserialize(SpillableMapUtils.readBytesFromDisk(file, entry.getOffsetOfValue(), entry.getSizeOfValue())); > ``` Thanks for your suggestion Vinod. BufferedFSInputStream allows seek forward (file pointer to make forward progress) and limited seek backwards functionality (seek to a previously set mark on the stream). Does not support seek anywhere, for random access of records (get() functionality). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] nbalajee commented on issue #1077: [HUDI-335] : Improvements to DiskbasedMap
nbalajee commented on issue #1077: [HUDI-335] : Improvements to DiskbasedMap URL: https://github.com/apache/incubator-hudi/pull/1077#issuecomment-566301347 > Is there a JIRA for this work? https://issues.apache.org/jira/browse/HUDI-335 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] nbalajee commented on issue #1077: [HUDI-335] : Improvements to DiskbasedMap
nbalajee commented on issue #1077: [HUDI-335] : Improvements to DiskbasedMap URL: https://github.com/apache/incubator-hudi/pull/1077#issuecomment-566294609 > Is there a JIRA for this work? Sorry. Missed your comment. Updated header with JIRA id. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
