jerqi commented on issue #198: URL: https://github.com/apache/incubator-uniffle/issues/198#issuecomment-1244806643
> Can you help me understand #issue_63? @jerqi `When we use the storageType MEMORY_HDFS, MEMORY_LOCALFILE_HDFS, after we read data from memory, the data may be flushed to HDFS. So we could read incomplete index data. ` `org.apache.uniffle.storage.handler.impl.HdfsFileWriter#writeIndex` is a complete operation, it looks like it shouldn't write an incomplete index file. If you use MEMORY_HDFS or MEMORY_LOCALFILE_HDFS, you can read data from memory and then you will read data from HDFS, but the data of memory may be flushed, so you can read the incomplete data of memory flushed to HDFS. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
