Hi Chris, Thanks for answering.
The underlying file system is HDFS, i use SequenceFile.Writer and Reader with protocol hdfs:// Sébastien. Le lun. 27 janv. 2025 à 18:45, Chris Nauroth <[email protected]> a écrit : > Hello Sébastien, > > I think hflush() on a sequence file should be working according to > existing unit tests that we have. What is the underlying file system you > are using (HDFS, S3A, etc.)? Support for hflush() varies depending on the > file system implementation. > > Chris Nauroth > > > On Mon, Jan 27, 2025 at 6:06 AM Sébastien Rebecchi > <[email protected]> wrote: > >> Hello, >> >> I got this issue with both hadoop 2.9.1 and 3.4.1. >> For SequenceFile.Writer class, after doing hflush(), data is not visible >> to new readers, it is visible only after doing close(). >> The doc asserts that data must be visible to readers >> https://hadoop.apache.org/docs/r3.4.1/hadoop-project-dist/hadoop-common/filesystem/outputstream.html >> >> What should we do for this? >> >> Thanks, >> Sébastien. >> >
