On Wed, Dec 21, 2016 at 7:26 AM, Ravikumar Govindarajan < [email protected]> wrote:
> On Tue, Dec 20, 2016 at 10:33 PM, Aaron McCurry <[email protected]> > wrote: > > > Perhaps if you really need a WAL that you could look at the sync/fencing > > code in the KV store to implement a WAL. > > > > Thanks for this. I can straightaway base my WAL impl on this.. > > Btw, I see a special reflective method HdfsUtil.getLength() & KV > InputStream does not read beyond this length. Any reasons for that? > The reason for it is to get the current posted length from the namenode. Newer hdfs implements may not need this but with the versions that Blur was tested against it was required to get an updated length. Basically the namenode will post the length of the file when the block barrier is crossed or when the file is closed. However the namenode metadata will not be reflected upon a fsync and to ensure the correct operation the most up to date length is needed. Hence the reflection call. Let me know if you have questions. Aaron > > -- > Ravi >
