Hi Mohammad, Like ATM said, HDFS-8965 is an important fix in this area. We have found that it prevents cases where INotify tries to read invalid sequences of bytes (sometimes because the edit log was truncated or corrupted; other times because it is in the middle of a write). HDFS-8964 fixes the "attempt to read edit log entries that are still being written." Both fixes have value when using INotify.
The other question is how many INotify clients you have simultaneously? Right now, INotify works best with a small number of clients, since it involves doing RPCs to the NameNode. There is some work going on to scale out INotify to support many clients in HDFS-8940. best, Colin On Tue, Sep 15, 2015 at 11:52 AM, Mohammad Islam <misla...@yahoo.com.invalid> wrote: > Hi,We were using INotify feature in one of our internal service. Looks like > it creates a lot of memory pressure on NN. Memory usage goes very high and > remains the same causing expensive GC. > Did anyone use this feature in any service? Is there any con to setup? We are > using latest CDH. > Regards,Mohammad >
