ayushtkn commented on PR #6624: URL: https://github.com/apache/hive/pull/6624#issuecomment-5043607843
> ipc.server.log.slow.rpc requires NameNode config changes and a restart. If my memory treats me well, that is reconfigurable and doesn't require a Restart, Hadoop has set of reconfigurable properties, which can be reconfigured without restarting the server https://github.com/apache/hadoop/blob/767c9da6f34f9ed57311ba9f43497563c05b9f46/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java#L2539-L2560 I do understand your use case, But I feel this is something to be dealt at the Hadoop RPC Client layer rather than the Hive as a FS Client doing this magic. If we want to implement it in Hive, which I seriously think we shouldn't, we should not just for `getFileInfo` or so. It should be a wrapper FS, which tracks the time of each call and delegates to the underlying FS. <Your Wrapper FS> -> start counter -> call any HCFS API-> stop counter -> log accordingly. This you can just hinge via fs.*.impl configs. Logging one API in one interface looks overkill to me. I won't block you if others are fine with it, but it won't help you much either IMO, `getFileInfo` might get pass at SBA, maybe some other write call is chocking, the write logic requires a write lock + writing to JN's that maybe slow, so I don't think it will help you in long run, or if the cluster isn't using SBA -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
