Indhumathi27 commented on PR #6624:
URL: https://github.com/apache/hive/pull/6624#issuecomment-5043344230
@ayushtkn Thanks for the review. I agree ipc.server.log.slow.rpc is the
right way at the IPC layer, but I would like to highlight the advantages of
having this in Hive as well.
**Client-side visibility without NameNode access**
*ipc.server.log.slow.rpc* requires NameNode config changes and a restart.
Hive/HMS operators in most enterprise environments control only the HMS process
— they cannot touch NameNode configs. This fix gives them actionable signal
without any dependency on the infrastructure team.
**Operation context**
The NameNode sees an anonymous *getFileInfo* RPC. It has no idea why it
was called. The Hive-side log tells you exactly which operation triggered it:
- WARN StorageBasedAuthorizationProvider - Slow getFileInfo during
- storage-based authorization check: path=hdfs://nn/warehouse/mydb,
elapsed=4523ms
Without this, even with *ipc.server.log.slow.rpc* enabled on the NameNode,
you still cannot correlate the slow RPC to a specific HMS operation (auth check
vs table scan vs partition listing).
**Path information for diagnosis**
The Hive-side log includes the HDFS path being authorized. This is
critical for diagnosis — it tells operators whether the slow call is on a
warehouse root, a specific database, or a partition path, helping narrow down
the issue to a specific table or misconfigured location.
**Lower blast radius restart**
Both require a process restart to change the config. However, restarting
the NameNode affects the entire HDFS cluster and all clients — it is a
high-impact operation that typically requires a change request and scheduled
maintenance window. Restarting HMS affects only Hive metadata operations and is
a routine, low-impact operation that the Hive team can do independently without
involving the infrastructure team or affecting other HDFS workloads.
**Threshold tuning per deployment**
Different deployments have different NN latency baselines. A threshold of
1s may be normal for one cluster and a warning sign for another. Having the
config on the Hive side lets operators tune it to their environment
independently.
**Filesystem agnostic — works for any future FileSystem**
ipc.server.log.slow.rpc is specific to Hadoop IPC and only covers HDFS
NameNode RPC calls. With storage backends like S3, Azure ADLS, GCS, or Vast S3
becoming increasingly common in Hive deployments, slow getFileInfo calls can
occur on any FileSystem implementation — not just HDFS. Since this fix
instruments FileUtils.getFileStatusOrNull() at the Hive level, it will
automatically provide slow-call visibility for any FileSystem that
storage-based authorization runs against today or in the future, without
requiring any equivalent mechanism to exist in those storage systems.
Both mechanisms complement each other — *ipc.server.log.slow.rpc* gives
NameNode-level visibility, while this fix gives HMS-level visibility with
operation context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
