[spark] branch master updated: [SPARK-45377][CORE] Handle InputStream in NettyLogger

[srowen]

This is an automated email from the ASF dual-hosted git repository.

srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new cdbb301143d [SPARK-45377][CORE] Handle InputStream in NettyLogger
cdbb301143d is described below

commit cdbb301143de2e9a0ea525d20867948f49863842
Author: Hasnain Lakhani <hasnain.lakh...@databricks.com>
AuthorDate: Mon Oct 2 08:27:50 2023 -0500

    [SPARK-45377][CORE] Handle InputStream in NettyLogger
    
    ### What changes were proposed in this pull request?
    
    Handle `InputStream`s in the `NettyLogger` so we can print out how many 
available bytes there are.
    
    ### Why are the changes needed?
    
    As part of the SSL support we are going to transfer `InputStream`s via 
Netty, and this functionality makes it easy to see the size of the streams in 
the log at a glance.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No
    
    ### How was this patch tested?
    
    CI. Tested as part of the changes in 
https://github.com/apache/spark/pull/42685 which this is split out of, I 
observed the logs there.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No
    
    Closes #43165 from hasnain-db/spark-tls-netty-logger.
    
    Authored-by: Hasnain Lakhani <hasnain.lakh...@databricks.com>
    Signed-off-by: Sean Owen <sro...@gmail.com>
---
 .../main/java/org/apache/spark/network/util/NettyLogger.java  | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git 
a/common/network-common/src/main/java/org/apache/spark/network/util/NettyLogger.java
 
b/common/network-common/src/main/java/org/apache/spark/network/util/NettyLogger.java
index 9398726a926..f4c0df6239d 100644
--- 
a/common/network-common/src/main/java/org/apache/spark/network/util/NettyLogger.java
+++ 
b/common/network-common/src/main/java/org/apache/spark/network/util/NettyLogger.java
@@ -17,6 +17,9 @@
 
 package org.apache.spark.network.util;
 
+import java.io.IOException;
+import java.io.InputStream;
+
 import io.netty.buffer.ByteBuf;
 import io.netty.buffer.ByteBufHolder;
 import io.netty.channel.ChannelHandlerContext;
@@ -42,6 +45,14 @@ public class NettyLogger {
       } else if (arg instanceof ByteBufHolder) {
         return format(ctx, eventName) + " " +
           ((ByteBufHolder) arg).content().readableBytes() + "B";
+      } else if (arg instanceof InputStream) {
+        int available = -1;
+        try {
+          available = ((InputStream) arg).available();
+        } catch (IOException ex) {
+          // Swallow, but return -1 to indicate an error happened
+        }
+        return format(ctx, eventName, arg) + " " + available + "B";
       } else {
         return super.format(ctx, eventName, arg);
       }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org