L. C. Hsieh created SPARK-39134: ----------------------------------- Summary: Add custom metric of skipped null values for stream join operator Key: SPARK-39134 URL: https://issues.apache.org/jira/browse/SPARK-39134 Project: Spark Issue Type: Improvement Components: Structured Streaming Affects Versions: 3.4.0 Reporter: L. C. Hsieh
When STATE_STORE_SKIP_NULLS_FOR_STREAM_STREAM_JOINS is enabled, Spark will skip null values in hash-based stream-stream joins. But currently we cannot track if this is happened and how many it is. For our customers, they have an interest to know the number of values skipped for better tracking purpose. -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org