Siying Dong created SPARK-44012:
-----------------------------------
Summary: KafkaDataConsumer to log some stats
Key: SPARK-44012
URL: https://issues.apache.org/jira/browse/SPARK-44012
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 3.4.0
Reporter: Siying Dong
For each task, Kafka source should report fraction of time spent in
KafkaConsumer to fetch records. It should also report overall read bandwidth
(bytes or records read / time spent fetching).
This will be useful in verifying if fetching is the bottleneck.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
