siying opened a new pull request, #40905: URL: https://github.com/apache/spark/pull/40905
### What changes were proposed in this pull request? We add a logging when creating the batch reader with task ID, topic, partition and offset range included. The log line looks like following: 23/04/18 22:35:38 INFO KafkaBatchReaderFactory: Creating Kafka reader partitionId=1 partition=StreamingDustTest-KafkaToKafkaTopic-4ccf8662-c3ca-4f3b-871e-1853c0e61765-source-2 fromOffset=0 untilOffset=3 queryId=b5b806c3-ebf3-432e-a9a7-d882d474c0f5 batchId=0 taskId=1 ### Why are the changes needed? Right now, for structure streaming from Kafka, it's hard to finding which task handling which topic/partition and offset range. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Run KafkaMicroBatchV2SourceSuite and watch logging outputs contain information needed. Also does a small cluster test and observe logs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
