HeartSaVioR commented on a change in pull request #27146: [SPARK-21869][SS][DOCS][FOLLOWUP] Document Kafka producer pool configuration URL: https://github.com/apache/spark/pull/27146#discussion_r365631595
########## File path: docs/structured-streaming-kafka-integration.md ########## @@ -802,6 +802,31 @@ df.selectExpr("topic", "CAST(key AS STRING)", "CAST(value AS STRING)") \ </div> </div> +### Producer Caching + +Given Kafka producer instance is designed to be thread-safe, Spark initializes a Kafka producer instance and co-use across tasks for same caching key. + +The caching key is built up from the following information: Review comment: Just added some explanation since it makes sense to let end users know about it to debug. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org