This is an automated email from the ASF dual-hosted git repository.
kabhwan pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from 921abc5 [SPARK-35636][PYTHON][DOCS][FOLLOW-UP] Restructure reference
API files according to the layout
add 2a33117 [SPARK-35312][SS] Introduce new Option in Kafka source to
specify minimum number of records to read per trigger
No new revisions were added by this update.
Summary of changes:
docs/structured-streaming-kafka-integration.md | 15 ++
.../spark/sql/kafka010/KafkaMicroBatchStream.scala | 87 +++++++-
.../apache/spark/sql/kafka010/KafkaSource.scala | 87 ++++++--
.../spark/sql/kafka010/KafkaSourceProvider.scala | 7 +
.../sql/kafka010/KafkaMicroBatchSourceSuite.scala | 245 ++++++++++++++++++---
.../sql/kafka010/KafkaSourceProviderSuite.scala | 6 +
.../{ReadMaxFiles.java => CompositeReadLimit.java} | 30 +--
.../sql/connector/read/streaming/ReadLimit.java | 8 +
.../{ReadMaxRows.java => ReadMinRows.java} | 32 ++-
9 files changed, 438 insertions(+), 79 deletions(-)
copy
sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/streaming/{ReadMaxFiles.java
=> CompositeReadLimit.java} (65%)
copy
sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/streaming/{ReadMaxRows.java
=> ReadMinRows.java} (61%)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
