This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.


    from 57450ed  [MINOR][SS] Rename `secondLatestBatchId` to 
`secondLatestOffsets`
     add 78a403f  [SPARK-27627][SQL] Make option "pathGlobFilter" as a general 
option for all file sources

No new revisions were added by this update.

Summary of changes:
 docs/sql-data-sources-binaryFile.md                |  16 +----
 docs/sql-data-sources-load-save-functions.md       |  21 +++++++
 .../examples/sql/JavaSQLDataSourceExample.java     |   5 ++
 examples/src/main/python/sql/datasource.py         |   5 ++
 examples/src/main/r/RSparkSQLExample.R             |   4 ++
 .../partitioned_users.orc/do_not_read_this.txt     |   1 +
 .../users.orc                                      | Bin 0 -> 448 bytes
 .../favorite_color=red/users.orc                   | Bin 0 -> 402 bytes
 .../spark/examples/sql/SQLDataSourceExample.scala  |   5 ++
 .../org/apache/spark/sql/avro/AvroFileFormat.scala |   4 ++
 .../org/apache/spark/sql/avro/AvroOptions.scala    |   5 +-
 python/pyspark/sql/readwriter.py                   |   6 ++
 python/pyspark/sql/streaming.py                    |   6 ++
 .../org/apache/spark/sql/DataFrameReader.scala     |   9 +++
 .../sql/execution/datasources/DataSource.scala     |   3 +-
 .../datasources/PartitioningAwareFileIndex.scala   |  11 +++-
 .../datasources/binaryfile/BinaryFileFormat.scala  |  70 ++++++++-------------
 .../sql/execution/streaming/FileStreamSource.scala |   4 +-
 .../execution/streaming/MetadataLogFileIndex.scala |   3 +-
 .../spark/sql/streaming/DataStreamReader.scala     |   9 +++
 .../spark/sql/FileBasedDataSourceSuite.scala       |  32 ++++++++++
 .../sql/streaming/FileStreamSourceSuite.scala      |  19 ++++++
 22 files changed, 173 insertions(+), 65 deletions(-)
 create mode 100644 
examples/src/main/resources/partitioned_users.orc/do_not_read_this.txt
 create mode 100644 
examples/src/main/resources/partitioned_users.orc/favorite_color=__HIVE_DEFAULT_PARTITION__/users.orc
 create mode 100644 
examples/src/main/resources/partitioned_users.orc/favorite_color=red/users.orc


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org

Reply via email to