(spark) branch master updated: [SPARK-48208][SS] Skip providing memory usage metrics from RocksDB if bounded memory usage is enabled

2024-05-09 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 045ec6a166c8 [SPARK-48208][SS] Skip providing

(spark) branch master updated: [SPARK-47960][SS] Allow chaining other stateful operators after transformWithState operator

2024-05-07 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 5e49665ac39b [SPARK-47960][SS] Allow chaining

(spark) branch master updated: [SPARK-48105][SS] Fix the race condition between state store unloading and snapshotting

2024-05-06 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 9dc5599b01b1 [SPARK-48105][SS] Fix the race

(spark) branch master updated: [SPARK-48102][SS] Track duration for acquiring source/sink metrics while reporting streaming query progress

2024-05-02 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d9d79a54a3cd [SPARK-48102][SS] Track duration

(spark) branch master updated: [SPARK-47793][SS][PYTHON] Implement SimpleDataSourceStreamReader for python streaming data source

2024-04-30 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new c8c249204178 [SPARK-47793][SS][PYTHON

(spark) branch master updated (332570f42203 -> 94763438943e)

2024-04-30 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 332570f42203 [SPARK-48052][PYTHON][CONNECT] Recover `pyspark-connect` CI by parent classes add 94763438943e

(spark) branch master updated (2b2a33cc35a8 -> d5712cea88cd)

2024-04-26 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 2b2a33cc35a8 [SPARK-48011][CORE] Store LogKey name as a value to avoid generating new string instances add

(spark) branch master updated (033ca3e7dd5b -> b0e03a193531)

2024-04-26 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 033ca3e7dd5b [SPARK-47922][SQL] Implement the try_parse_json expression add b0e03a193531 [SPARK-47999][SS

(spark) branch master updated: [SPARK-47805][SS] Implementing TTL for MapState

2024-04-23 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 61ac3421651c [SPARK-47805][SS] Implementing TTL

(spark) branch branch-3.5 updated: [SPARK-47840][SS] Disable foldable propagation across Streaming Aggregate/Join nodes

2024-04-15 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 6c67c61bfd21 [SPARK-47840][SS] Disable

(spark) branch master updated (51d3efcead5b -> f21719346eb0)

2024-04-15 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 51d3efcead5b [SPARK-47233][CONNECT][SS][2/2] Client & Server logic for Client side streaming query listener

(spark) branch master updated: [SPARK-47673][SS] Implementing TTL for ListState

2024-04-15 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new be080703688f [SPARK-47673][SS] Implementing TTL

(spark) branch master updated (f86dc2b7e5c3 -> e6b7950f553c)

2024-04-15 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from f86dc2b7e5c3 [SPARK-47848][SS] Fix thread safety issue around access to loadedMaps in close function for hdfs store

(spark) branch master updated: [SPARK-47848][SS] Fix thread safety issue around access to loadedMaps in close function for hdfs store provider

2024-04-15 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f86dc2b7e5c3 [SPARK-47848][SS] Fix thread

(spark) branch master updated (27987536be38 -> ae93b46b8c2b)

2024-04-12 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 27987536be38 [SPARK-47800][SQL] Create new method for identifier to tableIdentifier conversion add ae93b46b8c2b

(spark) branch master updated: [SPARK-47784][SS] Merge TTLMode and TimeoutMode into a single TimeMode

2024-04-11 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 9ffdbc65029a [SPARK-47784][SS] Merge TTLMode

(spark) branch master updated: [SPARK-47776][SS] Disallow binary inequality collation be used in key schema of stateful operator

2024-04-09 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 2f7cdc9cdaf4 [SPARK-47776][SS] Disallow binary

(spark) branch master updated: [SPARK-47746] Implement ordinal-based range encoding in the RocksDBStateEncoder

2024-04-08 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 60806c63d97b [SPARK-47746] Implement ordinal

(spark) branch master updated: [SPARK-47558][SS] State TTL support for ValueState

2024-04-07 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d55bb617a135 [SPARK-47558][SS] State TTL

(spark) branch branch-3.5 updated: [SPARK-47299][PYTHON][DOCS] Use the same `versions.json` in the dropdown of different versions of PySpark documents

2024-04-07 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 850ec0b4adcb [SPARK-47299][PYTHON][DOCS

(spark) branch master updated (0c992b205946 -> b299b2bc06a9)

2024-04-07 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 0c992b205946 [SPARK-47755][CONNECT] Pivot should fail when the number of distinct values is too large add

(spark) branch branch-3.4 updated: [SPARK-47734][PYTHON][TESTS][3.4] Fix flaky DataFrame.writeStream doctest by stopping streaming query

2024-04-07 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.4 by this push: new 1f66a40e3b85 [SPARK-47734][PYTHON

(spark) branch master updated: [SPARK-47744] Add support for negative-valued bytes in range encoder

2024-04-07 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e92e8f5441a7 [SPARK-47744] Add support

(spark) branch master updated: [SPARK-47310][SS] Add micro-benchmark for merge operations for multiple values in value portion of state store

2024-04-05 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 1efbf43160aa [SPARK-47310][SS] Add micro

(spark) branch master updated (a427a4586177 -> 1515c567e4f0)

2024-04-03 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from a427a4586177 [SPARK-47710][SQL][DOCS] Postgres: Document Mapping Spark SQL Data Types from PostgreSQL add

(spark) branch master updated: [SPARK-47653][SS] Add support for negative numeric types and range scan key encoder

2024-04-03 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d16183d044a6 [SPARK-47653][SS] Add support

(spark) branch master updated (49b7b6b9fe6b -> 2c2a2adc3275)

2024-04-02 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 49b7b6b9fe6b [SPARK-47691][SQL] Postgres: Support multi dimensional array on the write side add 2c2a2adc3275

(spark) branch master updated: [SPARK-47568][SS] Fix race condition between maintenance thread and load/commit for snapshot files

2024-03-28 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 0b844e52b35b [SPARK-47568][SS] Fix race

(spark) branch master updated: [SPARK-47363][SS] Initial State without state reader implementation for State API v2

2024-03-27 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4d72be3abdc4 [SPARK-47363][SS] Initial State

(spark) branch master updated: [SPARK-47107][SS][PYTHON] Implement partition reader for python streaming data source

2024-03-27 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e3e7135af4df [SPARK-47107][SS][PYTHON

(spark) branch master updated: [SPARK-47570][SS] Integrate range scan encoder changes with timer implementation

2024-03-26 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 88b29c5076d4 [SPARK-47570][SS] Integrate range

(spark) branch master updated: [SPARK-47273][SS][PYTHON] implement Python data stream writer interface

2024-03-26 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 5ac39181fe87 [SPARK-47273][SS][PYTHON

(spark) branch master updated: [SPARK-47469][SS][TESTS] Add `Trigger.AvailableNow` tests for `transformWithState` operator

2024-03-26 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 3e8343205c4d [SPARK-47469][SS][TESTS] Add

(spark) branch master updated: [SPARK-47372][SS] Add support for range scan based key state encoder for use with state store provider

2024-03-25 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new ff38378d7e42 [SPARK-47372][SS] Add support

(spark) branch master updated: [SPARK-47512][SS] Tag operation type used with RocksDB state store instance lock acquisition/release

2024-03-21 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 057acf986d78 [SPARK-47512][SS] Tag operation

(spark) branch master updated: [SPARK-47449][SS] Refactor and split list/timer unit tests

2024-03-19 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f0061dbe856a [SPARK-47449][SS] Refactor

(spark) branch master updated (acf17fd67217 -> 9f8147c2a8d2)

2024-03-18 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from acf17fd67217 [SPARK-47450][INFRA][R] Use R 4.3.3 in `windows` R GitHub Action job add 9f8147c2a8d2 [SPARK-47329

(spark) branch master updated: [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator

2024-03-13 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 839bc9f9c264 [SPARK-46913][SS] Add support

(spark) branch master updated: [SPARK-47272][SS] Add MapState implementation for State API v2.

2024-03-12 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 29e91d0e48a3 [SPARK-47272][SS] Add MapState

(spark) branch master updated: [SPARK-47250][SS] Add additional validations and NERF changes for RocksDB state provider and use of column families

2024-03-11 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e778ce689dcb [SPARK-47250][SS] Add additional

(spark) branch master updated: [SPARK-46962][SS][PYTHON] Add interface for python streaming data source API and implement python worker to run python streaming data source

2024-03-11 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f53dc08230b7 [SPARK-46962][SS][PYTHON] Add

(spark) branch master updated: [SPARK-47331][SS] Serialization using case classes/primitives/POJO based on SQL encoder for Arbitrary State API v2

2024-03-10 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new afbebfbadc4b [SPARK-47331][SS] Serialization

(spark) branch master updated: [SPARK-47200][SS] Make Foreach batch sink user function error handling backward compatible

2024-03-10 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new edb970b8a73e [SPARK-47200][SS] Make Foreach

(spark) branch branch-3.4 updated: [SPARK-47305][SQL] Fix PruneFilters to tag the isStreaming flag of LocalRelation correctly when the plan has both batch and streaming

2024-03-06 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.4 by this push: new 7e5d59289870 [SPARK-47305][SQL] Fix

(spark) branch branch-3.5 updated: [SPARK-47305][SQL] Fix PruneFilters to tag the isStreaming flag of LocalRelation correctly when the plan has both batch and streaming

2024-03-06 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 679f3b1e5e96 [SPARK-47305][SQL] Fix

(spark) branch master updated (3ec79a9df06f -> 8d6bd9bbd29d)

2024-03-06 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 3ec79a9df06f [SPARK-47238][SQL] Reduce executor memory usage by making generated code in WSCG a broadcast variable

(spark-website) branch asf-site updated: MINOR: Update the link of latest to 3.5.1

2024-02-29 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/spark-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 53982b743a MINOR: Update the link

(spark) branch master updated (9ce43c85a5d2 -> 7447172ffd5c)

2024-02-29 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 9ce43c85a5d2 [SPARK-47229][CORE][SQL][SS][YARN][CONNECT] Change the never changed `var` to `val` add

(spark) branch master updated: [SPARK-47135][SS] Implement error classes for Kafka data loss exceptions

2024-02-28 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 36df0a63a139 [SPARK-47135][SS] Implement error

(spark-website) branch asf-site updated: Slight correction of introduction for 3.5.1

2024-02-28 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/spark-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 5cf6d34d00 Slight correction

(spark) branch branch-3.5 updated: [SPARK-47036][SS][3.5] Cleanup RocksDB file tracking for previously uploaded files if files were deleted from local directory

2024-02-21 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 3f4425b4880d [SPARK-47036][SS][3.5

(spark) branch master updated: [SPARK-47036][SS] Cleanup RocksDB file tracking for previously uploaded files if files were deleted from local directory

2024-02-21 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new af28aebfa17c [SPARK-47036][SS] Cleanup RocksDB

(spark) branch master updated: [SPARK-46928][SS] Add support for ListState in Arbitrary State API v2

2024-02-20 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 0b907ed11e6e [SPARK-46928][SS] Add support

(spark) branch master updated: [SPARK-47052][SS] Separate state tracking variables from MicroBatchExecution/StreamExecution

2024-02-20 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new bffa92c838d6 [SPARK-47052][SS] Separate state

(spark) branch master updated (eb71268a6a38 -> 8ede494ad6c7)

2024-02-20 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from eb71268a6a38 [SPARK-47108][CORE] Set `derby.connection.requireAuthentication` to `false` explicitly in CLIs add

svn commit: r67358 - in /dev/spark/v3.5.1-rc2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/R/articles/ _site/api/R/deps/ _site/api/R/deps/bootstrap-5.3.1/ _site/api/R/deps/bootstrap-5.3.1/fonts/

2024-02-15 Thread kabhwan
Author: kabhwan Date: Thu Feb 15 14:08:37 2024 New Revision: 67358 Log: Apache Spark v3.5.1-rc2 docs [This commit notification would consist of 4713 parts, which exceeds the limit of 50 ones, so it was shortened to the summary

svn commit: r67355 - /dev/spark/v3.5.1-rc2-bin/

2024-02-15 Thread kabhwan
Author: kabhwan Date: Thu Feb 15 11:39:51 2024 New Revision: 67355 Log: Apache Spark v3.5.1-rc2 Added: dev/spark/v3.5.1-rc2-bin/ dev/spark/v3.5.1-rc2-bin/SparkR_3.5.1.tar.gz (with props) dev/spark/v3.5.1-rc2-bin/SparkR_3.5.1.tar.gz.asc dev/spark/v3.5.1-rc2-bin/SparkR_3.5.1

(spark) 01/01: Preparing development version 3.5.2-SNAPSHOT

2024-02-15 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git commit cacb6fa0868a8741cb67ef705375ac378adaeebb Author: Jungtaek Lim AuthorDate: Thu Feb 15 10:56:51 2024 +

(spark) branch branch-3.5 updated (9b4778fc1dc7 -> cacb6fa0868a)

2024-02-15 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git from 9b4778fc1dc7 [SPARK-46906][INFRA][3.5] Bump python libraries (pandas, pyarrow) in Docker image for release

(spark) tag v3.5.1-rc2 created (now fd86f85e181f)

2024-02-15 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to tag v3.5.1-rc2 in repository https://gitbox.apache.org/repos/asf/spark.git at fd86f85e181f (commit) This tag includes the following new commits: new fd86f85e181f Preparing Spark release

(spark) 01/01: Preparing Spark release v3.5.1-rc2

2024-02-15 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to tag v3.5.1-rc2 in repository https://gitbox.apache.org/repos/asf/spark.git commit fd86f85e181fc2dc0f50a096855acf83a6cc5d9c Author: Jungtaek Lim AuthorDate: Thu Feb 15 10:56:47 2024 +

(spark) branch branch-3.5 updated: [SPARK-46906][INFRA][3.5] Bump python libraries (pandas, pyarrow) in Docker image for release script

2024-02-14 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 9b4778fc1dc7 [SPARK-46906][INFRA][3.5

(spark) branch master updated: [SPARK-46979][SS] Add support for specifying key and value encoder separately and also for each col family in RocksDB state store provider

2024-02-13 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 63b97c6ad82a [SPARK-46979][SS] Add support

svn commit: r67285 - /dev/spark/v3.5.1-rc1-bin/

2024-02-11 Thread kabhwan
Author: kabhwan Date: Mon Feb 12 04:25:21 2024 New Revision: 67285 Log: Apache Spark v3.5.1-rc1 Added: dev/spark/v3.5.1-rc1-bin/ dev/spark/v3.5.1-rc1-bin/SparkR_3.5.1.tar.gz (with props) dev/spark/v3.5.1-rc1-bin/SparkR_3.5.1.tar.gz.asc dev/spark/v3.5.1-rc1-bin/SparkR_3.5.1

(spark) 01/01: Preparing Spark release v3.5.1-rc1

2024-02-11 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to tag v3.5.1-rc1 in repository https://gitbox.apache.org/repos/asf/spark.git commit 08fe67b9ebf656b6ae7c44163bffba247061aa42 Author: Jungtaek Lim AuthorDate: Mon Feb 12 03:39:11 2024 +

(spark) 01/01: Preparing development version 3.5.2-SNAPSHOT

2024-02-11 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git commit d27bdbeae9c5b634702ad20a58b9d3c68ac9d39d Author: Jungtaek Lim AuthorDate: Mon Feb 12 03:39:15 2024 +

(spark) branch branch-3.5 updated (4e4d9f07d095 -> d27bdbeae9c5)

2024-02-11 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git from 4e4d9f07d095 [SPARK-47022][CONNECT][TESTS][3.5] Fix `connect/client/jvm` to have explicit `commons-(io|lang3

(spark) tag v3.5.1-rc1 created (now 08fe67b9ebf6)

2024-02-11 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to tag v3.5.1-rc1 in repository https://gitbox.apache.org/repos/asf/spark.git at 08fe67b9ebf6 (commit) This tag includes the following new commits: new 08fe67b9ebf6 Preparing Spark release

(spark) branch master updated: [SPARK-46994][PYTHON] Refactor PythonWrite to prepare for supporting python data source streaming write

2024-02-07 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new abf8770ffac7 [SPARK-46994][PYTHON] Refactor

(spark) branch master updated: [SPARK-46865][SS] Add Batch Support for TransformWithState Operator

2024-02-07 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new becfb94e1c71 [SPARK-46865][SS] Add Batch

(spark) branch master updated: [SPARK-46960][SS] Testing Multiple Input Streams with TransformWithState operator

2024-02-05 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 5100b2e5d1aa [SPARK-46960][SS] Testing Multiple

svn commit: r67146 - /dev/spark/KEYS

2024-02-03 Thread kabhwan
Author: kabhwan Date: Sat Feb 3 11:42:23 2024 New Revision: 67146 Log: Update KEYS Modified: dev/spark/KEYS Modified: dev/spark/KEYS == --- dev/spark/KEYS (original) +++ dev/spark/KEYS Sat Feb 3 11:42:23 2024

(spark) branch master updated: [SPARK-46911][SS] Adding deleteIfExists operator to StatefulProcessorHandleImpl

2024-02-02 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 25d96f7bacb4 [SPARK-46911][SS] Adding

(spark) branch master updated: [SPARK-46864][SS] Onboard Arbitrary StateV2 onto New Error Class Framework

2024-02-01 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 00e63d63f9af [SPARK-46864][SS] Onboard

(spark) branch master updated: [SPARK-46852][SS] Remove use of explicit key encoder and pass it implicitly to the operator for transformWithState operator

2024-02-01 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e610d1d8f79b [SPARK-46852][SS] Remove use

(spark) branch master updated: [SPARK-46736][PROTOBUF] retain empty message field in protobuf connector

2024-01-29 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 3f7994217d5a [SPARK-46736][PROTOBUF] retain

(spark) branch branch-3.5 updated: [SPARK-46796][SS] Ensure the correct remote files (mentioned in metadata.zip) are used on RocksDB version load

2024-01-24 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new ef33b9c50806 [SPARK-46796][SS] Ensure

(spark) branch master updated: [SPARK-46796][SS] Ensure the correct remote files (mentioned in metadata.zip) are used on RocksDB version load

2024-01-24 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f25ebe52b9b8 [SPARK-46796][SS] Ensure

(spark) branch master updated: [SPARK-46777][SS] Refactor `StreamingDataSourceV2Relation` catalyst structure to be more on-par with the batch version

2024-01-22 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 02533d71806e [SPARK-46777][SS] Refactor

(spark) branch master updated: [SPARK-46731][SS] Manage state store provider instance by state data source - reader

2024-01-19 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 56730f6390a1 [SPARK-46731][SS] Manage state

(spark) branch branch-3.5 updated: [SPARK-46676][SS] dropDuplicatesWithinWatermark should not fail on canonicalization of the plan

2024-01-18 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new fa6bf22112b4 [SPARK-46676][SS

(spark) branch master updated: [SPARK-46676][SS] dropDuplicatesWithinWatermark should not fail on canonicalization of the plan

2024-01-18 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new c1ed3e60e67f [SPARK-46676][SS

(spark) branch master updated: [SPARK-46722][CONNECT] Add a test regarding to backward compatibility check for StreamingQueryListener in Spark Connect (Scala/PySpark)

2024-01-15 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new a58362ecbbf7 [SPARK-46722][CONNECT] Add a test

(spark) branch master updated: [SPARK-46711][SS] Fix RocksDB state provider race condition during rollback

2024-01-14 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 86bac8911a31 [SPARK-46711][SS] Fix RocksDB

(spark) branch master updated: [MINOR][SS] Fix indent for streaming aggregation operator

2024-01-14 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 173ba2c6327f [MINOR][SS] Fix indent

(spark) branch branch-3.5 updated: [SPARK-46547][SS] Swallow non-fatal exception in maintenance task to avoid deadlock between maintenance thread and streaming aggregation operator

2024-01-10 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new a4b184d8db28 [SPARK-46547][SS] Swallow

(spark) branch master updated: [SPARK-46547][SS] Swallow non-fatal exception in maintenance task to avoid deadlock between maintenance thread and streaming aggregation operator

2024-01-10 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f7b0b4537917 [SPARK-46547][SS] Swallow non

(spark) branch master updated (96bf373e9002 -> d163b319e21a)

2023-12-17 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 96bf373e9002 [SPARK-46430][CONNECT][TESTS] Add test for `ProtoUtils.abbreviate` add d163b319e21a [SPARK-46431

(spark) branch master updated: [SPARK-45888][SS] Apply error class framework to State (Metadata) Data Source

2023-12-06 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 59777222e72 [SPARK-45888][SS] Apply error class

(spark) branch master updated: [SPARK-46249][SS] Require instance lock for acquiring RocksDB metrics to prevent race with background operations

2023-12-04 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 1d80d80a56c4 [SPARK-46249][SS] Require instance

(spark) branch master updated: [SPARK-45833][SS][DOCS] Document the new introduction of state data source

2023-11-28 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 04c9583cecdf [SPARK-45833][SS][DOCS] Document

(spark) branch branch-3.4 updated: [SPARK-46062][SQL] Sync the isStreaming flag between CTE definition and reference

2023-11-23 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.4 by this push: new e87d166a81c [SPARK-46062][SQL] Sync

(spark) branch branch-3.5 updated: [SPARK-46062][SQL] Sync the isStreaming flag between CTE definition and reference

2023-11-23 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new a8552627cdc [SPARK-46062][SQL] Sync

(spark) branch master updated: [SPARK-46062][SQL] Sync the isStreaming flag between CTE definition and reference

2023-11-23 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 43046631a5d [SPARK-46062][SQL] Sync

(spark) branch branch-3.4 updated: [SPARK-46064][SQL][SS] Move out EliminateEventTimeWatermark to the analyzer and change to only take effect on resolved child

2023-11-23 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.4 by this push: new cfe072a904b [SPARK-46064][SQL][SS] Move

(spark) branch branch-3.5 updated: [SPARK-46064][SQL][SS] Move out EliminateEventTimeWatermark to the analyzer and change to only take effect on resolved child

2023-11-23 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new c3671942abb [SPARK-46064][SQL][SS] Move

(spark) branch master updated: [SPARK-46064][SQL][SS] Move out EliminateEventTimeWatermark to the analyzer and change to only take effect on resolved child

2023-11-23 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new a703dace0aa [SPARK-46064][SQL][SS] Move out

(spark) branch master updated: [SPARK-45511][SS] Fix state reader suite flakiness by clean up resources after each test run

2023-11-16 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new aff9eab9039 [SPARK-45511][SS] Fix state reader

(spark) branch master updated: [SPARK-45946][SS] Fix use of deprecated FileUtils write to pass default charset in RocksDBSuite

2023-11-16 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new bbe95cfcd05 [SPARK-45946][SS] Fix use

(spark) branch master updated: [SPARK-45747][SS] Use prefix key information in state metadata to handle reading state for session window aggregation

2023-11-15 Thread kabhwan
This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new bd14136b397 [SPARK-45747][SS] Use prefix key

  1   2   3   4   5   6   >