[jira] [Resolved] (SPARK-47920) Add documentation for python streaming data source

[ https://issues.apache.org/jira/browse/SPARK-47920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47920. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46139

[jira] [Resolved] (SPARK-48314) FileStreamSource shouldn't double cache files for availableNow

[ https://issues.apache.org/jira/browse/SPARK-48314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48314. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46627

[jira] [Assigned] (SPARK-48314) FileStreamSource shouldn't double cache files for availableNow

[ https://issues.apache.org/jira/browse/SPARK-48314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-48314: Assignee: Adam Binford > FileStreamSource shouldn't double cache files for availableNow

[jira] [Assigned] (SPARK-48330) Fix the python streaming data source timeout issue for large trigger interval

[ https://issues.apache.org/jira/browse/SPARK-48330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-48330: Assignee: Chaoqin Li > Fix the python streaming data source timeout issue for large

[jira] [Resolved] (SPARK-48330) Fix the python streaming data source timeout issue for large trigger interval

[ https://issues.apache.org/jira/browse/SPARK-48330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48330. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46651

[jira] [Assigned] (SPARK-44924) Add configurations for FileStreamSource cached files

[ https://issues.apache.org/jira/browse/SPARK-44924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-44924: Assignee: kevin nacios > Add configurations for FileStreamSource cached files >

[jira] [Resolved] (SPARK-44924) Add configurations for FileStreamSource cached files

[ https://issues.apache.org/jira/browse/SPARK-44924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-44924. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45362

[jira] [Updated] (SPARK-48105) Fix the data corruption issue when state store unload and snapshotting happens concurrently for HDFS state store

[ https://issues.apache.org/jira/browse/SPARK-48105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48105: - Labels: correctness pull-request-available (was: pull-request-available) > Fix the data

[jira] [Updated] (SPARK-48105) Fix the data corruption issue when state store unload and snapshotting happens concurrently for HDFS state store

[ https://issues.apache.org/jira/browse/SPARK-48105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48105: - Issue Type: Bug (was: Improvement) > Fix the data corruption issue when state store unload

[jira] [Updated] (SPARK-48105) Fix the data corruption issue when state store unload and snapshotting happens concurrently for HDFS state store

[ https://issues.apache.org/jira/browse/SPARK-48105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48105: - Priority: Blocker (was: Major) > Fix the data corruption issue when state store unload and

[jira] [Updated] (SPARK-48105) Fix the data corruption issue when state store unload and snapshotting happens concurrently for HDFS state store

[ https://issues.apache.org/jira/browse/SPARK-48105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48105: - Fix Version/s: 3.4.4 > Fix the data corruption issue when state store unload and snapshotting

[jira] [Resolved] (SPARK-48293) Add test for when ForeachBatchUserFuncException wraps interrupted exception due to query stop

[ https://issues.apache.org/jira/browse/SPARK-48293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48293. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46601

[jira] [Updated] (SPARK-48105) Fix the data corruption issue when state store unload and snapshotting happens concurrently for HDFS state store

[ https://issues.apache.org/jira/browse/SPARK-48105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48105: - Fix Version/s: 3.5.2 > Fix the data corruption issue when state store unload and snapshotting

[jira] [Resolved] (SPARK-48233) Tests for non-stateful streaming with collations

[ https://issues.apache.org/jira/browse/SPARK-48233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48233. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46247

[jira] [Updated] (SPARK-48267) Regression e2e test with SPARK-47305

[ https://issues.apache.org/jira/browse/SPARK-48267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48267: - Fix Version/s: 3.5.2 > Regression e2e test with SPARK-47305 >

[jira] [Resolved] (SPARK-48267) Regression e2e test with SPARK-47305

[ https://issues.apache.org/jira/browse/SPARK-48267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48267. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46569

[jira] [Assigned] (SPARK-48267) Regression e2e test with SPARK-47305

[ https://issues.apache.org/jira/browse/SPARK-48267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-48267: Assignee: Jungtaek Lim > Regression e2e test with SPARK-47305 >

[jira] [Created] (SPARK-48267) Regression e2e test with SPARK-47305

Jungtaek Lim created SPARK-48267: Summary: Regression e2e test with SPARK-47305 Key: SPARK-48267 URL: https://issues.apache.org/jira/browse/SPARK-48267 Project: Spark Issue Type: Test

[jira] [Assigned] (SPARK-48208) Skip reporting memory usage metrics if bounded memory usage is enabled

[ https://issues.apache.org/jira/browse/SPARK-48208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-48208: Assignee: Anish Shrigondekar > Skip reporting memory usage metrics if bounded memory

[jira] [Resolved] (SPARK-48208) Skip reporting memory usage metrics if bounded memory usage is enabled

[ https://issues.apache.org/jira/browse/SPARK-48208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48208. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46491

[jira] [Assigned] (SPARK-47960) Support Chaining Stateful Operators in TransformWithState

[ https://issues.apache.org/jira/browse/SPARK-47960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47960: Assignee: Bhuwan Sahni > Support Chaining Stateful Operators in TransformWithState >

[jira] [Resolved] (SPARK-47960) Support Chaining Stateful Operators in TransformWithState

[ https://issues.apache.org/jira/browse/SPARK-47960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47960. -- Resolution: Fixed Issue resolved by pull request 45376

[jira] [Updated] (SPARK-48105) Fix the data corruption issue when state store unload and snapshotting happens concurrently for HDFS state store

[ https://issues.apache.org/jira/browse/SPARK-48105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48105: - Affects Version/s: 3.5.2 3.4.4 > Fix the data corruption issue when

[jira] [Resolved] (SPARK-48105) Fix the data corruption issue when state store unload and snapshotting happens concurrently for HDFS state store

[ https://issues.apache.org/jira/browse/SPARK-48105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48105. -- Fix Version/s: 4.0.0 Assignee: Huanli Wang Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-48102) Track time to acquire source progress metrics for streaming triggers

[ https://issues.apache.org/jira/browse/SPARK-48102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48102. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46350

[jira] [Assigned] (SPARK-48102) Track time to acquire source progress metrics for streaming triggers

[ https://issues.apache.org/jira/browse/SPARK-48102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-48102: Assignee: Anish Shrigondekar > Track time to acquire source progress metrics for

[jira] [Updated] (SPARK-47920) Add documentation for python streaming data source

[ https://issues.apache.org/jira/browse/SPARK-47920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-47920: - Affects Version/s: 4.0.0 (was: 3.5.1) > Add documentation for python

[jira] [Commented] (SPARK-48073) StateStore schema incompatibility between 3.2 and 3.4

[ https://issues.apache.org/jira/browse/SPARK-48073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842785#comment-17842785 ] Jungtaek Lim commented on SPARK-48073: -- I roughly remember that Encoder.bean() had changed to

[jira] [Resolved] (SPARK-47793) Implement SimpleDataSourceStreamReader for python streaming data source

[ https://issues.apache.org/jira/browse/SPARK-47793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47793. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45977

[jira] [Assigned] (SPARK-47793) Implement SimpleDataSourceStreamReader for python streaming data source

[ https://issues.apache.org/jira/browse/SPARK-47793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47793: Assignee: Chaoqin Li > Implement SimpleDataSourceStreamReader for python streaming data

[jira] [Assigned] (SPARK-48050) Log logical plan at query start

[ https://issues.apache.org/jira/browse/SPARK-48050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-48050: Assignee: Fanyue Xia > Log logical plan at query start > ---

[jira] [Resolved] (SPARK-48050) Log logical plan at query start

[ https://issues.apache.org/jira/browse/SPARK-48050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48050. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46292

[jira] [Assigned] (SPARK-48018) Null groupId causing missing param error when throwing KafkaException.couldNotReadOffsetRange

[ https://issues.apache.org/jira/browse/SPARK-48018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-48018: Assignee: B. Micheal Okutubo > Null groupId causing missing param error when throwing >

[jira] [Resolved] (SPARK-48018) Null groupId causing missing param error when throwing KafkaException.couldNotReadOffsetRange

[ https://issues.apache.org/jira/browse/SPARK-48018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48018. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46253

[jira] [Resolved] (SPARK-47805) [Arbitrary State Support] State TTL support - MapState

[ https://issues.apache.org/jira/browse/SPARK-47805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47805. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45991

[jira] [Assigned] (SPARK-47840) Remove foldable propagation across Streaming Aggregate/Join nodes

[ https://issues.apache.org/jira/browse/SPARK-47840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47840: Assignee: Bhuwan Sahni > Remove foldable propagation across Streaming Aggregate/Join

[jira] [Resolved] (SPARK-47840) Remove foldable propagation across Streaming Aggregate/Join nodes

[ https://issues.apache.org/jira/browse/SPARK-47840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47840. -- Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-47673) [Arbitrary State Support] State TTL support - ListState

[ https://issues.apache.org/jira/browse/SPARK-47673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47673. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45932

[jira] [Resolved] (SPARK-47788) Ensure the same hash partitioning scheme/hash function is used across batches

[ https://issues.apache.org/jira/browse/SPARK-47788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47788. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45971

[jira] [Assigned] (SPARK-47788) Ensure the same hash partitioning scheme/hash function is used across batches

[ https://issues.apache.org/jira/browse/SPARK-47788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47788: Assignee: Fanyue Xia > Ensure the same hash partitioning scheme/hash function is used

[jira] [Resolved] (SPARK-47848) Fix thread safe access for loadedMaps in close

[ https://issues.apache.org/jira/browse/SPARK-47848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47848. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46048

[jira] [Assigned] (SPARK-47733) Add operational metrics for TWS operators

[ https://issues.apache.org/jira/browse/SPARK-47733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47733: Assignee: Anish Shrigondekar (was: Jing Zhan) > Add operational metrics for TWS

[jira] [Assigned] (SPARK-47733) Add operational metrics for TWS operators

[ https://issues.apache.org/jira/browse/SPARK-47733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47733: Assignee: Jing Zhan > Add operational metrics for TWS operators >

[jira] [Resolved] (SPARK-47784) [State API v2] Merge TimeoutMode and TTLMode into TimeMode

[ https://issues.apache.org/jira/browse/SPARK-47784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47784. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45960

[jira] [Assigned] (SPARK-47776) State store operation cannot work properly with binary inequality collation

[ https://issues.apache.org/jira/browse/SPARK-47776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47776: Assignee: Jungtaek Lim > State store operation cannot work properly with binary

[jira] [Resolved] (SPARK-47776) State store operation cannot work properly with binary inequality collation

[ https://issues.apache.org/jira/browse/SPARK-47776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47776. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45951

[jira] [Commented] (SPARK-47718) .sql() does not recognize watermark defined upstream

[ https://issues.apache.org/jira/browse/SPARK-47718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835303#comment-17835303 ] Jungtaek Lim commented on SPARK-47718: -- window('createTime', '1 hour', '30 minutes') 'createTime'

[jira] [Resolved] (SPARK-47718) .sql() does not recognize watermark defined upstream

[ https://issues.apache.org/jira/browse/SPARK-47718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47718. -- Resolution: Not A Bug > .sql() does not recognize watermark defined upstream >

[jira] [Created] (SPARK-47776) State store operation cannot work properly with binary inequality collation

Jungtaek Lim created SPARK-47776: Summary: State store operation cannot work properly with binary inequality collation Key: SPARK-47776 URL: https://issues.apache.org/jira/browse/SPARK-47776 Project:

[jira] [Commented] (SPARK-47718) .sql() does not recognize watermark defined upstream

[ https://issues.apache.org/jira/browse/SPARK-47718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835175#comment-17835175 ] Jungtaek Lim commented on SPARK-47718: -- I've lowered down to major - this is neither a regression

[jira] [Updated] (SPARK-47718) .sql() does not recognize watermark defined upstream

[ https://issues.apache.org/jira/browse/SPARK-47718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-47718: - Priority: Major (was: Blocker) > .sql() does not recognize watermark defined upstream >

[jira] [Resolved] (SPARK-47746) Use column ordinals instead of prefix ordering columns in the range scan encoder

[ https://issues.apache.org/jira/browse/SPARK-47746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47746. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45905

[jira] [Resolved] (SPARK-47558) [Arbitrary State Support] State TTL support - ValueState

[ https://issues.apache.org/jira/browse/SPARK-47558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47558. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45674

[jira] [Assigned] (SPARK-47299) Use the same `versions. json` in the dropdown of different versions of PySpark documents

[ https://issues.apache.org/jira/browse/SPARK-47299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47299: Assignee: BingKun Pan > Use the same `versions. json` in the dropdown of different

[jira] [Resolved] (SPARK-47299) Use the same `versions. json` in the dropdown of different versions of PySpark documents

[ https://issues.apache.org/jira/browse/SPARK-47299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47299. -- Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-47734) Fix flaky pyspark.sql.dataframe.DataFrame.writeStream doctest by stopping streaming query

[ https://issues.apache.org/jira/browse/SPARK-47734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-47734: - Fix Version/s: 3.4.3 > Fix flaky pyspark.sql.dataframe.DataFrame.writeStream doctest by

[jira] [Resolved] (SPARK-47744) Add support for negative byte types in RocksDB range scan encoder

[ https://issues.apache.org/jira/browse/SPARK-47744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47744. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45906

[jira] [Assigned] (SPARK-47744) Add support for negative byte types in RocksDB range scan encoder

[ https://issues.apache.org/jira/browse/SPARK-47744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47744: Assignee: Neil Ramaswamy > Add support for negative byte types in RocksDB range scan

[jira] [Resolved] (SPARK-47653) Add support for negative numbers with range scan encoder

[ https://issues.apache.org/jira/browse/SPARK-47653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47653. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45778

[jira] [Resolved] (SPARK-47655) Integrate timer with Initial State handling for state-v2

[ https://issues.apache.org/jira/browse/SPARK-47655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47655. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45780

[jira] [Resolved] (SPARK-47568) Fix race condition between maintenance thread and task thead for RocksDB snapshot

[ https://issues.apache.org/jira/browse/SPARK-47568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47568. -- Fix Version/s: 4.0.0 Assignee: Bhuwan Sahni Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-47363) Initial State without state reader implementation for State API v2.

[ https://issues.apache.org/jira/browse/SPARK-47363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47363. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45467

[jira] [Resolved] (SPARK-47107) Implement partition reader for python streaming data source

[ https://issues.apache.org/jira/browse/SPARK-47107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47107. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45485

[jira] [Assigned] (SPARK-47107) Implement partition reader for python streaming data source

[ https://issues.apache.org/jira/browse/SPARK-47107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47107: Assignee: Chaoqin Li > Implement partition reader for python streaming data source >

[jira] [Assigned] (SPARK-47570) Integrate range scan encoder changes with timer implementation

[ https://issues.apache.org/jira/browse/SPARK-47570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47570: Assignee: Jing Zhan > Integrate range scan encoder changes with timer implementation >

[jira] [Resolved] (SPARK-47570) Integrate range scan encoder changes with timer implementation

[ https://issues.apache.org/jira/browse/SPARK-47570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47570. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45709

[jira] [Resolved] (SPARK-47273) Implement python stream writer interface

[ https://issues.apache.org/jira/browse/SPARK-47273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47273. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45305

[jira] [Assigned] (SPARK-47273) Implement python stream writer interface

[ https://issues.apache.org/jira/browse/SPARK-47273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47273: Assignee: Chaoqin Li > Implement python stream writer interface >

[jira] [Resolved] (SPARK-47512) Tag operation type for RocksDB instance lock acquisition

[ https://issues.apache.org/jira/browse/SPARK-47512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47512. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45651

[jira] [Resolved] (SPARK-47449) Refactor and split list/timer unit tests

[ https://issues.apache.org/jira/browse/SPARK-47449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47449. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45573

[jira] [Resolved] (SPARK-47329) Persist df while using foreachbatch and stateful streaming query to prevent state from being re-loaded in each batch

[ https://issues.apache.org/jira/browse/SPARK-47329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47329. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45432

[jira] [Resolved] (SPARK-46913) Implement timer functionality for transformWithState operator

[ https://issues.apache.org/jira/browse/SPARK-46913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-46913. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45051

[jira] [Resolved] (SPARK-47272) MapState Implementation for State V2

[ https://issues.apache.org/jira/browse/SPARK-47272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47272. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45341

[jira] [Assigned] (SPARK-46962) Implement python worker to run python streaming data source

[ https://issues.apache.org/jira/browse/SPARK-46962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-46962: Assignee: Chaoqin Li > Implement python worker to run python streaming data source >

[jira] [Resolved] (SPARK-46962) Implement python worker to run python streaming data source

[ https://issues.apache.org/jira/browse/SPARK-46962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-46962. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45023

[jira] [Resolved] (SPARK-47331) Serialization using case classes/primitives/POJO based on SQL encoder for Arbitrary State API v2.

[ https://issues.apache.org/jira/browse/SPARK-47331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47331. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45447

[jira] [Assigned] (SPARK-47331) Serialization using case classes/primitives/POJO based on SQL encoder for Arbitrary State API v2.

[ https://issues.apache.org/jira/browse/SPARK-47331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47331: Assignee: Jing Zhan > Serialization using case classes/primitives/POJO based on SQL

[jira] [Resolved] (SPARK-47305) PruneFilters incorrectly tags isStreaming flag when replacing child of Filter with LocalRelation

[ https://issues.apache.org/jira/browse/SPARK-47305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47305. -- Fix Version/s: 3.4.3 3.5.2 4.0.0 Resolution:

[jira] [Assigned] (SPARK-47305) PruneFilters incorrectly tags isStreaming flag when replacing child of Filter with LocalRelation

[ https://issues.apache.org/jira/browse/SPARK-47305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47305: Assignee: Jungtaek Lim > PruneFilters incorrectly tags isStreaming flag when replacing

[jira] [Created] (SPARK-47305) PruneFilters incorrectly tags isStreaming flag when replacing child of Filter with LocalRelation

Jungtaek Lim created SPARK-47305: Summary: PruneFilters incorrectly tags isStreaming flag when replacing child of Filter with LocalRelation Key: SPARK-47305 URL: https://issues.apache.org/jira/browse/SPARK-47305

[jira] [Resolved] (SPARK-47200) Handle and classify errors from ForEachBatchSink user function

[ https://issues.apache.org/jira/browse/SPARK-47200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47200. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45299

[jira] [Resolved] (SPARK-47135) Implement error classes for Kafka data loss exceptions

[ https://issues.apache.org/jira/browse/SPARK-47135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47135. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45221

[jira] [Assigned] (SPARK-47135) Implement error classes for Kafka data loss exceptions

[ https://issues.apache.org/jira/browse/SPARK-47135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47135: Assignee: B. Micheal Okutubo > Implement error classes for Kafka data loss exceptions >

[jira] [Updated] (SPARK-45599) Percentile can produce a wrong answer if -0.0 and 0.0 are mixed in the dataset

[ https://issues.apache.org/jira/browse/SPARK-45599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-45599: - Fix Version/s: 3.5.2 (was: 3.5.1) > Percentile can produce a wrong

[jira] [Updated] (SPARK-47023) Upgrade `aircompressor` to 0.26

[ https://issues.apache.org/jira/browse/SPARK-47023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-47023: - Description: `aircompressor` is a transitive dependency from Apache ORC and Parquet.

[jira] [Updated] (SPARK-47023) Upgrade `aircompressor` to 0.26

[ https://issues.apache.org/jira/browse/SPARK-47023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-47023: - Summary: Upgrade `aircompressor` to 0.26 (was: Upgrade `aircompressor` to 1.26) > Upgrade

[jira] [Updated] (SPARK-47036) RocksDB versionID Mismatch in SST files with Compaction

[ https://issues.apache.org/jira/browse/SPARK-47036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-47036: - Fix Version/s: 3.5.2 > RocksDB versionID Mismatch in SST files with Compaction >

[jira] [Assigned] (SPARK-47036) RocksDB versionID Mismatch in SST files with Compaction

[ https://issues.apache.org/jira/browse/SPARK-47036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47036: Assignee: Bhuwan Sahni > RocksDB versionID Mismatch in SST files with Compaction >

[jira] [Resolved] (SPARK-47036) RocksDB versionID Mismatch in SST files with Compaction

[ https://issues.apache.org/jira/browse/SPARK-47036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47036. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45092

[jira] [Resolved] (SPARK-46928) Support ListState in Arbitrary State API v2

[ https://issues.apache.org/jira/browse/SPARK-46928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-46928. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44961

[jira] [Assigned] (SPARK-46928) Support ListState in Arbitrary State API v2

[ https://issues.apache.org/jira/browse/SPARK-46928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-46928: Assignee: Bhuwan Sahni > Support ListState in Arbitrary State API v2 >

[jira] [Assigned] (SPARK-47052) Separate state tracking variables from MicroBatchExecution/StreamExecution

[ https://issues.apache.org/jira/browse/SPARK-47052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47052: Assignee: Boyang Jerry Peng > Separate state tracking variables from

[jira] [Resolved] (SPARK-47052) Separate state tracking variables from MicroBatchExecution/StreamExecution

[ https://issues.apache.org/jira/browse/SPARK-47052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47052. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45109

[jira] [Resolved] (SPARK-46906) Add a check for stateful operator change for streaming

[ https://issues.apache.org/jira/browse/SPARK-46906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-46906. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44927

[jira] [Commented] (SPARK-46934) Unable to create Hive View from certain Spark Dataframe StructType

[ https://issues.apache.org/jira/browse/SPARK-46934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17818634#comment-17818634 ] Jungtaek Lim commented on SPARK-46934: -- Maybe the priority also has to be updated as well - we

[jira] [Assigned] (SPARK-47053) Docker image for release has to bump versions of some python libraries for 3.5.1

[ https://issues.apache.org/jira/browse/SPARK-47053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47053: Assignee: Jungtaek Lim > Docker image for release has to bump versions of some python

[jira] [Resolved] (SPARK-47053) Docker image for release has to bump versions of some python libraries for 3.5.1

[ https://issues.apache.org/jira/browse/SPARK-47053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47053. -- Fix Version/s: 3.5.1 Resolution: Fixed Issue resolved by pull request 45111

[jira] [Created] (SPARK-47053) Docker image for release has to bump versions of some python libraries for 3.5.1

Jungtaek Lim created SPARK-47053: Summary: Docker image for release has to bump versions of some python libraries for 3.5.1 Key: SPARK-47053 URL: https://issues.apache.org/jira/browse/SPARK-47053

[jira] [Assigned] (SPARK-46979) Add support for defining state encoder for key/value and col family independently

[ https://issues.apache.org/jira/browse/SPARK-46979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-46979: Assignee: Anish Shrigondekar > Add support for defining state encoder for key/value and

[jira] [Resolved] (SPARK-46979) Add support for defining state encoder for key/value and col family independently

[ https://issues.apache.org/jira/browse/SPARK-46979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-46979. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45038

  1   2   3   4   5   6   7   8   9   10   >