[jira] [Assigned] (SPARK-38941) Skip RocksDB-based test case in StreamingJoinSuite on Apple Silicon

2022-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38941: - Assignee: William Hyun > Skip RocksDB-based test case in StreamingJoinSuite on Apple

[jira] [Resolved] (SPARK-38941) Skip RocksDB-based test case in StreamingJoinSuite on Apple Silicon

2022-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38941. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36254

[jira] [Assigned] (SPARK-38941) Skip RocksDB-based test case in StreamingJoinSuite on Apple Silicon

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38941: Assignee: (was: Apache Spark) > Skip RocksDB-based test case in StreamingJoinSuite

[jira] [Commented] (SPARK-38941) Skip RocksDB-based test case in StreamingJoinSuite on Apple Silicon

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17524063#comment-17524063 ] Apache Spark commented on SPARK-38941: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-38941) Skip RocksDB-based test case in StreamingJoinSuite on Apple Silicon

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38941: Assignee: Apache Spark > Skip RocksDB-based test case in StreamingJoinSuite on Apple

[jira] [Created] (SPARK-38941) Skip RocksDB-based test case in StreamingJoinSuite on Apple Silicon

2022-04-18 Thread William Hyun (Jira)
William Hyun created SPARK-38941: Summary: Skip RocksDB-based test case in StreamingJoinSuite on Apple Silicon Key: SPARK-38941 URL: https://issues.apache.org/jira/browse/SPARK-38941 Project: Spark

[jira] [Assigned] (SPARK-38932) Datasource v2 support report unique keys

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38932: Assignee: Apache Spark > Datasource v2 support report unique keys >

[jira] [Commented] (SPARK-38932) Datasource v2 support report unique keys

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17524051#comment-17524051 ] Apache Spark commented on SPARK-38932: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-38932) Datasource v2 support report unique keys

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17524052#comment-17524052 ] Apache Spark commented on SPARK-38932: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-38932) Datasource v2 support report unique keys

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38932: Assignee: (was: Apache Spark) > Datasource v2 support report unique keys >

[jira] [Resolved] (SPARK-37670) Support predicate pushdown and column pruning for de-duped CTEs

2022-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37670. - Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-37670) Support predicate pushdown and column pruning for de-duped CTEs

2022-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37670: --- Assignee: Wei Xue > Support predicate pushdown and column pruning for de-duped CTEs >

[jira] [Resolved] (SPARK-38940) Test Series' anchor frame for in-place updates on Series

2022-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38940. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36250

[jira] [Assigned] (SPARK-38903) Implement `ignore_index` of `Series.sort_values` and `Series.sort_index`

2022-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38903: Assignee: Xinrong Meng > Implement `ignore_index` of `Series.sort_values` and

[jira] [Resolved] (SPARK-38903) Implement `ignore_index` of `Series.sort_values` and `Series.sort_index`

2022-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38903. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36186

[jira] [Assigned] (SPARK-38940) Test Series' anchor frame for in-place updates on Series

2022-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38940: Assignee: Xinrong Meng > Test Series' anchor frame for in-place updates on Series >

[jira] [Assigned] (SPARK-38880) Implement `numeric_only` parameter of `GroupBy.max/min`

2022-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38880: Assignee: Xinrong Meng > Implement `numeric_only` parameter of `GroupBy.max/min` >

[jira] [Resolved] (SPARK-38880) Implement `numeric_only` parameter of `GroupBy.max/min`

2022-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38880. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36148

[jira] [Commented] (SPARK-38856) Fix a rejectedExecutionException error when push-based shuffle is enabled

2022-04-18 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523981#comment-17523981 ] weixiuli commented on SPARK-38856: -- OK, done.  [~srowen]  > Fix a rejectedExecutionException error

[jira] [Updated] (SPARK-38856) Fix a rejectedExecutionException error when push-based shuffle is enabled

2022-04-18 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-38856: - Description: When enabled push-based shuffle in our production, there will be a

[jira] [Resolved] (SPARK-38933) Add examples of window functions into SQL docs

2022-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38933. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36243

[jira] [Updated] (SPARK-38933) Add examples of window functions into SQL docs

2022-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38933: - Fix Version/s: 3.3.0 (was: 3.4.0) > Add examples of window functions

[jira] [Assigned] (SPARK-38933) Add examples of window functions into SQL docs

2022-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38933: Assignee: jiaan.geng > Add examples of window functions into SQL docs >

[jira] [Commented] (SPARK-38939) Support ALTER TABLE ... DROP [IF EXISTS] COLUMN .. syntax

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523958#comment-17523958 ] Apache Spark commented on SPARK-38939: -- User 'jackierwzhang' has created a pull request for this

[jira] [Commented] (SPARK-38940) Test Series' anchor frame for in-place updates on Series

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523957#comment-17523957 ] Apache Spark commented on SPARK-38940: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-38940) Test Series' anchor frame for in-place updates on Series

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38940: Assignee: Apache Spark > Test Series' anchor frame for in-place updates on Series >

[jira] [Commented] (SPARK-38940) Test Series' anchor frame for in-place updates on Series

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523956#comment-17523956 ] Apache Spark commented on SPARK-38940: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-38940) Test Series' anchor frame for in-place updates on Series

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38940: Assignee: (was: Apache Spark) > Test Series' anchor frame for in-place updates on

[jira] [Created] (SPARK-38940) Test Series' anchor frame for in-place updates on Series

2022-04-18 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-38940: Summary: Test Series' anchor frame for in-place updates on Series Key: SPARK-38940 URL: https://issues.apache.org/jira/browse/SPARK-38940 Project: Spark

[jira] [Commented] (SPARK-38939) Support ALTER TABLE ... DROP [IF EXISTS] COLUMN .. syntax

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523954#comment-17523954 ] Apache Spark commented on SPARK-38939: -- User 'jackierwzhang' has created a pull request for this

[jira] [Commented] (SPARK-38939) Support ALTER TABLE ... DROP [IF EXISTS] COLUMN .. syntax

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523953#comment-17523953 ] Apache Spark commented on SPARK-38939: -- User 'jackierwzhang' has created a pull request for this

[jira] [Assigned] (SPARK-38939) Support ALTER TABLE ... DROP [IF EXISTS] COLUMN .. syntax

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38939: Assignee: (was: Apache Spark) > Support ALTER TABLE ... DROP [IF EXISTS] COLUMN ..

[jira] [Commented] (SPARK-38939) Support ALTER TABLE ... DROP [IF EXISTS] COLUMN .. syntax

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523946#comment-17523946 ] Apache Spark commented on SPARK-38939: -- User 'jackierwzhang' has created a pull request for this

[jira] [Assigned] (SPARK-38939) Support ALTER TABLE ... DROP [IF EXISTS] COLUMN .. syntax

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38939: Assignee: Apache Spark > Support ALTER TABLE ... DROP [IF EXISTS] COLUMN .. syntax >

[jira] [Updated] (SPARK-38939) Support ALTER TABLE ... DROP [IF EXISTS] COLUMN .. syntax

2022-04-18 Thread Jackie Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackie Zhang updated SPARK-38939: - Summary: Support ALTER TABLE ... DROP [IF EXISTS] COLUMN .. syntax (was: Support ALTER TABLE

[jira] [Commented] (SPARK-38825) Add a test to cover parquet notIn filter

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523909#comment-17523909 ] Apache Spark commented on SPARK-38825: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-38825) Add a test to cover parquet notIn filter

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523907#comment-17523907 ] Apache Spark commented on SPARK-38825: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-38330) Certificate doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]

2022-04-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523891#comment-17523891 ] Steve Loughran commented on SPARK-38330: FWIW I'm not 100% sure this is fixed, as we've had

[jira] [Resolved] (SPARK-38856) Fix a rejectedExecutionException error when push-based shuffle is enabled

2022-04-18 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-38856. - Target Version/s: 3.4.0 Assignee: weixiuli Resolution:

[jira] [Updated] (SPARK-38939) Support ALTER TABLE ... DROP COLUMN [IF EXISTS] .. syntax

2022-04-18 Thread Jackie Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackie Zhang updated SPARK-38939: - Description: Currently `ALTER TABLE ... DROP COLUMN(s) ...` syntax will always throw error if

[jira] [Assigned] (SPARK-38851) Refactor `HistoryServerSuite` to add UTs for RocksDB

2022-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38851: - Assignee: Yang Jie > Refactor `HistoryServerSuite` to add UTs for RocksDB >

[jira] [Resolved] (SPARK-38851) Refactor `HistoryServerSuite` to add UTs for RocksDB

2022-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38851. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36138

[jira] [Updated] (SPARK-38939) Support ALTER TABLE ... DROP COLUMN [IF EXISTS] .. syntax

2022-04-18 Thread Jackie Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackie Zhang updated SPARK-38939: - Description: Currently `ALTER TABLE ... DROP COLUMN(s) ...` syntax will always throw error if

[jira] [Created] (SPARK-38939) Support ALTER TABLE ... DROP COLUMN [IF EXISTS] .. syntax

2022-04-18 Thread Jackie Zhang (Jira)
Jackie Zhang created SPARK-38939: Summary: Support ALTER TABLE ... DROP COLUMN [IF EXISTS] .. syntax Key: SPARK-38939 URL: https://issues.apache.org/jira/browse/SPARK-38939 Project: Spark

[jira] [Commented] (SPARK-38810) Upgrade Python black in /dev/reformat-python

2022-04-18 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-38810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523847#comment-17523847 ] Bjørn Jørgensen commented on SPARK-38810: - [Black

[jira] [Updated] (SPARK-38868) `assert_true` fails unconditionnaly after `left_outer` joins

2022-04-18 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-38868: -- Component/s: SQL > `assert_true` fails unconditionnaly after `left_outer` joins >

[jira] [Assigned] (SPARK-38938) Implement `inplace` and `columns` parameters of `Series.drop`

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38938: Assignee: Apache Spark > Implement `inplace` and `columns` parameters of `Series.drop` >

[jira] [Assigned] (SPARK-38938) Implement `inplace` and `columns` parameters of `Series.drop`

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38938: Assignee: (was: Apache Spark) > Implement `inplace` and `columns` parameters of

[jira] [Commented] (SPARK-38938) Implement `inplace` and `columns` parameters of `Series.drop`

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523796#comment-17523796 ] Apache Spark commented on SPARK-38938: -- User 'xinrong-databricks' has created a pull request for

[jira] [Created] (SPARK-38938) Implement `inplace` and `columns` parameters of `Series.drop`

2022-04-18 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-38938: Summary: Implement `inplace` and `columns` parameters of `Series.drop` Key: SPARK-38938 URL: https://issues.apache.org/jira/browse/SPARK-38938 Project: Spark

[jira] [Assigned] (SPARK-38910) Clean sparkStaging dir should before unregister()

2022-04-18 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38910: Assignee: angerszhu > Clean sparkStaging dir should before unregister() >

[jira] [Updated] (SPARK-38910) Clean sparkStaging dir should before unregister()

2022-04-18 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38910: - Priority: Minor (was: Major) > Clean sparkStaging dir should before unregister() >

[jira] [Resolved] (SPARK-38910) Clean sparkStaging dir should before unregister()

2022-04-18 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38910. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36207

[jira] [Resolved] (SPARK-38885) Upgrade netty to 4.1.76

2022-04-18 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38885. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36173

[jira] [Assigned] (SPARK-38885) Upgrade netty to 4.1.76

2022-04-18 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38885: Assignee: Yang Jie > Upgrade netty to 4.1.76 > --- > >

[jira] [Resolved] (SPARK-37015) Inline type hints for python/pyspark/streaming/dstream.py

2022-04-18 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37015. Fix Version/s: 3.3.0 Resolution: Fixed > Inline type hints for

[jira] [Assigned] (SPARK-37015) Inline type hints for python/pyspark/streaming/dstream.py

2022-04-18 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37015: -- Assignee: dch nguyen > Inline type hints for

[jira] [Resolved] (SPARK-37093) Inline type hints python/pyspark/streaming

2022-04-18 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37093. Fix Version/s: 3.3.0 Resolution: Fixed > Inline type hints

[jira] [Commented] (SPARK-37015) Inline type hints for python/pyspark/streaming/dstream.py

2022-04-18 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523761#comment-17523761 ] Maciej Szymkiewicz commented on SPARK-37015: Issue resolved by pull request 34324.

[jira] [Commented] (SPARK-38926) Output types in error messages in SQL style

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523760#comment-17523760 ] Apache Spark commented on SPARK-38926: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38886) Remove outer join if aggregate functions are duplicate agnostic on streamed side

2022-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38886: --- Assignee: XiDuo You > Remove outer join if aggregate functions are duplicate agnostic on

[jira] [Resolved] (SPARK-38886) Remove outer join if aggregate functions are duplicate agnostic on streamed side

2022-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38886. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36177

[jira] [Resolved] (SPARK-37643) when charVarcharAsString is true, char datatype partition table query incorrect

2022-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37643. - Fix Version/s: 3.3.0 3.2.2 3.1.3 Resolution: Fixed

[jira] [Assigned] (SPARK-37643) when charVarcharAsString is true, char datatype partition table query incorrect

2022-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37643: --- Assignee: YuanGuanhu > when charVarcharAsString is true, char datatype partition table

[jira] [Updated] (SPARK-38935) Improve the exception type and message of casting string to numbers

2022-04-18 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-38935: --- Priority: Minor (was: Major) > Improve the exception type and message of casting string to

[jira] [Commented] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

2022-04-18 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523711#comment-17523711 ] Sean R. Owen commented on SPARK-38816: -- Hm, you're right, I didn't see that before. The blocks are

[jira] [Assigned] (SPARK-38937) interpolate support param `limit_direction`

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38937: Assignee: (was: Apache Spark) > interpolate support param `limit_direction` >

[jira] [Assigned] (SPARK-38937) interpolate support param `limit_direction`

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38937: Assignee: Apache Spark > interpolate support param `limit_direction` >

[jira] [Commented] (SPARK-38937) interpolate support param `limit_direction`

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523696#comment-17523696 ] Apache Spark commented on SPARK-38937: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-38937) interpolate support param `limit_direction`

2022-04-18 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-38937: Summary: interpolate support param `limit_direction` Key: SPARK-38937 URL: https://issues.apache.org/jira/browse/SPARK-38937 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-38936) Script transform feed thread should have name

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38936: Assignee: (was: Apache Spark) > Script transform feed thread should have name >

[jira] [Assigned] (SPARK-38936) Script transform feed thread should have name

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38936: Assignee: Apache Spark > Script transform feed thread should have name >

[jira] [Commented] (SPARK-38936) Script transform feed thread should have name

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523686#comment-17523686 ] Apache Spark commented on SPARK-38936: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Commented] (SPARK-38693) Spark does not use SessionManager

2022-04-18 Thread Brad Solomon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523685#comment-17523685 ] Brad Solomon commented on SPARK-38693: -- Another argument in favor of supporting is that Spark

[jira] [Comment Edited] (SPARK-38693) Spark does not use SessionManager

2022-04-18 Thread Brad Solomon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523685#comment-17523685 ] Brad Solomon edited comment on SPARK-38693 at 4/18/22 1:00 PM: --- Another

[jira] [Created] (SPARK-38936) Script transform feed thread should have name

2022-04-18 Thread dzcxzl (Jira)
dzcxzl created SPARK-38936: -- Summary: Script transform feed thread should have name Key: SPARK-38936 URL: https://issues.apache.org/jira/browse/SPARK-38936 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

2022-04-18 Thread Nikolay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay updated SPARK-38816: Attachment: (was: image-2022-04-18-15-54-15-474.png) > Wrong comment in random matrix generator in

[jira] [Updated] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

2022-04-18 Thread Nikolay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay updated SPARK-38816: Attachment: image-2022-04-18-15-54-06-679.png > Wrong comment in random matrix generator in spark-als

[jira] [Commented] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

2022-04-18 Thread Nikolay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523684#comment-17523684 ] Nikolay commented on SPARK-38816: - Its not use QR decomposition. From discription we have "The method

[jira] [Updated] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

2022-04-18 Thread Nikolay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay updated SPARK-38816: Attachment: image-2022-04-18-15-54-15-474.png > Wrong comment in random matrix generator in spark-als

[jira] [Updated] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

2022-04-18 Thread Nikolay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay updated SPARK-38816: Attachment: (was: image-2022-04-18-15-54-06-679.png) > Wrong comment in random matrix generator in

[jira] [Commented] (SPARK-38693) Spark does not use SessionManager

2022-04-18 Thread Brad Solomon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523674#comment-17523674 ] Brad Solomon commented on SPARK-38693: -- Keycloak is an open source IAM solution that allows

[jira] [Commented] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

2022-04-18 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523669#comment-17523669 ] Sean R. Owen commented on SPARK-38816: -- I don't think so - ALS doesn't need nonnegative weights. It

[jira] [Assigned] (SPARK-38935) Improve the exception type and message of casting string to numbers

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38935: Assignee: Apache Spark (was: Gengliang Wang) > Improve the exception type and message

[jira] [Assigned] (SPARK-38935) Improve the exception type and message of casting string to numbers

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38935: Assignee: Gengliang Wang (was: Apache Spark) > Improve the exception type and message

[jira] [Assigned] (SPARK-38935) Improve the exception type and message of casting string to numbers

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38935: Assignee: Apache Spark (was: Gengliang Wang) > Improve the exception type and message

[jira] [Commented] (SPARK-38935) Improve the exception type and message of casting string to numbers

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523668#comment-17523668 ] Apache Spark commented on SPARK-38935: -- User 'gengliangwang' has created a pull request for this

[jira] [Updated] (SPARK-38935) Improve the exception type and message of casting string to numbers

2022-04-18 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-38935: --- Description: # change the exception type from "java.lang.NumberFormatException" to 

[jira] [Created] (SPARK-38935) Improve the exception type and message of casting string to numbers

2022-04-18 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-38935: -- Summary: Improve the exception type and message of casting string to numbers Key: SPARK-38935 URL: https://issues.apache.org/jira/browse/SPARK-38935 Project:

[jira] [Updated] (SPARK-38934) Provider TemporaryAWSCredentialsProvider has no credentials

2022-04-18 Thread Lily (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lily updated SPARK-38934: - Priority: Critical (was: Major) > Provider TemporaryAWSCredentialsProvider has no credentials >

[jira] [Updated] (SPARK-38934) Provider TemporaryAWSCredentialsProvider has no credentials

2022-04-18 Thread Lily (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lily updated SPARK-38934: - Priority: Major (was: Critical) > Provider TemporaryAWSCredentialsProvider has no credentials >

[jira] [Comment Edited] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

2022-04-18 Thread Nickolay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523657#comment-17523657 ] Nickolay edited comment on SPARK-38816 at 4/18/22 11:13 AM: Dear, [~srowen].

[jira] [Commented] (SPARK-38816) Wrong comment in random matrix generator in spark-als algorithm

2022-04-18 Thread Nickolay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523657#comment-17523657 ] Nickolay commented on SPARK-38816: -- Dear, [~srowen]. We call this method before choosing a solver.

[jira] [Commented] (SPARK-38904) Low cost DataFrame schema swap util

2022-04-18 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523636#comment-17523636 ] Rafal Wojdyla commented on SPARK-38904: --- [~hyukjin.kwon] ok, will give it a shot, and ping you if

[jira] [Updated] (SPARK-38934) Provider TemporaryAWSCredentialsProvider has no credentials

2022-04-18 Thread Lily (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lily updated SPARK-38934: - Description:   We are using Jupyter Hub on K8s as a notebook based development environment and Spark on K8s

[jira] [Updated] (SPARK-38934) Provider TemporaryAWSCredentialsProvider has no credentials

2022-04-18 Thread Lily (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lily updated SPARK-38934: - Description:   We are using Jupyter Hub on K8s as a notebook based development environment and Spark on K8s

[jira] [Created] (SPARK-38934) Provider TemporaryAWSCredentialsProvider has no credentials

2022-04-18 Thread Lily (Jira)
Lily created SPARK-38934: Summary: Provider TemporaryAWSCredentialsProvider has no credentials Key: SPARK-38934 URL: https://issues.apache.org/jira/browse/SPARK-38934 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-38933) Add examples of window functions into SQL docs

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38933: Assignee: Apache Spark > Add examples of window functions into SQL docs >

[jira] [Assigned] (SPARK-38933) Add examples of window functions into SQL docs

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38933: Assignee: (was: Apache Spark) > Add examples of window functions into SQL docs >

[jira] [Commented] (SPARK-38933) Add examples of window functions into SQL docs

2022-04-18 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523628#comment-17523628 ] Apache Spark commented on SPARK-38933: -- User 'beliefer' has created a pull request for this issue:

  1   2   >