[jira] [Commented] (SPARK-33807) Data Source V2: Remove read specific distributions

2021-08-11 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397875#comment-17397875 ] Anton Okolnychyi commented on SPARK-33807: -- I don't think it should be a blocke

[jira] [Commented] (SPARK-33807) Data Source V2: Remove read specific distributions

2021-08-11 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397876#comment-17397876 ] Anton Okolnychyi commented on SPARK-33807: -- Sorry for the delay commenting! >

[jira] [Updated] (SPARK-36488) "Invalid usage of '*' in expression" error due to the feature of 'quotedRegexColumnNames' in some scenarios.

2021-08-11 Thread merrily01 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] merrily01 updated SPARK-36488: -- Description:  In some cases, the error happens when the following property is set. {code:java} spark.s

[jira] [Updated] (SPARK-36488) "Invalid usage of '*' in expression" error due to the feature of 'quotedRegexColumnNames' in some scenarios.

2021-08-11 Thread merrily01 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] merrily01 updated SPARK-36488: -- Description:   In some cases, the error happens when the following property is set. {code:java} spark

[jira] [Updated] (SPARK-36488) "Invalid usage of '*' in expression" error due to the feature of 'quotedRegexColumnNames' in some scenarios.

2021-08-11 Thread merrily01 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] merrily01 updated SPARK-36488: -- Environment: (was: In some cases, the error happens when the following property is set. {code:java

[jira] [Updated] (SPARK-36488) "Invalid usage of '*' in expression" error due to the feature of 'quotedRegexColumnNames' in some scenarios.

2021-08-11 Thread merrily01 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] merrily01 updated SPARK-36488: -- Description: In some cases, the error happens when the following property is set. {code:java} spark.sq

[jira] [Created] (SPARK-36488) "Invalid usage of '*' in expression" error due to the feature of 'quotedRegexColumnNames' in some scenarios.

2021-08-11 Thread merrily01 (Jira)
merrily01 created SPARK-36488: - Summary: "Invalid usage of '*' in expression" error due to the feature of 'quotedRegexColumnNames' in some scenarios. Key: SPARK-36488 URL: https://issues.apache.org/jira/browse/SPARK-3

[jira] [Commented] (SPARK-36459) Date Value '0001-01-01' changes to '0001-12-30' when inserted into a parquet hive table

2021-08-11 Thread sindhura alluri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397843#comment-17397843 ] sindhura alluri commented on SPARK-36459: - [~hyukjin.kwon],  In our case we mig

[jira] [Assigned] (SPARK-36487) modify exit executor log logic

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36487: Assignee: (was: Apache Spark) > modify exit executor log logic >

[jira] [Commented] (SPARK-36487) modify exit executor log logic

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397841#comment-17397841 ] Apache Spark commented on SPARK-36487: -- User 'zwangsheng' has created a pull reques

[jira] [Commented] (SPARK-36487) modify exit executor log logic

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397840#comment-17397840 ] Apache Spark commented on SPARK-36487: -- User 'zwangsheng' has created a pull reques

[jira] [Assigned] (SPARK-36487) modify exit executor log logic

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36487: Assignee: Apache Spark > modify exit executor log logic > --

[jira] [Created] (SPARK-36487) modify exit executor log logic

2021-08-11 Thread binjie yang (Jira)
binjie yang created SPARK-36487: --- Summary: modify exit executor log logic Key: SPARK-36487 URL: https://issues.apache.org/jira/browse/SPARK-36487 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-36387) Fix Series.astype from datetime to nullable string

2021-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397828#comment-17397828 ] Hyukjin Kwon commented on SPARK-36387: -- If he's inactive, let's go ahead. [~itholic

[jira] [Commented] (SPARK-36387) Fix Series.astype from datetime to nullable string

2021-08-11 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397825#comment-17397825 ] Haejoon Lee commented on SPARK-36387: - [~dc-heros], are you still working on this??

[jira] [Updated] (SPARK-36387) Fix Series.astype from datetime to nullable string

2021-08-11 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-36387: Attachment: image-2021-08-12-14-24-31-321.png > Fix Series.astype from datetime to nullable string

[jira] [Resolved] (SPARK-36479) improve datetime test coverage in SQL files

2021-08-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36479. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33707 [https://gith

[jira] [Assigned] (SPARK-36479) improve datetime test coverage in SQL files

2021-08-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36479: --- Assignee: Wenchen Fan > improve datetime test coverage in SQL files > -

[jira] [Updated] (SPARK-33807) Data Source V2: Remove read specific distributions

2021-08-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-33807: Affects Version/s: (was: 3.2.0) 3.3.0 > Data Source V2: Remove read spe

[jira] [Assigned] (SPARK-36486) Support type constructed string as dat time interval

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36486: Assignee: (was: Apache Spark) > Support type constructed string as dat time interval

[jira] [Assigned] (SPARK-36486) Support type constructed string as dat time interval

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36486: Assignee: Apache Spark > Support type constructed string as dat time interval > -

[jira] [Commented] (SPARK-36486) Support type constructed string as dat time interval

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397804#comment-17397804 ] Apache Spark commented on SPARK-36486: -- User 'AngersZh' has created a pull requ

[jira] [Commented] (SPARK-36485) Support cast type constructed string to year month interval

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1739#comment-1739 ] Apache Spark commented on SPARK-36485: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-36485) Support cast type constructed string to year month interval

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36485: Assignee: (was: Apache Spark) > Support cast type constructed string to year month in

[jira] [Assigned] (SPARK-36485) Support cast type constructed string to year month interval

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36485: Assignee: Apache Spark > Support cast type constructed string to year month interval > --

[jira] [Commented] (SPARK-36485) Support cast type constructed string to year month interval

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397778#comment-17397778 ] Apache Spark commented on SPARK-36485: -- User 'AngersZh' has created a pull requ

[jira] [Created] (SPARK-36486) Support type constructed string as dat time interval

2021-08-11 Thread angerszhu (Jira)
angerszhu created SPARK-36486: - Summary: Support type constructed string as dat time interval Key: SPARK-36486 URL: https://issues.apache.org/jira/browse/SPARK-36486 Project: Spark Issue Type: Su

[jira] [Assigned] (SPARK-36480) SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-36480: --- Assignee: Jungtaek Lim > SessionWindowStateStoreSaveExec should not filter input rows again

[jira] [Commented] (SPARK-36480) SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397775#comment-17397775 ] L. C. Hsieh commented on SPARK-36480: - The issue was resolved at https://github.com/

[jira] [Resolved] (SPARK-36480) SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-36480. - Resolution: Fixed > SessionWindowStateStoreSaveExec should not filter input rows against waterma

[jira] [Updated] (SPARK-36480) SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-36480: Fix Version/s: 3.2.0 > SessionWindowStateStoreSaveExec should not filter input rows against waterm

[jira] [Created] (SPARK-36485) Support cast type constructed string to year month interval

2021-08-11 Thread angerszhu (Jira)
angerszhu created SPARK-36485: - Summary: Support cast type constructed string to year month interval Key: SPARK-36485 URL: https://issues.apache.org/jira/browse/SPARK-36485 Project: Spark Issue

[jira] [Assigned] (SPARK-32986) Add bucket scan info in explain output of FileSourceScanExec

2021-08-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32986: - Assignee: Cheng Su > Add bucket scan info in explain output of FileSourceScanExec > ---

[jira] [Resolved] (SPARK-32986) Add bucket scan info in explain output of FileSourceScanExec

2021-08-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32986. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33698 [https://

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2021-08-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397747#comment-17397747 ] Dongjoon Hyun commented on SPARK-18105: --- I did only the above code I posted becaus

[jira] [Updated] (SPARK-34081) Only pushdown LeftSemi/LeftAnti over Aggregate if join can be planned as broadcast join

2021-08-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-34081: --- Issue Type: Improvement (was: Bug) > Only pushdown LeftSemi/LeftAnti over Aggregate if join

[jira] [Commented] (SPARK-32285) Add PySpark support for nested timestamps with arrow

2021-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397729#comment-17397729 ] Hyukjin Kwon commented on SPARK-32285: -- Arrow version is 2.0.0 now. Can we fix this

[jira] [Commented] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

2021-08-11 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397725#comment-17397725 ] Mridul Muralidharan commented on SPARK-36378: - Thanks for marking this resol

[jira] [Created] (SPARK-36484) Handle Stale block fetch failure on the client side by not retrying the requests

2021-08-11 Thread Venkata krishnan Sowrirajan (Jira)
Venkata krishnan Sowrirajan created SPARK-36484: --- Summary: Handle Stale block fetch failure on the client side by not retrying the requests Key: SPARK-36484 URL: https://issues.apache.org/jira/browse

[jira] [Resolved] (SPARK-36481) Expose LogisticRegression.setInitialModel

2021-08-11 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-36481. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33710 [https://github.com/a

[jira] [Assigned] (SPARK-36483) Fix intermittent test failure due to netty dependency version bump

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36483: Assignee: Apache Spark > Fix intermittent test failure due to netty dependency version bu

[jira] [Assigned] (SPARK-36483) Fix intermittent test failure due to netty dependency version bump

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36483: Assignee: (was: Apache Spark) > Fix intermittent test failure due to netty dependency

[jira] [Commented] (SPARK-36483) Fix intermittent test failure due to netty dependency version bump

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397639#comment-17397639 ] Apache Spark commented on SPARK-36483: -- User 'Victsm' has created a pull request fo

[jira] [Updated] (SPARK-36483) Fix intermittent test failure due to netty dependency version bump

2021-08-11 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Shen updated SPARK-36483: - Description: In SPARK-35132, Spark's netty dependency version was bumped from 4.1.51 to 4.1.63. Since

[jira] [Created] (SPARK-36483) Fix intermittent test failure due to netty dependency version bump

2021-08-11 Thread Min Shen (Jira)
Min Shen created SPARK-36483: Summary: Fix intermittent test failure due to netty dependency version bump Key: SPARK-36483 URL: https://issues.apache.org/jira/browse/SPARK-36483 Project: Spark I

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2021-08-11 Thread Cameron Todd (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397603#comment-17397603 ] Cameron Todd commented on SPARK-18105: -- Good to hear. Also the count of 136,935,074

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2021-08-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397585#comment-17397585 ] Dongjoon Hyun commented on SPARK-18105: --- BTW, I checked that a single zip file con

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2021-08-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397584#comment-17397584 ] Dongjoon Hyun commented on SPARK-18105: --- Thank you, [~cameron.todd]. I downloaded

[jira] [Updated] (SPARK-34827) Support fetching shuffle blocks in batch with i/o encryption

2021-08-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34827: -- Affects Version/s: 3.3.0 > Support fetching shuffle blocks in batch with i/o encryption >

[jira] [Commented] (SPARK-34827) Support fetching shuffle blocks in batch with i/o encryption

2021-08-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397569#comment-17397569 ] Dongjoon Hyun commented on SPARK-34827: --- Thank you. I moved the Target Version to

[jira] [Updated] (SPARK-34827) Support fetching shuffle blocks in batch with i/o encryption

2021-08-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34827: -- Target Version/s: 3.3.0 (was: 3.2.0) > Support fetching shuffle blocks in batch with i/o encr

[jira] [Commented] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397563#comment-17397563 ] Apache Spark commented on SPARK-36378: -- User 'Victsm' has created a pull request fo

[jira] [Resolved] (SPARK-36482) Bump orc to 1.6.10

2021-08-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36482. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33712 [https://

[jira] [Assigned] (SPARK-36482) Bump orc to 1.6.10

2021-08-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36482: - Assignee: William Hyun > Bump orc to 1.6.10 > -- > > Ke

[jira] [Assigned] (SPARK-36482) Bump orc to 1.6.10

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36482: Assignee: Apache Spark > Bump orc to 1.6.10 > -- > > Key:

[jira] [Commented] (SPARK-36482) Bump orc to 1.6.10

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397486#comment-17397486 ] Apache Spark commented on SPARK-36482: -- User 'williamhyun' has created a pull reque

[jira] [Assigned] (SPARK-36482) Bump orc to 1.6.10

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36482: Assignee: (was: Apache Spark) > Bump orc to 1.6.10 > -- > >

[jira] [Created] (SPARK-36482) Bump orc to 1.6.10

2021-08-11 Thread William Hyun (Jira)
William Hyun created SPARK-36482: Summary: Bump orc to 1.6.10 Key: SPARK-36482 URL: https://issues.apache.org/jira/browse/SPARK-36482 Project: Spark Issue Type: Improvement Componen

[jira] [Commented] (SPARK-36446) YARN shuffle server restart crashes all dynamic allocation jobs that have deallocated an executor

2021-08-11 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397461#comment-17397461 ] Thomas Graves commented on SPARK-36446: --- [~adamkennedy77] ^ > YARN shuffle server

[jira] [Assigned] (SPARK-36481) Expose LogisticRegression.setInitialModel

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36481: Assignee: Sean R. Owen (was: Apache Spark) > Expose LogisticRegression.setInitialModel >

[jira] [Assigned] (SPARK-36481) Expose LogisticRegression.setInitialModel

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36481: Assignee: Apache Spark (was: Sean R. Owen) > Expose LogisticRegression.setInitialModel >

[jira] [Commented] (SPARK-36481) Expose LogisticRegression.setInitialModel

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397445#comment-17397445 ] Apache Spark commented on SPARK-36481: -- User 'srowen' has created a pull request fo

[jira] [Created] (SPARK-36481) Expose LogisticRegression.setInitialModel

2021-08-11 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-36481: Summary: Expose LogisticRegression.setInitialModel Key: SPARK-36481 URL: https://issues.apache.org/jira/browse/SPARK-36481 Project: Spark Issue Type: Improve

[jira] [Updated] (SPARK-36360) Delete appName from StreamingSource

2021-08-11 Thread Marcel Neumann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Neumann updated SPARK-36360: --- Description: The StreamingSource includes the appName in its sourceName. However, the appNa

[jira] [Updated] (SPARK-36360) Delete appName from StreamingSource

2021-08-11 Thread Marcel Neumann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Neumann updated SPARK-36360: --- Description: The StreamingSource includes the appName in its sourceName. The appName should

[jira] [Updated] (SPARK-36478) Removes outer join if all grouping and aggregate expressions are from the streamed side

2021-08-11 Thread Wan Kun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-36478: Description: Removes outer join if all grouping and aggregate expressions are from the streamed side. Fo

[jira] [Updated] (SPARK-36360) Delete appName from StreamingSource

2021-08-11 Thread Marcel Neumann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Neumann updated SPARK-36360: --- Description: The StreamingSource includes the appName in its sourceName. This is not desired

[jira] [Updated] (SPARK-36360) Delete appName from StreamingSource

2021-08-11 Thread Marcel Neumann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Neumann updated SPARK-36360: --- Summary: Delete appName from StreamingSource (was: StreamingSource duplicates appName) >

[jira] [Updated] (SPARK-36360) StreamingSource duplicates appName

2021-08-11 Thread Marcel Neumann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Neumann updated SPARK-36360: --- Description: The StreamingSource includes the appName in its sourceName. This is not desired

[jira] [Updated] (SPARK-36360) StreamingSource duplicates appName

2021-08-11 Thread Marcel Neumann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Neumann updated SPARK-36360: --- Description: The StreamingSource includes the appName in its sourceName. This is not desired

[jira] [Updated] (SPARK-36360) StreamingSource duplicates appName

2021-08-11 Thread Marcel Neumann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Neumann updated SPARK-36360: --- Priority: Blocker (was: Trivial) > StreamingSource duplicates appName > ---

[jira] [Updated] (SPARK-36360) StreamingSource duplicates appName

2021-08-11 Thread Marcel Neumann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Neumann updated SPARK-36360: --- Priority: Trivial (was: Minor) > StreamingSource duplicates appName > -

[jira] [Updated] (SPARK-36478) Removes outer join if all grouping and aggregate expressions are from the streamed side

2021-08-11 Thread Wan Kun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-36478: Description: Removes outer join if all grouping and aggregate expressions are from the streamed side. Fo

[jira] [Assigned] (SPARK-35030) ANSI SQL compliance

2021-08-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-35030: -- Assignee: Apache Spark > ANSI SQL compliance > --- > >

[jira] [Assigned] (SPARK-36418) Use CAST in parsing of dates/timestamps with default pattern

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36418: Assignee: (was: Apache Spark) > Use CAST in parsing of dates/timestamps with default

[jira] [Assigned] (SPARK-36418) Use CAST in parsing of dates/timestamps with default pattern

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36418: Assignee: Apache Spark > Use CAST in parsing of dates/timestamps with default pattern > -

[jira] [Commented] (SPARK-36418) Use CAST in parsing of dates/timestamps with default pattern

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397376#comment-17397376 ] Apache Spark commented on SPARK-36418: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-36474) Mention pandas API on Spark in Spark overview pages

2021-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36474: Assignee: Hyukjin Kwon > Mention pandas API on Spark in Spark overview pages > --

[jira] [Resolved] (SPARK-36474) Mention pandas API on Spark in Spark overview pages

2021-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36474. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33699 [https://gi

[jira] [Resolved] (SPARK-36378) Minor changes to address a few identified server side inefficiencies

2021-08-11 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-36378. -- Fix Version/s: 3.3.0 3.2.0 Assignee: Min Shen Resolution: Fixed Issue resol

[jira] [Commented] (SPARK-36480) SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397336#comment-17397336 ] Apache Spark commented on SPARK-36480: -- User 'HeartSaVioR' has created a pull reque

[jira] [Assigned] (SPARK-36480) SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36480: Assignee: Apache Spark > SessionWindowStateStoreSaveExec should not filter input rows aga

[jira] [Assigned] (SPARK-36480) SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36480: Assignee: (was: Apache Spark) > SessionWindowStateStoreSaveExec should not filter inp

[jira] [Commented] (SPARK-36480) SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397334#comment-17397334 ] Apache Spark commented on SPARK-36480: -- User 'HeartSaVioR' has created a pull reque

[jira] [Created] (SPARK-36480) SessionWindowStateStoreSaveExec should not filter input rows against watermark

2021-08-11 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-36480: Summary: SessionWindowStateStoreSaveExec should not filter input rows against watermark Key: SPARK-36480 URL: https://issues.apache.org/jira/browse/SPARK-36480 Projec

[jira] [Resolved] (SPARK-36132) Support initial state for flatMapGroupsWithState in batch mode

2021-08-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-36132. Resolution: Done This issue is resolved in https://github.com/apache/spark/pull/6 > S

[jira] [Assigned] (SPARK-36132) Support initial state for flatMapGroupsWithState in batch mode

2021-08-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-36132: -- Assignee: Rahul Shivu Mahadev > Support initial state for flatMapGroupsWithState in b

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2021-08-11 Thread Cameron Todd (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397301#comment-17397301 ] Cameron Todd commented on SPARK-18105: -- Ok I added the zip file on this public S3 b

[jira] [Updated] (SPARK-36413) Spark and Hive are inconsistent in inferring the expression type in the view, resulting in AnalysisException

2021-08-11 Thread bingfeng.guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bingfeng.guo updated SPARK-36413: - Priority: Blocker (was: Critical) > Spark and Hive are inconsistent in inferring the expression

[jira] [Commented] (SPARK-36478) Removes outer join if all grouping and aggregate expressions are from the streamed side

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397279#comment-17397279 ] Apache Spark commented on SPARK-36478: -- User 'wankunde' has created a pull request

[jira] [Commented] (SPARK-36479) improve datetime test coverage in SQL files

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397278#comment-17397278 ] Apache Spark commented on SPARK-36479: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-36479) improve datetime test coverage in SQL files

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36479: Assignee: Apache Spark > improve datetime test coverage in SQL files > --

[jira] [Commented] (SPARK-36478) Removes outer join if all grouping and aggregate expressions are from the streamed side

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17397276#comment-17397276 ] Apache Spark commented on SPARK-36478: -- User 'wankunde' has created a pull request

[jira] [Assigned] (SPARK-36478) Removes outer join if all grouping and aggregate expressions are from the streamed side

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36478: Assignee: Apache Spark > Removes outer join if all grouping and aggregate expressions are

[jira] [Assigned] (SPARK-36479) improve datetime test coverage in SQL files

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36479: Assignee: (was: Apache Spark) > improve datetime test coverage in SQL files > ---

[jira] [Assigned] (SPARK-36478) Removes outer join if all grouping and aggregate expressions are from the streamed side

2021-08-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36478: Assignee: (was: Apache Spark) > Removes outer join if all grouping and aggregate expr

[jira] [Created] (SPARK-36479) improve datetime test coverage in SQL files

2021-08-11 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-36479: --- Summary: improve datetime test coverage in SQL files Key: SPARK-36479 URL: https://issues.apache.org/jira/browse/SPARK-36479 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-36478) Removes outer join if all grouping and aggregate expressions are from the streamed side

2021-08-11 Thread Wan Kun (Jira)
Wan Kun created SPARK-36478: --- Summary: Removes outer join if all grouping and aggregate expressions are from the streamed side Key: SPARK-36478 URL: https://issues.apache.org/jira/browse/SPARK-36478 Project

[jira] [Resolved] (SPARK-36468) Update docs about ANSI interval literals

2021-08-11 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36468. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33693 [https://github.com

[jira] [Updated] (SPARK-36472) Improve SQL syntax for MERGE

2021-08-11 Thread Denis Krivenko (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Krivenko updated SPARK-36472: --- Description: Existing SQL syntax for *MEGRE* (see Delta Lake examples [here|https://docs.de

  1   2   >