[jira] [Assigned] (SPARK-33904) Recognize `spark_catalog` in `saveAsTable()` and `insertInto()`

2020-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33904: --- Assignee: Maxim Gekk > Recognize `spark_catalog` in `saveAsTable()` and `insertInto()` >

[jira] [Resolved] (SPARK-33904) Recognize `spark_catalog` in `saveAsTable()` and `insertInto()`

2020-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33904. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30919

[jira] [Resolved] (SPARK-33926) Improve the error message in resolving of DSv1 multi-part identifiers

2020-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33926. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30963

[jira] [Assigned] (SPARK-33926) Improve the error message in resolving of DSv1 multi-part identifiers

2020-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33926: --- Assignee: Maxim Gekk > Improve the error message in resolving of DSv1 multi-part

[jira] [Commented] (SPARK-33940) allow configuring the max column name length in csv writer

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256357#comment-17256357 ] Apache Spark commented on SPARK-33940: -- User 'CodingCat' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33940) allow configuring the max column name length in csv writer

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33940: Assignee: (was: Apache Spark) > allow configuring the max column name length in csv

[jira] [Assigned] (SPARK-33940) allow configuring the max column name length in csv writer

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33940: Assignee: Apache Spark > allow configuring the max column name length in csv writer >

[jira] [Created] (SPARK-33940) allow configuring the max column name length in csv writer

2020-12-29 Thread Nan Zhu (Jira)
Nan Zhu created SPARK-33940: --- Summary: allow configuring the max column name length in csv writer Key: SPARK-33940 URL: https://issues.apache.org/jira/browse/SPARK-33940 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-33927) Fix Spark Release image

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33927: Assignee: Hyukjin Kwon > Fix Spark Release image > --- > >

[jira] [Resolved] (SPARK-33927) Fix Spark Release image

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33927. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30971

[jira] [Commented] (SPARK-33907) Only prune columns of from_json if parsing options is empty

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256336#comment-17256336 ] Apache Spark commented on SPARK-33907: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33927) Fix Spark Release image

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33927: Assignee: (was: Apache Spark) > Fix Spark Release image > --- >

[jira] [Assigned] (SPARK-33927) Fix Spark Release image

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33927: Assignee: Apache Spark > Fix Spark Release image > --- > >

[jira] [Commented] (SPARK-33927) Fix Spark Release image

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256335#comment-17256335 ] Apache Spark commented on SPARK-33927: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-33890) Improve the implement of trim/trimleft/trimright

2020-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33890. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30905

[jira] [Assigned] (SPARK-33890) Improve the implement of trim/trimleft/trimright

2020-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33890: --- Assignee: jiaan.geng > Improve the implement of trim/trimleft/trimright >

[jira] [Assigned] (SPARK-32684) Add a test case for hive serde/default-serde mode's null value '\\N'

2020-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32684: --- Assignee: angerszhu > Add a test case for hive serde/default-serde mode's null value '\\N'

[jira] [Resolved] (SPARK-32684) Add a test case for hive serde/default-serde mode's null value '\\N'

2020-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32684. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30946

[jira] [Resolved] (SPARK-33874) Spark may report PodRunning if there is a sidecar that has not exited

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33874. -- Fix Version/s: 3.1.0 Resolution: Fixed Fixed in

[jira] [Commented] (SPARK-33938) Optimize Like Any/All by LikeSimplification

2020-12-29 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256285#comment-17256285 ] jiaan.geng commented on SPARK-33938: I'm working on. > Optimize Like Any/All by LikeSimplification

[jira] [Assigned] (SPARK-33932) Clean up KafkaOffsetReader API document

2020-12-29 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-33932: --- Assignee: L. C. Hsieh (was: Apache Spark) > Clean up KafkaOffsetReader API document >

[jira] [Commented] (SPARK-31946) Failed to register SIGPWR handler on MacOS

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256249#comment-17256249 ] Apache Spark commented on SPARK-31946: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31946) Failed to register SIGPWR handler on MacOS

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31946: Assignee: Apache Spark > Failed to register SIGPWR handler on MacOS >

[jira] [Assigned] (SPARK-31946) Failed to register SIGPWR handler on MacOS

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31946: Assignee: (was: Apache Spark) > Failed to register SIGPWR handler on MacOS >

[jira] [Created] (SPARK-33939) Do not display the seed of uuid/shuffle if user not specify

2020-12-29 Thread ulysses you (Jira)
ulysses you created SPARK-33939: --- Summary: Do not display the seed of uuid/shuffle if user not specify Key: SPARK-33939 URL: https://issues.apache.org/jira/browse/SPARK-33939 Project: Spark

[jira] [Created] (SPARK-33938) Optimize Like Any/All by LikeSimplification

2020-12-29 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33938: --- Summary: Optimize Like Any/All by LikeSimplification Key: SPARK-33938 URL: https://issues.apache.org/jira/browse/SPARK-33938 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33934) Support automatically identify the Python file and execute it

2020-12-29 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256228#comment-17256228 ] angerszhu commented on SPARK-33934: --- raise a pr soon > Support automatically identify the Python file

[jira] [Resolved] (SPARK-33932) Clean up KafkaOffsetReader API document

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33932. -- Fix Version/s: 3.2.0 Resolution: Fixed Fixed in

[jira] [Commented] (SPARK-33937) Move the old partition data to trash instead of deleting it when inserting rewrite hive table

2020-12-29 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-33937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256196#comment-17256196 ] 黄海升 commented on SPARK-33937: - I'm sorry. It looks like it is. > Move the old partition data to trash

[jira] [Resolved] (SPARK-33937) Move the old partition data to trash instead of deleting it when inserting rewrite hive table

2020-12-29 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-33937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 黄海升 resolved SPARK-33937. - Resolution: Duplicate > Move the old partition data to trash instead of deleting it when inserting > rewrite

[jira] [Commented] (SPARK-33937) Move the old partition data to trash instead of deleting it when inserting rewrite hive table

2020-12-29 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256192#comment-17256192 ] Chao Sun commented on SPARK-33937: -- This looks like a duplicate of SPARK-32480. > Move the old

[jira] [Created] (SPARK-33937) Move the old partition data to trash instead of deleting it when inserting rewrite hive table

2020-12-29 Thread Jira
黄海升 created SPARK-33937: --- Summary: Move the old partition data to trash instead of deleting it when inserting rewrite hive table Key: SPARK-33937 URL: https://issues.apache.org/jira/browse/SPARK-33937 Project:

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

2020-12-29 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256046#comment-17256046 ] David Wyles edited comment on SPARK-33635 at 12/29/20, 8:48 PM:

[jira] [Updated] (SPARK-33936) Add the version when connector methods and interfaces were added

2020-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33936: -- Fix Version/s: (was: 3.2.0) 3.1.0 > Add the version when connector

[jira] [Updated] (SPARK-33936) Add the version when connector methods and interfaces were added

2020-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33936: -- Fix Version/s: 3.2.0 > Add the version when connector methods and interfaces were added >

[jira] [Resolved] (SPARK-33936) Add the version when connector methods and interfaces were added

2020-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33936. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30966

[jira] [Assigned] (SPARK-33936) Add the version when connector methods and interfaces were added

2020-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33936: - Assignee: Maxim Gekk > Add the version when connector methods and interfaces were

[jira] [Commented] (SPARK-33936) Add the version when connector methods and interfaces were added

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256116#comment-17256116 ] Apache Spark commented on SPARK-33936: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-33915) Allow json expression to be pushable column

2020-12-29 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255389#comment-17255389 ] Ted Yu edited comment on SPARK-33915 at 12/29/20, 6:51 PM: --- Here is the plan

[jira] [Assigned] (SPARK-33936) Add the version when connector methods and interfaces were added

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33936: Assignee: (was: Apache Spark) > Add the version when connector methods and

[jira] [Assigned] (SPARK-33936) Add the version when connector methods and interfaces were added

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33936: Assignee: Apache Spark > Add the version when connector methods and interfaces were

[jira] [Commented] (SPARK-33936) Add the version when connector methods and interfaces were added

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256107#comment-17256107 ] Apache Spark commented on SPARK-33936: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Created] (SPARK-33936) Add the version when connector methods and interfaces were added

2020-12-29 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33936: -- Summary: Add the version when connector methods and interfaces were added Key: SPARK-33936 URL: https://issues.apache.org/jira/browse/SPARK-33936 Project: Spark

[jira] [Commented] (SPARK-33935) Fix CBOs cost function

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256103#comment-17256103 ] Apache Spark commented on SPARK-33935: -- User 'tanelk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33935) Fix CBOs cost function

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256101#comment-17256101 ] Apache Spark commented on SPARK-33935: -- User 'tanelk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33935) Fix CBOs cost function

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33935: Assignee: (was: Apache Spark) > Fix CBOs cost function > --- >

[jira] [Assigned] (SPARK-33935) Fix CBOs cost function

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33935: Assignee: Apache Spark > Fix CBOs cost function > --- > >

[jira] [Updated] (SPARK-33935) Fix CBOs cost function

2020-12-29 Thread Tanel Kiis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-33935: --- Issue Type: Bug (was: Improvement) > Fix CBOs cost function > --- > >

[jira] [Created] (SPARK-33935) Fix CBOs cost function

2020-12-29 Thread Tanel Kiis (Jira)
Tanel Kiis created SPARK-33935: -- Summary: Fix CBOs cost function Key: SPARK-33935 URL: https://issues.apache.org/jira/browse/SPARK-33935 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

2020-12-29 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256046#comment-17256046 ] David Wyles edited comment on SPARK-33635 at 12/29/20, 4:54 PM:

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

2020-12-29 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256046#comment-17256046 ] David Wyles edited comment on SPARK-33635 at 12/29/20, 4:34 PM:

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

2020-12-29 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256046#comment-17256046 ] David Wyles commented on SPARK-33635: - [~gsomogyi] I now have my results. I was so unhappy about

[jira] [Resolved] (SPARK-33910) Simplify/Optimize conditional expressions

2020-12-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33910. - Fix Version/s: 3.2.0 Resolution: Fixed > Simplify/Optimize conditional expressions >

[jira] [Assigned] (SPARK-33910) Simplify/Optimize conditional expressions

2020-12-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-33910: --- Assignee: Yuming Wang > Simplify/Optimize conditional expressions >

[jira] [Updated] (SPARK-33934) Support automatically identify the Python file and execute it

2020-12-29 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-33934: -- Parent: SPARK-31936 Issue Type: Sub-task (was: Improvement) > Support automatically identify

[jira] [Updated] (SPARK-33934) Support automatically identify the Python file and execute it

2020-12-29 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-33934: -- Description: In Hive script transform, we can use `USING xxx.py` but in Spark we will got error 

[jira] [Created] (SPARK-33934) Support automatically identify the Python file and execute it

2020-12-29 Thread angerszhu (Jira)
angerszhu created SPARK-33934: - Summary: Support automatically identify the Python file and execute it Key: SPARK-33934 URL: https://issues.apache.org/jira/browse/SPARK-33934 Project: Spark

[jira] [Assigned] (SPARK-32684) Add a test case for hive serde/default-serde mode's null value '\\N'

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32684: Assignee: (was: Apache Spark) > Add a test case for hive serde/default-serde mode's

[jira] [Assigned] (SPARK-32684) Add a test case for hive serde/default-serde mode's null value '\\N'

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32684: Assignee: Apache Spark > Add a test case for hive serde/default-serde mode's null value

[jira] [Updated] (SPARK-32684) Add a test case for hive serde/default-serde mode's null value '\\N'

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32684: - Priority: Minor (was: Major) > Add a test case for hive serde/default-serde mode's null value

[jira] [Updated] (SPARK-32684) Add a test case for hive serde/default-serde mode's null value '\\N'

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32684: - Summary: Add a test case for hive serde/default-serde mode's null value '\\N' (was: Scrip

[jira] [Commented] (SPARK-33859) Support V2 ALTER TABLE .. RENAME PARTITION

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256015#comment-17256015 ] Apache Spark commented on SPARK-33859: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33859) Support V2 ALTER TABLE .. RENAME PARTITION

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256014#comment-17256014 ] Apache Spark commented on SPARK-33859: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Updated] (SPARK-32684) Add a test case for hive serde/default-serde mode's null value '\\N'

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32684: - Affects Version/s: (was: 3.0.0) 3.2.0 > Add a test case for hive

[jira] [Reopened] (SPARK-32684) Scrip transform hive serde/default-serde mode null value keep same with hive as '\\N'

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-32684: -- > Scrip transform hive serde/default-serde mode null value keep same with hive > as '\\N' >

[jira] [Resolved] (SPARK-33930) Spark SQL no serde row format field delimit default is '\u0001'

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33930. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30958

[jira] [Assigned] (SPARK-33930) Spark SQL no serde row format field delimit default is '\u0001'

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33930: Assignee: angerszhu > Spark SQL no serde row format field delimit default is '\u0001' >

[jira] [Resolved] (SPARK-33909) Check rand functions seed is legal at analyer side

2020-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33909. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30923

[jira] [Assigned] (SPARK-33909) Check rand functions seed is legal at analyer side

2020-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33909: --- Assignee: ulysses you > Check rand functions seed is legal at analyer side >

[jira] [Assigned] (SPARK-33859) Support V2 ALTER TABLE .. RENAME PARTITION

2020-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33859: --- Assignee: Maxim Gekk > Support V2 ALTER TABLE .. RENAME PARTITION >

[jira] [Resolved] (SPARK-33859) Support V2 ALTER TABLE .. RENAME PARTITION

2020-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33859. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30935

[jira] [Commented] (SPARK-33871) Cannot access to column after left semi join and left join

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255978#comment-17255978 ] Hyukjin Kwon commented on SPARK-33871: -- +1 for [~viirya]'s advice here. > Cannot access to column

[jira] [Commented] (SPARK-33927) Fix Spark Release image

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255977#comment-17255977 ] Hyukjin Kwon commented on SPARK-33927: -- Thanks for letting me know [~dongjoon]. I will likely have

[jira] [Updated] (SPARK-33929) Spark-submit with --package deequ doesn't pull all jars

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33929: - Affects Version/s: (was: 2.3.4) (was: 2.3.3)

[jira] [Updated] (SPARK-33929) Spark-submit with --package deequ doesn't pull all jars

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33929: - Target Version/s: (was: 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4) > Spark-submit with --package

[jira] [Resolved] (SPARK-32968) Column pruning for CsvToStructs

2020-12-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32968. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30912

[jira] [Commented] (SPARK-33926) Improve the error message in resolving of DSv1 multi-part identifiers

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255953#comment-17255953 ] Apache Spark commented on SPARK-33926: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33933) Broadcast timeout happened unexpectedly in AQE

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255904#comment-17255904 ] Apache Spark commented on SPARK-33933: -- User 'zhongyu09' has created a pull request for this issue:

[jira] [Commented] (SPARK-33933) Broadcast timeout happened unexpectedly in AQE

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255903#comment-17255903 ] Apache Spark commented on SPARK-33933: -- User 'zhongyu09' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33933) Broadcast timeout happened unexpectedly in AQE

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33933: Assignee: (was: Apache Spark) > Broadcast timeout happened unexpectedly in AQE >

[jira] [Assigned] (SPARK-33933) Broadcast timeout happened unexpectedly in AQE

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33933: Assignee: Apache Spark > Broadcast timeout happened unexpectedly in AQE >

[jira] [Updated] (SPARK-31936) Implement ScriptTransform in sql/core

2020-12-29 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-31936: -- Affects Version/s: 3.2.0 3.1.0 > Implement ScriptTransform in sql/core >

[jira] [Updated] (SPARK-33933) Broadcast timeout happened unexpectedly in AQE

2020-12-29 Thread Yu Zhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Zhong updated SPARK-33933: - Description: In Spark 3.0, when AQE is enabled, there is often broadcast timeout in normal queries as

[jira] [Created] (SPARK-33933) Broadcast timeout happened unexpectedly in AQE

2020-12-29 Thread Yu Zhong (Jira)
Yu Zhong created SPARK-33933: Summary: Broadcast timeout happened unexpectedly in AQE Key: SPARK-33933 URL: https://issues.apache.org/jira/browse/SPARK-33933 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-33932) Clean up KafkaOffsetReader API document

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33932: Assignee: Apache Spark (was: L. C. Hsieh) > Clean up KafkaOffsetReader API document >

[jira] [Assigned] (SPARK-33932) Clean up KafkaOffsetReader API document

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33932: Assignee: Apache Spark (was: L. C. Hsieh) > Clean up KafkaOffsetReader API document >

[jira] [Assigned] (SPARK-33932) Clean up KafkaOffsetReader API document

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33932: Assignee: L. C. Hsieh (was: Apache Spark) > Clean up KafkaOffsetReader API document >

[jira] [Commented] (SPARK-33932) Clean up KafkaOffsetReader API document

2020-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255862#comment-17255862 ] Apache Spark commented on SPARK-33932: -- User 'viirya' has created a pull request for this issue:

[jira] [Created] (SPARK-33932) Clean up KafkaOffsetReader API document

2020-12-29 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-33932: --- Summary: Clean up KafkaOffsetReader API document Key: SPARK-33932 URL: https://issues.apache.org/jira/browse/SPARK-33932 Project: Spark Issue Type: