[jira] [Commented] (SPARK-32481) Support truncate table to move the data to trash

[ https://issues.apache.org/jira/browse/SPARK-32481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173757#comment-17173757 ] L. C. Hsieh commented on SPARK-32481: - Why this needs to be subtask of SPARK-32480.

[jira] [Resolved] (SPARK-32563) spark-sql doesn't support insert into mixed static & dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32563. -- Resolution: Not A Problem > spark-sql doesn't support insert into mixed static & dynam

[jira] [Commented] (SPARK-32563) spark-sql doesn't support insert into mixed static & dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173752#comment-17173752 ] Takeshi Yamamuro commented on SPARK-32563: -- Yea, I think so. Then I will close

[jira] [Commented] (SPARK-32563) spark-sql doesn't support insert into mixed static & dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173751#comment-17173751 ] L. C. Hsieh commented on SPARK-32563: - So it is not an issue anymore in 2.4, 3.0 bra

[jira] [Commented] (SPARK-32571) yarnClient.killApplication(appId) is never called

[ https://issues.apache.org/jira/browse/SPARK-32571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173750#comment-17173750 ] L. C. Hsieh commented on SPARK-32571: - I think by design in cluster mode the spark a

[jira] [Comment Edited] (SPARK-32563) spark-sql doesn't support insert into mixed static & dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173746#comment-17173746 ] yx91490 edited comment on SPARK-32563 at 8/9/20, 5:17 AM: -- I tr

[jira] [Commented] (SPARK-32563) spark-sql doesn't support insert into mixed static & dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173746#comment-17173746 ] yx91490 commented on SPARK-32563: - I tried in apache-spark-3.0 and cannot reproduce it.

[jira] [Updated] (SPARK-32462) Reset previous search text for datatable

[ https://issues.apache.org/jira/browse/SPARK-32462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32462: -- Summary: Reset previous search text for datatable (was: Don't save the previous search text f

[jira] [Resolved] (SPARK-32462) Don't save the previous search text for datatable

[ https://issues.apache.org/jira/browse/SPARK-32462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32462. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29265 [https://

[jira] [Commented] (SPARK-32502) Please fix CVE related to Guava 14.0.1

[ https://issues.apache.org/jira/browse/SPARK-32502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173744#comment-17173744 ] L. C. Hsieh commented on SPARK-32502: - Currently I'm working on some changes at Hive

[jira] [Commented] (SPARK-32536) deleted not existing hdfs locations when use spark sql to execute "insert overwrite" statement to dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173728#comment-17173728 ] yx91490 commented on SPARK-32536: - I found that I mistake the issue condition, I reprodu

[jira] [Comment Edited] (SPARK-32536) deleted not existing hdfs locations when use spark sql to execute "insert overwrite" statement to dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17172889#comment-17172889 ] yx91490 edited comment on SPARK-32536 at 8/9/20, 2:55 AM: -- the

[jira] [Commented] (SPARK-32559) Fix the trim logic in UTF8String.toInt/toLong did't handle Chinese characters correctly

[ https://issues.apache.org/jira/browse/SPARK-32559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173723#comment-17173723 ] Apache Spark commented on SPARK-32559: -- User 'WangGuangxin' has created a pull requ

[jira] [Commented] (SPARK-31703) Changes made by SPARK-26985 break reading parquet files correctly in BigEndian architectures (AIX + LinuxPPC64)

[ https://issues.apache.org/jira/browse/SPARK-31703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173721#comment-17173721 ] Dongjoon Hyun commented on SPARK-31703: --- I raised the priority to `Blocker` with `

[jira] [Updated] (SPARK-31703) Changes made by SPARK-26985 break reading parquet files correctly in BigEndian architectures (AIX + LinuxPPC64)

[ https://issues.apache.org/jira/browse/SPARK-31703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31703: -- Target Version/s: 2.4.7, 3.0.1 > Changes made by SPARK-26985 break reading parquet files corre

[jira] [Updated] (SPARK-31703) Changes made by SPARK-26985 break reading parquet files correctly in BigEndian architectures (AIX + LinuxPPC64)

[ https://issues.apache.org/jira/browse/SPARK-31703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31703: -- Priority: Blocker (was: Critical) > Changes made by SPARK-26985 break reading parquet files c

[jira] [Updated] (SPARK-32432) Add support for reading ORC/Parquet files with SymlinkTextInputFormat

[ https://issues.apache.org/jira/browse/SPARK-32432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32432: -- Affects Version/s: (was: 3.0.0) 3.1.0 > Add support for reading ORC

[jira] [Assigned] (SPARK-32555) Add unique ID on QueryExecution to enable listeners to deduplicate

[ https://issues.apache.org/jira/browse/SPARK-32555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32555: - Assignee: Jungtaek Lim > Add unique ID on QueryExecution to enable listeners to dedupli

[jira] [Resolved] (SPARK-32555) Add unique ID on QueryExecution to enable listeners to deduplicate

[ https://issues.apache.org/jira/browse/SPARK-32555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32555. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29372 [https://

[jira] [Commented] (SPARK-32558) ORC target files that Spark_3.0 produces does not work with Hive_2.1.1 (work-around of using spark.sql.orc.impl=hive is also not working)

[ https://issues.apache.org/jira/browse/SPARK-32558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173710#comment-17173710 ] Dongjoon Hyun commented on SPARK-32558: --- [~ramks]. Please see HIVE-16683. You are

[jira] [Comment Edited] (SPARK-32558) ORC target files that Spark_3.0 produces does not work with Hive_2.1.1 (work-around of using spark.sql.orc.impl=hive is also not working)

[ https://issues.apache.org/jira/browse/SPARK-32558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173710#comment-17173710 ] Dongjoon Hyun edited comment on SPARK-32558 at 8/8/20, 9:09 PM: --

[jira] [Resolved] (SPARK-32558) ORC target files that Spark_3.0 produces does not work with Hive_2.1.1 (work-around of using spark.sql.orc.impl=hive is also not working)

[ https://issues.apache.org/jira/browse/SPARK-32558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32558. --- Resolution: Invalid > ORC target files that Spark_3.0 produces does not work with Hive_2.1.1

[jira] [Commented] (SPARK-32558) ORC target files that Spark_3.0 produces does not work with Hive_2.1.1 (work-around of using spark.sql.orc.impl=hive is also not working)

[ https://issues.apache.org/jira/browse/SPARK-32558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173709#comment-17173709 ] Dongjoon Hyun commented on SPARK-32558: --- Hi, [~ramks]. Thank you for reporting. F

[jira] [Commented] (SPARK-32570) Thriftserver LDAP failed

[ https://issues.apache.org/jira/browse/SPARK-32570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173708#comment-17173708 ] Rohit Mishra commented on SPARK-32570: -- Thanks, [~maropu]. If I may add one more po

[jira] [Updated] (SPARK-32570) Thriftserver LDAP failed

[ https://issues.apache.org/jira/browse/SPARK-32570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32570: - Priority: Major (was: Critical) > Thriftserver LDAP failed > > >

[jira] [Updated] (SPARK-32571) yarnClient.killApplication(appId) is never called

[ https://issues.apache.org/jira/browse/SPARK-32571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32571: -- Priority: Major (was: Critical) > yarnClient.killApplication(appId) is never called > ---

[jira] [Updated] (SPARK-32564) Inject data statistics to simulate plan generation on actual TPCDS data

[ https://issues.apache.org/jira/browse/SPARK-32564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32564: -- Fix Version/s: 3.0.1 > Inject data statistics to simulate plan generation on actual TPCDS dat

[jira] [Resolved] (SPARK-32319) Disallow the use of unused imports

[ https://issues.apache.org/jira/browse/SPARK-32319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32319. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29121 [https://

[jira] [Assigned] (SPARK-32319) Disallow the use of unused imports

[ https://issues.apache.org/jira/browse/SPARK-32319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32319: - Assignee: Fokko Driesprong > Disallow the use of unused imports > -

[jira] [Commented] (SPARK-32574) Race condition in FsHistoryProvider listing iteration

[ https://issues.apache.org/jira/browse/SPARK-32574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173647#comment-17173647 ] Apache Spark commented on SPARK-32574: -- User 'yanxiaole' has created a pull request

[jira] [Assigned] (SPARK-32574) Race condition in FsHistoryProvider listing iteration

[ https://issues.apache.org/jira/browse/SPARK-32574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32574: Assignee: (was: Apache Spark) > Race condition in FsHistoryProvider listing iteration

[jira] [Assigned] (SPARK-32574) Race condition in FsHistoryProvider listing iteration

[ https://issues.apache.org/jira/browse/SPARK-32574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32574: Assignee: Apache Spark > Race condition in FsHistoryProvider listing iteration >

[jira] [Commented] (SPARK-32574) Race condition in FsHistoryProvider listing iteration

[ https://issues.apache.org/jira/browse/SPARK-32574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173646#comment-17173646 ] Apache Spark commented on SPARK-32574: -- User 'yanxiaole' has created a pull request

[jira] [Commented] (SPARK-32564) Inject data statistics to simulate plan generation on actual TPCDS data

[ https://issues.apache.org/jira/browse/SPARK-32564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173639#comment-17173639 ] Apache Spark commented on SPARK-32564: -- User 'maropu' has created a pull request fo

[jira] [Commented] (SPARK-32564) Inject data statistics to simulate plan generation on actual TPCDS data

[ https://issues.apache.org/jira/browse/SPARK-32564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173638#comment-17173638 ] Apache Spark commented on SPARK-32564: -- User 'maropu' has created a pull request fo

[jira] [Created] (SPARK-32574) Race condition in FsHistoryProvider listing iteration

Yan Xiaole created SPARK-32574: -- Summary: Race condition in FsHistoryProvider listing iteration Key: SPARK-32574 URL: https://issues.apache.org/jira/browse/SPARK-32574 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32564) Inject data statistics to simulate plan generation on actual TPCDS data

[ https://issues.apache.org/jira/browse/SPARK-32564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173635#comment-17173635 ] Apache Spark commented on SPARK-32564: -- User 'maropu' has created a pull request fo

[jira] [Commented] (SPARK-32564) Inject data statistics to simulate plan generation on actual TPCDS data

[ https://issues.apache.org/jira/browse/SPARK-32564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173634#comment-17173634 ] Apache Spark commented on SPARK-32564: -- User 'maropu' has created a pull request fo

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173633#comment-17173633 ] JinxinTang commented on SPARK-32500: I have found the root cause is `org.apache.spar

[jira] [Commented] (SPARK-32573) Eliminate Anti Join when BuildSide is Empty

[ https://issues.apache.org/jira/browse/SPARK-32573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173624#comment-17173624 ] Apache Spark commented on SPARK-32573: -- User 'leanken' has created a pull request f

[jira] [Assigned] (SPARK-32573) Eliminate Anti Join when BuildSide is Empty

[ https://issues.apache.org/jira/browse/SPARK-32573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32573: Assignee: (was: Apache Spark) > Eliminate Anti Join when BuildSide is Empty > ---

[jira] [Commented] (SPARK-32573) Eliminate Anti Join when BuildSide is Empty

[ https://issues.apache.org/jira/browse/SPARK-32573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173623#comment-17173623 ] Apache Spark commented on SPARK-32573: -- User 'leanken' has created a pull request f

[jira] [Assigned] (SPARK-32573) Eliminate Anti Join when BuildSide is Empty

[ https://issues.apache.org/jira/browse/SPARK-32573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32573: Assignee: Apache Spark > Eliminate Anti Join when BuildSide is Empty > --

[jira] [Created] (SPARK-32573) Eliminate Anti Join when BuildSide is Empty

Leanken.Lin created SPARK-32573: --- Summary: Eliminate Anti Join when BuildSide is Empty Key: SPARK-32573 URL: https://issues.apache.org/jira/browse/SPARK-32573 Project: Spark Issue Type: Improve

[jira] [Updated] (SPARK-32572) Run all the tests at once, instead of having separate entrypoints.

[ https://issues.apache.org/jira/browse/SPARK-32572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated SPARK-32572: - Description: Started with this comment thread:  https://github.com/apache/spark/pull/291

[jira] [Created] (SPARK-32572) Run all the tests at once, instead of having separate entrypoints.

Fokko Driesprong created SPARK-32572: Summary: Run all the tests at once, instead of having separate entrypoints. Key: SPARK-32572 URL: https://issues.apache.org/jira/browse/SPARK-32572 Project: S