[jira] [Created] (SPARK-34535) Cleanup unused symbol in Orc related code

2021-02-24 Thread Yang Jie (Jira)
Yang Jie created SPARK-34535: Summary: Cleanup unused symbol in Orc related code Key: SPARK-34535 URL: https://issues.apache.org/jira/browse/SPARK-34535 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290744#comment-17290744 ] Apache Spark commented on SPARK-34534: -- User 'seayoun' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34534: Assignee: (was: Apache Spark) > New protocol FetchShuffleBlocks in

[jira] [Assigned] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34534: Assignee: Apache Spark > New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead

[jira] [Commented] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290743#comment-17290743 ] Apache Spark commented on SPARK-34534: -- User 'seayoun' has created a pull request for this issue:

[jira] [Commented] (SPARK-33212) Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290726#comment-17290726 ] Apache Spark commented on SPARK-33212: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-33212) Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile

2021-02-24 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290707#comment-17290707 ] Chao Sun commented on SPARK-33212: -- Yes. I think the only class Spark needs from this jar is

[jira] [Commented] (SPARK-34533) Eliminate LEFT ANTI join to empty relation in AQE

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290676#comment-17290676 ] Apache Spark commented on SPARK-34533: -- User 'c21' has created a pull request for this issue:

[jira] [Commented] (SPARK-34533) Eliminate LEFT ANTI join to empty relation in AQE

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290675#comment-17290675 ] Apache Spark commented on SPARK-34533: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34533) Eliminate LEFT ANTI join to empty relation in AQE

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34533: Assignee: Apache Spark > Eliminate LEFT ANTI join to empty relation in AQE >

[jira] [Assigned] (SPARK-34533) Eliminate LEFT ANTI join to empty relation in AQE

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34533: Assignee: (was: Apache Spark) > Eliminate LEFT ANTI join to empty relation in AQE >

[jira] [Assigned] (SPARK-34520) Remove unused SecurityManager references

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34520: - Assignee: Hyukjin Kwon > Remove unused SecurityManager references >

[jira] [Resolved] (SPARK-34520) Remove unused SecurityManager references

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34520. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31636

[jira] [Commented] (SPARK-33212) Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile

2021-02-24 Thread Xiaochen Ouyang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290656#comment-17290656 ] Xiaochen Ouyang commented on SPARK-33212: - {{Maybe we should confirm there are no Guava direct

[jira] [Commented] (SPARK-34529) spark.read.csv is throwing exception ,"lineSep' can contain only 1 character" when parsing windows line feed (CR LF)

2021-02-24 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290652#comment-17290652 ] Yang Jie commented on SPARK-34529: -- There seems to be some discussion

[jira] [Updated] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread haiyangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] haiyangyu updated SPARK-34534: -- Description: We will build a new rpc message `FetchShuffleBlocks` when `OneForOneBlockFetcher` init

[jira] [Updated] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread haiyangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] haiyangyu updated SPARK-34534: -- Description: We will build a new rpc message `FetchShuffleBlocks` when `OneForOneBlockFetcher` init

[jira] [Updated] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread haiyangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] haiyangyu updated SPARK-34534: -- Attachment: image-2021-02-25-11-31-59-110.png > New protocol FetchShuffleBlocks in

[jira] [Updated] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread haiyangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] haiyangyu updated SPARK-34534: -- Attachment: image-2021-02-25-11-30-03-834.png > New protocol FetchShuffleBlocks in

[jira] [Commented] (SPARK-33212) Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290650#comment-17290650 ] Dongjoon Hyun commented on SPARK-33212: --- Thanks! > Upgrade to Hadoop 3.2.2 and move to shaded

[jira] [Updated] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread haiyangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] haiyangyu updated SPARK-34534: -- Attachment: image-2021-02-25-11-28-31-255.png > New protocol FetchShuffleBlocks in

[jira] [Updated] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread haiyangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] haiyangyu updated SPARK-34534: -- Attachment: image-2021-02-25-11-27-34-429.png > New protocol FetchShuffleBlocks in

[jira] [Commented] (SPARK-34532) IntervalUtils.add() may result in 'long overflow'

2021-02-24 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290646#comment-17290646 ] Ted Yu commented on SPARK-34532: Included the test command and some more information in the description.

[jira] [Updated] (SPARK-34532) IntervalUtils.add() may result in 'long overflow'

2021-02-24 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-34532: --- Description: I noticed the following when running test suite: build/sbt "sql/testOnly *SQLQueryTestSuite"

[jira] [Updated] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread haiyangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] haiyangyu updated SPARK-34534: -- Description: We will build a new rpc message `FetchShuffleBlocks` when `OneForOneBlockFetcher` init

[jira] [Updated] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread haiyangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] haiyangyu updated SPARK-34534: -- Attachment: image-2021-02-25-11-17-12-714.png > New protocol FetchShuffleBlocks in

[jira] [Commented] (SPARK-34448) Binary logistic regression incorrectly computes the intercept and coefficients when data is not centered

2021-02-24 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290642#comment-17290642 ] zhengruifeng commented on SPARK-34448: -- [~srowen] Thanks for pinging me, I am going to look into

[jira] [Updated] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread haiyangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] haiyangyu updated SPARK-34534: -- Description: We will build a new rpc message  {code:java} FetchShuffleBlocks{code} when {code:java}

[jira] [Updated] (SPARK-34534) New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread haiyangyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] haiyangyu updated SPARK-34534: -- Summary: New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

[jira] [Created] (SPARK-34534) FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness

2021-02-24 Thread haiyangyu (Jira)
haiyangyu created SPARK-34534: - Summary: FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness Key: SPARK-34534 URL: https://issues.apache.org/jira/browse/SPARK-34534 Project:

[jira] [Commented] (SPARK-34530) logError for interrupting block migrations is too high

2021-02-24 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290633#comment-17290633 ] Yang Jie commented on SPARK-34530: -- [~holden] Can you add some descriptions? > logError for

[jira] [Commented] (SPARK-34532) IntervalUtils.add() may result in 'long overflow'

2021-02-24 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290632#comment-17290632 ] Yang Jie commented on SPARK-34532: -- Which case have this problem? > IntervalUtils.add() may result in

[jira] [Comment Edited] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290631#comment-17290631 ] Kent Yao edited comment on SPARK-34523 at 2/25/21, 2:56 AM: Hi [~dongjoon],

[jira] [Commented] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290631#comment-17290631 ] Kent Yao commented on SPARK-34523: -- Hi [~dongjoon], thanks for your suggestions. When the problem

[jira] [Created] (SPARK-34533) Eliminate LEFT ANTI join to empty relation in AQE

2021-02-24 Thread Cheng Su (Jira)
Cheng Su created SPARK-34533: Summary: Eliminate LEFT ANTI join to empty relation in AQE Key: SPARK-34533 URL: https://issues.apache.org/jira/browse/SPARK-34533 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-33212) Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile

2021-02-24 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290613#comment-17290613 ] Chao Sun edited comment on SPARK-33212 at 2/25/21, 2:21 AM: I was able to

[jira] [Updated] (SPARK-34497) JDBC connection provider is not removing kerberos credentials from JVM security context

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34497: -- Affects Version/s: (was: 3.1.2) 3.1.1 > JDBC connection provider

[jira] [Commented] (SPARK-33212) Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile

2021-02-24 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290613#comment-17290613 ] Chao Sun commented on SPARK-33212: -- I was able to reproduce the error in my local environment, and find

[jira] [Updated] (SPARK-34531) Remove Experimental API tag in PrometheusServlet

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34531: -- Issue Type: Bug (was: Improvement) > Remove Experimental API tag in PrometheusServlet >

[jira] [Updated] (SPARK-34531) Remove Experimental API tag in PrometheusServlet

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34531: -- Affects Version/s: (was: 3.1.2) 3.1.1 > Remove Experimental API

[jira] [Updated] (SPARK-34531) Remove Experimental API tag in PrometheusServlet

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34531: -- Affects Version/s: 3.0.2 > Remove Experimental API tag in PrometheusServlet >

[jira] [Updated] (SPARK-34531) Remove Experimental API tag in PrometheusServlet

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34531: -- Affects Version/s: 3.1.2 > Remove Experimental API tag in PrometheusServlet >

[jira] [Assigned] (SPARK-34531) Remove Experimental API tag in PrometheusServlet

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34531: - Assignee: Hyukjin Kwon > Remove Experimental API tag in PrometheusServlet >

[jira] [Resolved] (SPARK-34531) Remove Experimental API tag in PrometheusServlet

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34531. --- Fix Version/s: 3.1.2 Resolution: Fixed Issue resolved by pull request 31640

[jira] [Updated] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34523: - Description: h2. Instruction This will cause deadlock and hangs concurrent tasks forever on the same

[jira] [Commented] (SPARK-34516) Spark 3.0.1 encounter parquet PageHerder IO issue

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290599#comment-17290599 ] Dongjoon Hyun commented on SPARK-34516: --- Hi, [~angerszhuuu]. Could you provide a reproducer? For

[jira] [Updated] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34523: - Description: h2. Instruction This will cause deadlock and hangs concurrent tasks forever on the same

[jira] [Updated] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34523: - Description: This will cause deadlock and hangs concurrent tasks forever on the same executor. for

[jira] [Updated] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34523: - Description: This will cause deadlock and hangs concurrent tasks forever on the same executor. for

[jira] [Updated] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34523: - Attachment: screenshot-2.png > JDK-8194653: Deadlock involving FileSystems.getDefault and >

[jira] [Assigned] (SPARK-34497) JDBC connection provider is not removing kerberos credentials from JVM security context

2021-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34497: Assignee: Gabor Somogyi > JDBC connection provider is not removing kerberos credentials

[jira] [Resolved] (SPARK-34497) JDBC connection provider is not removing kerberos credentials from JVM security context

2021-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34497. -- Fix Version/s: 3.1.2 Resolution: Fixed Issue resolved by pull request 31622

[jira] [Created] (SPARK-34532) IntervalUtils.add() may result in 'long overflow'

2021-02-24 Thread Ted Yu (Jira)
Ted Yu created SPARK-34532: -- Summary: IntervalUtils.add() may result in 'long overflow' Key: SPARK-34532 URL: https://issues.apache.org/jira/browse/SPARK-34532 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34523: - Description: This will cause deadlock and hangs concurrent tasks forever on the same executor. for

[jira] [Updated] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34523: - Attachment: screenshot-1.png > JDK-8194653: Deadlock involving FileSystems.getDefault and >

[jira] [Assigned] (SPARK-34531) Remove Experimental API tag in PrometheusServlet

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34531: Assignee: (was: Apache Spark) > Remove Experimental API tag in PrometheusServlet >

[jira] [Commented] (SPARK-34531) Remove Experimental API tag in PrometheusServlet

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290561#comment-17290561 ] Apache Spark commented on SPARK-34531: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-34531) Remove Experimental API tag in PrometheusServlet

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34531: Assignee: Apache Spark > Remove Experimental API tag in PrometheusServlet >

[jira] [Commented] (SPARK-34531) Remove Experimental API tag in PrometheusServlet

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290563#comment-17290563 ] Apache Spark commented on SPARK-34531: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-34531) Remove Experimental API tag in PrometheusServlet

2021-02-24 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-34531: Summary: Remove Experimental API tag in PrometheusServlet Key: SPARK-34531 URL: https://issues.apache.org/jira/browse/SPARK-34531 Project: Spark Issue Type:

[jira] [Created] (SPARK-34530) logError for interrupting block migrations is too high

2021-02-24 Thread Holden Karau (Jira)
Holden Karau created SPARK-34530: Summary: logError for interrupting block migrations is too high Key: SPARK-34530 URL: https://issues.apache.org/jira/browse/SPARK-34530 Project: Spark Issue

[jira] [Commented] (SPARK-34529) spark.read.csv is throwing exception ,"lineSep' can contain only 1 character" when parsing windows line feed (CR LF)

2021-02-24 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290539#comment-17290539 ] Takeshi Yamamuro commented on SPARK-34529: -- Since I think this is not a bug but improvement, I

[jira] [Updated] (SPARK-34529) spark.read.csv is throwing exception ,"lineSep' can contain only 1 character" when parsing windows line feed (CR LF)

2021-02-24 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-34529: - Component/s: (was: Spark Core) SQL > spark.read.csv is throwing

[jira] [Updated] (SPARK-34529) spark.read.csv is throwing exception ,"lineSep' can contain only 1 character" when parsing windows line feed (CR LF)

2021-02-24 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-34529: - Affects Version/s: (was: 3.0.1) 3.0.3

[jira] [Updated] (SPARK-34529) spark.read.csv is throwing exception ,"lineSep' can contain only 1 character" when parsing windows line feed (CR LF)

2021-02-24 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-34529: - Issue Type: Improvement (was: Bug) > spark.read.csv is throwing exception ,"lineSep'

[jira] [Assigned] (SPARK-34528) View result are not consistent after a modification inside a struct of the table

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34528: Assignee: Apache Spark > View result are not consistent after a modification inside a

[jira] [Assigned] (SPARK-34528) View result are not consistent after a modification inside a struct of the table

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34528: Assignee: (was: Apache Spark) > View result are not consistent after a modification

[jira] [Commented] (SPARK-34528) View result are not consistent after a modification inside a struct of the table

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290275#comment-17290275 ] Apache Spark commented on SPARK-34528: -- User 'tprelle' has created a pull request for this issue:

[jira] [Created] (SPARK-34529) spark.read.csv is throwing exception ,"lineSep' can contain only 1 character" when parsing windows line feed (CR LF)

2021-02-24 Thread Shanmugavel Kuttiyandi Chandrakasu (Jira)
Shanmugavel Kuttiyandi Chandrakasu created SPARK-34529: -- Summary: spark.read.csv is throwing exception ,"lineSep' can contain only 1 character" when parsing windows line feed (CR LF) Key: SPARK-34529

[jira] [Created] (SPARK-34528) View result are not consistent after a modification inside a struct of the table

2021-02-24 Thread Thomas Prelle (Jira)
Thomas Prelle created SPARK-34528: - Summary: View result are not consistent after a modification inside a struct of the table Key: SPARK-34528 URL: https://issues.apache.org/jira/browse/SPARK-34528

[jira] [Resolved] (SPARK-32617) Upgrade kubernetes client version to support latest minikube version.

2021-02-24 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-32617. -- Fix Version/s: 3.2.0 Assignee: Attila Zsolt Piros Resolution: Fixed > Upgrade

[jira] [Commented] (SPARK-34527) De-duplicated common columns cannot be resolved from USING/NATURAL JOIN

2021-02-24 Thread Karen Feng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290211#comment-17290211 ] Karen Feng commented on SPARK-34527: I've implemented a fix for this, will push a PR. >

[jira] [Created] (SPARK-34527) De-duplicated common columns cannot be resolved from USING/NATURAL JOIN

2021-02-24 Thread Karen Feng (Jira)
Karen Feng created SPARK-34527: -- Summary: De-duplicated common columns cannot be resolved from USING/NATURAL JOIN Key: SPARK-34527 URL: https://issues.apache.org/jira/browse/SPARK-34527 Project: Spark

[jira] [Assigned] (SPARK-34526) Add a flag to skip checking file sink format and handle glob path

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34526: Assignee: (was: Apache Spark) > Add a flag to skip checking file sink format and

[jira] [Commented] (SPARK-34526) Add a flag to skip checking file sink format and handle glob path

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290177#comment-17290177 ] Apache Spark commented on SPARK-34526: -- User 'xuanyuanking' has created a pull request for this

[jira] [Assigned] (SPARK-34526) Add a flag to skip checking file sink format and handle glob path

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34526: Assignee: Apache Spark > Add a flag to skip checking file sink format and handle glob

[jira] [Updated] (SPARK-34526) Add a flag to skip checking file sink format and handle glob path

2021-02-24 Thread Yuanjian Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanjian Li updated SPARK-34526: Description: This ticket fixes the following issues related to file sink format checking

[jira] [Created] (SPARK-34526) Add a flag to skip checking file sink format and handle glob path

2021-02-24 Thread Yuanjian Li (Jira)
Yuanjian Li created SPARK-34526: --- Summary: Add a flag to skip checking file sink format and handle glob path Key: SPARK-34526 URL: https://issues.apache.org/jira/browse/SPARK-34526 Project: Spark

[jira] [Commented] (SPARK-34448) Binary logistic regression incorrectly computes the intercept and coefficients when data is not centered

2021-02-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290158#comment-17290158 ] Sean R. Owen commented on SPARK-34448: -- I crudely ported the test setup to a Scala test, and tried

[jira] [Commented] (SPARK-33212) Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile

2021-02-24 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290127#comment-17290127 ] Chao Sun commented on SPARK-33212: -- Thanks again [~ouyangxc.zte].

[jira] [Commented] (SPARK-34524) simplify v2 partition commands resolution

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290120#comment-17290120 ] Apache Spark commented on SPARK-34524: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34524) simplify v2 partition commands resolution

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34524: Assignee: (was: Apache Spark) > simplify v2 partition commands resolution >

[jira] [Assigned] (SPARK-34524) simplify v2 partition commands resolution

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34524: Assignee: Apache Spark > simplify v2 partition commands resolution >

[jira] [Commented] (SPARK-34524) simplify v2 partition commands resolution

2021-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290119#comment-17290119 ] Apache Spark commented on SPARK-34524: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Updated] (SPARK-34525) Update Spark Create Table DDL Docs

2021-02-24 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-34525: Labels: starter (was: ) > Update Spark Create Table DDL Docs > -- > >

[jira] [Created] (SPARK-34525) Update Spark Create Table DDL Docs

2021-02-24 Thread Miklos Christine (Jira)
Miklos Christine created SPARK-34525: Summary: Update Spark Create Table DDL Docs Key: SPARK-34525 URL: https://issues.apache.org/jira/browse/SPARK-34525 Project: Spark Issue Type:

[jira] [Commented] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290087#comment-17290087 ] Dongjoon Hyun commented on SPARK-34523: --- I'd like to recommend to make a documentation PR instead.

[jira] [Created] (SPARK-34524) simplify v2 partition commands resolution

2021-02-24 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-34524: --- Summary: simplify v2 partition commands resolution Key: SPARK-34524 URL: https://issues.apache.org/jira/browse/SPARK-34524 Project: Spark Issue Type:

[jira] [Commented] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290084#comment-17290084 ] Dongjoon Hyun commented on SPARK-34523: --- Hi, [~Qin Yao]. This looks like a duplicate JDK

[jira] [Commented] (SPARK-34448) Binary logistic regression incorrectly computes the intercept and coefficients when data is not centered

2021-02-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290052#comment-17290052 ] Sean R. Owen commented on SPARK-34448: -- Yes I believe you're definitely correct there's a problem

[jira] [Issue Comment Deleted] (SPARK-34521) spark.createDataFrame does not support Pandas StringDtype extension type

2021-02-24 Thread Pavel Ganelin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Ganelin updated SPARK-34521: -- Comment: was deleted (was: Originally submitted to ARROW:

[jira] [Created] (SPARK-34523) JDK-8194653

2021-02-24 Thread Kent Yao (Jira)
Kent Yao created SPARK-34523: Summary: JDK-8194653 Key: SPARK-34523 URL: https://issues.apache.org/jira/browse/SPARK-34523 Project: Spark Issue Type: Sub-task Components: Spark Core

[jira] [Resolved] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-34523. -- Resolution: Information Provided > JDK-8194653: Deadlock involving FileSystems.getDefault and >

[jira] [Updated] (SPARK-34523) JDK-8194653: JDK-8194653 Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34523: - Summary: JDK-8194653: JDK-8194653 Deadlock involving FileSystems.getDefault and System.loadLibrary call

[jira] [Updated] (SPARK-34523) JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34523: - Summary: JDK-8194653: Deadlock involving FileSystems.getDefault and System.loadLibrary call (was:

[jira] [Updated] (SPARK-34523) JDK-8194653

2021-02-24 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34523: - Attachment: 4303.log > JDK-8194653 > --- > > Key: SPARK-34523 >

[jira] [Created] (SPARK-34522) Issue Tracker for JDK related Bugs

2021-02-24 Thread Kent Yao (Jira)
Kent Yao created SPARK-34522: Summary: Issue Tracker for JDK related Bugs Key: SPARK-34522 URL: https://issues.apache.org/jira/browse/SPARK-34522 Project: Spark Issue Type: Documentation

[jira] [Commented] (SPARK-34521) spark.createDataFrame does not support Pandas StringDtype extension type

2021-02-24 Thread Pavel Ganelin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17289983#comment-17289983 ] Pavel Ganelin commented on SPARK-34521: --- Originally submitted to ARROW:

[jira] [Created] (SPARK-34521) spark.createDataFrame does not support Pandas StringDtype extension type

2021-02-24 Thread Pavel Ganelin (Jira)
Pavel Ganelin created SPARK-34521: - Summary: spark.createDataFrame does not support Pandas StringDtype extension type Key: SPARK-34521 URL: https://issues.apache.org/jira/browse/SPARK-34521 Project:

[jira] [Resolved] (SPARK-34515) Fix NPE if InSet contains null value during getPartitionsByFilter

2021-02-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34515. - Fix Version/s: 3.1.2 3.2.0 Resolution: Fixed Issue resolved by pull

  1   2   >