[jira] [Resolved] (SPARK-36825) Read/write dataframes with ANSI intervals from/to parquet files

2021-09-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36825. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34057 [https://github.com

[jira] [Commented] (SPARK-36840) Support DPP if there is no selective predicate on the filtering side

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419604#comment-17419604 ] Apache Spark commented on SPARK-36840: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-36840) Support DPP if there is no selective predicate on the filtering side

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36840: Assignee: Apache Spark > Support DPP if there is no selective predicate on the filtering

[jira] [Assigned] (SPARK-36840) Support DPP if there is no selective predicate on the filtering side

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36840: Assignee: (was: Apache Spark) > Support DPP if there is no selective predicate on the

[jira] [Updated] (SPARK-36838) Keep In/InSet use same nullSafeEval

2021-09-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36838: -- Summary: Keep In/InSet use same nullSafeEval (was: In/InSet should handle NULL correctly ) > Keep In

[jira] [Reopened] (SPARK-36838) Keep In/InSet use same nullSafeEval

2021-09-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu reopened SPARK-36838: --- > Keep In/InSet use same nullSafeEval > --- > > Key: SPA

[jira] [Created] (SPARK-36840) Support DPP if there is no selective predicate on the filtering side

2021-09-23 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36840: --- Summary: Support DPP if there is no selective predicate on the filtering side Key: SPARK-36840 URL: https://issues.apache.org/jira/browse/SPARK-36840 Project: Spark

[jira] [Commented] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419561#comment-17419561 ] Apache Spark commented on SPARK-36827: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-36681) Fail to load Snappy codec

2021-09-23 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419557#comment-17419557 ] Xiao Li commented on SPARK-36681: - Can you link the related Jira in Apache Hadoop? > Fa

[jira] [Resolved] (SPARK-36838) In/InSet should handle NULL correctly

2021-09-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-36838. --- Resolution: Not A Problem > In/InSet should handle NULL correctly > ---

[jira] [Commented] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419554#comment-17419554 ] Hyukjin Kwon commented on SPARK-35672: -- Reverted at: https://github.com/apache/spa

[jira] [Updated] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35672: - Fix Version/s: (was: 3.1.3) (was: 3.2.0) > Spark fails to launch exec

[jira] [Commented] (SPARK-36839) Add daily build with Hadoop 2 profile in GitHub Actions build

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419552#comment-17419552 ] Apache Spark commented on SPARK-36839: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-36839) Add daily build with Hadoop 2 profile in GitHub Actions build

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36839: Assignee: (was: Apache Spark) > Add daily build with Hadoop 2 profile in GitHub Actio

[jira] [Assigned] (SPARK-36839) Add daily build with Hadoop 2 profile in GitHub Actions build

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36839: Assignee: Apache Spark > Add daily build with Hadoop 2 profile in GitHub Actions build >

[jira] [Commented] (SPARK-36839) Add daily build with Hadoop 2 profile in GitHub Actions build

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419551#comment-17419551 ] Apache Spark commented on SPARK-36839: -- User 'HyukjinKwon' has created a pull reque

[jira] [Created] (SPARK-36839) Add daily build with Hadoop 2 profile in GitHub Actions build

2021-09-23 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-36839: Summary: Add daily build with Hadoop 2 profile in GitHub Actions build Key: SPARK-36839 URL: https://issues.apache.org/jira/browse/SPARK-36839 Project: Spark

[jira] [Assigned] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36827: Assignee: (was: Apache Spark) > Task/Stage/Job data remain in memory leads memory lea

[jira] [Assigned] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36827: Assignee: Apache Spark > Task/Stage/Job data remain in memory leads memory leak > ---

[jira] [Issue Comment Deleted] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

2021-09-23 Thread Kohki Nishio (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kohki Nishio updated SPARK-36827: - Comment: was deleted (was: https://github.com/apache/spark/pull/34090) > Task/Stage/Job data re

[jira] [Commented] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

2021-09-23 Thread Kohki Nishio (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419548#comment-17419548 ] Kohki Nishio commented on SPARK-36827: -- https://github.com/apache/spark/pull/34090

[jira] [Commented] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419547#comment-17419547 ] Apache Spark commented on SPARK-36827: -- User 'taroplus' has created a pull request

[jira] [Resolved] (SPARK-36504) Improve test coverage for pandas API on Spark

2021-09-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36504. -- Fix Version/s: 3.3.0 Assignee: Haejoon Lee Resolution: Done > Improve test cov

[jira] [Commented] (SPARK-36838) In/InSet should handle NULL correctly

2021-09-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419532#comment-17419532 ] angerszhu commented on SPARK-36838: --- raising a  pr   > In/InSet should handle NULL c

[jira] [Created] (SPARK-36838) In/InSet should handle NULL correctly

2021-09-23 Thread angerszhu (Jira)
angerszhu created SPARK-36838: - Summary: In/InSet should handle NULL correctly Key: SPARK-36838 URL: https://issues.apache.org/jira/browse/SPARK-36838 Project: Spark Issue Type: Bug Co

[jira] [Assigned] (SPARK-36837) Upgrade Kafka to 3.0.0

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36837: Assignee: (was: Apache Spark) > Upgrade Kafka to 3.0.0 > -- > >

[jira] [Assigned] (SPARK-36837) Upgrade Kafka to 3.0.0

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36837: Assignee: Apache Spark > Upgrade Kafka to 3.0.0 > -- > >

[jira] [Commented] (SPARK-36837) Upgrade Kafka to 3.0.0

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419528#comment-17419528 ] Apache Spark commented on SPARK-36837: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-36837) Upgrade Kafka to 3.0.0

2021-09-23 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-36837: - Summary: Upgrade Kafka to 3.0.0 Key: SPARK-36837 URL: https://issues.apache.org/jira/browse/SPARK-36837 Project: Spark Issue Type: Improvement Co

[jira] [Assigned] (SPARK-36835) Spark 3.2.0 POMs are no longer "dependency reduced"

2021-09-23 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-36835: -- Assignee: Chao Sun > Spark 3.2.0 POMs are no longer "dependency reduced" > --

[jira] [Resolved] (SPARK-36835) Spark 3.2.0 POMs are no longer "dependency reduced"

2021-09-23 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-36835. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 34085 [https:

[jira] [Commented] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419522#comment-17419522 ] Apache Spark commented on SPARK-35672: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-36752) Support ILIKE by Scala/Java, PySpark and R APIs

2021-09-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419520#comment-17419520 ] Hyukjin Kwon commented on SPARK-36752: -- [~yoda-mon] just cheking. any update on R a

[jira] [Assigned] (SPARK-36836) "sha2" expression with bit_length of 224 returns incorrect results

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36836: Assignee: Apache Spark > "sha2" expression with bit_length of 224 returns incorrect resul

[jira] [Commented] (SPARK-36835) Spark 3.2.0 POMs are no longer "dependency reduced"

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419503#comment-17419503 ] Apache Spark commented on SPARK-36835: -- User 'sunchao' has created a pull request f

[jira] [Commented] (SPARK-36836) "sha2" expression with bit_length of 224 returns incorrect results

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419504#comment-17419504 ] Apache Spark commented on SPARK-36836: -- User 'richardc-db' has created a pull reque

[jira] [Assigned] (SPARK-36835) Spark 3.2.0 POMs are no longer "dependency reduced"

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36835: Assignee: (was: Apache Spark) > Spark 3.2.0 POMs are no longer "dependency reduced" >

[jira] [Assigned] (SPARK-36835) Spark 3.2.0 POMs are no longer "dependency reduced"

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36835: Assignee: Apache Spark > Spark 3.2.0 POMs are no longer "dependency reduced" > --

[jira] [Assigned] (SPARK-36836) "sha2" expression with bit_length of 224 returns incorrect results

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36836: Assignee: (was: Apache Spark) > "sha2" expression with bit_length of 224 returns inco

[jira] [Commented] (SPARK-36835) Spark 3.2.0 POMs are no longer "dependency reduced"

2021-09-23 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419499#comment-17419499 ] Chao Sun commented on SPARK-36835: -- Sorry for the regression [~joshrosen]. I forgot exa

[jira] [Created] (SPARK-36836) "sha2" expression with bit_length of 224 returns incorrect results

2021-09-23 Thread Richard Chen (Jira)
Richard Chen created SPARK-36836: Summary: "sha2" expression with bit_length of 224 returns incorrect results Key: SPARK-36836 URL: https://issues.apache.org/jira/browse/SPARK-36836 Project: Spark

[jira] [Updated] (SPARK-36835) Spark 3.2.0 POMs are no longer "dependency reduced"

2021-09-23 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-36835: --- Priority: Blocker (was: Major) > Spark 3.2.0 POMs are no longer "dependency reduced" >

[jira] [Created] (SPARK-36835) Spark 3.2.0 POMs are no longer "dependency reduced"

2021-09-23 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-36835: -- Summary: Spark 3.2.0 POMs are no longer "dependency reduced" Key: SPARK-36835 URL: https://issues.apache.org/jira/browse/SPARK-36835 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419493#comment-17419493 ] Apache Spark commented on SPARK-35672: -- User 'xkrogen' has created a pull request f

[jira] [Commented] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419492#comment-17419492 ] Apache Spark commented on SPARK-35672: -- User 'xkrogen' has created a pull request f

[jira] [Assigned] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35672: Assignee: Erik Krogen (was: Apache Spark) > Spark fails to launch executors with very la

[jira] [Assigned] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35672: Assignee: Apache Spark (was: Erik Krogen) > Spark fails to launch executors with very la

[jira] [Reopened] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reopened SPARK-35672: --- > Spark fails to launch executors with very large user classpath lists on YARN > ---

[jira] [Commented] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419392#comment-17419392 ] Apache Spark commented on SPARK-35672: -- User 'peter-toth' has created a pull reques

[jira] [Commented] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419390#comment-17419390 ] Apache Spark commented on SPARK-35672: -- User 'peter-toth' has created a pull reques

[jira] [Commented] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419388#comment-17419388 ] Peter Toth commented on SPARK-35672: I put up a revert PR: https://github.com/apache

[jira] [Commented] (SPARK-36757) countDistinct becomes count when converted to SQL string

2021-09-23 Thread Paul Deng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419386#comment-17419386 ] Paul Deng commented on SPARK-36757: --- Just adding that this behavior also occurs when c

[jira] [Commented] (SPARK-36747) Do not collapse Project with Aggregate when correlated subqueries are present in the project list

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419365#comment-17419365 ] Apache Spark commented on SPARK-36747: -- User 'allisonwang-db' has created a pull re

[jira] [Commented] (SPARK-33832) Add an option in AQE to mitigate skew even if it causes an new shuffle

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419358#comment-17419358 ] Apache Spark commented on SPARK-33832: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-33832) Add an option in AQE to mitigate skew even if it causes an new shuffle

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419360#comment-17419360 ] Apache Spark commented on SPARK-33832: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-36834) Namespace log lines in External Shuffle Service

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419331#comment-17419331 ] Apache Spark commented on SPARK-36834: -- User 'thejdeep' has created a pull request

[jira] [Assigned] (SPARK-36834) Namespace log lines in External Shuffle Service

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36834: Assignee: Apache Spark > Namespace log lines in External Shuffle Service > --

[jira] [Assigned] (SPARK-36834) Namespace log lines in External Shuffle Service

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36834: Assignee: (was: Apache Spark) > Namespace log lines in External Shuffle Service > ---

[jira] [Commented] (SPARK-36834) Namespace log lines in External Shuffle Service

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419329#comment-17419329 ] Apache Spark commented on SPARK-36834: -- User 'thejdeep' has created a pull request

[jira] [Commented] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419312#comment-17419312 ] Thomas Graves commented on SPARK-35672: --- Ok, sounds like we should revert then so

[jira] [Created] (SPARK-36834) Namespace log lines in External Shuffle Service

2021-09-23 Thread Thejdeep Gudivada (Jira)
Thejdeep Gudivada created SPARK-36834: - Summary: Namespace log lines in External Shuffle Service Key: SPARK-36834 URL: https://issues.apache.org/jira/browse/SPARK-36834 Project: Spark Iss

[jira] [Commented] (SPARK-36760) Add interface SupportsPushDownV2Filters

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419310#comment-17419310 ] Apache Spark commented on SPARK-36760: -- User 'sarutak' has created a pull request f

[jira] [Commented] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-23 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419285#comment-17419285 ] Peter Toth commented on SPARK-35672: [~xkrogen], [~tgraves], unfortunately, I think

[jira] [Updated] (SPARK-36833) Can't use SSL with spark on kubernetes on service level

2021-09-23 Thread zoli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zoli updated SPARK-36833: - Description: Currently it seems impossible to create the correct cert for driver's pod because of the random na

[jira] [Updated] (SPARK-36833) Can't use SSL with spark on kubernetes on service level

2021-09-23 Thread zoli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zoli updated SPARK-36833: - Description: Currently it seems impossible to create the correct cert for driver's pod because of the random na

[jira] [Updated] (SPARK-36833) Can't use SSL with spark on kubernetes on service level

2021-09-23 Thread zoli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zoli updated SPARK-36833: - Priority: Critical (was: Major) > Can't use SSL with spark on kubernetes on service level > ---

[jira] [Updated] (SPARK-36832) Add LauncherProtocol to K8s resource manager client

2021-09-23 Thread timothy65535 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] timothy65535 updated SPARK-36832: - Description: Refer to: org.apache.spark.deploy.yarn.Client {code:java} private val launcherBacke

[jira] [Commented] (SPARK-36832) Add LauncherProtocol to K8s resource manager client

2021-09-23 Thread timothy65535 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419167#comment-17419167 ] timothy65535 commented on SPARK-36832: -- Feel free to take this Jira :) > Add Launc

[jira] [Updated] (SPARK-36832) Add LauncherProtocol to K8s resource manager client

2021-09-23 Thread timothy65535 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] timothy65535 updated SPARK-36832: - Description: Refer to org.apache.spark.deploy.yarn.Client {code:java} private val launcherBacke

[jira] [Updated] (SPARK-36833) Can't use SSL with spark on kubernetes on service level

2021-09-23 Thread zoli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zoli updated SPARK-36833: - Description: Currently it seems impossible to generate the correct cert for driver's pod because of the random

[jira] [Updated] (SPARK-36833) Can't use SSL with spark on kubernetes on service level

2021-09-23 Thread zoli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zoli updated SPARK-36833: - Description: Currently it seems impossible to generate the correct cert for driver's pod because of the random

[jira] [Updated] (SPARK-36833) Can't use SSL with spark on kubernetes on service level

2021-09-23 Thread zoli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zoli updated SPARK-36833: - Description: Currently it seems impossible to generate the correct cert for driver's pod because of the random

[jira] [Updated] (SPARK-36833) Can't use SSL with spark on kubernetes on service level

2021-09-23 Thread zoli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zoli updated SPARK-36833: - Priority: Major (was: Blocker) > Can't use SSL with spark on kubernetes on service level >

[jira] [Updated] (SPARK-36833) Can't use SSL with spark on kubernetes on service level

2021-09-23 Thread zoli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zoli updated SPARK-36833: - Description: Currently it seems impossible to generate the correct cert for driver's pod because of the random

[jira] [Updated] (SPARK-36833) Can't use SSL with spark on kubernetes on service level

2021-09-23 Thread zoli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zoli updated SPARK-36833: - Description: Currently it seems impossible to generate the correct cert for driver's pod because of the random

[jira] [Updated] (SPARK-36833) Can't use SSL with spark on kubernetes on service level

2021-09-23 Thread zoli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zoli updated SPARK-36833: - Description: Currently seems impossible to generate the correct cert for driver's service because of the random

[jira] [Created] (SPARK-36833) Can't use SSL with spark on kubernetes on service level

2021-09-23 Thread zoli (Jira)
zoli created SPARK-36833: Summary: Can't use SSL with spark on kubernetes on service level Key: SPARK-36833 URL: https://issues.apache.org/jira/browse/SPARK-36833 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-36832) Add LauncherProtocol to K8s resource manager client

2021-09-23 Thread timothy65535 (Jira)
timothy65535 created SPARK-36832: Summary: Add LauncherProtocol to K8s resource manager client Key: SPARK-36832 URL: https://issues.apache.org/jira/browse/SPARK-36832 Project: Spark Issue Typ

[jira] [Commented] (SPARK-27780) Shuffle server & client should be versioned to enable smoother upgrade

2021-09-23 Thread sooyeon shin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419135#comment-17419135 ] sooyeon shin commented on SPARK-27780: -- [~shyamabigd...@gmail.com]  I meant that yo

[jira] [Created] (SPARK-36831) Read/write dataframes with ANSI intervals from/to CSV files

2021-09-23 Thread Max Gekk (Jira)
Max Gekk created SPARK-36831: Summary: Read/write dataframes with ANSI intervals from/to CSV files Key: SPARK-36831 URL: https://issues.apache.org/jira/browse/SPARK-36831 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-36830) Read/write dataframes with ANSI intervals from/to JSON files

2021-09-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-36830: Assignee: (was: Max Gekk) > Read/write dataframes with ANSI intervals from/to JSON files > --

[jira] [Created] (SPARK-36830) Read/write dataframes with ANSI intervals from/to JSON files

2021-09-23 Thread Max Gekk (Jira)
Max Gekk created SPARK-36830: Summary: Read/write dataframes with ANSI intervals from/to JSON files Key: SPARK-36830 URL: https://issues.apache.org/jira/browse/SPARK-36830 Project: Spark Issue T

[jira] [Assigned] (SPARK-36829) Refactor collectionOperation related Null check related code

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36829: Assignee: (was: Apache Spark) > Refactor collectionOperation related Null check relat

[jira] [Assigned] (SPARK-36829) Refactor collectionOperation related Null check related code

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36829: Assignee: Apache Spark > Refactor collectionOperation related Null check related code > -

[jira] [Commented] (SPARK-36829) Refactor collectionOperation related Null check related code

2021-09-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419068#comment-17419068 ] Apache Spark commented on SPARK-36829: -- User 'AngersZh' has created a pull requ

[jira] [Commented] (SPARK-4036) Add Conditional Random Fields (CRF) algorithm to Spark MLlib

2021-09-23 Thread Martin Wunderlich (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419064#comment-17419064 ] Martin Wunderlich commented on SPARK-4036: -- Hello, This issue was closed 5 years

[jira] [Updated] (SPARK-36829) Refactor collectionOperation related Null check related code

2021-09-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36829: -- Summary: Refactor collectionOperation related Null check related code (was: Refactor Null check relat

[jira] [Assigned] (SPARK-36782) Deadlock between map-output-dispatcher and dispatcher-BlockManagerMaster upon migrating shuffle blocks

2021-09-23 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-36782: -- Assignee: Fabian Thiele > Deadlock between map-output-dispatcher and dispatcher-Block

[jira] [Created] (SPARK-36829) Refactor Null check related code

2021-09-23 Thread angerszhu (Jira)
angerszhu created SPARK-36829: - Summary: Refactor Null check related code Key: SPARK-36829 URL: https://issues.apache.org/jira/browse/SPARK-36829 Project: Spark Issue Type: Improvement