[jira] [Resolved] (SPARK-33913) Upgrade Kafka to 2.8.0

2021-04-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33913. -- Resolution: Fixed Fixed in [https://github.com/apache/spark/pull/32325] > Upgrade Kafka to 2.

[jira] [Resolved] (SPARK-35209) CLONE - CatalystTypeConverters of date/timestamp should accept both the old and new Java time classes

2021-04-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35209. -- Resolution: Invalid > CLONE - CatalystTypeConverters of date/timestamp should accept both the

[jira] [Created] (SPARK-35216) a general auto merge output files feature for datasource api

2021-04-25 Thread qian wang (Jira)
qian wang created SPARK-35216: - Summary: a general auto merge output files feature for datasource api Key: SPARK-35216 URL: https://issues.apache.org/jira/browse/SPARK-35216 Project: Spark Issue

[jira] [Created] (SPARK-35217) com.google.protobuf.Parser.parseFrom() method Can't use in spark

2021-04-25 Thread ShuDaoNan (Jira)
ShuDaoNan created SPARK-35217: - Summary: com.google.protobuf.Parser.parseFrom() method Can't use in spark Key: SPARK-35217 URL: https://issues.apache.org/jira/browse/SPARK-35217 Project: Spark

[jira] [Updated] (SPARK-35217) com.google.protobuf.Parser.parseFrom() method Can't use in spark

2021-04-25 Thread ShuDaoNan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ShuDaoNan updated SPARK-35217: -- Description: *platform:* Linux ip-172-17-1-1 4.14.219-164.354.amzn2.x86_64 [#1|https://github.com/ap

[jira] [Updated] (SPARK-35216) a general auto merge output small files feature for datasource api

2021-04-25 Thread qian wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] qian wang updated SPARK-35216: -- Summary: a general auto merge output small files feature for datasource api (was: a general auto merg

[jira] [Closed] (SPARK-33378) Update doc about grouping analytics

2021-04-25 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu closed SPARK-33378. - Fix in other pr > Update doc about grouping analytics > > >

[jira] [Closed] (SPARK-33377) Support composite columns in cube/rollup

2021-04-25 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu closed SPARK-33377. - > Support composite columns in cube/rollup > > > Ke

[jira] [Assigned] (SPARK-35216) a general auto merge output small files feature for datasource api

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35216: Assignee: Apache Spark > a general auto merge output small files feature for datasource a

[jira] [Closed] (SPARK-33296) Format exception when use cube func && with cube

2021-04-25 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu closed SPARK-33296. - Not a problem now > Format exception when use cube func && with cube >

[jira] [Assigned] (SPARK-35216) a general auto merge output small files feature for datasource api

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35216: Assignee: (was: Apache Spark) > a general auto merge output small files feature for d

[jira] [Closed] (SPARK-33309) Replace origin GROUPING SETS with new expression

2021-04-25 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu closed SPARK-33309. - Fixed in other pr > Replace origin GROUPING SETS with new expression >

[jira] [Commented] (SPARK-35211) Support UDT for Pandas with Arrow Disabled

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331457#comment-17331457 ] Apache Spark commented on SPARK-35211: -- User 'sadhen' has created a pull request fo

[jira] [Updated] (SPARK-35217) com.google.protobuf.Parser.parseFrom() method Can't use in spark

2021-04-25 Thread ShuDaoNan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ShuDaoNan updated SPARK-35217: -- Priority: Major (was: Minor) > com.google.protobuf.Parser.parseFrom() method Can't use in spark > -

[jira] [Created] (SPARK-35218) In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to better track Spark

2021-04-25 Thread Runlin Zhang (Jira)
Runlin Zhang created SPARK-35218: Summary: In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to better track Spark Job Key: SPARK-3

[jira] [Updated] (SPARK-35218) In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to better track Spark

2021-04-25 Thread Runlin Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Runlin Zhang updated SPARK-35218: - Attachment: screenshot-1.png > In YARN mode, for better user experience, when Spark is started,

[jira] [Updated] (SPARK-35218) [SPARK-35218] In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to bett

2021-04-25 Thread Runlin Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Runlin Zhang updated SPARK-35218: - Summary: [SPARK-35218] In YARN mode, for better user experience, when Spark is started, not only

[jira] [Commented] (SPARK-35218) [SPARK-35218] In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to be

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331476#comment-17331476 ] Apache Spark commented on SPARK-35218: -- User 'runlin512' has created a pull request

[jira] [Assigned] (SPARK-33985) Transform with clusterby/orderby/sortby

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33985: Assignee: Apache Spark > Transform with clusterby/orderby/sortby > --

[jira] [Assigned] (SPARK-33985) Transform with clusterby/orderby/sortby

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33985: Assignee: (was: Apache Spark) > Transform with clusterby/orderby/sortby > ---

[jira] [Commented] (SPARK-33985) Transform with clusterby/orderby/sortby

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331475#comment-17331475 ] Apache Spark commented on SPARK-33985: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-35218) [SPARK-35218] In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to bet

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35218: Assignee: (was: Apache Spark) > [SPARK-35218] In YARN mode, for better user experienc

[jira] [Assigned] (SPARK-35218) [SPARK-35218] In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to bet

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35218: Assignee: Apache Spark > [SPARK-35218] In YARN mode, for better user experience, when Spa

[jira] [Resolved] (SPARK-32390) TRANSFORM with hive serde support CalendarIntervalType and UserDefinedType

2021-04-25 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-32390. --- Resolution: Won't Fix > TRANSFORM with hive serde support CalendarIntervalType and UserDefinedType >

[jira] [Created] (SPARK-35219) YearMonthIntervalType/DayTimeIntervalType 's string value not same in spark default serde and hive serde

2021-04-25 Thread angerszhu (Jira)
angerszhu created SPARK-35219: - Summary: YearMonthIntervalType/DayTimeIntervalType 's string value not same in spark default serde and hive serde Key: SPARK-35219 URL: https://issues.apache.org/jira/browse/SPARK-35219

[jira] [Created] (SPARK-35220) DayTimeIntervalType/YearMonthIntervalString show different in hive serde and row formet delimited

2021-04-25 Thread angerszhu (Jira)
angerszhu created SPARK-35220: - Summary: DayTimeIntervalType/YearMonthIntervalString show different in hive serde and row formet delimited Key: SPARK-35220 URL: https://issues.apache.org/jira/browse/SPARK-35220

[jira] [Updated] (SPARK-35220) DayTimeIntervalType/YearMonthIntervalString show different in hive serde and row formet delimited

2021-04-25 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-35220: -- Parent: SPARK-31936 Issue Type: Sub-task (was: Bug) > DayTimeIntervalType/YearMonthIntervalSt

[jira] [Created] (SPARK-35221) Add join hint build side check

2021-04-25 Thread ulysses you (Jira)
ulysses you created SPARK-35221: --- Summary: Add join hint build side check Key: SPARK-35221 URL: https://issues.apache.org/jira/browse/SPARK-35221 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35221) Add join hint build side check

2021-04-25 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-35221: Description: Currently we support specify the join implementation with hint, but Spark did not pr

[jira] [Assigned] (SPARK-35220) DayTimeIntervalType/YearMonthIntervalString show different in hive serde and row formet delimited

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35220: Assignee: (was: Apache Spark) > DayTimeIntervalType/YearMonthIntervalString show diff

[jira] [Commented] (SPARK-35220) DayTimeIntervalType/YearMonthIntervalString show different in hive serde and row formet delimited

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331492#comment-17331492 ] Apache Spark commented on SPARK-35220: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-35220) DayTimeIntervalType/YearMonthIntervalString show different in hive serde and row formet delimited

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35220: Assignee: Apache Spark > DayTimeIntervalType/YearMonthIntervalString show different in hi

[jira] [Commented] (SPARK-35220) DayTimeIntervalType/YearMonthIntervalString show different in hive serde and row formet delimited

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331495#comment-17331495 ] Apache Spark commented on SPARK-35220: -- User 'AngersZh' has created a pull requ

[jira] [Updated] (SPARK-35218) abandoned

2021-04-25 Thread Runlin Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Runlin Zhang updated SPARK-35218: - Summary: abandoned (was: [SPARK-35218] In YARN mode, for better user experience, when Spark is

[jira] [Updated] (SPARK-35222) [SPARK-35222] In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to bett

2021-04-25 Thread Runlin Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Runlin Zhang updated SPARK-35222: - Summary: [SPARK-35222] In YARN mode, for better user experience, when Spark is started, not only

[jira] [Created] (SPARK-35222) In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to better track Spark

2021-04-25 Thread Runlin Zhang (Jira)
Runlin Zhang created SPARK-35222: Summary: In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to better track Spark Job Key: SPARK-3

[jira] [Assigned] (SPARK-35222) [SPARK-35222] In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to bet

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35222: Assignee: (was: Apache Spark) > [SPARK-35222] In YARN mode, for better user experienc

[jira] [Assigned] (SPARK-35222) [SPARK-35222] In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to bet

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35222: Assignee: (was: Apache Spark) > [SPARK-35222] In YARN mode, for better user experienc

[jira] [Assigned] (SPARK-35222) [SPARK-35222] In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to bet

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35222: Assignee: Apache Spark > [SPARK-35222] In YARN mode, for better user experience, when Spa

[jira] [Commented] (SPARK-35222) [SPARK-35222] In YARN mode, for better user experience, when Spark is started, not only the AppID is printed, but the Tracking URL is also printed to allow users to be

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331499#comment-17331499 ] Apache Spark commented on SPARK-35222: -- User 'runlin512' has created a pull request

[jira] [Assigned] (SPARK-13587) Support virtualenv in PySpark

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13587: Assignee: Apache Spark > Support virtualenv in PySpark > - >

[jira] [Assigned] (SPARK-13587) Support virtualenv in PySpark

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13587: Assignee: (was: Apache Spark) > Support virtualenv in PySpark > -

[jira] [Assigned] (SPARK-35168) mapred.reduce.tasks should be shuffle.partitions not adaptive.coalescePartitions.initialPartitionNum

2021-04-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-35168: Assignee: Kent Yao > mapred.reduce.tasks should be shuffle.partitions not > adaptive.coalescePar

[jira] [Resolved] (SPARK-35168) mapred.reduce.tasks should be shuffle.partitions not adaptive.coalescePartitions.initialPartitionNum

2021-04-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-35168. -- Fix Version/s: 3.0.3 3.1.2 3.2.0 Resolution: Fixed Issue

[jira] [Created] (SPARK-35223) Add IssueNavigationLink for IDEA

2021-04-25 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-35223: - Summary: Add IssueNavigationLink for IDEA Key: SPARK-35223 URL: https://issues.apache.org/jira/browse/SPARK-35223 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35223) Add IssueNavigationLink for IDEA

2021-04-25 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-35223: -- Description: !image-2021-04-25-22-26-52-006.png! (was: !image-2021-04-25-22-24-00-868.png!) > Add Is

[jira] [Updated] (SPARK-35223) Add IssueNavigationLink for IDEA

2021-04-25 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-35223: -- Attachment: image-2021-04-25-22-26-52-006.png > Add IssueNavigationLink for IDEA > ---

[jira] [Commented] (SPARK-35223) Add IssueNavigationLink for IDEA

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331540#comment-17331540 ] Apache Spark commented on SPARK-35223: -- User 'pan3793' has created a pull request f

[jira] [Assigned] (SPARK-35223) Add IssueNavigationLink for IDEA

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35223: Assignee: (was: Apache Spark) > Add IssueNavigationLink for IDEA > --

[jira] [Assigned] (SPARK-35223) Add IssueNavigationLink for IDEA

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35223: Assignee: Apache Spark > Add IssueNavigationLink for IDEA > -

[jira] [Commented] (SPARK-35223) Add IssueNavigationLink for IDEA

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331541#comment-17331541 ] Apache Spark commented on SPARK-35223: -- User 'pan3793' has created a pull request f

[jira] [Updated] (SPARK-35223) Add IssueNavigationLink for IDEA

2021-04-25 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-35223: -- Affects Version/s: (was: 3.1.1) 3.2.0 > Add IssueNavigationLink for IDEA >

[jira] [Commented] (SPARK-35213) Corrupt DataFrame for certain withField patterns

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331552#comment-17331552 ] Apache Spark commented on SPARK-35213: -- User 'Kimahriman' has created a pull reques

[jira] [Assigned] (SPARK-35213) Corrupt DataFrame for certain withField patterns

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35213: Assignee: Apache Spark > Corrupt DataFrame for certain withField patterns > -

[jira] [Assigned] (SPARK-35213) Corrupt DataFrame for certain withField patterns

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35213: Assignee: (was: Apache Spark) > Corrupt DataFrame for certain withField patterns > --

[jira] [Commented] (SPARK-35213) Corrupt DataFrame for certain withField patterns

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331553#comment-17331553 ] Apache Spark commented on SPARK-35213: -- User 'Kimahriman' has created a pull reques

[jira] [Created] (SPARK-35224) Flaky MutableProjectionSuite

2021-04-25 Thread Max Gekk (Jira)
Max Gekk created SPARK-35224: Summary: Flaky MutableProjectionSuite Key: SPARK-35224 URL: https://issues.apache.org/jira/browse/SPARK-35224 Project: Spark Issue Type: Test Components: S

[jira] [Commented] (SPARK-35224) Flaky MutableProjectionSuite

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331619#comment-17331619 ] Apache Spark commented on SPARK-35224: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-35224) Flaky MutableProjectionSuite

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35224: Assignee: Max Gekk (was: Apache Spark) > Flaky MutableProjectionSuite >

[jira] [Assigned] (SPARK-35224) Flaky MutableProjectionSuite

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35224: Assignee: Apache Spark (was: Max Gekk) > Flaky MutableProjectionSuite >

[jira] [Commented] (SPARK-35224) Flaky MutableProjectionSuite

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331621#comment-17331621 ] Apache Spark commented on SPARK-35224: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-35054) Getting Critical Vulnerability CVE-2021-20231 on spark 3.0.0 branch

2021-04-25 Thread Shashank Jain (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331634#comment-17331634 ] Shashank Jain commented on SPARK-35054: --- Hi [~srowen] i have creating docker for s

[jira] [Commented] (SPARK-35054) Getting Critical Vulnerability CVE-2021-20231 on spark 3.0.0 branch

2021-04-25 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331640#comment-17331640 ] Sean R. Owen commented on SPARK-35054: -- Then this is not related to Spark, but to O

[jira] [Commented] (SPARK-33223) Expose state information on SS UI

2021-04-25 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331657#comment-17331657 ] Xiao Li commented on SPARK-33223: - [~gsomogyi] Could you add the new metrics to the doc?

[jira] [Commented] (SPARK-35139) Support ANSI intervals as Arrow Column vectors

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331672#comment-17331672 ] Apache Spark commented on SPARK-35139: -- User 'Peng-Lei' has created a pull request

[jira] [Assigned] (SPARK-35139) Support ANSI intervals as Arrow Column vectors

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35139: Assignee: Apache Spark > Support ANSI intervals as Arrow Column vectors > ---

[jira] [Assigned] (SPARK-35139) Support ANSI intervals as Arrow Column vectors

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35139: Assignee: (was: Apache Spark) > Support ANSI intervals as Arrow Column vectors >

[jira] [Commented] (SPARK-35139) Support ANSI intervals as Arrow Column vectors

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331673#comment-17331673 ] Apache Spark commented on SPARK-35139: -- User 'Peng-Lei' has created a pull request

[jira] [Resolved] (SPARK-35220) DayTimeIntervalType/YearMonthIntervalString show different in hive serde and row formet delimited

2021-04-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35220. -- Fix Version/s: 3.2.0 Assignee: angerszhu Resolution: Fixed Fixed in [https://g

[jira] [Commented] (SPARK-35212) Spark Streaming LocationStrategy should provide a random option that mapping kafka partitions randomly to spark executors

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331695#comment-17331695 ] Apache Spark commented on SPARK-35212: -- User 'tiehexue' has created a pull request

[jira] [Commented] (SPARK-35212) Spark Streaming LocationStrategy should provide a random option that mapping kafka partitions randomly to spark executors

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331696#comment-17331696 ] Apache Spark commented on SPARK-35212: -- User 'tiehexue' has created a pull request

[jira] [Created] (SPARK-35225) EXPLAIN command should handle empty output of an analyzed plan

2021-04-25 Thread Terry Kim (Jira)
Terry Kim created SPARK-35225: - Summary: EXPLAIN command should handle empty output of an analyzed plan Key: SPARK-35225 URL: https://issues.apache.org/jira/browse/SPARK-35225 Project: Spark Iss

[jira] [Updated] (SPARK-35087) Some columns in table ` Aggregated Metrics by Executor` of stage-detail page shows incorrectly.

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35087: --- Fix Version/s: 3.2.0 3.1.2 > Some columns in table ` Aggregated Metrics

[jira] [Assigned] (SPARK-35087) Some columns in table ` Aggregated Metrics by Executor` of stage-detail page shows incorrectly.

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta reassigned SPARK-35087: -- Assignee: akiyamaneko > Some columns in table ` Aggregated Metrics by Executor` of s

[jira] [Resolved] (SPARK-35087) Some columns in table ` Aggregated Metrics by Executor` of stage-detail page shows incorrectly.

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35087. Resolution: Fixed This issue was resolved in https://github.com/apache/spark/pull/32190.

[jira] [Assigned] (SPARK-35225) EXPLAIN command should handle empty output of an analyzed plan

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35225: Assignee: Apache Spark > EXPLAIN command should handle empty output of an analyzed plan >

[jira] [Commented] (SPARK-35225) EXPLAIN command should handle empty output of an analyzed plan

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331710#comment-17331710 ] Apache Spark commented on SPARK-35225: -- User 'imback82' has created a pull request

[jira] [Assigned] (SPARK-35225) EXPLAIN command should handle empty output of an analyzed plan

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35225: Assignee: (was: Apache Spark) > EXPLAIN command should handle empty output of an anal

[jira] [Commented] (SPARK-35225) EXPLAIN command should handle empty output of an analyzed plan

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331711#comment-17331711 ] Apache Spark commented on SPARK-35225: -- User 'imback82' has created a pull request

[jira] [Commented] (SPARK-35220) DayTimeIntervalType/YearMonthIntervalString show different in hive serde and row formet delimited

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331720#comment-17331720 ] Apache Spark commented on SPARK-35220: -- User 'AngersZh' has created a pull requ

[jira] [Commented] (SPARK-35220) DayTimeIntervalType/YearMonthIntervalString show different in hive serde and row formet delimited

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331721#comment-17331721 ] Apache Spark commented on SPARK-35220: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-32921) Extend MapOutputTracker to support tracking and serving the metadata about each merged shuffle partitions for a given shuffle in push-based shuffle scenario

2021-04-25 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-32921: --- Assignee: Venkata krishnan Sowrirajan > Extend MapOutputTracker to support

[jira] [Resolved] (SPARK-32921) Extend MapOutputTracker to support tracking and serving the metadata about each merged shuffle partitions for a given shuffle in push-based shuffle scenario

2021-04-25 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-32921. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 304

[jira] [Created] (SPARK-35226) JDBCOption should accept refreshKrb5Config parameter

2021-04-25 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35226: -- Summary: JDBCOption should accept refreshKrb5Config parameter Key: SPARK-35226 URL: https://issues.apache.org/jira/browse/SPARK-35226 Project: Spark Issu

[jira] [Updated] (SPARK-35226) JDBCOptions should accept refreshKrb5Config parameter

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35226: --- Summary: JDBCOptions should accept refreshKrb5Config parameter (was: JDBCOption should acce

[jira] [Updated] (SPARK-35226) JDBCOption should accept refreshKrb5Config parameter

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35226: --- Description: In the current master, JDBCOptions can't accept refreshKrb5Config which is def

[jira] [Updated] (SPARK-35226) JDBC datasources should accept refreshKrb5Config parameter

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35226: --- Description: In the current master, JDBC datasources can't accept refreshKrb5Config which is

[jira] [Updated] (SPARK-35226) JDBC datasources should accept refreshKrb5Config parameter

2021-04-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35226: --- Summary: JDBC datasources should accept refreshKrb5Config parameter (was: JDBCOptions shoul

[jira] [Assigned] (SPARK-35226) JDBC datasources should accept refreshKrb5Config parameter

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35226: Assignee: Apache Spark (was: Kousuke Saruta) > JDBC datasources should accept refreshKrb

[jira] [Assigned] (SPARK-35226) JDBC datasources should accept refreshKrb5Config parameter

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35226: Assignee: Kousuke Saruta (was: Apache Spark) > JDBC datasources should accept refreshKrb

[jira] [Commented] (SPARK-35226) JDBC datasources should accept refreshKrb5Config parameter

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331772#comment-17331772 ] Apache Spark commented on SPARK-35226: -- User 'sarutak' has created a pull request f

[jira] [Commented] (SPARK-35085) Get columns operation should handle ANSI interval column properly

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331783#comment-17331783 ] Apache Spark commented on SPARK-35085: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-35085) Get columns operation should handle ANSI interval column properly

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35085: Assignee: (was: Apache Spark) > Get columns operation should handle ANSI interval col

[jira] [Assigned] (SPARK-35085) Get columns operation should handle ANSI interval column properly

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35085: Assignee: Apache Spark > Get columns operation should handle ANSI interval column properl

[jira] [Commented] (SPARK-35085) Get columns operation should handle ANSI interval column properly

2021-04-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331784#comment-17331784 ] Apache Spark commented on SPARK-35085: -- User 'beliefer' has created a pull request

[jira] [Resolved] (SPARK-35224) Flaky MutableProjectionSuite

2021-04-25 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35224. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32339 [https://github.com

[jira] [Resolved] (SPARK-35213) Corrupt DataFrame for certain withField patterns

2021-04-25 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35213. - Fix Version/s: 3.1.2 3.2.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-35213) Corrupt DataFrame for certain withField patterns

2021-04-25 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35213: --- Assignee: Adam Binford > Corrupt DataFrame for certain withField patterns > ---

[jira] [Created] (SPARK-35227) Replace Bintray with the new repository service for the spark-packages resolver in SparkSubmit

2021-04-25 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-35227: Summary: Replace Bintray with the new repository service for the spark-packages resolver in SparkSubmit Key: SPARK-35227 URL: https://issues.apache.org/jira/browse/SPARK-35227

[jira] [Updated] (SPARK-35227) Replace Bintray with the new repository service for the spark-packages resolver in SparkSubmit

2021-04-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35227: - Affects Version/s: (was: 3.1.1) (was: 3.0.2)

  1   2   >