[jira] [Commented] (SPARK-36560) Deflake PySpark coverage report

[ https://issues.apache.org/jira/browse/SPARK-36560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402987#comment-17402987 ] Apache Spark commented on SPARK-36560: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-36560) Deflake PySpark coverage report

[ https://issues.apache.org/jira/browse/SPARK-36560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36560: Assignee: Apache Spark > Deflake PySpark coverage report > --

[jira] [Commented] (SPARK-36560) Deflake PySpark coverage report

[ https://issues.apache.org/jira/browse/SPARK-36560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402986#comment-17402986 ] Apache Spark commented on SPARK-36560: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-36560) Deflake PySpark coverage report

[ https://issues.apache.org/jira/browse/SPARK-36560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36560: Assignee: (was: Apache Spark) > Deflake PySpark coverage report > ---

[jira] [Updated] (SPARK-36560) Deflake PySpark coverage report

[ https://issues.apache.org/jira/browse/SPARK-36560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36560: - Priority: Minor (was: Major) > Deflake PySpark coverage report > --

[jira] [Updated] (SPARK-36560) Deflake PySpark coverage report

[ https://issues.apache.org/jira/browse/SPARK-36560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36560: - Description: https://github.com/apache/spark/runs/3388727798?check_suite_focus=true https://gith

[jira] [Created] (SPARK-36560) Deflake PySpark coverage report

Hyukjin Kwon created SPARK-36560: Summary: Deflake PySpark coverage report Key: SPARK-36560 URL: https://issues.apache.org/jira/browse/SPARK-36560 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-36559) Allow column pruning on distributed sequence index (pandas API on Spark)

[ https://issues.apache.org/jira/browse/SPARK-36559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36559: Assignee: (was: Apache Spark) > Allow column pruning on distributed sequence index (p

[jira] [Assigned] (SPARK-36559) Allow column pruning on distributed sequence index (pandas API on Spark)

[ https://issues.apache.org/jira/browse/SPARK-36559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36559: Assignee: Apache Spark > Allow column pruning on distributed sequence index (pandas API o

[jira] [Commented] (SPARK-36559) Allow column pruning on distributed sequence index (pandas API on Spark)

[ https://issues.apache.org/jira/browse/SPARK-36559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402972#comment-17402972 ] Apache Spark commented on SPARK-36559: -- User 'HyukjinKwon' has created a pull reque

[jira] [Created] (SPARK-36559) Allow column pruning on distributed sequence index (pandas API on Spark)

Hyukjin Kwon created SPARK-36559: Summary: Allow column pruning on distributed sequence index (pandas API on Spark) Key: SPARK-36559 URL: https://issues.apache.org/jira/browse/SPARK-36559 Project: Spa

[jira] [Updated] (SPARK-36457) Review and fix issues in API docs

[ https://issues.apache.org/jira/browse/SPARK-36457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-36457: Priority: Blocker (was: Critical) > Review and fix issues in API docs > -

[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402964#comment-17402964 ] Dongjoon Hyun commented on SPARK-25075: --- [~LuciferYang]. Not yet although I believ

[jira] [Commented] (SPARK-36558) Stage has all tasks finished but with ongoing finalization can cause job hang

[ https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402960#comment-17402960 ] Gengliang Wang commented on SPARK-36558: [~Ngone51] Nice catch. I will cut RC2 a

[jira] [Assigned] (SPARK-36304) Refactor fifteenth set of 20 query execution errors to use error classes

[ https://issues.apache.org/jira/browse/SPARK-36304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36304: Assignee: Apache Spark > Refactor fifteenth set of 20 query execution errors to use error

[jira] [Assigned] (SPARK-36304) Refactor fifteenth set of 20 query execution errors to use error classes

[ https://issues.apache.org/jira/browse/SPARK-36304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36304: Assignee: (was: Apache Spark) > Refactor fifteenth set of 20 query execution errors t

[jira] [Commented] (SPARK-36304) Refactor fifteenth set of 20 query execution errors to use error classes

[ https://issues.apache.org/jira/browse/SPARK-36304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402953#comment-17402953 ] Apache Spark commented on SPARK-36304: -- User 'dgd-contributor' has created a pull r

[jira] [Commented] (SPARK-36429) JacksonParser should throw exception when data type unsupported.

[ https://issues.apache.org/jira/browse/SPARK-36429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402951#comment-17402951 ] Kousuke Saruta commented on SPARK-36429: Issue resolved in https://github.com/ap

[jira] [Updated] (SPARK-36558) Stage has all tasks finished but with ongoing finalization can cause job hang

[ https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-36558: - Description:   For a stage that all tasks are finished but with ongoing finalization can lead to job hang. The

[jira] [Updated] (SPARK-36558) Stage has all tasks finished but with ongoing finalization can cause job hang

[ https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-36558: - Description:   For a stage that all tasks are finished but with ongoing finalization can lead to job hang. The

[jira] [Commented] (SPARK-36558) Stage has all tasks finished but with ongoing finalization can cause job hang

[ https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402944#comment-17402944 ] Min Shen commented on SPARK-36558: -- This is an issue we previously resolved internally.

[jira] [Commented] (SPARK-36558) Stage has all tasks finished but with ongoing finalization can cause job hang

[ https://issues.apache.org/jira/browse/SPARK-36558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402942#comment-17402942 ] wuyi commented on SPARK-36558: -- cc [~mridul] [~mshen] [~csingh] > Stage has all tasks fini

[jira] [Created] (SPARK-36558) Stage has all tasks finished but with ongoing finalization can cause job hang

wuyi created SPARK-36558: Summary: Stage has all tasks finished but with ongoing finalization can cause job hang Key: SPARK-36558 URL: https://issues.apache.org/jira/browse/SPARK-36558 Project: Spark

[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402927#comment-17402927 ] Yang Jie commented on SPARK-25075: -- Do we plan to use Scala 2.13 as the default version

[jira] [Commented] (SPARK-36302) Refactor thirteenth set of 20 query execution errors to use error classes

[ https://issues.apache.org/jira/browse/SPARK-36302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402911#comment-17402911 ] Apache Spark commented on SPARK-36302: -- User 'dgd-contributor' has created a pull r

[jira] [Commented] (SPARK-36302) Refactor thirteenth set of 20 query execution errors to use error classes

[ https://issues.apache.org/jira/browse/SPARK-36302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402910#comment-17402910 ] Apache Spark commented on SPARK-36302: -- User 'dgd-contributor' has created a pull r

[jira] [Resolved] (SPARK-36542) Unable to create executor due to unable to connect to external shuffle server

[ https://issues.apache.org/jira/browse/SPARK-36542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36542. -- Resolution: Invalid > Unable to create executor due to unable to connect to external shuffle s

[jira] [Commented] (SPARK-36542) Unable to create executor due to unable to connect to external shuffle server

[ https://issues.apache.org/jira/browse/SPARK-36542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402908#comment-17402908 ] Hyukjin Kwon commented on SPARK-36542: -- sounds more like a question instead of an i

[jira] [Commented] (SPARK-36546) Make unionByName null-filling behavior work with array of struct columns

[ https://issues.apache.org/jira/browse/SPARK-36546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402907#comment-17402907 ] Hyukjin Kwon commented on SPARK-36546: -- I think this is fixed in the latest master

[jira] [Resolved] (SPARK-36544) Special strings for dates are no longer automatically converted without the "date" keyword

[ https://issues.apache.org/jira/browse/SPARK-36544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36544. -- Resolution: Duplicate > Special strings for dates are no longer automatically converted withou

[jira] [Commented] (SPARK-36545) sparkstreaming input rate exceed spark.streaming.kafka.maxRatePerPartition

[ https://issues.apache.org/jira/browse/SPARK-36545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402906#comment-17402906 ] Hyukjin Kwon commented on SPARK-36545: -- Seems like more a question. Let's ask it in

[jira] [Resolved] (SPARK-36545) sparkstreaming input rate exceed spark.streaming.kafka.maxRatePerPartition

[ https://issues.apache.org/jira/browse/SPARK-36545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36545. -- Resolution: Invalid > sparkstreaming input rate exceed spark.streaming.kafka.maxRatePerPartit

[jira] [Commented] (SPARK-36554) Error message while trying to use spark sql functions directly on dataframe columns without using select expression

[ https://issues.apache.org/jira/browse/SPARK-36554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402905#comment-17402905 ] Hyukjin Kwon commented on SPARK-36554: -- {{make_date}} is simply missing under {{fun

[jira] [Updated] (SPARK-36470) Implement `CategoricalIndex.map` and `DatetimeIndx.map`

[ https://issues.apache.org/jira/browse/SPARK-36470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36470: - Fix Version/s: (was: 3.3.0) 3.2.0 > Implement `CategoricalIndex.map` and

[jira] [Resolved] (SPARK-36470) Implement `CategoricalIndex.map` and `DatetimeIndx.map`

[ https://issues.apache.org/jira/browse/SPARK-36470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36470. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33756 [https://gi

[jira] [Assigned] (SPARK-36470) Implement `CategoricalIndex.map` and `DatetimeIndx.map`

[ https://issues.apache.org/jira/browse/SPARK-36470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36470: Assignee: Xinrong Meng > Implement `CategoricalIndex.map` and `DatetimeIndx.map` > --

[jira] [Resolved] (SPARK-36557) Update the MAVEN_OPTS in Spark build docs

[ https://issues.apache.org/jira/browse/SPARK-36557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36557. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33804 [https://gi

[jira] [Commented] (SPARK-36557) Update the MAVEN_OPTS in Spark build docs

[ https://issues.apache.org/jira/browse/SPARK-36557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402865#comment-17402865 ] Apache Spark commented on SPARK-36557: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-36557) Update the MAVEN_OPTS in Spark build docs

[ https://issues.apache.org/jira/browse/SPARK-36557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36557: Assignee: Apache Spark (was: Gengliang Wang) > Update the MAVEN_OPTS in Spark build docs

[jira] [Assigned] (SPARK-36557) Update the MAVEN_OPTS in Spark build docs

[ https://issues.apache.org/jira/browse/SPARK-36557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36557: Assignee: Gengliang Wang (was: Apache Spark) > Update the MAVEN_OPTS in Spark build docs

[jira] [Updated] (SPARK-36557) Update the MAVEN_OPTS in Spark build docs

[ https://issues.apache.org/jira/browse/SPARK-36557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-36557: --- Summary: Update the MAVEN_OPTS in Spark build docs (was: Update the docs for MAVEN_OPTS of

[jira] [Created] (SPARK-36557) Update the docs for MAVEN_OPTS of Spark build

Gengliang Wang created SPARK-36557: -- Summary: Update the docs for MAVEN_OPTS of Spark build Key: SPARK-36557 URL: https://issues.apache.org/jira/browse/SPARK-36557 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36557) Update the docs for MAVEN_OPTS of Spark build

[ https://issues.apache.org/jira/browse/SPARK-36557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-36557: --- Issue Type: Task (was: Bug) > Update the docs for MAVEN_OPTS of Spark build > -

[jira] [Assigned] (SPARK-36556) Add DSV2 filters

[ https://issues.apache.org/jira/browse/SPARK-36556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36556: Assignee: (was: Apache Spark) > Add DSV2 filters > > >

[jira] [Assigned] (SPARK-36556) Add DSV2 filters

[ https://issues.apache.org/jira/browse/SPARK-36556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36556: Assignee: Apache Spark > Add DSV2 filters > > > Key: SPA

[jira] [Commented] (SPARK-36556) Add DSV2 filters

[ https://issues.apache.org/jira/browse/SPARK-36556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402831#comment-17402831 ] Apache Spark commented on SPARK-36556: -- User 'huaxingao' has created a pull request

[jira] [Created] (SPARK-36556) Add DSV2 filters

Huaxin Gao created SPARK-36556: -- Summary: Add DSV2 filters Key: SPARK-36556 URL: https://issues.apache.org/jira/browse/SPARK-36556 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Created] (SPARK-36555) DS V2 Filter support

Huaxin Gao created SPARK-36555: -- Summary: DS V2 Filter support Key: SPARK-36555 URL: https://issues.apache.org/jira/browse/SPARK-36555 Project: Spark Issue Type: Improvement Components