[jira] [Resolved] (SPARK-38423) Support priority scheduling with volcano implementations

[ https://issues.apache.org/jira/browse/SPARK-38423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38423. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35639

[jira] [Assigned] (SPARK-38423) Support priority scheduling with volcano implementations

[ https://issues.apache.org/jira/browse/SPARK-38423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38423: - Assignee: Yikun Jiang > Support priority scheduling with volcano implementations >

[jira] [Commented] (SPARK-38432) Reactor framework so as JDBC dialect could compile filter by self way

[ https://issues.apache.org/jira/browse/SPARK-38432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502112#comment-17502112 ] Apache Spark commented on SPARK-38432: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38432) Reactor framework so as JDBC dialect could compile filter by self way

[ https://issues.apache.org/jira/browse/SPARK-38432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38432: Assignee: (was: Apache Spark) > Reactor framework so as JDBC dialect could compile

[jira] [Assigned] (SPARK-38432) Reactor framework so as JDBC dialect could compile filter by self way

[ https://issues.apache.org/jira/browse/SPARK-38432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38432: Assignee: Apache Spark > Reactor framework so as JDBC dialect could compile filter by

[jira] [Created] (SPARK-38432) Reactor framework so as JDBC dialect could compile filter by self way

jiaan.geng created SPARK-38432: -- Summary: Reactor framework so as JDBC dialect could compile filter by self way Key: SPARK-38432 URL: https://issues.apache.org/jira/browse/SPARK-38432 Project: Spark

[jira] [Created] (SPARK-38431) Support to delete matched rows from jdbc tables

caican created SPARK-38431: -- Summary: Support to delete matched rows from jdbc tables Key: SPARK-38431 URL: https://issues.apache.org/jira/browse/SPARK-38431 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-38430) Add SBT commands to K8s IT readme

[ https://issues.apache.org/jira/browse/SPARK-38430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38430: - Assignee: William Hyun > Add SBT commands to K8s IT readme >

[jira] [Resolved] (SPARK-38430) Add SBT commands to K8s IT readme

[ https://issues.apache.org/jira/browse/SPARK-38430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38430. --- Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-38430) Add SBT commands to K8s IT readme

[ https://issues.apache.org/jira/browse/SPARK-38430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38430: Assignee: Apache Spark > Add SBT commands to K8s IT readme >

[jira] [Commented] (SPARK-38430) Add SBT commands to K8s IT readme

[ https://issues.apache.org/jira/browse/SPARK-38430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502081#comment-17502081 ] Apache Spark commented on SPARK-38430: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-38430) Add SBT commands to K8s IT readme

[ https://issues.apache.org/jira/browse/SPARK-38430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38430: Assignee: (was: Apache Spark) > Add SBT commands to K8s IT readme >

[jira] [Created] (SPARK-38430) Add SBT commands to K8s IT readme

William Hyun created SPARK-38430: Summary: Add SBT commands to K8s IT readme Key: SPARK-38430 URL: https://issues.apache.org/jira/browse/SPARK-38430 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-37383) Print the parsing time for each phase of a SQL

[ https://issues.apache.org/jira/browse/SPARK-37383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502080#comment-17502080 ] Apache Spark commented on SPARK-37383: -- User 'caican00' has created a pull request for this issue:

[jira] [Commented] (SPARK-37383) Print the parsing time for each phase of a SQL

[ https://issues.apache.org/jira/browse/SPARK-37383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502079#comment-17502079 ] Apache Spark commented on SPARK-37383: -- User 'caican00' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38428) Check the FetchShuffleBlocks message only once to improve iteration in external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-38428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38428: Assignee: (was: Apache Spark) > Check the FetchShuffleBlocks message only once to

[jira] [Commented] (SPARK-38428) Check the FetchShuffleBlocks message only once to improve iteration in external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-38428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502072#comment-17502072 ] Apache Spark commented on SPARK-38428: -- User 'weixiuli' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38428) Check the FetchShuffleBlocks message only once to improve iteration in external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-38428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38428: Assignee: Apache Spark > Check the FetchShuffleBlocks message only once to improve

[jira] [Assigned] (SPARK-38428) Check the FetchShuffleBlocks message only once to improve iteration in external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-38428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38428: Assignee: Apache Spark > Check the FetchShuffleBlocks message only once to improve

[jira] [Commented] (SPARK-38429) Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1

[ https://issues.apache.org/jira/browse/SPARK-38429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502071#comment-17502071 ] JacobZheng commented on SPARK-38429: The hadoop jars inside package is version 3.3.1 > Why the

[jira] [Updated] (SPARK-38428) Check the FetchShuffleBlocks message only once to improve iteration in external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-38428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-38428: - Summary: Check the FetchShuffleBlocks message only once to improve iteration in external shuffle

[jira] [Updated] (SPARK-38429) Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1

[ https://issues.apache.org/jira/browse/SPARK-38429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-38429: --- Description: [https://spark.apache.org/downloads.html]  !11271646623853_.pic.jpg! was:

[jira] [Updated] (SPARK-38429) Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1

[ https://issues.apache.org/jira/browse/SPARK-38429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-38429: --- Issue Type: Question (was: Improvement) > Why the latest release named

[jira] [Updated] (SPARK-38429) Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1

[ https://issues.apache.org/jira/browse/SPARK-38429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-38429: --- Attachment: 11271646623853_.pic.jpg > Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz

[jira] [Created] (SPARK-38429) Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1

JacobZheng created SPARK-38429: -- Summary: Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1 Key: SPARK-38429 URL: https://issues.apache.org/jira/browse/SPARK-38429

[jira] [Created] (SPARK-38428) Improve FetchShuffleBlocks in External shuffle service

weixiuli created SPARK-38428: Summary: Improve FetchShuffleBlocks in External shuffle service Key: SPARK-38428 URL: https://issues.apache.org/jira/browse/SPARK-38428 Project: Spark Issue Type:

[jira] [Created] (SPARK-38427) DataFilter pushed down with PartitionFilter for Orc

Jackey Lee created SPARK-38427: -- Summary: DataFilter pushed down with PartitionFilter for Orc Key: SPARK-38427 URL: https://issues.apache.org/jira/browse/SPARK-38427 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-38405) Supports customization of the entire optimizer and planner.

[ https://issues.apache.org/jira/browse/SPARK-38405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38405: Assignee: (was: Apache Spark) > Supports customization of the entire optimizer and

[jira] [Commented] (SPARK-38405) Supports customization of the entire optimizer and planner.

[ https://issues.apache.org/jira/browse/SPARK-38405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502062#comment-17502062 ] Apache Spark commented on SPARK-38405: -- User 'chenjunbiao001' has created a pull request for this

[jira] [Assigned] (SPARK-38405) Supports customization of the entire optimizer and planner.

[ https://issues.apache.org/jira/browse/SPARK-38405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38405: Assignee: Apache Spark > Supports customization of the entire optimizer and planner. >

[jira] [Commented] (SPARK-38399) Why doesn't shuffle hash join support build left table for left-outer-join but full-outer-join?

[ https://issues.apache.org/jira/browse/SPARK-38399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502059#comment-17502059 ] Hyukjin Kwon commented on SPARK-38399: -- [~mengdou] let's ask questions to the Spark user/dev

[jira] [Resolved] (SPARK-38399) Why doesn't shuffle hash join support build left table for left-outer-join but full-outer-join?

[ https://issues.apache.org/jira/browse/SPARK-38399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38399. -- Resolution: Invalid > Why doesn't shuffle hash join support build left table for

[jira] [Commented] (SPARK-38402) Improve user experience when working on data frames created from CSV and JSON in PERMISSIVE mode.

[ https://issues.apache.org/jira/browse/SPARK-38402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502058#comment-17502058 ] Hyukjin Kwon commented on SPARK-38402: -- {quote} we currently cache the data frame before using it ,

[jira] [Commented] (SPARK-38413) Invalid call to exprId

[ https://issues.apache.org/jira/browse/SPARK-38413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502053#comment-17502053 ] Hyukjin Kwon commented on SPARK-38413: -- How do you reproduce this? Also Spark 2.4 is EOL. Can you

[jira] [Commented] (SPARK-38420) Upgrade bcprov-jdk15on from 1.60 to 1.67

[ https://issues.apache.org/jira/browse/SPARK-38420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502052#comment-17502052 ] Hyukjin Kwon commented on SPARK-38420: -- bcprov-jdk15 is not bundled with Spark. we only use it for

[jira] [Resolved] (SPARK-38420) Upgrade bcprov-jdk15on from 1.60 to 1.67

[ https://issues.apache.org/jira/browse/SPARK-38420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38420. -- Resolution: Invalid > Upgrade bcprov-jdk15on from 1.60 to 1.67 >

[jira] [Resolved] (SPARK-38419) Remove tab character and trailing space in script

[ https://issues.apache.org/jira/browse/SPARK-38419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38419. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35738

[jira] [Assigned] (SPARK-38419) Remove tab character and trailing space in script

[ https://issues.apache.org/jira/browse/SPARK-38419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38419: Assignee: Jackey Lee > Remove tab character and trailing space in script >

[jira] [Updated] (SPARK-38419) Remove tab character and trailing space in script

[ https://issues.apache.org/jira/browse/SPARK-38419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38419: - Issue Type: Task (was: Bug) Priority: Trivial (was: Major) > Remove tab character and

[jira] [Resolved] (SPARK-38267) Replace pattern matches on boolean expressions with conditional statements

[ https://issues.apache.org/jira/browse/SPARK-38267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38267. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35589

[jira] [Assigned] (SPARK-38267) Replace pattern matches on boolean expressions with conditional statements

[ https://issues.apache.org/jira/browse/SPARK-38267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38267: Assignee: Yang Jie > Replace pattern matches on boolean expressions with conditional

[jira] [Updated] (SPARK-38267) Replace pattern matches on boolean expressions with conditional statements

[ https://issues.apache.org/jira/browse/SPARK-38267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38267: - Priority: Trivial (was: Minor) > Replace pattern matches on boolean expressions with

[jira] [Resolved] (SPARK-38394) build of spark sql against hadoop-3.4.0-snapshot failing with bouncycastle classpath error

[ https://issues.apache.org/jira/browse/SPARK-38394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38394. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35725

[jira] [Assigned] (SPARK-38394) build of spark sql against hadoop-3.4.0-snapshot failing with bouncycastle classpath error

[ https://issues.apache.org/jira/browse/SPARK-38394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38394: Assignee: Steve Loughran > build of spark sql against hadoop-3.4.0-snapshot failing with

[jira] [Resolved] (SPARK-38416) Duplicate functions

[ https://issues.apache.org/jira/browse/SPARK-38416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38416. -- Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-38416) Duplicate functions

[ https://issues.apache.org/jira/browse/SPARK-38416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38416: Assignee: Bjørn Jørgensen > Duplicate functions > --- > >

[jira] [Assigned] (SPARK-38411) Use UTF-8 when doMergeApplicationListingInternal reads event logs

[ https://issues.apache.org/jira/browse/SPARK-38411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38411: - Assignee: Cheng Pan > Use UTF-8 when doMergeApplicationListingInternal reads event

[jira] [Resolved] (SPARK-38411) Use UTF-8 when doMergeApplicationListingInternal reads event logs

[ https://issues.apache.org/jira/browse/SPARK-38411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38411. --- Fix Version/s: 3.3.0 3.0.4 3.2.2

[jira] [Updated] (SPARK-38411) Use UTF-8 when doMergeApplicationListingInternal reads event logs

[ https://issues.apache.org/jira/browse/SPARK-38411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38411: -- Affects Version/s: 3.0.3 > Use UTF-8 when doMergeApplicationListingInternal reads event logs

[jira] [Updated] (SPARK-38411) Use UTF-8 when doMergeApplicationListingInternal reads event logs

[ https://issues.apache.org/jira/browse/SPARK-38411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38411: -- Summary: Use UTF-8 when doMergeApplicationListingInternal reads event logs (was: Use UTF-8

[jira] [Resolved] (SPARK-38239) AttributeError: 'LogisticRegressionModel' object has no attribute '_call_java'

[ https://issues.apache.org/jira/browse/SPARK-38239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38239. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35554

[jira] [Assigned] (SPARK-38239) AttributeError: 'LogisticRegressionModel' object has no attribute '_call_java'

[ https://issues.apache.org/jira/browse/SPARK-38239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38239: Assignee: Maciej Szymkiewicz > AttributeError: 'LogisticRegressionModel' object has no

[jira] [Assigned] (SPARK-38426) Fix the permissions for GitHub workflows

[ https://issues.apache.org/jira/browse/SPARK-38426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38426: Assignee: Apache Spark > Fix the permissions for GitHub workflows >

[jira] [Commented] (SPARK-38426) Fix the permissions for GitHub workflows

[ https://issues.apache.org/jira/browse/SPARK-38426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17501987#comment-17501987 ] Apache Spark commented on SPARK-38426: -- User 'naveensrinivasan' has created a pull request for this

[jira] [Assigned] (SPARK-38426) Fix the permissions for GitHub workflows

[ https://issues.apache.org/jira/browse/SPARK-38426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38426: Assignee: (was: Apache Spark) > Fix the permissions for GitHub workflows >

[jira] [Created] (SPARK-38426) Fix the permissions for GitHub workflows

Naveen S created SPARK-38426: Summary: Fix the permissions for GitHub workflows Key: SPARK-38426 URL: https://issues.apache.org/jira/browse/SPARK-38426 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-38416) Duplicate functions

[ https://issues.apache.org/jira/browse/SPARK-38416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38416: Assignee: Apache Spark > Duplicate functions > --- > >

[jira] [Assigned] (SPARK-38416) Duplicate functions

[ https://issues.apache.org/jira/browse/SPARK-38416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38416: Assignee: (was: Apache Spark) > Duplicate functions > --- > >

[jira] [Commented] (SPARK-38416) Duplicate functions

[ https://issues.apache.org/jira/browse/SPARK-38416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17501965#comment-17501965 ] Apache Spark commented on SPARK-38416: -- User 'bjornjorgensen' has created a pull request for this

[jira] [Created] (SPARK-38425) Avoid possible errors due to incorrect file size or type supplied in spark conf.

luyuan created SPARK-38425: -- Summary: Avoid possible errors due to incorrect file size or type supplied in spark conf. Key: SPARK-38425 URL: https://issues.apache.org/jira/browse/SPARK-38425 Project: Spark

[jira] [Commented] (SPARK-38355) Change mktemp() to mkstemp()

[ https://issues.apache.org/jira/browse/SPARK-38355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17501958#comment-17501958 ] Bjørn Jørgensen commented on SPARK-38355: - Well I did guess that this one was an easy one. But

[jira] [Commented] (SPARK-38424) Disallow unused casts and ignores

[ https://issues.apache.org/jira/browse/SPARK-38424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17501950#comment-17501950 ] Apache Spark commented on SPARK-38424: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38424) Disallow unused casts and ignores

[ https://issues.apache.org/jira/browse/SPARK-38424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38424: Assignee: (was: Apache Spark) > Disallow unused casts and ignores >

[jira] [Assigned] (SPARK-38424) Disallow unused casts and ignores

[ https://issues.apache.org/jira/browse/SPARK-38424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38424: Assignee: Apache Spark > Disallow unused casts and ignores >

[jira] [Created] (SPARK-38424) Disallow unused casts and ignores

Maciej Szymkiewicz created SPARK-38424: -- Summary: Disallow unused casts and ignores Key: SPARK-38424 URL: https://issues.apache.org/jira/browse/SPARK-38424 Project: Spark Issue Type:

[jira] [Commented] (SPARK-38416) Duplicate functions

[ https://issues.apache.org/jira/browse/SPARK-38416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17501907#comment-17501907 ] Yikun Jiang commented on SPARK-38416: - Yep, this might be a bad copy paste, are you going to submit

[jira] [Commented] (SPARK-38388) Repartition + Stage retries could lead to incorrect data

[ https://issues.apache.org/jira/browse/SPARK-38388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17501894#comment-17501894 ] Jason Xu commented on SPARK-38388: -- [~jiangxb1987] thanks for reply, above repo example uses DataFrame