[jira] [Resolved] (SPARK-34585) Remove BatchWriteHelper completely

[ https://issues.apache.org/jira/browse/SPARK-34585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34585. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31699 [https://gi

[jira] [Assigned] (SPARK-34585) Remove BatchWriteHelper completely

[ https://issues.apache.org/jira/browse/SPARK-34585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34585: Assignee: Anton Okolnychyi > Remove BatchWriteHelper completely > ---

[jira] [Commented] (SPARK-34580) Provide the relationship between batch ID and SQL executions (and/or Jobs) in SS UI page

[ https://issues.apache.org/jira/browse/SPARK-34580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293476#comment-17293476 ] Hyukjin Kwon commented on SPARK-34580: -- Thanks [~gsomogyi]! > Provide the relation

[jira] [Resolved] (SPARK-34578) Ignore ORC encryption tests when ORC is loaded by old Hadoop library by other tests

[ https://issues.apache.org/jira/browse/SPARK-34578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34578. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31697 [https://gi

[jira] [Assigned] (SPARK-32703) Replace deprecated API calls from SpecificParquetRecordReaderBase

[ https://issues.apache.org/jira/browse/SPARK-32703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32703: Assignee: Chao Sun > Replace deprecated API calls from SpecificParquetRecordReaderBase >

[jira] [Resolved] (SPARK-32703) Replace deprecated API calls from SpecificParquetRecordReaderBase

[ https://issues.apache.org/jira/browse/SPARK-32703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32703. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31667 [https://gi

[jira] [Assigned] (SPARK-34578) Ignore ORC encryption tests when ORC is loaded by old Hadoop library by other tests

[ https://issues.apache.org/jira/browse/SPARK-34578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34578: Assignee: Dongjoon Hyun > Ignore ORC encryption tests when ORC is loaded by old Hadoop li

[jira] [Assigned] (SPARK-33678) Numerical product aggregation

[ https://issues.apache.org/jira/browse/SPARK-33678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33678: Assignee: Richard Penney > Numerical product aggregation > -

[jira] [Resolved] (SPARK-33678) Numerical product aggregation

[ https://issues.apache.org/jira/browse/SPARK-33678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33678. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30745 [https://gi

[jira] [Commented] (SPARK-34590) Allow JDWP debug for tests with sbt

[ https://issues.apache.org/jira/browse/SPARK-34590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293469#comment-17293469 ] Apache Spark commented on SPARK-34590: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-34590) Allow JDWP debug for tests with sbt

[ https://issues.apache.org/jira/browse/SPARK-34590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34590: Assignee: Apache Spark (was: Kousuke Saruta) > Allow JDWP debug for tests with sbt > ---

[jira] [Assigned] (SPARK-34590) Allow JDWP debug for tests with sbt

[ https://issues.apache.org/jira/browse/SPARK-34590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34590: Assignee: Kousuke Saruta (was: Apache Spark) > Allow JDWP debug for tests with sbt > ---

[jira] [Commented] (SPARK-34590) Allow JDWP debug for tests with sbt

[ https://issues.apache.org/jira/browse/SPARK-34590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293468#comment-17293468 ] Apache Spark commented on SPARK-34590: -- User 'sarutak' has created a pull request f

[jira] [Resolved] (SPARK-34589) Make BypassMergeSortShuffleWriter close partitionWriters in case of success

[ https://issues.apache.org/jira/browse/SPARK-34589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34589. --- Resolution: Invalid > Make BypassMergeSortShuffleWriter close partitionWriters in case of su

[jira] [Created] (SPARK-34590) Allow JDWP debug for tests with sbt

Kousuke Saruta created SPARK-34590: -- Summary: Allow JDWP debug for tests with sbt Key: SPARK-34590 URL: https://issues.apache.org/jira/browse/SPARK-34590 Project: Spark Issue Type: Improveme

[jira] [Commented] (SPARK-34577) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE NAMESPACE`

[ https://issues.apache.org/jira/browse/SPARK-34577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293429#comment-17293429 ] Apache Spark commented on SPARK-34577: -- User 'AngersZh' has created a pull requ

[jira] [Commented] (SPARK-34577) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE NAMESPACE`

[ https://issues.apache.org/jira/browse/SPARK-34577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293426#comment-17293426 ] Apache Spark commented on SPARK-34577: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-34577) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE NAMESPACE`

[ https://issues.apache.org/jira/browse/SPARK-34577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34577: Assignee: (was: Apache Spark) > Cannot drop/add columns from/to a dataset of v2 `DESC

[jira] [Assigned] (SPARK-34577) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE NAMESPACE`

[ https://issues.apache.org/jira/browse/SPARK-34577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34577: Assignee: Apache Spark > Cannot drop/add columns from/to a dataset of v2 `DESCRIBE NAMESP

[jira] [Updated] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34544: - Target Version/s: 3.2.0 (was: 3.2.0, 3.1.2) > pyspark toPandas() should return pd.DataFrame > -

[jira] [Commented] (SPARK-34588) Support int64 buffer lengths in Java for pyspark Pandas UDF as buffer expanding

[ https://issues.apache.org/jira/browse/SPARK-34588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293421#comment-17293421 ] Dmitry Kravchuk commented on SPARK-34588: - [~apachespark] ping > Support int64

[jira] [Commented] (SPARK-34589) Make BypassMergeSortShuffleWriter close partitionWriters in case of success

[ https://issues.apache.org/jira/browse/SPARK-34589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293420#comment-17293420 ] Apache Spark commented on SPARK-34589: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-34589) Make BypassMergeSortShuffleWriter close partitionWriters in case of success

[ https://issues.apache.org/jira/browse/SPARK-34589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293419#comment-17293419 ] Apache Spark commented on SPARK-34589: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-34589) Make BypassMergeSortShuffleWriter close partitionWriters in case of success

[ https://issues.apache.org/jira/browse/SPARK-34589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34589: Assignee: Apache Spark > Make BypassMergeSortShuffleWriter close partitionWriters in case

[jira] [Assigned] (SPARK-34589) Make BypassMergeSortShuffleWriter close partitionWriters in case of success

[ https://issues.apache.org/jira/browse/SPARK-34589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34589: Assignee: (was: Apache Spark) > Make BypassMergeSortShuffleWriter close partitionWrit

[jira] [Commented] (SPARK-34587) Replaces map and getOrElse(false/true) with exists/forall

[ https://issues.apache.org/jira/browse/SPARK-34587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293417#comment-17293417 ] Apache Spark commented on SPARK-34587: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-34587) Replaces map and getOrElse(false/true) with exists/forall

[ https://issues.apache.org/jira/browse/SPARK-34587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293415#comment-17293415 ] Apache Spark commented on SPARK-34587: -- User 'LuciferYang' has created a pull reque

[jira] [Created] (SPARK-34589) Make BypassMergeSortShuffleWriter close partitionWriters in case of success

Dongjoon Hyun created SPARK-34589: - Summary: Make BypassMergeSortShuffleWriter close partitionWriters in case of success Key: SPARK-34589 URL: https://issues.apache.org/jira/browse/SPARK-34589 Project

[jira] [Commented] (SPARK-34587) Replaces map and getOrElse(false/true) with exists/forall

[ https://issues.apache.org/jira/browse/SPARK-34587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293408#comment-17293408 ] Apache Spark commented on SPARK-34587: -- User 'LuciferYang' has created a pull reque

[jira] [Resolved] (SPARK-34573) SQLConf sqlConfEntries map has a global lock, should not lock on get

[ https://issues.apache.org/jira/browse/SPARK-34573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-34573. -- Fix Version/s: 3.2.0 Assignee: Gabriele Nizzoli Resolution: Fixed Reso

[jira] [Commented] (SPARK-34587) Replaces map and getOrElse(false/true) with exists/forall

[ https://issues.apache.org/jira/browse/SPARK-34587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293402#comment-17293402 ] Apache Spark commented on SPARK-34587: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-34183) DataSource V2: Support required distribution and ordering in SS

[ https://issues.apache.org/jira/browse/SPARK-34183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293401#comment-17293401 ] Apache Spark commented on SPARK-34183: -- User 'aokolnychyi' has created a pull reque

[jira] [Assigned] (SPARK-34183) DataSource V2: Support required distribution and ordering in SS

[ https://issues.apache.org/jira/browse/SPARK-34183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34183: Assignee: (was: Apache Spark) > DataSource V2: Support required distribution and orde

[jira] [Commented] (SPARK-34183) DataSource V2: Support required distribution and ordering in SS

[ https://issues.apache.org/jira/browse/SPARK-34183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293399#comment-17293399 ] Apache Spark commented on SPARK-34183: -- User 'aokolnychyi' has created a pull reque

[jira] [Assigned] (SPARK-34183) DataSource V2: Support required distribution and ordering in SS

[ https://issues.apache.org/jira/browse/SPARK-34183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34183: Assignee: Apache Spark > DataSource V2: Support required distribution and ordering in SS

[jira] [Assigned] (SPARK-34585) Remove BatchWriteHelper completely

[ https://issues.apache.org/jira/browse/SPARK-34585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34585: Assignee: (was: Apache Spark) > Remove BatchWriteHelper completely >

[jira] [Assigned] (SPARK-34587) Replaces map and getOrElse(false/true) with exists/forall

[ https://issues.apache.org/jira/browse/SPARK-34587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34587: Assignee: Apache Spark > Replaces map and getOrElse(false/true) with exists/forall >

[jira] [Assigned] (SPARK-34587) Replaces map and getOrElse(false/true) with exists/forall

[ https://issues.apache.org/jira/browse/SPARK-34587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34587: Assignee: (was: Apache Spark) > Replaces map and getOrElse(false/true) with exists/fo

[jira] [Assigned] (SPARK-34585) Remove BatchWriteHelper completely

[ https://issues.apache.org/jira/browse/SPARK-34585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34585: Assignee: Apache Spark > Remove BatchWriteHelper completely > ---

[jira] [Commented] (SPARK-34585) Remove BatchWriteHelper completely

[ https://issues.apache.org/jira/browse/SPARK-34585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293393#comment-17293393 ] Apache Spark commented on SPARK-34585: -- User 'aokolnychyi' has created a pull reque

[jira] [Commented] (SPARK-34587) Replaces map and getOrElse(false/true) with exists/forall

[ https://issues.apache.org/jira/browse/SPARK-34587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293392#comment-17293392 ] Apache Spark commented on SPARK-34587: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-34587) Replaces map and getOrElse(false/true) with exists/forall

[ https://issues.apache.org/jira/browse/SPARK-34587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293394#comment-17293394 ] Apache Spark commented on SPARK-34587: -- User 'LuciferYang' has created a pull reque

[jira] [Created] (SPARK-34588) Support int64 buffer lengths in Java for pyspark Pandas UDF as buffer expanding

Dmitry Kravchuk created SPARK-34588: --- Summary: Support int64 buffer lengths in Java for pyspark Pandas UDF as buffer expanding Key: SPARK-34588 URL: https://issues.apache.org/jira/browse/SPARK-34588

[jira] [Commented] (SPARK-3458) enable use of python's "with" statements for SparkContext management

[ https://issues.apache.org/jira/browse/SPARK-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293385#comment-17293385 ] Apache Spark commented on SPARK-3458: - User 'LuciferYang' has created a pull request

[jira] [Commented] (SPARK-3458) enable use of python's "with" statements for SparkContext management

[ https://issues.apache.org/jira/browse/SPARK-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293383#comment-17293383 ] Apache Spark commented on SPARK-3458: - User 'LuciferYang' has created a pull request

[jira] [Created] (SPARK-34587) Replaces map and getOrElse(false/true) with exists/forall

Yang Jie created SPARK-34587: Summary: Replaces map and getOrElse(false/true) with exists/forall Key: SPARK-34587 URL: https://issues.apache.org/jira/browse/SPARK-34587 Project: Spark Issue Type:

[jira] [Commented] (SPARK-34578) Ignore ORC encryption tests when ORC is loaded by old Hadoop library by other tests

[ https://issues.apache.org/jira/browse/SPARK-34578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293368#comment-17293368 ] Apache Spark commented on SPARK-34578: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-34586) Consider supporting distribution and ordering definitions during table creation

Anton Okolnychyi created SPARK-34586: Summary: Consider supporting distribution and ordering definitions during table creation Key: SPARK-34586 URL: https://issues.apache.org/jira/browse/SPARK-34586

[jira] [Created] (SPARK-34585) Remove BatchWriteHelper completely

Anton Okolnychyi created SPARK-34585: Summary: Remove BatchWriteHelper completely Key: SPARK-34585 URL: https://issues.apache.org/jira/browse/SPARK-34585 Project: Spark Issue Type: Improv

[jira] [Commented] (SPARK-34576) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE COLUMN`

[ https://issues.apache.org/jira/browse/SPARK-34576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293315#comment-17293315 ] Apache Spark commented on SPARK-34576: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-34576) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE COLUMN`

[ https://issues.apache.org/jira/browse/SPARK-34576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34576: Assignee: (was: Apache Spark) > Cannot drop/add columns from/to a dataset of v2 `DESC

[jira] [Assigned] (SPARK-34576) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE COLUMN`

[ https://issues.apache.org/jira/browse/SPARK-34576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34576: Assignee: Apache Spark > Cannot drop/add columns from/to a dataset of v2 `DESCRIBE COLUMN

[jira] [Commented] (SPARK-34576) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE COLUMN`

[ https://issues.apache.org/jira/browse/SPARK-34576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293314#comment-17293314 ] Apache Spark commented on SPARK-34576: -- User 'AngersZh' has created a pull requ

[jira] [Comment Edited] (SPARK-3159) Check for reducible DecisionTree

[ https://issues.apache.org/jira/browse/SPARK-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293294#comment-17293294 ] Carlos Bribiescas edited comment on SPARK-3159 at 3/2/21, 2:20 AM:

[jira] [Commented] (SPARK-3159) Check for reducible DecisionTree

[ https://issues.apache.org/jira/browse/SPARK-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293294#comment-17293294 ] Carlos Bribiescas commented on SPARK-3159: -- Would setting `canMergeChildren` in

[jira] [Commented] (SPARK-34579) Fix wrong test in SQLQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-34579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293262#comment-17293262 ] Takeshi Yamamuro commented on SPARK-34579: -- Could we resolve this? Btw, [~anger

[jira] [Updated] (SPARK-34573) SQLConf sqlConfEntries map has a global lock, should not lock on get

[ https://issues.apache.org/jira/browse/SPARK-34573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-34573: - Affects Version/s: 3.1.2 3.2.0 > SQLConf sqlConfEntries map has a

[jira] [Commented] (SPARK-3159) Check for reducible DecisionTree

[ https://issues.apache.org/jira/browse/SPARK-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293259#comment-17293259 ] Julian King commented on SPARK-3159: I also need the probability estimates for the tr

[jira] [Commented] (SPARK-34503) Use zstd for spark.eventLog.compression.codec by default

[ https://issues.apache.org/jira/browse/SPARK-34503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293196#comment-17293196 ] Apache Spark commented on SPARK-34503: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-34503) Use zstd for spark.eventLog.compression.codec by default

[ https://issues.apache.org/jira/browse/SPARK-34503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293195#comment-17293195 ] Apache Spark commented on SPARK-34503: -- User 'dongjoon-hyun' has created a pull req

[jira] [Updated] (SPARK-34573) SQLConf sqlConfEntries map has a global lock, should not lock on get

[ https://issues.apache.org/jira/browse/SPARK-34573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabriele Nizzoli updated SPARK-34573: - Description: SQLConf object has a sqlConfEntries map, which has a global lock (since it

[jira] [Updated] (SPARK-34573) SQLConf sqlConfEntries map has a global lock, should not lock on get

[ https://issues.apache.org/jira/browse/SPARK-34573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabriele Nizzoli updated SPARK-34573: - Description: SQLConf object has a sqlConfEntries map, which has a global lock (since it

[jira] [Assigned] (SPARK-34560) Cannot join datasets of SHOW TABLES

[ https://issues.apache.org/jira/browse/SPARK-34560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34560: --- Assignee: Maxim Gekk > Cannot join datasets of SHOW TABLES > --

[jira] [Resolved] (SPARK-34560) Cannot join datasets of SHOW TABLES

[ https://issues.apache.org/jira/browse/SPARK-34560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34560. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31675 [https://gith

[jira] [Commented] (SPARK-33084) Add jar support ivy path

[ https://issues.apache.org/jira/browse/SPARK-33084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293059#comment-17293059 ] Apache Spark commented on SPARK-33084: -- User 'xkrogen' has created a pull request f

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Commented] (SPARK-34448) Binary logistic regression incorrectly computes the intercept and coefficients when data is not centered

[ https://issues.apache.org/jira/browse/SPARK-34448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293027#comment-17293027 ] Yakov Kerzhner commented on SPARK-34448: I will try to do a code review, but wil

[jira] [Resolved] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-27495. --- Fix Version/s: 3.1.1 Target Version/s: (was: 3.2.0) Resolution: Fixed the

[jira] [Commented] (SPARK-34448) Binary logistic regression incorrectly computes the intercept and coefficients when data is not centered

[ https://issues.apache.org/jira/browse/SPARK-34448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293017#comment-17293017 ] Yakov Kerzhner commented on SPARK-34448: I took a look over the weekend.  It see

[jira] [Comment Edited] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292987#comment-17292987 ] Rafal Wojdyla edited comment on SPARK-34544 at 3/1/21, 4:40 PM: --

[jira] [Commented] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293013#comment-17293013 ] Rafal Wojdyla commented on SPARK-34544: --- [~zero323] I appreciate your prompt answe

[jira] [Commented] (SPARK-29329) maven incremental builds not working

[ https://issues.apache.org/jira/browse/SPARK-29329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293005#comment-17293005 ] Thomas Graves commented on SPARK-29329: --- Yeah I know 4.3.0 made things better. Its

[jira] [Commented] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292996#comment-17292996 ] Maciej Szymkiewicz commented on SPARK-34544: [~ravwojdyla] > But until pysp

[jira] [Commented] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292989#comment-17292989 ] Maciej Szymkiewicz commented on SPARK-34544: Overall: * {{DataFrameLike}} s

[jira] [Commented] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292987#comment-17292987 ] Rafal Wojdyla commented on SPARK-34544: --- 👋 [~zero323] > it is more a dev utility

[jira] [Comment Edited] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292977#comment-17292977 ] Maciej Szymkiewicz edited comment on SPARK-34544 at 3/1/21, 3:49 PM: -

[jira] [Commented] (SPARK-34544) pyspark toPandas() should return pd.DataFrame

[ https://issues.apache.org/jira/browse/SPARK-34544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292977#comment-17292977 ] Maciej Szymkiewicz commented on SPARK-34544: The problem we're dealing with

[jira] [Comment Edited] (SPARK-34584) When insert into a partition table with a illegal partition value, DSV2 behavior different as others

[ https://issues.apache.org/jira/browse/SPARK-34584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292963#comment-17292963 ] Gengliang Wang edited comment on SPARK-34584 at 3/1/21, 3:36 PM: -

[jira] [Comment Edited] (SPARK-34584) When insert into a partition table with a illegal partition value, DSV2 behavior different as others

[ https://issues.apache.org/jira/browse/SPARK-34584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292963#comment-17292963 ] Gengliang Wang edited comment on SPARK-34584 at 3/1/21, 3:35 PM: -

[jira] [Commented] (SPARK-34584) When insert into a partition table with a illegal partition value, DSV2 behavior different as others

[ https://issues.apache.org/jira/browse/SPARK-34584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292963#comment-17292963 ] Gengliang Wang commented on SPARK-34584: The difference is from https://github.c

[jira] [Commented] (SPARK-34409) haversine distance spark function

[ https://issues.apache.org/jira/browse/SPARK-34409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292930#comment-17292930 ] Sean R. Owen commented on SPARK-34409: -- I don't think codegen helps here? that won'

[jira] [Resolved] (SPARK-34410) Vincenty distance spark function

[ https://issues.apache.org/jira/browse/SPARK-34410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34410. -- Resolution: Won't Fix > Vincenty distance spark function > >

[jira] [Resolved] (SPARK-34409) haversine distance spark function

[ https://issues.apache.org/jira/browse/SPARK-34409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34409. -- Resolution: Won't Fix > haversine distance spark function > -

[jira] [Created] (SPARK-34584) When insert into a partition table with a illegal partition value, DSV2 behavior different as others

angerszhu created SPARK-34584: - Summary: When insert into a partition table with a illegal partition value, DSV2 behavior different as others Key: SPARK-34584 URL: https://issues.apache.org/jira/browse/SPARK-34584

[jira] [Resolved] (SPARK-33490) calling system.exit(-1) from a future does not result in the driver reflecting the fact that the spark app failed.

[ https://issues.apache.org/jira/browse/SPARK-33490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-33490. -- Resolution: Not A Problem > calling system.exit(-1) from a future does not result in the drive

[jira] [Resolved] (SPARK-33686) Spark SQL unrecognized `db.tbl`

[ https://issues.apache.org/jira/browse/SPARK-33686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-33686. -- Resolution: Won't Fix > Spark SQL unrecognized `db.tbl` > --- > >

[jira] [Resolved] (SPARK-34561) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE TABLE`

[ https://issues.apache.org/jira/browse/SPARK-34561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34561. - Fix Version/s: 3.1.2 3.2.0 Assignee: Maxim Gekk Resolution: Fi

[jira] [Commented] (SPARK-34448) Binary logistic regression incorrectly computes the intercept and coefficients when data is not centered

[ https://issues.apache.org/jira/browse/SPARK-34448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292924#comment-17292924 ] Sean R. Owen commented on SPARK-34448: -- [~ykerzhner] you may be interested in revie

[jira] [Assigned] (SPARK-34539) Zinc standalone server is useless after scala-maven-plugin 4.x

[ https://issues.apache.org/jira/browse/SPARK-34539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-34539: Assignee: Yikun Jiang > Zinc standalone server is useless after scala-maven-plugin 4.x >

[jira] [Resolved] (SPARK-34539) Zinc standalone server is useless after scala-maven-plugin 4.x

[ https://issues.apache.org/jira/browse/SPARK-34539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34539. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31647 [https://gi

[jira] [Updated] (SPARK-34539) Zinc standalone server is useless after scala-maven-plugin 4.x

[ https://issues.apache.org/jira/browse/SPARK-34539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-34539: - Priority: Minor (was: Major) > Zinc standalone server is useless after scala-maven-plugin 4.x >

[jira] [Updated] (SPARK-34581) BoundAttribute issue when pushing foldable into branches

[ https://issues.apache.org/jira/browse/SPARK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34581: Description: How to reproduce this issue: {code:scala} spark.sql("CREATE TABLE t1 (a INT, b INT) U

[jira] [Updated] (SPARK-34583) typed udf fails when it refers to type member in abstract class

[ https://issues.apache.org/jira/browse/SPARK-34583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kondziolka9ld updated SPARK-34583: -- Description:  Please consider a following scenario:  {code:java} scala> abstract class SomeAbs

[jira] [Created] (SPARK-34583) typed udf fails when it refers to type member in abstract class

kondziolka9ld created SPARK-34583: - Summary: typed udf fails when it refers to type member in abstract class Key: SPARK-34583 URL: https://issues.apache.org/jira/browse/SPARK-34583 Project: Spark

[jira] [Issue Comment Deleted] (SPARK-34569) java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.

[ https://issues.apache.org/jira/browse/SPARK-34569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34569: -- Comment: was deleted (was: User 'AngersZh' has created a pull request for this issue: http

[jira] [Updated] (SPARK-34579) Fix wrong test in SQLQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-34579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34579: -- Component/s: Tests > Fix wrong test in SQLQuerySuite > --- > >

[jira] [Commented] (SPARK-34579) Fix wrong test in SQLQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-34579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292850#comment-17292850 ] Dongjoon Hyun commented on SPARK-34579: --- Hi, [~vvin] and [~angerszhu]. I switched

[jira] [Updated] (SPARK-34579) Fix wrong test in SQLQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-34579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34579: -- Affects Version/s: (was: 3.0.1) 3.1.1 > Fix wrong test in SQLQueryS

[jira] [Updated] (SPARK-34569) java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.

[ https://issues.apache.org/jira/browse/SPARK-34569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34569: -- Affects Version/s: (was: 3.1.1) 3.0.1 > java.lang.NoSuchMethodError

[jira] [Updated] (SPARK-34579) Fix wrong test in SQLQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-34579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34579: -- Priority: Minor (was: Major) > Fix wrong test in SQLQuerySuite >

  1   2   >