[jira] [Created] (SPARK-40768) Migrate type check failures of bloom_filter_agg() onto error classes

2022-10-12 Thread Max Gekk (Jira)
Max Gekk created SPARK-40768: Summary: Migrate type check failures of bloom_filter_agg() onto error classes Key: SPARK-40768 URL: https://issues.apache.org/jira/browse/SPARK-40768 Project: Spark

[jira] [Updated] (SPARK-40768) Migrate type check failures of bloom_filter_agg() onto error classes

2022-10-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-40768: - Description: Replace TypeCheckFailure by DataTypeMismatch in type checks in bloom_filter_agg(): https:/

[jira] [Created] (SPARK-40769) Migrate type check failures of collection actions onto error classes

2022-10-12 Thread Max Gekk (Jira)
Max Gekk created SPARK-40769: Summary: Migrate type check failures of collection actions onto error classes Key: SPARK-40769 URL: https://issues.apache.org/jira/browse/SPARK-40769 Project: Spark

[jira] [Updated] (SPARK-40769) Migrate type check failures of aggregate expressions onto error classes

2022-10-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-40769: - Summary: Migrate type check failures of aggregate expressions onto error classes (was: Migrate type che

[jira] [Updated] (SPARK-40769) Migrate type check failures of aggregate expressions onto error classes

2022-10-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-40769: - Description: Replace TypeCheckFailure by DataTypeMismatch in type checks in the aggregate expressions:

[jira] [Updated] (SPARK-40769) Migrate type check failures of collection actions onto error classes

2022-10-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-40769: - Description: Replace TypeCheckFailure by DataTypeMismatch in type checks in the collection actions: 1.

[jira] [Updated] (SPARK-40767) Compile Spark example module will hang

2022-10-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-40767: - Attachment: screenshot-2.png > Compile Spark example module will hang >

[jira] [Updated] (SPARK-40767) Compile Spark example module will hang

2022-10-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-40767: - Attachment: image-2022-10-12-15-10-31-650.png > Compile Spark example module will hang > ---

[jira] [Updated] (SPARK-40769) Migrate type check failures of aggregate expressions onto error classes

2022-10-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-40769: - Description: Replace TypeCheckFailure by DataTypeMismatch in type checks in the aggregate expressions:

[jira] [Commented] (SPARK-40767) Compile Spark example module will hang

2022-10-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616243#comment-17616243 ] Yang Jie commented on SPARK-40767: -- !image-2022-10-12-15-10-31-650.png!    I have the

[jira] [Comment Edited] (SPARK-40767) Compile Spark example module will hang

2022-10-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616243#comment-17616243 ] Yang Jie edited comment on SPARK-40767 at 10/12/22 7:12 AM:

[jira] [Assigned] (SPARK-40361) Migrate arithmetic type check failures onto error classes

2022-10-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-40361: Assignee: Shaokang Lv > Migrate arithmetic type check failures onto error classes > -

[jira] [Resolved] (SPARK-40361) Migrate arithmetic type check failures onto error classes

2022-10-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-40361. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38208 [https://github.com

[jira] [Commented] (SPARK-40457) upgrade jackson data mapper to latest

2022-10-12 Thread Bilna (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616249#comment-17616249 ] Bilna commented on SPARK-40457: --- Are we going to remove this dependency from spark's pom f

[jira] [Updated] (SPARK-40736) Spark 3.3.0 doesn't works with Hive 3.1.2

2022-10-12 Thread Pratik Malani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratik Malani updated SPARK-40736: -- Component/s: Spark Core > Spark 3.3.0 doesn't works with Hive 3.1.2 >

[jira] [Assigned] (SPARK-40464) Support automatic data format conversion for shuffle state db

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40464: Assignee: (was: Apache Spark) > Support automatic data format conversion for shuffle

[jira] [Commented] (SPARK-40464) Support automatic data format conversion for shuffle state db

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616278#comment-17616278 ] Apache Spark commented on SPARK-40464: -- User 'panbingkun' has created a pull reques

[jira] [Assigned] (SPARK-40464) Support automatic data format conversion for shuffle state db

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40464: Assignee: Apache Spark > Support automatic data format conversion for shuffle state db >

[jira] [Commented] (SPARK-40764) Extract partitioning through all children output expressions

2022-10-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616301#comment-17616301 ] Yuming Wang commented on SPARK-40764: - Another case: {code:scala} withSQLConf(SQ

[jira] [Commented] (SPARK-40764) Extract partitioning through all children output expressions

2022-10-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616303#comment-17616303 ] Yuming Wang commented on SPARK-40764: - Another case if pull out join condition: {co

[jira] [Commented] (SPARK-40724) Simplify `corr` with method `inline`

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616332#comment-17616332 ] Apache Spark commented on SPARK-40724: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-40724) Simplify `corr` with method `inline`

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616333#comment-17616333 ] Apache Spark commented on SPARK-40724: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-40764) Extract partitioning through all children output expressions

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616347#comment-17616347 ] Apache Spark commented on SPARK-40764: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-40764) Extract partitioning through all children output expressions

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40764: Assignee: Apache Spark > Extract partitioning through all children output expressions > -

[jira] [Assigned] (SPARK-40764) Extract partitioning through all children output expressions

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40764: Assignee: (was: Apache Spark) > Extract partitioning through all children output expr

[jira] [Updated] (SPARK-40497) Upgrade Scala to 2.13.11

2022-10-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-40497: - Summary: Upgrade Scala to 2.13.11 (was: Upgrade Scala to 2.13.9) > Upgrade Scala to 2.13.11 > -

[jira] [Commented] (SPARK-40497) Upgrade Scala to 2.13.11

2022-10-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616398#comment-17616398 ] Yang Jie commented on SPARK-40497: -- Change te Jira title due to [https://github.com/sca

[jira] [Updated] (SPARK-40736) Spark 3.3.0 doesn't works with Hive 3.1.2

2022-10-12 Thread Pratik Malani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratik Malani updated SPARK-40736: -- Attachment: image-2022-10-12-18-19-24-455.png > Spark 3.3.0 doesn't works with Hive 3.1.2 > --

[jira] [Commented] (SPARK-40736) Spark 3.3.0 doesn't works with Hive 3.1.2

2022-10-12 Thread Pratik Malani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616399#comment-17616399 ] Pratik Malani commented on SPARK-40736: --- Hi All, Removed hive-service jar from th

[jira] [Resolved] (SPARK-40762) Check error classes in ErrorParserSuite

2022-10-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-40762. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38204 [https://github.com

[jira] [Commented] (SPARK-35084) [k8s] On Spark 3, jars listed in spark.jars and spark.jars.packages are not added to sparkContext

2022-10-12 Thread Gabor Roczei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616444#comment-17616444 ] Gabor Roczei commented on SPARK-35084: -- Hi [~ocworld], [Your pull request|https://

[jira] [Created] (SPARK-40770) Improved error messages for applyInPandas for schema mismatch

2022-10-12 Thread Enrico Minack (Jira)
Enrico Minack created SPARK-40770: - Summary: Improved error messages for applyInPandas for schema mismatch Key: SPARK-40770 URL: https://issues.apache.org/jira/browse/SPARK-40770 Project: Spark

[jira] [Commented] (SPARK-40770) Improved error messages for applyInPandas for schema mismatch

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616478#comment-17616478 ] Apache Spark commented on SPARK-40770: -- User 'EnricoMi' has created a pull request

[jira] [Assigned] (SPARK-40770) Improved error messages for applyInPandas for schema mismatch

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40770: Assignee: (was: Apache Spark) > Improved error messages for applyInPandas for schema

[jira] [Assigned] (SPARK-40770) Improved error messages for applyInPandas for schema mismatch

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40770: Assignee: Apache Spark > Improved error messages for applyInPandas for schema mismatch >

[jira] [Commented] (SPARK-40770) Improved error messages for applyInPandas for schema mismatch

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616483#comment-17616483 ] Apache Spark commented on SPARK-40770: -- User 'EnricoMi' has created a pull request

[jira] [Created] (SPARK-40771) Estimated size in log message can overflow Int

2022-10-12 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-40771: -- Summary: Estimated size in log message can overflow Int Key: SPARK-40771 URL: https://issues.apache.org/jira/browse/SPARK-40771 Project: Spark Issue Type

[jira] [Assigned] (SPARK-40771) Estimated size in log message can overflow Int

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40771: Assignee: Apache Spark > Estimated size in log message can overflow Int > ---

[jira] [Assigned] (SPARK-40771) Estimated size in log message can overflow Int

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40771: Assignee: (was: Apache Spark) > Estimated size in log message can overflow Int >

[jira] [Commented] (SPARK-40771) Estimated size in log message can overflow Int

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616495#comment-17616495 ] Apache Spark commented on SPARK-40771: -- User 'eejbyfeldt' has created a pull reques

[jira] [Commented] (SPARK-40771) Estimated size in log message can overflow Int

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616496#comment-17616496 ] Apache Spark commented on SPARK-40771: -- User 'eejbyfeldt' has created a pull reques

[jira] [Created] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-40772: - Summary: Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values Key: SPARK-40772 URL: https://issues.apache.org/jira/browse/SPARK-40772 P

[jira] [Assigned] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40772: Assignee: (was: Apache Spark) > Improve spark.sql.adaptive.skewJoin.skewedPartitionFa

[jira] [Assigned] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40772: Assignee: Apache Spark > Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to sup

[jira] [Commented] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616553#comment-17616553 ] Apache Spark commented on SPARK-40772: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616552#comment-17616552 ] Apache Spark commented on SPARK-40772: -- User 'dongjoon-hyun' has created a pull req

[jira] [Resolved] (SPARK-40721) Use `||` instead of `|` in TimSort

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40721. -- Resolution: Not A Problem > Use `||` instead of `|` in TimSort > -

[jira] [Commented] (SPARK-40758) Upgrade Apache zookeeper to get rid of CVE-2020-10663

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616612#comment-17616612 ] Sean R. Owen commented on SPARK-40758: -- Does the CVE affect Spark? Can we update ZK

[jira] [Updated] (SPARK-40681) Update gson transitive dependency to 2.8.9 or later

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-40681: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) Go ahead and try updatin

[jira] [Updated] (SPARK-40685) Update kryo transitive dependency to 5.2.0 or later

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-40685: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) Does kryo 5 work with Sp

[jira] [Resolved] (SPARK-40650) Infer date type for Json schema inference

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40650. -- Fix Version/s: (was: 3.4.0) Resolution: Duplicate > Infer date type for Json schema

[jira] [Resolved] (SPARK-40526) Upgrade Scala to 2.13.9

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40526. -- Resolution: Won't Fix > Upgrade Scala to 2.13.9 > --- > >

[jira] [Resolved] (SPARK-40499) Spark 3.2.1 percentlie_approx query much slower than Spark 2.4.0

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40499. -- Resolution: Invalid It's hard to action this; this sounds like a question for the mailing list

[jira] [Resolved] (SPARK-40422) Upgrade hive to 4.0.0

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40422. -- Resolution: Won't Fix > Upgrade hive to 4.0.0 > - > > Key:

[jira] [Commented] (SPARK-40422) Upgrade hive to 4.0.0

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616623#comment-17616623 ] Sean R. Owen commented on SPARK-40422: -- This would be a massively breaking change -

[jira] [Resolved] (SPARK-40489) Spark 3.3.0 breaks with SFL4J 2.

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40489. -- Resolution: Duplicate Spark is on slf4j 2.x now, so I think SPARK-40511 supersedes this > Spa

[jira] [Commented] (SPARK-40511) Upgrade slf4j to 2.x

2022-10-12 Thread Garret Wilson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616642#comment-17616642 ] Garret Wilson commented on SPARK-40511: --- Thank you for your work on this! It appea

[jira] [Commented] (SPARK-40706) IllegalStateException when querying array values inside a nested struct

2022-10-12 Thread Rohan Barman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616654#comment-17616654 ] Rohan Barman commented on SPARK-40706: -- thanks [~bersprockets]. Those 2 spark setti

[jira] [Created] (SPARK-40773) Refactor checkCorrelationsInSubquery

2022-10-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-40773: Summary: Refactor checkCorrelationsInSubquery Key: SPARK-40773 URL: https://issues.apache.org/jira/browse/SPARK-40773 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-40773) Refactor checkCorrelationsInSubquery

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40773: Assignee: Apache Spark > Refactor checkCorrelationsInSubquery > -

[jira] [Assigned] (SPARK-40773) Refactor checkCorrelationsInSubquery

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40773: Assignee: (was: Apache Spark) > Refactor checkCorrelationsInSubquery > --

[jira] [Commented] (SPARK-40773) Refactor checkCorrelationsInSubquery

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616661#comment-17616661 ] Apache Spark commented on SPARK-40773: -- User 'allisonwang-db' has created a pull re

[jira] [Updated] (SPARK-40760) Migrate type check failures of interval expressions onto error classes

2022-10-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-40760: - Description: Replace TypeCheckFailure by DataTypeMismatch in type checks in the interval expressions: 1

[jira] [Created] (SPARK-40774) Add Sample to proto and DSL

2022-10-12 Thread Rui Wang (Jira)
Rui Wang created SPARK-40774: Summary: Add Sample to proto and DSL Key: SPARK-40774 URL: https://issues.apache.org/jira/browse/SPARK-40774 Project: Spark Issue Type: Sub-task Components

[jira] [Assigned] (SPARK-40774) Add Sample to proto and DSL

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40774: Assignee: (was: Apache Spark) > Add Sample to proto and DSL > ---

[jira] [Commented] (SPARK-40774) Add Sample to proto and DSL

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616682#comment-17616682 ] Apache Spark commented on SPARK-40774: -- User 'amaliujia' has created a pull request

[jira] [Assigned] (SPARK-40774) Add Sample to proto and DSL

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40774: Assignee: Apache Spark > Add Sample to proto and DSL > --- > >

[jira] [Commented] (SPARK-40774) Add Sample to proto and DSL

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616683#comment-17616683 ] Apache Spark commented on SPARK-40774: -- User 'amaliujia' has created a pull request

[jira] [Created] (SPARK-40775) V2 file scans have duplicative descriptions

2022-10-12 Thread Adam Binford (Jira)
Adam Binford created SPARK-40775: Summary: V2 file scans have duplicative descriptions Key: SPARK-40775 URL: https://issues.apache.org/jira/browse/SPARK-40775 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-40739) "sbt packageBin" fails in cygwin or other windows bash session

2022-10-12 Thread Phil Walker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Walker updated SPARK-40739: External issue ID: 38228 (was: 38167) Description:  In a Windows _*SHELL*_ environment,

[jira] [Commented] (SPARK-40739) "sbt packageBin" fails in cygwin or other windows bash session

2022-10-12 Thread Phil Walker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616693#comment-17616693 ] Phil Walker commented on SPARK-40739: - The actual spark PR is now 38228, not sure wh

[jira] [Updated] (SPARK-40738) spark-shell fails with "bad array subscript" in cygwin or msys bash session

2022-10-12 Thread Phil Walker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Walker updated SPARK-40738: External issue ID: 38228 (was: 38167) > spark-shell fails with "bad array subscript" in cygwin or

[jira] [Updated] (SPARK-40738) spark-shell fails with "bad array subscript" in cygwin or msys bash session

2022-10-12 Thread Phil Walker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Walker updated SPARK-40738: External issue URL: https://github.com/apache/spark/pull/38228 (was: https://github.com/apache/sp

[jira] (SPARK-40739) "sbt packageBin" fails in cygwin or other windows bash session

2022-10-12 Thread Phil Walker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40739 ] Phil Walker deleted comment on SPARK-40739: - was (Author: JIRAUSER296755): The actual spark PR is now 38228, not sure why I wasn't able to modify the links above. > "sbt packageBin" fails i

[jira] [Updated] (SPARK-40739) "sbt packageBin" fails in cygwin or other windows bash session

2022-10-12 Thread Phil Walker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Walker updated SPARK-40739: External issue URL: https://github.com/apache/spark/pull/38228 (was: https://github.com/apache/sp

[jira] [Resolved] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40772. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38225 [https://

[jira] [Assigned] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40772: - Assignee: Dongjoon Hyun > Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to

[jira] [Commented] (SPARK-40657) Add support for compiled classes (Java classes)

2022-10-12 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616711#comment-17616711 ] Raghu Angadi commented on SPARK-40657: -- I will add this.  cc: [~sanysand...@gmail.

[jira] [Created] (SPARK-40776) Add documentation (similar Avro functions).

2022-10-12 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-40776: Summary: Add documentation (similar Avro functions). Key: SPARK-40776 URL: https://issues.apache.org/jira/browse/SPARK-40776 Project: Spark Issue Type: Impro

[jira] [Created] (SPARK-40777) Use error classes for Protobuf exceptions

2022-10-12 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-40777: Summary: Use error classes for Protobuf exceptions Key: SPARK-40777 URL: https://issues.apache.org/jira/browse/SPARK-40777 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-40775) V2 file scans have duplicative descriptions

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616738#comment-17616738 ] Apache Spark commented on SPARK-40775: -- User 'Kimahriman' has created a pull reques

[jira] [Assigned] (SPARK-40775) V2 file scans have duplicative descriptions

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40775: Assignee: Apache Spark > V2 file scans have duplicative descriptions > --

[jira] [Assigned] (SPARK-40775) V2 file scans have duplicative descriptions

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40775: Assignee: (was: Apache Spark) > V2 file scans have duplicative descriptions > ---

[jira] [Resolved] (SPARK-40766) Upgrade the guava defined in `plugins.sbt` to `31.0.1-jre`

2022-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40766. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38217 [https://gi

[jira] [Assigned] (SPARK-40766) Upgrade the guava defined in `plugins.sbt` to `31.0.1-jre`

2022-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40766: Assignee: Yang Jie > Upgrade the guava defined in `plugins.sbt` to `31.0.1-jre` > ---

[jira] [Commented] (SPARK-40738) spark-shell fails with "bad array subscript" in cygwin or msys bash session

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616751#comment-17616751 ] Apache Spark commented on SPARK-40738: -- User 'philwalk' has created a pull request

[jira] [Commented] (SPARK-40739) "sbt packageBin" fails in cygwin or other windows bash session

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616750#comment-17616750 ] Apache Spark commented on SPARK-40739: -- User 'philwalk' has created a pull request

[jira] [Assigned] (SPARK-40765) Optimize redundant fs operations in `CommandUtils#calculateSingleLocationSize#getPathSize` method

2022-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40765: Assignee: Yang Jie > Optimize redundant fs operations in > `CommandUtils#calculateSingle

[jira] [Resolved] (SPARK-40765) Optimize redundant fs operations in `CommandUtils#calculateSingleLocationSize#getPathSize` method

2022-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40765. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38214 [https://gi

[jira] [Commented] (SPARK-40616) Loss of precision using SparkSQL shell on high-precision DECIMAL types

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616754#comment-17616754 ] Apache Spark commented on SPARK-40616: -- User 'xsystem2022' has created a pull reque

[jira] [Created] (SPARK-40778) Make HeartbeatReceiver as an IsolatedRpcEndpoint

2022-10-12 Thread Zhongwei Zhu (Jira)
Zhongwei Zhu created SPARK-40778: Summary: Make HeartbeatReceiver as an IsolatedRpcEndpoint Key: SPARK-40778 URL: https://issues.apache.org/jira/browse/SPARK-40778 Project: Spark Issue Type:

[jira] [Created] (SPARK-40779) Fix `corrwith` to work properly with different anchor.

2022-10-12 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-40779: --- Summary: Fix `corrwith` to work properly with different anchor. Key: SPARK-40779 URL: https://issues.apache.org/jira/browse/SPARK-40779 Project: Spark Issue Ty

[jira] [Commented] (SPARK-40778) Make HeartbeatReceiver as an IsolatedRpcEndpoint

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616766#comment-17616766 ] Apache Spark commented on SPARK-40778: -- User 'warrenzhu25' has created a pull reque

[jira] [Assigned] (SPARK-40778) Make HeartbeatReceiver as an IsolatedRpcEndpoint

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40778: Assignee: Apache Spark > Make HeartbeatReceiver as an IsolatedRpcEndpoint > -

[jira] [Assigned] (SPARK-40778) Make HeartbeatReceiver as an IsolatedRpcEndpoint

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40778: Assignee: (was: Apache Spark) > Make HeartbeatReceiver as an IsolatedRpcEndpoint > --

[jira] [Commented] (SPARK-40778) Make HeartbeatReceiver as an IsolatedRpcEndpoint

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616767#comment-17616767 ] Apache Spark commented on SPARK-40778: -- User 'warrenzhu25' has created a pull reque

[jira] [Commented] (SPARK-40779) Fix `corrwith` to work properly with different anchor.

2022-10-12 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616768#comment-17616768 ] Haejoon Lee commented on SPARK-40779: - I'm taking a look. Seems like it's different

[jira] [Comment Edited] (SPARK-40779) Fix `corrwith` to work properly with different anchor.

2022-10-12 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616768#comment-17616768 ] Haejoon Lee edited comment on SPARK-40779 at 10/13/22 4:07 AM: ---

[jira] [Created] (SPARK-40780) Add WHERE to Connect proto and DSL

2022-10-12 Thread Rui Wang (Jira)
Rui Wang created SPARK-40780: Summary: Add WHERE to Connect proto and DSL Key: SPARK-40780 URL: https://issues.apache.org/jira/browse/SPARK-40780 Project: Spark Issue Type: Sub-task Com

[jira] [Created] (SPARK-40781) Explain exit code 137 as killed due to OOM

2022-10-12 Thread Zhongwei Zhu (Jira)
Zhongwei Zhu created SPARK-40781: Summary: Explain exit code 137 as killed due to OOM Key: SPARK-40781 URL: https://issues.apache.org/jira/browse/SPARK-40781 Project: Spark Issue Type: Improv

  1   2   >