[jira] [Updated] (SPARK-37592) Improve performance of JoinSelection

[ https://issues.apache.org/jira/browse/SPARK-37592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37592: --- Summary: Improve performance of JoinSelection (was: Improve performance for JoinSelection) >

[jira] [Created] (SPARK-37592) Improve performance for JoinSelection

jiaan.geng created SPARK-37592: -- Summary: Improve performance for JoinSelection Key: SPARK-37592 URL: https://issues.apache.org/jira/browse/SPARK-37592 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-37581) sql hang at planning stage

[ https://issues.apache.org/jira/browse/SPARK-37581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456193#comment-17456193 ] ocean commented on SPARK-37581: --- I have narrowed down in the comment.please have a look. > sql hang at

[jira] (SPARK-37581) sql hang at planning stage

[ https://issues.apache.org/jira/browse/SPARK-37581 ] ocean deleted comment on SPARK-37581: --- was (Author: oceaneast): I have narrowed down in the comment.please have a look. > sql hang at planning stage > -- > >

[jira] [Commented] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456172#comment-17456172 ] Hyukjin Kwon commented on SPARK-37575: -- We don't usually assign issues to someone. Once you open a

[jira] [Commented] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456168#comment-17456168 ] Guo Wei commented on SPARK-37575: - I  have made some bug fix in my local repo and add some test cases

[jira] [Comment Edited] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456167#comment-17456167 ] Guo Wei edited comment on SPARK-37575 at 12/9/21, 6:52 AM: --- I also found that

[jira] [Comment Edited] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456167#comment-17456167 ] Guo Wei edited comment on SPARK-37575 at 12/9/21, 6:49 AM: --- I also found that

[jira] [Commented] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456167#comment-17456167 ] Guo Wei commented on SPARK-37575: - I also found that if emptyValueInRead set to "\"\"", reading csv data

[jira] [Updated] (SPARK-37581) sql hang at planning stage

[ https://issues.apache.org/jira/browse/SPARK-37581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ocean updated SPARK-37581: -- Description: when exec a sql, this sql hang at planning stage. when disable DPP, sql can finish very

[jira] [Comment Edited] (SPARK-37581) sql hang at planning stage

[ https://issues.apache.org/jira/browse/SPARK-37581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456154#comment-17456154 ] ocean edited comment on SPARK-37581 at 12/9/21, 6:12 AM: - Hi [~hyukjin.kwon].

[jira] [Comment Edited] (SPARK-37581) sql hang at planning stage

[ https://issues.apache.org/jira/browse/SPARK-37581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456154#comment-17456154 ] ocean edited comment on SPARK-37581 at 12/9/21, 6:06 AM: - Hi [~hyukjin.kwon].

[jira] [Commented] (SPARK-37581) sql hang at planning stage

[ https://issues.apache.org/jira/browse/SPARK-37581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456154#comment-17456154 ] ocean commented on SPARK-37581: --- Hi [~hyukjin.kwon]. This sql have 19 join operators.But these join have

[jira] [Commented] (SPARK-37591) Support the GCM mode by aes_encrypt()/aes_decrypt()

[ https://issues.apache.org/jira/browse/SPARK-37591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456153#comment-17456153 ] Max Gekk commented on SPARK-37591: -- I am working on this. > Support the GCM mode by

[jira] [Created] (SPARK-37591) Support the GCM mode by aes_encrypt()/aes_decrypt()

Max Gekk created SPARK-37591: Summary: Support the GCM mode by aes_encrypt()/aes_decrypt() Key: SPARK-37591 URL: https://issues.apache.org/jira/browse/SPARK-37591 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-37586) Add cipher mode option and set default cipher mode for aes_encrypt and aes_decrypt

[ https://issues.apache.org/jira/browse/SPARK-37586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-37586. Fix Version/s: 3.3.0 Assignee: Max Gekk Resolution: Fixed Issue resolved

[jira] [Commented] (SPARK-36137) HiveShim always fallback to getAllPartitionsOf regardless of whether directSQL is enabled in remote HMS

[ https://issues.apache.org/jira/browse/SPARK-36137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456141#comment-17456141 ] Apache Spark commented on SPARK-36137: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-37590) Unify v1 and v2 ALTER NAMESPACE ... SET PROPERTIES tests

[ https://issues.apache.org/jira/browse/SPARK-37590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456135#comment-17456135 ] Apache Spark commented on SPARK-37590: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37590) Unify v1 and v2 ALTER NAMESPACE ... SET PROPERTIES tests

[ https://issues.apache.org/jira/browse/SPARK-37590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37590: Assignee: (was: Apache Spark) > Unify v1 and v2 ALTER NAMESPACE ... SET PROPERTIES

[jira] [Assigned] (SPARK-37590) Unify v1 and v2 ALTER NAMESPACE ... SET PROPERTIES tests

[ https://issues.apache.org/jira/browse/SPARK-37590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37590: Assignee: Apache Spark > Unify v1 and v2 ALTER NAMESPACE ... SET PROPERTIES tests >

[jira] [Commented] (SPARK-37590) Unify v1 and v2 ALTER NAMESPACE ... SET PROPERTIES tests

[ https://issues.apache.org/jira/browse/SPARK-37590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456134#comment-17456134 ] Apache Spark commented on SPARK-37590: -- User 'imback82' has created a pull request for this issue:

[jira] [Created] (SPARK-37590) Unify v1 and v2 ALTER NAMESPACE ... SET PROPERTIES tests

Terry Kim created SPARK-37590: - Summary: Unify v1 and v2 ALTER NAMESPACE ... SET PROPERTIES tests Key: SPARK-37590 URL: https://issues.apache.org/jira/browse/SPARK-37590 Project: Spark Issue

[jira] [Resolved] (SPARK-37579) Called spark.sql multiple times，union multiple DataFrame, groupBy and pivot, join other table view cause exception

[ https://issues.apache.org/jira/browse/SPARK-37579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37579. -- Resolution: Incomplete > Called spark.sql multiple times，union multiple DataFrame, groupBy

[jira] [Commented] (SPARK-37579) Called spark.sql multiple times，union multiple DataFrame, groupBy and pivot, join other table view cause exception

[ https://issues.apache.org/jira/browse/SPARK-37579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456121#comment-17456121 ] Hyukjin Kwon commented on SPARK-37579: -- For that, you can leverage Spark dev mailing list instead

[jira] [Commented] (SPARK-37579) Called spark.sql multiple times，union multiple DataFrame, groupBy and pivot, join other table view cause exception

[ https://issues.apache.org/jira/browse/SPARK-37579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456120#comment-17456120 ] page commented on SPARK-37579: -- No current plans to use Spark 3+. Can I still get advice or help on

[jira] [Assigned] (SPARK-37589) Improve the performance of PlanParserSuite

[ https://issues.apache.org/jira/browse/SPARK-37589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37589: Assignee: Apache Spark > Improve the performance of PlanParserSuite >

[jira] [Commented] (SPARK-37589) Improve the performance of PlanParserSuite

[ https://issues.apache.org/jira/browse/SPARK-37589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456116#comment-17456116 ] Apache Spark commented on SPARK-37589: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37589) Improve the performance of PlanParserSuite

[ https://issues.apache.org/jira/browse/SPARK-37589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37589: Assignee: Apache Spark > Improve the performance of PlanParserSuite >

[jira] [Assigned] (SPARK-37589) Improve the performance of PlanParserSuite

[ https://issues.apache.org/jira/browse/SPARK-37589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37589: Assignee: (was: Apache Spark) > Improve the performance of PlanParserSuite >

[jira] [Created] (SPARK-37589) Improve the performance of PlanParserSuite

jiaan.geng created SPARK-37589: -- Summary: Improve the performance of PlanParserSuite Key: SPARK-37589 URL: https://issues.apache.org/jira/browse/SPARK-37589 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37577) ClassCastException: ArrayType cannot be cast to StructType

[ https://issues.apache.org/jira/browse/SPARK-37577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456103#comment-17456103 ] L. C. Hsieh commented on SPARK-37577: - Thanks [~hyukjin.kwon]. I will take a look. >

[jira] [Updated] (SPARK-37572) Flexible ways of launching executors

[ https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dagang Wei updated SPARK-37572: --- Description: Currently Spark launches executor processes by constructing and running commands [1],

[jira] [Updated] (SPARK-37588) lot of strings get accumulated in the heap dump of spark thrift server

[ https://issues.apache.org/jira/browse/SPARK-37588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramakrishna chilaka updated SPARK-37588: Environment: Open JDK (8 build 1.8.0_312-b07) and scala 2.12 OS: Red Hat

[jira] [Updated] (SPARK-37572) Flexible ways of launching executors

[ https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dagang Wei updated SPARK-37572: --- Description: Currently Spark launches executor processes by constructing and running commands [1],

[jira] [Updated] (SPARK-37572) Flexible ways of launching executors

[ https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dagang Wei updated SPARK-37572: --- Description: Currently Spark launches executor processes by constructing and running commands [1],

[jira] [Updated] (SPARK-37572) Flexible ways of launching executors

[ https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dagang Wei updated SPARK-37572: --- Description: Currently Spark launches executor processes by constructing and running commands [1],

[jira] [Updated] (SPARK-37588) lot of strings get accumulated in the heap dump of spark thrift server

[ https://issues.apache.org/jira/browse/SPARK-37588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramakrishna chilaka updated SPARK-37588: Description: I am starting spark thrift server using the following options ```

[jira] [Created] (SPARK-37588) lot of strings get accumulated in the heap dump of spark thrift server

ramakrishna chilaka created SPARK-37588: --- Summary: lot of strings get accumulated in the heap dump of spark thrift server Key: SPARK-37588 URL: https://issues.apache.org/jira/browse/SPARK-37588

[jira] [Comment Edited] (SPARK-37577) ClassCastException: ArrayType cannot be cast to StructType

[ https://issues.apache.org/jira/browse/SPARK-37577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456093#comment-17456093 ] Hyukjin Kwon edited comment on SPARK-37577 at 12/9/21, 2:16 AM: It works

[jira] [Commented] (SPARK-37577) ClassCastException: ArrayType cannot be cast to StructType

[ https://issues.apache.org/jira/browse/SPARK-37577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456093#comment-17456093 ] Hyukjin Kwon commented on SPARK-37577: -- It works if you disable: {code}

[jira] [Updated] (SPARK-37577) ClassCastException: ArrayType cannot be cast to StructType

[ https://issues.apache.org/jira/browse/SPARK-37577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37577: - Priority: Major (was: Critical) > ClassCastException: ArrayType cannot be cast to StructType >

[jira] [Commented] (SPARK-37579) Called spark.sql multiple times，union multiple DataFrame, groupBy and pivot, join other table view cause exception

[ https://issues.apache.org/jira/browse/SPARK-37579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456092#comment-17456092 ] Hyukjin Kwon commented on SPARK-37579: -- Spark 2.4 is EOL. mind checking if that passes with Spark

[jira] [Updated] (SPARK-37581) sql hang at planning stage

[ https://issues.apache.org/jira/browse/SPARK-37581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37581: - Priority: Major (was: Critical) > sql hang at planning stage > -- > >

[jira] [Commented] (SPARK-37581) sql hang at planning stage

[ https://issues.apache.org/jira/browse/SPARK-37581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456091#comment-17456091 ] Hyukjin Kwon commented on SPARK-37581: -- [~oceaneast] do you mind narrowing down and make the

[jira] [Commented] (SPARK-31785) Add a helper function to test all parquet readers

[ https://issues.apache.org/jira/browse/SPARK-31785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456090#comment-17456090 ] Apache Spark commented on SPARK-31785: -- User 'beliefer' has created a pull request for this issue:

[jira] [Commented] (SPARK-31785) Add a helper function to test all parquet readers

[ https://issues.apache.org/jira/browse/SPARK-31785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456088#comment-17456088 ] Apache Spark commented on SPARK-31785: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37561) Avoid loading all functions when obtaining hive's DelegationToken

[ https://issues.apache.org/jira/browse/SPARK-37561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-37561: Assignee: dzcxzl > Avoid loading all functions when obtaining hive's DelegationToken >

[jira] [Resolved] (SPARK-37561) Avoid loading all functions when obtaining hive's DelegationToken

[ https://issues.apache.org/jira/browse/SPARK-37561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-37561. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34822

[jira] [Updated] (SPARK-37581) sql hang at planning stage

[ https://issues.apache.org/jira/browse/SPARK-37581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ocean updated SPARK-37581: -- Priority: Critical (was: Major) > sql hang at planning stage > -- > >

[jira] [Assigned] (SPARK-37569) View Analysis incorrectly marks nested fields as nullable

[ https://issues.apache.org/jira/browse/SPARK-37569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37569: Assignee: Apache Spark > View Analysis incorrectly marks nested fields as nullable >

[jira] [Assigned] (SPARK-37569) View Analysis incorrectly marks nested fields as nullable

[ https://issues.apache.org/jira/browse/SPARK-37569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37569: Assignee: (was: Apache Spark) > View Analysis incorrectly marks nested fields as

[jira] [Commented] (SPARK-37569) View Analysis incorrectly marks nested fields as nullable

[ https://issues.apache.org/jira/browse/SPARK-37569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455950#comment-17455950 ] Apache Spark commented on SPARK-37569: -- User 'shardulm94' has created a pull request for this

[jira] [Created] (SPARK-37587) Forever-running streams get completed with finished status when k8s worker is lost

In-Ho Yi created SPARK-37587: Summary: Forever-running streams get completed with finished status when k8s worker is lost Key: SPARK-37587 URL: https://issues.apache.org/jira/browse/SPARK-37587 Project:

[jira] [Commented] (SPARK-37570) mypy breaks on pyspark.pandas.plot.core.Bucketizer

[ https://issues.apache.org/jira/browse/SPARK-37570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455897#comment-17455897 ] Rafal Wojdyla commented on SPARK-37570: --- A workaround in {{setup.cfg}}: {noformat}

[jira] [Resolved] (SPARK-37205) Support mapreduce.job.send-token-conf when starting containers in YARN

[ https://issues.apache.org/jira/browse/SPARK-37205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-37205. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34635

[jira] [Assigned] (SPARK-37205) Support mapreduce.job.send-token-conf when starting containers in YARN

[ https://issues.apache.org/jira/browse/SPARK-37205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-37205: Assignee: Chao Sun > Support mapreduce.job.send-token-conf when starting containers in YARN >

[jira] [Updated] (SPARK-37572) Flexible ways of launching executors

[ https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dagang Wei updated SPARK-37572: --- Affects Version/s: 3.2.0 (was: 3.3.0) > Flexible ways of launching

[jira] [Updated] (SPARK-37572) Flexible ways of launching executors

[ https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dagang Wei updated SPARK-37572: --- Affects Version/s: 3.3.0 (was: 3.2.0) > Flexible ways of launching

[jira] [Updated] (SPARK-37451) Performance improvement regressed String to Decimal cast

[ https://issues.apache.org/jira/browse/SPARK-37451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37451: -- Target Version/s: 3.1.3, 3.2.1 (was: 3.2.0, 3.1.3) > Performance improvement regressed

[jira] [Comment Edited] (SPARK-37451) Performance improvement regressed String to Decimal cast

[ https://issues.apache.org/jira/browse/SPARK-37451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455357#comment-17455357 ] Dongjoon Hyun edited comment on SPARK-37451 at 12/8/21, 4:16 PM: -

[jira] [Commented] (SPARK-37451) Performance improvement regressed String to Decimal cast

[ https://issues.apache.org/jira/browse/SPARK-37451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455357#comment-17455357 ] Dongjoon Hyun commented on SPARK-37451: --- According to the JIRA description, I added a correctness

[jira] [Updated] (SPARK-37451) Performance improvement regressed String to Decimal cast

[ https://issues.apache.org/jira/browse/SPARK-37451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37451: -- Labels: correctness (was: ) > Performance improvement regressed String to Decimal cast >

[jira] [Updated] (SPARK-37451) Performance improvement regressed String to Decimal cast

[ https://issues.apache.org/jira/browse/SPARK-37451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37451: -- Priority: Blocker (was: Major) > Performance improvement regressed String to Decimal cast >

[jira] [Updated] (SPARK-37451) Performance improvement regressed String to Decimal cast

[ https://issues.apache.org/jira/browse/SPARK-37451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37451: -- Target Version/s: 3.2.0, 3.1.3 Description: A performance improvement to how Spark

[jira] [Commented] (SPARK-37586) Add cipher mode option and set default cipher mode for aes_encrypt and aes_decrypt

[ https://issues.apache.org/jira/browse/SPARK-37586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455351#comment-17455351 ] Apache Spark commented on SPARK-37586: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37586) Add cipher mode option and set default cipher mode for aes_encrypt and aes_decrypt

[ https://issues.apache.org/jira/browse/SPARK-37586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37586: Assignee: (was: Apache Spark) > Add cipher mode option and set default cipher mode

[jira] [Assigned] (SPARK-37586) Add cipher mode option and set default cipher mode for aes_encrypt and aes_decrypt

[ https://issues.apache.org/jira/browse/SPARK-37586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37586: Assignee: Apache Spark > Add cipher mode option and set default cipher mode for

[jira] [Created] (SPARK-37586) Add cipher mode option and set default cipher mode for aes_encrypt and aes_decrypt

Max Gekk created SPARK-37586: Summary: Add cipher mode option and set default cipher mode for aes_encrypt and aes_decrypt Key: SPARK-37586 URL: https://issues.apache.org/jira/browse/SPARK-37586 Project:

[jira] [Resolved] (SPARK-37576) Support built-in K8s executor roll plugin

[ https://issues.apache.org/jira/browse/SPARK-37576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37576. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34832

[jira] [Assigned] (SPARK-37576) Support built-in K8s executor roll plugin

[ https://issues.apache.org/jira/browse/SPARK-37576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37576: - Assignee: Dongjoon Hyun > Support built-in K8s executor roll plugin >

[jira] [Commented] (SPARK-37583) Support the binary type by startswith() and endswith()

[ https://issues.apache.org/jira/browse/SPARK-37583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455290#comment-17455290 ] Max Gekk commented on SPARK-37583: -- [~angerszhuuu] Sure, you can combine it to one PR. > Support the

[jira] [Commented] (SPARK-37583) Support the binary type by startswith() and endswith()

[ https://issues.apache.org/jira/browse/SPARK-37583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455262#comment-17455262 ] angerszhu commented on SPARK-37583: --- I think this should be done together with contains. > Support

[jira] [Created] (SPARK-37585) DSV2 InputMetrics are not getting update in corner case

Sandeep Katta created SPARK-37585: - Summary: DSV2 InputMetrics are not getting update in corner case Key: SPARK-37585 URL: https://issues.apache.org/jira/browse/SPARK-37585 Project: Spark

[jira] [Assigned] (SPARK-37584) New SQL function: map_contains_key

[ https://issues.apache.org/jira/browse/SPARK-37584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37584: Assignee: Apache Spark (was: Gengliang Wang) > New SQL function: map_contains_key >

[jira] [Assigned] (SPARK-37584) New SQL function: map_contains_key

[ https://issues.apache.org/jira/browse/SPARK-37584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37584: Assignee: Apache Spark (was: Gengliang Wang) > New SQL function: map_contains_key >

[jira] [Commented] (SPARK-37584) New SQL function: map_contains_key

[ https://issues.apache.org/jira/browse/SPARK-37584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455204#comment-17455204 ] Apache Spark commented on SPARK-37584: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-37584) New SQL function: map_contains_key

[ https://issues.apache.org/jira/browse/SPARK-37584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37584: Assignee: Gengliang Wang (was: Apache Spark) > New SQL function: map_contains_key >

[jira] [Updated] (SPARK-37584) New SQL function: map_contains_key

[ https://issues.apache.org/jira/browse/SPARK-37584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-37584: --- Summary: New SQL function: map_contains_key (was: New function: map_contains_key) > New

[jira] [Commented] (SPARK-37445) Update hadoop-profile

[ https://issues.apache.org/jira/browse/SPARK-37445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455202#comment-17455202 ] Apache Spark commented on SPARK-37445: -- User 'sarutak' has created a pull request for this issue:

[jira] [Created] (SPARK-37584) New function: map_contains_key

Gengliang Wang created SPARK-37584: -- Summary: New function: map_contains_key Key: SPARK-37584 URL: https://issues.apache.org/jira/browse/SPARK-37584 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-37582) Support the binary type by contains()

[ https://issues.apache.org/jira/browse/SPARK-37582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455200#comment-17455200 ] angerszhu commented on SPARK-37582: --- Sure, will raise a pr soon > Support the binary type by

[jira] [Created] (SPARK-37583) Support the binary type by startswith() and endswith()

Max Gekk created SPARK-37583: Summary: Support the binary type by startswith() and endswith() Key: SPARK-37583 URL: https://issues.apache.org/jira/browse/SPARK-37583 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37582) Support the binary type by contains()

[ https://issues.apache.org/jira/browse/SPARK-37582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455194#comment-17455194 ] Max Gekk commented on SPARK-37582: -- [~angerszhuuu] Would you like to work on this? If so, please, leave

[jira] [Created] (SPARK-37582) Support the binary type by contains()

Max Gekk created SPARK-37582: Summary: Support the binary type by contains() Key: SPARK-37582 URL: https://issues.apache.org/jira/browse/SPARK-37582 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-37581) sql hang at planning stage

ocean created SPARK-37581: - Summary: sql hang at planning stage Key: SPARK-37581 URL: https://issues.apache.org/jira/browse/SPARK-37581 Project: Spark Issue Type: Bug Components: SQL

[jira] [Assigned] (SPARK-37580) Optimize current TaskSetManager abort logic when task failed count reach the threshold

[ https://issues.apache.org/jira/browse/SPARK-37580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37580: Assignee: Apache Spark > Optimize current TaskSetManager abort logic when task failed

[jira] [Assigned] (SPARK-37580) Optimize current TaskSetManager abort logic when task failed count reach the threshold

[ https://issues.apache.org/jira/browse/SPARK-37580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37580: Assignee: (was: Apache Spark) > Optimize current TaskSetManager abort logic when

[jira] [Commented] (SPARK-37580) Optimize current TaskSetManager abort logic when task failed count reach the threshold

[ https://issues.apache.org/jira/browse/SPARK-37580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455094#comment-17455094 ] Apache Spark commented on SPARK-37580: -- User 'wangshengjie123' has created a pull request for this

[jira] [Created] (SPARK-37580) Optimize current TaskSetManager abort logic when task failed count reach the threshold

wangshengjie created SPARK-37580: Summary: Optimize current TaskSetManager abort logic when task failed count reach the threshold Key: SPARK-37580 URL: https://issues.apache.org/jira/browse/SPARK-37580

[jira] [Created] (SPARK-37579) Called spark.sql multiple times，union multiple DataFrame, groupBy and pivot, join other table view cause exception

page created SPARK-37579: Summary: Called spark.sql multiple times，union multiple DataFrame, groupBy and pivot, join other table view cause exception Key: SPARK-37579 URL:

[jira] [Created] (SPARK-37578) DSV2 is not updating Output Metrics

Sandeep Katta created SPARK-37578: - Summary: DSV2 is not updating Output Metrics Key: SPARK-37578 URL: https://issues.apache.org/jira/browse/SPARK-37578 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455055#comment-17455055 ] Hyukjin Kwon commented on SPARK-37575: -- Yeah, it would be great to fix it, and upgrade the version

[jira] [Commented] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455053#comment-17455053 ] Guo Wei commented on SPARK-37575: - I think I can make a fast repair to verify my conclusion and add some

[jira] [Comment Edited] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455043#comment-17455043 ] Guo Wei edited comment on SPARK-37575 at 12/8/21, 8:41 AM: --- I think I maybe

[jira] [Comment Edited] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455043#comment-17455043 ] Guo Wei edited comment on SPARK-37575 at 12/8/21, 8:40 AM: --- I think I maybe

[jira] [Comment Edited] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455043#comment-17455043 ] Guo Wei edited comment on SPARK-37575 at 12/8/21, 8:40 AM: --- I think I maybe

[jira] [Commented] (SPARK-37575) Empty strings and null values are both saved as quoted empty Strings "" rather than "" (for empty strings) and nothing(for null values)

[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455043#comment-17455043 ] Guo Wei commented on SPARK-37575: - I think I maybe have found the root cause through debug Spark source

[jira] [Commented] (SPARK-37533) New SQL function: try_element_at

[ https://issues.apache.org/jira/browse/SPARK-37533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455039#comment-17455039 ] Apache Spark commented on SPARK-37533: -- User 'gengliangwang' has created a pull request for this

[jira] [Updated] (SPARK-37570) mypy breaks on pyspark.pandas.plot.core.Bucketizer

[ https://issues.apache.org/jira/browse/SPARK-37570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafal Wojdyla updated SPARK-37570: -- Environment: Mypy version: v0.910-1 Py: 3.9 > mypy breaks on

[jira] [Updated] (SPARK-37570) mypy breaks on pyspark.pandas.plot.core.Bucketizer

[ https://issues.apache.org/jira/browse/SPARK-37570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafal Wojdyla updated SPARK-37570: -- Environment: Mypy: v0.910-1 Py: 3.9 was: Mypy version: v0.910-1 Py: 3.9 > mypy breaks on

  1   2   >