[jira] [Updated] (SPARK-35690) Stream-stream join keys should be reordered properly

2021-06-08 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-35690: - Issue Type: Improvement (was: Documentation) > Stream-stream join keys should be reordered properly >

[jira] [Comment Edited] (SPARK-35688) GeneratePredicate eliminate will fail in some case

2021-06-08 Thread todd.chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359775#comment-17359775 ] todd.chen edited comment on SPARK-35688 at 6/9/21, 5:47 AM: in this case

[jira] [Commented] (SPARK-35688) GeneratePredicate eliminate will fail in some case

2021-06-08 Thread todd.chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359775#comment-17359775 ] todd.chen commented on SPARK-35688: --- in this case ,because get_json_object is  CodegenFallback, so in 

[jira] [Commented] (SPARK-35673) Spark fails on unrecognized hint in subquery

2021-06-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359773#comment-17359773 ] Wenchen Fan commented on SPARK-35673: - Can we simply change `UnresolvedHint.resolved` to true? >

[jira] [Commented] (SPARK-35690) Stream-stream join keys should be reordered properly

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359772#comment-17359772 ] Apache Spark commented on SPARK-35690: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35690) Stream-stream join keys should be reordered properly

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35690: Assignee: (was: Apache Spark) > Stream-stream join keys should be reordered properly

[jira] [Commented] (SPARK-35690) Stream-stream join keys should be reordered properly

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359771#comment-17359771 ] Apache Spark commented on SPARK-35690: -- User 'c21' has created a pull request for this issue:

[jira] [Updated] (SPARK-35688) GeneratePredicate eliminate will fail in some case

2021-06-08 Thread todd.chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] todd.chen updated SPARK-35688: -- Docs Text: code:  ```scala val sparkSession = SparkSession .builder() .config(sparkConf)

[jira] [Assigned] (SPARK-35690) Stream-stream join keys should be reordered properly

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35690: Assignee: Apache Spark > Stream-stream join keys should be reordered properly >

[jira] [Updated] (SPARK-35688) GeneratePredicate eliminate will fail in some case

2021-06-08 Thread todd.chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] todd.chen updated SPARK-35688: -- Docs Text: code:  ``` val sparkSession = SparkSession .builder() .config(sparkConf) .getOrCreate()

[jira] [Assigned] (SPARK-35689) Add logging for null value retrieval for SymmetricHashJoinStateManager

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35689: Assignee: Apache Spark (was: L. C. Hsieh) > Add logging for null value retrieval for

[jira] [Commented] (SPARK-35689) Add logging for null value retrieval for SymmetricHashJoinStateManager

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359768#comment-17359768 ] Apache Spark commented on SPARK-35689: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35689) Add logging for null value retrieval for SymmetricHashJoinStateManager

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35689: Assignee: L. C. Hsieh (was: Apache Spark) > Add logging for null value retrieval for

[jira] [Created] (SPARK-35690) Stream-stream join keys should be reordered properly

2021-06-08 Thread Cheng Su (Jira)
Cheng Su created SPARK-35690: Summary: Stream-stream join keys should be reordered properly Key: SPARK-35690 URL: https://issues.apache.org/jira/browse/SPARK-35690 Project: Spark Issue Type:

[jira] [Created] (SPARK-35689) Add logging for null value retrieval for SymmetricHashJoinStateManager

2021-06-08 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-35689: --- Summary: Add logging for null value retrieval for SymmetricHashJoinStateManager Key: SPARK-35689 URL: https://issues.apache.org/jira/browse/SPARK-35689 Project: Spark

[jira] [Created] (SPARK-35688) GeneratePredicate eliminate will fail in some case

2021-06-08 Thread todd.chen (Jira)
todd.chen created SPARK-35688: - Summary: GeneratePredicate eliminate will fail in some case Key: SPARK-35688 URL: https://issues.apache.org/jira/browse/SPARK-35688 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-35436) RocksDBFileManager - save checkpoint to DFS

2021-06-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-35436: Assignee: Yuanjian Li > RocksDBFileManager - save checkpoint to DFS >

[jira] [Resolved] (SPARK-35436) RocksDBFileManager - save checkpoint to DFS

2021-06-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-35436. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32582

[jira] [Commented] (SPARK-35687) PythonUDFSuite move assume into its methods

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359759#comment-17359759 ] Apache Spark commented on SPARK-35687: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-35687) PythonUDFSuite move assume into its methods

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35687: Assignee: Apache Spark > PythonUDFSuite move assume into its methods >

[jira] [Assigned] (SPARK-35687) PythonUDFSuite move assume into its methods

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35687: Assignee: (was: Apache Spark) > PythonUDFSuite move assume into its methods >

[jira] [Created] (SPARK-35687) PythonUDFSuite move assume into its methods

2021-06-08 Thread XiDuo You (Jira)
XiDuo You created SPARK-35687: - Summary: PythonUDFSuite move assume into its methods Key: SPARK-35687 URL: https://issues.apache.org/jira/browse/SPARK-35687 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-35378) Eagerly execute commands in QueryExecution instead of caller sides

2021-06-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35378. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32513

[jira] [Assigned] (SPARK-35378) Eagerly execute commands in QueryExecution instead of caller sides

2021-06-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35378: --- Assignee: jiaan.geng > Eagerly execute commands in QueryExecution instead of caller sides

[jira] [Assigned] (SPARK-35686) Avoid using auto generated alias when creating view

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35686: Assignee: Apache Spark > Avoid using auto generated alias when creating view >

[jira] [Assigned] (SPARK-35686) Avoid using auto generated alias when creating view

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35686: Assignee: Apache Spark > Avoid using auto generated alias when creating view >

[jira] [Commented] (SPARK-35686) Avoid using auto generated alias when creating view

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359742#comment-17359742 ] Apache Spark commented on SPARK-35686: -- User 'linhongliu-db' has created a pull request for this

[jira] [Commented] (SPARK-35686) Avoid using auto generated alias when creating view

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359743#comment-17359743 ] Apache Spark commented on SPARK-35686: -- User 'linhongliu-db' has created a pull request for this

[jira] [Assigned] (SPARK-35686) Avoid using auto generated alias when creating view

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35686: Assignee: (was: Apache Spark) > Avoid using auto generated alias when creating view

[jira] [Commented] (SPARK-35602) Job crashes with java.io.UTFDataFormatException: encoded string too long

2021-06-08 Thread dejan miljkovic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359741#comment-17359741 ] dejan miljkovic commented on SPARK-35602: - [~kabhwan], Sorry can not share the schema. I am

[jira] [Commented] (SPARK-35679) Overflow on converting valid Timestamp to Microseconds

2021-06-08 Thread dc-heros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359730#comment-17359730 ] dc-heros commented on SPARK-35679: -- I will create a pull request soon!   > Overflow on converting

[jira] [Assigned] (SPARK-35647) Merge contents and remove obsolete pages in User Guide section

2021-06-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35647: Assignee: Hyukjin Kwon > Merge contents and remove obsolete pages in User Guide section

[jira] [Resolved] (SPARK-35647) Merge contents and remove obsolete pages in User Guide section

2021-06-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35647. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32820

[jira] [Issue Comment Deleted] (SPARK-35679) Overflow on converting valid Timestamp to Microseconds

2021-06-08 Thread Kevin Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Su updated SPARK-35679: - Comment: was deleted (was: Long.MinValue = -9223372036854775808 I think the time should not be

[jira] [Commented] (SPARK-35679) Overflow on converting valid Timestamp to Microseconds

2021-06-08 Thread Kevin Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359727#comment-17359727 ] Kevin Su commented on SPARK-35679: -- Long.MinValue = -9223372036854775808 I think the time should not

[jira] [Created] (SPARK-35686) Avoid using auto generated alias when creating view

2021-06-08 Thread Linhong Liu (Jira)
Linhong Liu created SPARK-35686: --- Summary: Avoid using auto generated alias when creating view Key: SPARK-35686 URL: https://issues.apache.org/jira/browse/SPARK-35686 Project: Spark Issue

[jira] [Commented] (SPARK-35685) Prompt recreating the View when there is a schema incompatible change

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359725#comment-17359725 ] Apache Spark commented on SPARK-35685: -- User 'linhongliu-db' has created a pull request for this

[jira] [Commented] (SPARK-35685) Prompt recreating the View when there is a schema incompatible change

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359722#comment-17359722 ] Apache Spark commented on SPARK-35685: -- User 'linhongliu-db' has created a pull request for this

[jira] [Assigned] (SPARK-35685) Prompt recreating the View when there is a schema incompatible change

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35685: Assignee: (was: Apache Spark) > Prompt recreating the View when there is a schema

[jira] [Assigned] (SPARK-35685) Prompt recreating the View when there is a schema incompatible change

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35685: Assignee: Apache Spark > Prompt recreating the View when there is a schema incompatible

[jira] [Commented] (SPARK-35673) Spark fails on unrecognized hint in subquery

2021-06-08 Thread Fu Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359716#comment-17359716 ] Fu Chen commented on SPARK-35673: - Hi, [~maropu]. I think a better implement for injecting user-defined

[jira] [Commented] (SPARK-35667) spark.speculation causes incorrect query results with TRANSFORM

2021-06-08 Thread yuanxm (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359710#comment-17359710 ] yuanxm commented on SPARK-35667: Tried 3.1.2, still have this problem [~angerszhuuu] [~maropu] >

[jira] [Created] (SPARK-35685) Prompt recreating the View when there is a schema incompatible change

2021-06-08 Thread Linhong Liu (Jira)
Linhong Liu created SPARK-35685: --- Summary: Prompt recreating the View when there is a schema incompatible change Key: SPARK-35685 URL: https://issues.apache.org/jira/browse/SPARK-35685 Project: Spark

[jira] [Commented] (SPARK-32975) Add config for driver readiness timeout before executors start

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359707#comment-17359707 ] Apache Spark commented on SPARK-32975: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-35673) Spark fails on unrecognized hint in subquery

2021-06-08 Thread Fu Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359704#comment-17359704 ] Fu Chen edited comment on SPARK-35673 at 6/9/21, 2:22 AM: -- Yes, we can't.  The

[jira] [Commented] (SPARK-35673) Spark fails on unrecognized hint in subquery

2021-06-08 Thread Fu Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359704#comment-17359704 ] Fu Chen commented on SPARK-35673: - Yes, we can't.  The user-defined hint is injected in batch

[jira] [Resolved] (SPARK-35682) Pin mypy version in GitHub Actions CI

2021-06-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35682. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32829

[jira] [Assigned] (SPARK-35682) Pin mypy version in GitHub Actions CI

2021-06-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35682: Assignee: Hyukjin Kwon > Pin mypy version in GitHub Actions CI >

[jira] [Updated] (SPARK-35601) Complete arithmetic operators involving bool literals, Series, and Index

2021-06-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35601: - Description: Completing arithmetic operators involving bool literals, Series, and Index

[jira] [Assigned] (SPARK-35512) pyspark partitionBy may encounter 'OverflowError: cannot convert float infinity to integer'

2021-06-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35512: Assignee: nolan liu > pyspark partitionBy may encounter 'OverflowError: cannot convert

[jira] [Resolved] (SPARK-35512) pyspark partitionBy may encounter 'OverflowError: cannot convert float infinity to integer'

2021-06-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35512. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32667

[jira] [Updated] (SPARK-35601) Complete arithmetic operators involving bool literals, Series, and Index

2021-06-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35601: - Summary: Complete arithmetic operators involving bool literals, Series, and Index (was:

[jira] [Resolved] (SPARK-35681) Support arithmetic operators (+, *) among bool Series/Index

2021-06-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-35681. -- Resolution: Duplicate > Support arithmetic operators (+, *) among bool Series/Index >

[jira] [Commented] (SPARK-35602) Job crashes with java.io.UTFDataFormatException: encoded string too long

2021-06-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359693#comment-17359693 ] Jungtaek Lim commented on SPARK-35602: -- Very interested that the json representation of the schema

[jira] [Assigned] (SPARK-35602) Job crashes with java.io.UTFDataFormatException: encoded string too long

2021-06-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-35602: Assignee: Kousuke Saruta > Job crashes with java.io.UTFDataFormatException: encoded

[jira] [Resolved] (SPARK-35602) Job crashes with java.io.UTFDataFormatException: encoded string too long

2021-06-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-35602. -- Fix Version/s: 3.1.3 3.2.0 Resolution: Fixed Issue resolved by pull

[jira] [Commented] (SPARK-29626) notEqual() should return true when the one is null, the other is not null

2021-06-08 Thread dc-heros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359686#comment-17359686 ] dc-heros commented on SPARK-29626: -- I think this behavior is acceptable as it's similar to the behavior

[jira] [Assigned] (SPARK-35682) Pin mypy version in GitHub Actions CI

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35682: Assignee: Apache Spark > Pin mypy version in GitHub Actions CI >

[jira] [Commented] (SPARK-35682) Pin mypy version in GitHub Actions CI

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359682#comment-17359682 ] Apache Spark commented on SPARK-35682: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-35682) Pin mypy version in GitHub Actions CI

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359681#comment-17359681 ] Apache Spark commented on SPARK-35682: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-35682) Pin mypy version in GitHub Actions CI

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35682: Assignee: (was: Apache Spark) > Pin mypy version in GitHub Actions CI >

[jira] [Created] (SPARK-35684) Bump up mypy version in GitHub Actions

2021-06-08 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-35684: Summary: Bump up mypy version in GitHub Actions Key: SPARK-35684 URL: https://issues.apache.org/jira/browse/SPARK-35684 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-35684) Bump up mypy version in GitHub Actions

2021-06-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35684: Assignee: Haejoon Lee > Bump up mypy version in GitHub Actions >

[jira] [Created] (SPARK-35683) Fix Index.difference to avoid collect 'other' to driver side

2021-06-08 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-35683: Summary: Fix Index.difference to avoid collect 'other' to driver side Key: SPARK-35683 URL: https://issues.apache.org/jira/browse/SPARK-35683 Project: Spark

[jira] [Created] (SPARK-35682) Pin mypy version in GitHub Actions CI

2021-06-08 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-35682: Summary: Pin mypy version in GitHub Actions CI Key: SPARK-35682 URL: https://issues.apache.org/jira/browse/SPARK-35682 Project: Spark Issue Type:

[jira] [Commented] (SPARK-35426) When addMergerLocation exceed the maxRetainedMergerLocations , we should remove the merger based on merged shuffle data size.

2021-06-08 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359673#comment-17359673 ] Min Shen commented on SPARK-35426: -- [~zhuqi], the retained mergers are meant for choosing merger

[jira] [Commented] (SPARK-35549) Register merge status even after shuffle dependency is merge finalized

2021-06-08 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359671#comment-17359671 ] Min Shen commented on SPARK-35549: -- It might not be straightforward to register later MergeStatus.

[jira] [Commented] (SPARK-35673) Spark fails on unrecognized hint in subquery

2021-06-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359663#comment-17359663 ] Takeshi Yamamuro commented on SPARK-35673: -- We cannot move `RemoveAllHints` just after the

[jira] [Commented] (SPARK-35659) Avoid write null to StateStore

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359646#comment-17359646 ] Apache Spark commented on SPARK-35659: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-35384) Improve performance for InvokeLike.invoke

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359598#comment-17359598 ] Apache Spark commented on SPARK-35384: -- User 'sunchao' has created a pull request for this issue:

[jira] [Updated] (SPARK-35659) Avoid write null to StateStore

2021-06-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35659: -- Fix Version/s: 3.0.3 > Avoid write null to StateStore > -- > >

[jira] [Updated] (SPARK-35681) Support arithmetic operators (+, *) among bool Series/Index

2021-06-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35681: - Description: We ought to support arithmetic operators (+, *) between bool Series/Indexes,

[jira] [Updated] (SPARK-35681) Support arithmetic operators (+, *) among bool Series/Index

2021-06-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35681: - Description: We ought to support arithmetic operators (+, *) between bool Series/Indexes,

[jira] [Created] (SPARK-35681) Support arithmetic operators (+, *) among bool Series/Index

2021-06-08 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35681: Summary: Support arithmetic operators (+, *) among bool Series/Index Key: SPARK-35681 URL: https://issues.apache.org/jira/browse/SPARK-35681 Project: Spark

[jira] [Commented] (SPARK-35670) Upgrade ZSTD-JNI to 1.5.0-1

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359553#comment-17359553 ] Apache Spark commented on SPARK-35670: -- User 'dchristle' has created a pull request for this issue:

[jira] [Commented] (SPARK-35670) Upgrade ZSTD-JNI to 1.5.0-1

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359552#comment-17359552 ] Apache Spark commented on SPARK-35670: -- User 'dchristle' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-35673) Spark fails on unrecognized hint in subquery

2021-06-08 Thread Fu Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359524#comment-17359524 ] Fu Chen edited comment on SPARK-35673 at 6/8/21, 6:49 PM: -- Hi, [~maropu]  This

[jira] [Comment Edited] (SPARK-35673) Spark fails on unrecognized hint in subquery

2021-06-08 Thread Fu Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359524#comment-17359524 ] Fu Chen edited comment on SPARK-35673 at 6/8/21, 6:49 PM: -- Hi, [~maropu]  This

[jira] [Commented] (SPARK-35673) Spark fails on unrecognized hint in subquery

2021-06-08 Thread Fu Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359524#comment-17359524 ] Fu Chen commented on SPARK-35673: - Hi, [~maropu]  This is because

[jira] [Commented] (SPARK-35680) Support units by year-month interval type

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359491#comment-17359491 ] Apache Spark commented on SPARK-35680: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-35680) Support units by year-month interval type

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359490#comment-17359490 ] Apache Spark commented on SPARK-35680: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35680) Support units by year-month interval type

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35680: Assignee: Apache Spark (was: Max Gekk) > Support units by year-month interval type >

[jira] [Assigned] (SPARK-35680) Support units by year-month interval type

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35680: Assignee: Max Gekk (was: Apache Spark) > Support units by year-month interval type >

[jira] [Created] (SPARK-35680) Support units by year-month interval type

2021-06-08 Thread Max Gekk (Jira)
Max Gekk created SPARK-35680: Summary: Support units by year-month interval type Key: SPARK-35680 URL: https://issues.apache.org/jira/browse/SPARK-35680 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-35659) Avoid write null to StateStore

2021-06-08 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359470#comment-17359470 ] L. C. Hsieh commented on SPARK-35659: - The issue was resolved at

[jira] [Resolved] (SPARK-35659) Avoid write null to StateStore

2021-06-08 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35659. - Resolution: Fixed > Avoid write null to StateStore > -- > >

[jira] [Updated] (SPARK-35659) Avoid write null to StateStore

2021-06-08 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35659: Fix Version/s: 3.1.3 3.2.0 > Avoid write null to StateStore >

[jira] [Commented] (SPARK-30993) GenerateUnsafeRowJoiner corrupts the value if the datatype is UDF and its sql type has fixed length

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359462#comment-17359462 ] Apache Spark commented on SPARK-30993: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-30993) GenerateUnsafeRowJoiner corrupts the value if the datatype is UDF and its sql type has fixed length

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359461#comment-17359461 ] Apache Spark commented on SPARK-30993: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-35659) Avoid write null to StateStore

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359459#comment-17359459 ] Apache Spark commented on SPARK-35659: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-35659) Avoid write null to StateStore

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359458#comment-17359458 ] Apache Spark commented on SPARK-35659: -- User 'viirya' has created a pull request for this issue:

[jira] [Updated] (SPARK-35679) Overflow on converting valid Timestamp to Microseconds

2021-06-08 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-35679: --- Summary: Overflow on converting valid Timestamp to Microseconds (was: Overflow on

[jira] [Updated] (SPARK-35679) Overflow on converting Timestamp to Microseconds

2021-06-08 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-35679: --- Summary: Overflow on converting Timestamp to Microseconds (was: Overflow on converting

[jira] [Created] (SPARK-35679) Overflow on converting Long.MinValue to Timestamp

2021-06-08 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35679: -- Summary: Overflow on converting Long.MinValue to Timestamp Key: SPARK-35679 URL: https://issues.apache.org/jira/browse/SPARK-35679 Project: Spark Issue

[jira] [Updated] (SPARK-35668) Use "concurrency" setting on Github Actions

2021-06-08 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-35668: Description: We are using

[jira] [Assigned] (SPARK-35312) Introduce new Option in Kafka source to specify minimum number of records to read per trigger

2021-06-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-35312: Assignee: Satish Gopalani > Introduce new Option in Kafka source to specify minimum

[jira] [Resolved] (SPARK-35312) Introduce new Option in Kafka source to specify minimum number of records to read per trigger

2021-06-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-35312. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32653

[jira] [Assigned] (SPARK-35678) add a common softmax function

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35678: Assignee: (was: Apache Spark) > add a common softmax function >

[jira] [Commented] (SPARK-35678) add a common softmax function

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359342#comment-17359342 ] Apache Spark commented on SPARK-35678: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-35678) add a common softmax function

2021-06-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35678: Assignee: Apache Spark > add a common softmax function > - >

  1   2   >