[jira] [Created] (SPARK-35871) Literal.create(value, dataType) should support fields

2021-06-23 Thread angerszhu (Jira)
angerszhu created SPARK-35871: - Summary: Literal.create(value, dataType) should support fields Key: SPARK-35871 URL: https://issues.apache.org/jira/browse/SPARK-35871 Project: Spark Issue Type: S

[jira] [Assigned] (SPARK-35870) Upgrade Jetty to 9.4.42

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35870: Assignee: Apache Spark (was: Kousuke Saruta) > Upgrade Jetty to 9.4.42 > ---

[jira] [Assigned] (SPARK-35870) Upgrade Jetty to 9.4.42

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35870: Assignee: Kousuke Saruta (was: Apache Spark) > Upgrade Jetty to 9.4.42 > ---

[jira] [Commented] (SPARK-35870) Upgrade Jetty to 9.4.42

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368657#comment-17368657 ] Apache Spark commented on SPARK-35870: -- User 'sarutak' has created a pull request f

[jira] [Commented] (SPARK-35778) Check multiply/divide of year-month intervals of any fields by numeric

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368656#comment-17368656 ] Apache Spark commented on SPARK-35778: -- User 'Peng-Lei' has created a pull request

[jira] [Commented] (SPARK-35778) Check multiply/divide of year-month intervals of any fields by numeric

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368655#comment-17368655 ] Apache Spark commented on SPARK-35778: -- User 'Peng-Lei' has created a pull request

[jira] [Commented] (SPARK-35696) Refine the code examples in the pandas API on Spark documentation.

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368654#comment-17368654 ] Apache Spark commented on SPARK-35696: -- User 'itholic' has created a pull request f

[jira] [Created] (SPARK-35870) Upgrade Jetty to 9.4.42

2021-06-23 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35870: -- Summary: Upgrade Jetty to 9.4.42 Key: SPARK-35870 URL: https://issues.apache.org/jira/browse/SPARK-35870 Project: Spark Issue Type: Bug Compone

[jira] [Updated] (SPARK-35870) Upgrade Jetty to 9.4.42

2021-06-23 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35870: --- Component/s: (was: Spark Core) Build > Upgrade Jetty to 9.4.42 > --

[jira] [Assigned] (SPARK-35747) Avoid printing full Exception stack trace, if HBase/Kafka/Hive services are not running in a secure cluster

2021-06-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35747: - Assignee: Vinod KC > Avoid printing full Exception stack trace, if HBase/Kafka/Hive ser

[jira] [Resolved] (SPARK-35747) Avoid printing full Exception stack trace, if HBase/Kafka/Hive services are not running in a secure cluster

2021-06-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35747. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32894 [https://

[jira] [Resolved] (SPARK-35777) Check all year-month interval types in UDF

2021-06-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35777. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32985 [https://github.com

[jira] [Assigned] (SPARK-35777) Check all year-month interval types in UDF

2021-06-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35777: Assignee: angerszhu > Check all year-month interval types in UDF > --

[jira] [Commented] (SPARK-35778) Check multiply/divide of year-month intervals of any fields by numeric

2021-06-23 Thread PengLei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368636#comment-17368636 ] PengLei commented on SPARK-35778: - [~angerszhuuu] May I continue to finish this question

[jira] [Resolved] (SPARK-34885) Port/integrate Koalas documentation into PySpark

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34885. -- Resolution: Done > Port/integrate Koalas documentation into PySpark >

[jira] [Resolved] (SPARK-35696) Refine the code examples in the pandas API on Spark documentation.

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35696. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33017 [https://gi

[jira] [Assigned] (SPARK-35696) Refine the code examples in the pandas API on Spark documentation.

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35696: Assignee: Haejoon Lee > Refine the code examples in the pandas API on Spark documentation

[jira] [Resolved] (SPARK-35868) Add fs.s3a.downgrade.syncable.exceptions if not set

2021-06-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35868. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33044 [https://

[jira] [Assigned] (SPARK-35852) Improve the implementation for DateType +/- DayTimeIntervalType(DAY)

2021-06-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35852: Assignee: PengLei > Improve the implementation for DateType +/- DayTimeIntervalType(DAY) > --

[jira] [Resolved] (SPARK-35852) Improve the implementation for DateType +/- DayTimeIntervalType(DAY)

2021-06-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35852. -- Resolution: Fixed Issue resolved by pull request 33033 [https://github.com/apache/spark/pull/33033] >

[jira] [Assigned] (SPARK-35868) Add fs.s3a.downgrade.syncable.exceptions if not set

2021-06-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35868: - Assignee: Dongjoon Hyun > Add fs.s3a.downgrade.syncable.exceptions if not set > ---

[jira] [Assigned] (SPARK-35301) Document migration guide from Koalas to pandas APIs on Spark

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35301: Assignee: (was: Apache Spark) > Document migration guide from Koalas to pandas APIs o

[jira] [Commented] (SPARK-35301) Document migration guide from Koalas to pandas APIs on Spark

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368632#comment-17368632 ] Apache Spark commented on SPARK-35301: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-35301) Document migration guide from Koalas to pandas APIs on Spark

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35301: Assignee: Apache Spark > Document migration guide from Koalas to pandas APIs on Spark > -

[jira] [Updated] (SPARK-35783) Set the list of read columns in the task configuration to reduce reading of ORC data.

2021-06-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35783: -- Fix Version/s: 3.0.4 3.1.3 > Set the list of read columns in the task confi

[jira] [Updated] (SPARK-35783) Set the list of read columns in the task configuration to reduce reading of ORC data.

2021-06-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35783: -- Issue Type: Bug (was: Improvement) > Set the list of read columns in the task configuration t

[jira] [Updated] (SPARK-35783) Set the list of read columns in the task configuration to reduce reading of ORC data.

2021-06-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35783: -- Affects Version/s: 3.0.3 3.1.2 > Set the list of read columns in the ta

[jira] [Updated] (SPARK-35605) Move to_pandas_on_spark to the Spark DataFrame.

2021-06-23 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35605: Summary: Move to_pandas_on_spark to the Spark DataFrame. (was: Move pandas_on_spark accessor to t

[jira] [Commented] (SPARK-35678) add a common softmax function

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368602#comment-17368602 ] Apache Spark commented on SPARK-35678: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-35678) add a common softmax function

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368603#comment-17368603 ] Apache Spark commented on SPARK-35678: -- User 'zhengruifeng' has created a pull requ

[jira] [Resolved] (SPARK-35869) Cannot run program "python" error when run do-release-docker.sh

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35869. -- Fix Version/s: 3.2.0 Assignee: wuyi Resolution: Fixed Fixed in https://github.

[jira] [Commented] (SPARK-35869) Cannot run program "python" error when run do-release-docker.sh

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368597#comment-17368597 ] Apache Spark commented on SPARK-35869: -- User 'Ngone51' has created a pull request f

[jira] [Assigned] (SPARK-35869) Cannot run program "python" error when run do-release-docker.sh

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35869: Assignee: (was: Apache Spark) > Cannot run program "python" error when run do-release

[jira] [Commented] (SPARK-35869) Cannot run program "python" error when run do-release-docker.sh

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368596#comment-17368596 ] Apache Spark commented on SPARK-35869: -- User 'Ngone51' has created a pull request f

[jira] [Assigned] (SPARK-35869) Cannot run program "python" error when run do-release-docker.sh

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35869: Assignee: Apache Spark > Cannot run program "python" error when run do-release-docker.sh

[jira] [Assigned] (SPARK-35730) Check all day-time interval types in UDF

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35730: Assignee: (was: Apache Spark) > Check all day-time interval types in UDF > --

[jira] [Assigned] (SPARK-35730) Check all day-time interval types in UDF

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35730: Assignee: Apache Spark > Check all day-time interval types in UDF > -

[jira] [Commented] (SPARK-35730) Check all day-time interval types in UDF

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368594#comment-17368594 ] Apache Spark commented on SPARK-35730: -- User 'AngersZh' has created a pull requ

[jira] [Created] (SPARK-35869) Cannot run program "python" error when run do-release-docker.sh

2021-06-23 Thread wuyi (Jira)
wuyi created SPARK-35869: Summary: Cannot run program "python" error when run do-release-docker.sh Key: SPARK-35869 URL: https://issues.apache.org/jira/browse/SPARK-35869 Project: Spark Issue Type:

[jira] [Commented] (SPARK-35282) Support AQE side shuffled hash join formula

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368563#comment-17368563 ] Apache Spark commented on SPARK-35282: -- User 'ulysses-you' has created a pull reque

[jira] [Resolved] (SPARK-35569) Document the deprecation of Koalas Accessor after porting the documentation.

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35569. -- Resolution: Duplicate I will do it in SPARK-35301 > Document the deprecation of Koalas Access

[jira] [Assigned] (SPARK-34807) Push down filter through window after TransposeWindow

2021-06-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-34807: --- Assignee: Tanel Kiis > Push down filter through window after TransposeWindow >

[jira] [Resolved] (SPARK-34807) Push down filter through window after TransposeWindow

2021-06-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-34807. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31980 [https://gith

[jira] [Commented] (SPARK-35730) Check all day-time interval types in UDF

2021-06-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368556#comment-17368556 ] angerszhu commented on SPARK-35730: --- working on this > Check all day-time interval ty

[jira] [Commented] (SPARK-35623) Volcano resource manager for Spark on Kubernetes

2021-06-23 Thread Kevin Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368553#comment-17368553 ] Kevin Su commented on SPARK-35623: -- [~dipanjanK] Here is my email address. pings...@gma

[jira] [Commented] (SPARK-33863) Pyspark UDF wrongly changes timestamps to UTC

2021-06-23 Thread Nasir Ali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368550#comment-17368550 ] Nasir Ali commented on SPARK-33863: --- [~dc-heros] Could you please share the output you

[jira] [Reopened] (SPARK-33863) Pyspark UDF wrongly changes timestamps to UTC

2021-06-23 Thread Nasir Ali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nasir Ali reopened SPARK-33863: --- Issue is not resolved.  > Pyspark UDF wrongly changes timestamps to UTC > -

[jira] [Updated] (SPARK-33863) Pyspark UDF wrongly changes timestamps to UTC

2021-06-23 Thread Nasir Ali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nasir Ali updated SPARK-33863: -- Affects Version/s: 3.1.2 > Pyspark UDF wrongly changes timestamps to UTC > ---

[jira] [Commented] (SPARK-35476) Enable disallow_untyped_defs mypy check for pyspark.pandas.series.

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368546#comment-17368546 ] Apache Spark commented on SPARK-35476: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-35476) Enable disallow_untyped_defs mypy check for pyspark.pandas.series.

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35476: Assignee: Apache Spark > Enable disallow_untyped_defs mypy check for pyspark.pandas.serie

[jira] [Commented] (SPARK-35476) Enable disallow_untyped_defs mypy check for pyspark.pandas.series.

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368545#comment-17368545 ] Apache Spark commented on SPARK-35476: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-35476) Enable disallow_untyped_defs mypy check for pyspark.pandas.series.

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35476: Assignee: (was: Apache Spark) > Enable disallow_untyped_defs mypy check for pyspark.p

[jira] [Resolved] (SPARK-35588) Merge Binder integration and quickstart notebook

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35588. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33041 [https://gi

[jira] [Assigned] (SPARK-35588) Merge Binder integration and quickstart notebook

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35588: Assignee: Hyukjin Kwon > Merge Binder integration and quickstart notebook > -

[jira] [Commented] (SPARK-35868) Add fs.s3a.downgrade.syncable.exceptions if not set

2021-06-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368541#comment-17368541 ] Dongjoon Hyun commented on SPARK-35868: --- I marked this as `Blocker` because this c

[jira] [Updated] (SPARK-35868) Add fs.s3a.downgrade.syncable.exceptions if not set

2021-06-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35868: -- Target Version/s: 3.2.0 > Add fs.s3a.downgrade.syncable.exceptions if not set > --

[jira] [Updated] (SPARK-35868) Add fs.s3a.downgrade.syncable.exceptions if not set

2021-06-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35868: -- Priority: Blocker (was: Major) > Add fs.s3a.downgrade.syncable.exceptions if not set > --

[jira] [Assigned] (SPARK-35868) Add fs.s3a.downgrade.syncable.exceptions if not set

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35868: Assignee: (was: Apache Spark) > Add fs.s3a.downgrade.syncable.exceptions if not set >

[jira] [Assigned] (SPARK-35868) Add fs.s3a.downgrade.syncable.exceptions if not set

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35868: Assignee: Apache Spark > Add fs.s3a.downgrade.syncable.exceptions if not set > --

[jira] [Commented] (SPARK-35868) Add fs.s3a.downgrade.syncable.exceptions if not set

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368537#comment-17368537 ] Apache Spark commented on SPARK-35868: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-35868) Add fs.s3a.downgrade.syncable.exceptions if not set

2021-06-23 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35868: - Summary: Add fs.s3a.downgrade.syncable.exceptions if not set Key: SPARK-35868 URL: https://issues.apache.org/jira/browse/SPARK-35868 Project: Spark Issue T

[jira] [Updated] (SPARK-35851) Wrong docs in function GraphGenerators.sampleLogNormal

2021-06-23 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-35851: - Description: In class GraphGenerators docs, function sampleLogNormal use wrong variable to compu

[jira] [Resolved] (SPARK-35851) Wrong docs in function GraphGenerators.sampleLogNormal

2021-06-23 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35851. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33010 [https://gi

[jira] [Assigned] (SPARK-35851) Wrong docs in function GraphGenerators.sampleLogNormal

2021-06-23 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-35851: Assignee: zengrui > Wrong docs in function GraphGenerators.sampleLogNormal >

[jira] [Updated] (SPARK-19256) Hive bucketing write support

2021-06-23 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-19256: - Affects Version/s: 3.2.0 > Hive bucketing write support > > >

[jira] [Commented] (SPARK-19256) Hive bucketing write support

2021-06-23 Thread Pushkar Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368523#comment-17368523 ] Pushkar Kumar commented on SPARK-19256: --- Thank you [~chengsu]!! > Hive bucketing

[jira] [Commented] (SPARK-19256) Hive bucketing write support

2021-06-23 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368522#comment-17368522 ] Cheng Su commented on SPARK-19256: -- [~pushkarcse] - we are currently working on https:

[jira] [Updated] (SPARK-35623) Volcano resource manager for Spark on Kubernetes

2021-06-23 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-35623: - Shepherd: Holden Karau > Volcano resource manager for Spark on Kubernetes >

[jira] [Commented] (SPARK-35623) Volcano resource manager for Spark on Kubernetes

2021-06-23 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368486#comment-17368486 ] Holden Karau commented on SPARK-35623: -- I'm also interested in this. I sent a messa

[jira] [Commented] (SPARK-35476) Enable disallow_untyped_defs mypy check for pyspark.pandas.series.

2021-06-23 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368476#comment-17368476 ] Takuya Ueshin commented on SPARK-35476: --- I'm working on this. > Enable disallow_u

[jira] [Created] (SPARK-35867) Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-06-23 Thread Chao Sun (Jira)
Chao Sun created SPARK-35867: Summary: Enable vectorized read for VectorizedPlainValuesReader.readBooleans Key: SPARK-35867 URL: https://issues.apache.org/jira/browse/SPARK-35867 Project: Spark

[jira] [Created] (SPARK-35866) Improve error message quality

2021-06-23 Thread Karen Feng (Jira)
Karen Feng created SPARK-35866: -- Summary: Improve error message quality Key: SPARK-35866 URL: https://issues.apache.org/jira/browse/SPARK-35866 Project: Spark Issue Type: Improvement C

[jira] [Resolved] (SPARK-35731) Check all day-time interval types in arrow

2021-06-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35731. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33039 [https://github.com

[jira] [Assigned] (SPARK-35731) Check all day-time interval types in arrow

2021-06-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35731: Assignee: angerszhu > Check all day-time interval types in arrow > --

[jira] [Resolved] (SPARK-35729) Check all day-time interval types in aggregate expressions

2021-06-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35729. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33042 [https://github.com

[jira] [Assigned] (SPARK-35729) Check all day-time interval types in aggregate expressions

2021-06-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35729: Assignee: Kousuke Saruta > Check all day-time interval types in aggregate expressions > -

[jira] [Resolved] (SPARK-34889) Introduce MergingSessionsIterator merging elements directly which belong to the same session

2021-06-23 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-34889. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31987 [https://gith

[jira] [Assigned] (SPARK-34889) Introduce MergingSessionsIterator merging elements directly which belong to the same session

2021-06-23 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-34889: --- Assignee: Jungtaek Lim > Introduce MergingSessionsIterator merging elements directly which

[jira] [Updated] (SPARK-35695) QueryExecutionListener does not see any observed metrics fired before persist/cache

2021-06-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35695: -- Fix Version/s: (was: 3.0.4) 3.0.3 > QueryExecutionListener does not see

[jira] [Updated] (SPARK-35695) QueryExecutionListener does not see any observed metrics fired before persist/cache

2021-06-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35695: -- Fix Version/s: (was: 3.0.3) 3.0.4 > QueryExecutionListener does not see

[jira] [Updated] (SPARK-35865) Remove await (syncMode) in ChunkFetchRequestHandler

2021-06-23 Thread Baohe Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baohe Zhang updated SPARK-35865: Description: SPARK-24355 introduces syncMode to mitigate the issue of sasl timeout by throting th

[jira] [Updated] (SPARK-35865) Remove await (syncMode) in ChunkFetchRequestHandler

2021-06-23 Thread Baohe Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baohe Zhang updated SPARK-35865: Description: SPARK-24355 introduces syncMode to mitigate the issue of sasl timeout by throting th

[jira] [Updated] (SPARK-35865) Remove await (syncMode) in ChunkFetchRequestHandler

2021-06-23 Thread Baohe Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baohe Zhang updated SPARK-35865: Attachment: openblock-compare.png > Remove await (syncMode) in ChunkFetchRequestHandler >

[jira] [Created] (SPARK-35865) Remove await (syncMode) in ChunkFetchRequestHandler

2021-06-23 Thread Baohe Zhang (Jira)
Baohe Zhang created SPARK-35865: --- Summary: Remove await (syncMode) in ChunkFetchRequestHandler Key: SPARK-35865 URL: https://issues.apache.org/jira/browse/SPARK-35865 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35865) Remove await (syncMode) in ChunkFetchRequestHandler

2021-06-23 Thread Baohe Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baohe Zhang updated SPARK-35865: Attachment: openblock.png > Remove await (syncMode) in ChunkFetchRequestHandler >

[jira] [Created] (SPARK-35864) CatalogFileIndex only used if Partition Columns Nonempty

2021-06-23 Thread Josh (Jira)
Josh created SPARK-35864: Summary: CatalogFileIndex only used if Partition Columns Nonempty Key: SPARK-35864 URL: https://issues.apache.org/jira/browse/SPARK-35864 Project: Spark Issue Type: Improvem

[jira] [Assigned] (SPARK-35729) Check all day-time interval types in aggregate expressions

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35729: Assignee: (was: Apache Spark) > Check all day-time interval types in aggregate expres

[jira] [Assigned] (SPARK-35729) Check all day-time interval types in aggregate expressions

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35729: Assignee: Apache Spark > Check all day-time interval types in aggregate expressions > ---

[jira] [Commented] (SPARK-35729) Check all day-time interval types in aggregate expressions

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368344#comment-17368344 ] Apache Spark commented on SPARK-35729: -- User 'sarutak' has created a pull request f

[jira] [Resolved] (SPARK-35733) Check all day-time interval types in HiveInspectors tests

2021-06-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35733. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33036 [https://github.com

[jira] [Assigned] (SPARK-35733) Check all day-time interval types in HiveInspectors tests

2021-06-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35733: Assignee: angerszhu > Check all day-time interval types in HiveInspectors tests > ---

[jira] [Assigned] (SPARK-35817) Queries against wide Avro tables can be slow

2021-06-23 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-35817: -- Assignee: Bruce Robbins > Queries against wide Avro tables can be slow >

[jira] [Updated] (SPARK-35817) Queries against wide Avro tables can be slow

2021-06-23 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-35817: --- Affects Version/s: 3.1.1 3.1.2 > Queries against wide Avro tables can

[jira] [Assigned] (SPARK-34849) SPIP: Support pandas API layer on PySpark

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34849: Assignee: Haejoon Lee > SPIP: Support pandas API layer on PySpark > -

[jira] [Assigned] (SPARK-35805) API auditing in Pandas API on Spark

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35805: Assignee: Haejoon Lee > API auditing in Pandas API on Spark > ---

[jira] [Assigned] (SPARK-35337) pandas APIs on Spark: Separate basic operations into data type based structures

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35337: Assignee: Xinrong Meng > pandas APIs on Spark: Separate basic operations into data type b

[jira] [Assigned] (SPARK-35464) pandas APIs on Spark: Enable mypy check "disallow_untyped_defs" for main codes.

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35464: Assignee: Takuya Ueshin > pandas APIs on Spark: Enable mypy check "disallow_untyped_defs"

[jira] [Resolved] (SPARK-32185) User Guide - Monitoring

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32185. -- Resolution: Later > User Guide - Monitoring > --- > > Key:

[jira] [Assigned] (SPARK-32666) Install ipython and nbsphinx in Jenkins for Binder integration

2021-06-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32666: Assignee: Shane Knapp > Install ipython and nbsphinx in Jenkins for Binder integration >

[jira] [Commented] (SPARK-35588) Merge Binder integration and quickstart notebook

2021-06-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368175#comment-17368175 ] Apache Spark commented on SPARK-35588: -- User 'HyukjinKwon' has created a pull reque

  1   2   >