[jira] [Assigned] (SPARK-35813) Add new adaptive config into sql-performance-tuning docs

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35813: Assignee: Apache Spark > Add new adaptive config into sql-performance-tuning docs > -

[jira] [Commented] (SPARK-35813) Add new adaptive config into sql-performance-tuning docs

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365252#comment-17365252 ] Apache Spark commented on SPARK-35813: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-35813) Add new adaptive config into sql-performance-tuning docs

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35813: Assignee: (was: Apache Spark) > Add new adaptive config into sql-performance-tuning d

[jira] [Commented] (SPARK-35813) Add new adaptive config into sql-performance-tuning docs

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365251#comment-17365251 ] Apache Spark commented on SPARK-35813: -- User 'ulysses-you' has created a pull reque

[jira] [Created] (SPARK-35813) Add new adaptive config into sql-performance-tuning docs

2021-06-17 Thread XiDuo You (Jira)
XiDuo You created SPARK-35813: - Summary: Add new adaptive config into sql-performance-tuning docs Key: SPARK-35813 URL: https://issues.apache.org/jira/browse/SPARK-35813 Project: Spark Issue Type

[jira] [Created] (SPARK-35812) Throw an error if `version` and `timestamp` are used together in DataFrame.to_delta.

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35812: --- Summary: Throw an error if `version` and `timestamp` are used together in DataFrame.to_delta. Key: SPARK-35812 URL: https://issues.apache.org/jira/browse/SPARK-35812 Pr

[jira] [Created] (SPARK-35811) Deprecate DataFrame.to_spark_io

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35811: --- Summary: Deprecate DataFrame.to_spark_io Key: SPARK-35811 URL: https://issues.apache.org/jira/browse/SPARK-35811 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-35810) Remove ps.broadcast API

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35810: --- Summary: Remove ps.broadcast API Key: SPARK-35810 URL: https://issues.apache.org/jira/browse/SPARK-35810 Project: Spark Issue Type: Sub-task Componen

[jira] [Created] (SPARK-35809) Add `index_col` argument for ps.sql.

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35809: --- Summary: Add `index_col` argument for ps.sql. Key: SPARK-35809 URL: https://issues.apache.org/jira/browse/SPARK-35809 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-35808) Always enable the `pandas_metadata` in DataFrame.parquet

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35808: --- Summary: Always enable the `pandas_metadata` in DataFrame.parquet Key: SPARK-35808 URL: https://issues.apache.org/jira/browse/SPARK-35808 Project: Spark Issue

[jira] [Updated] (SPARK-35807) Rename the `num_files` argument

2021-06-17 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35807: Description: We should rename the num_files argument in [DataFrame.to_csv  |https://koalas.readthe

[jira] [Created] (SPARK-35807) Rename the `num_files` argument

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35807: --- Summary: Rename the `num_files` argument Key: SPARK-35807 URL: https://issues.apache.org/jira/browse/SPARK-35807 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-35806) Rename the `mode` argument to avoid confusion with `mode` argument in pandas

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35806: --- Summary: Rename the `mode` argument to avoid confusion with `mode` argument in pandas Key: SPARK-35806 URL: https://issues.apache.org/jira/browse/SPARK-35806 Project: S

[jira] [Updated] (SPARK-35747) Avoid printing full Exception stack trace, if HBase/Kafka/Hive services are not running in a secure cluster

2021-06-17 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-35747: - Description: In a secure Yarn cluster, even though HBase or Kafka, or Hive services are not used in the

[jira] [Assigned] (SPARK-35780) Support DATE/TIMESTAMP literals across the full range

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35780: Assignee: (was: Apache Spark) > Support DATE/TIMESTAMP literals across the full range

[jira] [Assigned] (SPARK-35780) Support DATE/TIMESTAMP literals across the full range

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35780: Assignee: Apache Spark > Support DATE/TIMESTAMP literals across the full range >

[jira] [Created] (SPARK-35805) Pandas API on Spark improvements

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35805: --- Summary: Pandas API on Spark improvements Key: SPARK-35805 URL: https://issues.apache.org/jira/browse/SPARK-35805 Project: Spark Issue Type: Umbrella

[jira] [Commented] (SPARK-35780) Support DATE/TIMESTAMP literals across the full range

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365234#comment-17365234 ] Apache Spark commented on SPARK-35780: -- User 'linhongliu-db' has created a pull req

[jira] [Issue Comment Deleted] (SPARK-35065) Group exception messages in spark/sql (core)

2021-06-17 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35065: --- Comment: was deleted (was: I'm working on.) > Group exception messages in spark/sql (core) > --

[jira] [Updated] (SPARK-35747) Avoid printing full Exception stack trace, if HBase/Kafka/Hive services are not running in a secure cluster

2021-06-17 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-35747: - Description: In a secure Yarn cluster, even though HBase or Kafka, or Hive services are not used in the

[jira] [Commented] (SPARK-35065) Group exception messages in spark/sql (core)

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365233#comment-17365233 ] Apache Spark commented on SPARK-35065: -- User 'beliefer' has created a pull request

[jira] [Commented] (SPARK-35065) Group exception messages in spark/sql (core)

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365232#comment-17365232 ] Apache Spark commented on SPARK-35065: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-35065) Group exception messages in spark/sql (core)

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35065: Assignee: Apache Spark > Group exception messages in spark/sql (core) > -

[jira] [Assigned] (SPARK-35065) Group exception messages in spark/sql (core)

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35065: Assignee: (was: Apache Spark) > Group exception messages in spark/sql (core) > --

[jira] [Updated] (SPARK-35747) Avoid printing full Exception stack trace, if HBase/Kafka/Hive services are not running in a secure cluster

2021-06-17 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-35747: - Summary: Avoid printing full Exception stack trace, if HBase/Kafka/Hive services are not running in a se

[jira] [Updated] (SPARK-30641) Project Matrix: Linear Models revisit and refactor

2021-06-17 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30641: - Description: We had been refactoring linear models for a long time, and there still are some wo

[jira] [Resolved] (SPARK-35303) Enable pinned thread mode by default

2021-06-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35303. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32429 [https://gi

[jira] [Assigned] (SPARK-35303) Enable pinned thread mode by default

2021-06-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35303: Assignee: Hyukjin Kwon > Enable pinned thread mode by default > -

[jira] [Updated] (SPARK-35804) can't read external hive table on spark

2021-06-17 Thread cao zhiyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cao zhiyu updated SPARK-35804: -- Description: I create a external hive table with hdfs file which is formatted as json string. I can r

[jira] [Created] (SPARK-35804) can't read external hive table on spark

2021-06-17 Thread cao zhiyu (Jira)
cao zhiyu created SPARK-35804: - Summary: can't read external hive table on spark Key: SPARK-35804 URL: https://issues.apache.org/jira/browse/SPARK-35804 Project: Spark Issue Type: Bug C

[jira] [Commented] (SPARK-35472) Enable disallow_untyped_defs mypy check for pyspark.pandas.generic.

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365215#comment-17365215 ] Apache Spark commented on SPARK-35472: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-35472) Enable disallow_untyped_defs mypy check for pyspark.pandas.generic.

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35472: Assignee: Apache Spark > Enable disallow_untyped_defs mypy check for pyspark.pandas.gener

[jira] [Commented] (SPARK-35472) Enable disallow_untyped_defs mypy check for pyspark.pandas.generic.

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365214#comment-17365214 ] Apache Spark commented on SPARK-35472: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-35472) Enable disallow_untyped_defs mypy check for pyspark.pandas.generic.

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35472: Assignee: (was: Apache Spark) > Enable disallow_untyped_defs mypy check for pyspark.p

[jira] [Created] (SPARK-35803) Spark SQL does not support creating views using DataSource v2 based data sources

2021-06-17 Thread David Rabinowitz (Jira)
David Rabinowitz created SPARK-35803: Summary: Spark SQL does not support creating views using DataSource v2 based data sources Key: SPARK-35803 URL: https://issues.apache.org/jira/browse/SPARK-35803

[jira] [Created] (SPARK-35802) Error loading the stages/stage/ page in spark UI

2021-06-17 Thread Helt Long (Jira)
Helt Long created SPARK-35802: - Summary: Error loading the stages/stage/ page in spark UI Key: SPARK-35802 URL: https://issues.apache.org/jira/browse/SPARK-35802 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-22674) PySpark breaks serialization of namedtuple subclasses

2021-06-17 Thread Sarth Frey (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365194#comment-17365194 ] Sarth Frey commented on SPARK-22674: I can confirm this is affecting PySpark 3.1.1

[jira] [Updated] (SPARK-22674) PySpark breaks serialization of namedtuple subclasses

2021-06-17 Thread Sarth Frey (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarth Frey updated SPARK-22674: --- Affects Version/s: 3.1.1 > PySpark breaks serialization of namedtuple subclasses > -

[jira] [Updated] (SPARK-35790) Spark Package Python Import does not work for namespace packages

2021-06-17 Thread Mark Hamilton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Hamilton updated SPARK-35790: -- Description: If one includes python files within several jars that comprise a python "namespa

[jira] [Commented] (SPARK-35469) Enable disallow_untyped_defs mypy check for pyspark.pandas.accessors.

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365138#comment-17365138 ] Apache Spark commented on SPARK-35469: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-35469) Enable disallow_untyped_defs mypy check for pyspark.pandas.accessors.

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35469: Assignee: (was: Apache Spark) > Enable disallow_untyped_defs mypy check for pyspark.p

[jira] [Assigned] (SPARK-35469) Enable disallow_untyped_defs mypy check for pyspark.pandas.accessors.

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35469: Assignee: Apache Spark > Enable disallow_untyped_defs mypy check for pyspark.pandas.acces

[jira] [Created] (SPARK-35801) SPIP: Support MERGE in Data Source V2

2021-06-17 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-35801: Summary: SPIP: Support MERGE in Data Source V2 Key: SPARK-35801 URL: https://issues.apache.org/jira/browse/SPARK-35801 Project: Spark Issue Type: Imp

[jira] [Resolved] (SPARK-35095) Use ANSI intervals in streaming join tests

2021-06-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35095. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32953 [https://github.com

[jira] [Assigned] (SPARK-35095) Use ANSI intervals in streaming join tests

2021-06-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35095: Assignee: Kousuke Saruta > Use ANSI intervals in streaming join tests > -

[jira] [Commented] (SPARK-18107) Insert overwrite statement runs much slower in spark-sql than it does in hive-client

2021-06-17 Thread Hemanth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365072#comment-17365072 ] Hemanth commented on SPARK-18107: - We are seeing this issue still exists on Spark versio

[jira] [Resolved] (SPARK-35670) Upgrade ZSTD-JNI to 1.5.0-1

2021-06-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35670. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32826 [https://

[jira] [Assigned] (SPARK-35670) Upgrade ZSTD-JNI to 1.5.0-1

2021-06-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35670: - Assignee: apache.org > Upgrade ZSTD-JNI to 1.5.0-1 > --- > >

[jira] [Assigned] (SPARK-35720) Support casting of String to timestamp without time zone type

2021-06-17 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-35720: -- Assignee: Gengliang Wang > Support casting of String to timestamp without time zone t

[jira] [Resolved] (SPARK-35720) Support casting of String to timestamp without time zone type

2021-06-17 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35720. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32936 [https:

[jira] [Assigned] (SPARK-35800) Improving testability of GroupState in streaming flatMapGroupsWithState

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35800: Assignee: Apache Spark > Improving testability of GroupState in streaming flatMapGroupsWi

[jira] [Commented] (SPARK-35800) Improving testability of GroupState in streaming flatMapGroupsWithState

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365056#comment-17365056 ] Apache Spark commented on SPARK-35800: -- User 'lizhangdatabricks' has created a pull

[jira] [Assigned] (SPARK-35800) Improving testability of GroupState in streaming flatMapGroupsWithState

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35800: Assignee: (was: Apache Spark) > Improving testability of GroupState in streaming flat

[jira] [Created] (SPARK-35800) Improving testability of GroupState in streaming flatMapGroupsWithState

2021-06-17 Thread Tathagata Das (Jira)
Tathagata Das created SPARK-35800: - Summary: Improving testability of GroupState in streaming flatMapGroupsWithState Key: SPARK-35800 URL: https://issues.apache.org/jira/browse/SPARK-35800 Project: Sp

[jira] [Assigned] (SPARK-35095) Use ANSI intervals in streaming join tests

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35095: Assignee: Apache Spark > Use ANSI intervals in streaming join tests > ---

[jira] [Commented] (SPARK-35095) Use ANSI intervals in streaming join tests

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365052#comment-17365052 ] Apache Spark commented on SPARK-35095: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-35095) Use ANSI intervals in streaming join tests

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35095: Assignee: (was: Apache Spark) > Use ANSI intervals in streaming join tests >

[jira] [Assigned] (SPARK-35799) Fix the allUpdatesTimeMs metric measuring in FlatMapGroupsWithStateExec

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35799: Assignee: (was: Apache Spark) > Fix the allUpdatesTimeMs metric measuring in FlatMapG

[jira] [Assigned] (SPARK-35799) Fix the allUpdatesTimeMs metric measuring in FlatMapGroupsWithStateExec

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35799: Assignee: Apache Spark > Fix the allUpdatesTimeMs metric measuring in FlatMapGroupsWithSt

[jira] [Commented] (SPARK-35799) Fix the allUpdatesTimeMs metric measuring in FlatMapGroupsWithStateExec

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365049#comment-17365049 ] Apache Spark commented on SPARK-35799: -- User 'vkorukanti' has created a pull reques

[jira] [Updated] (SPARK-35799) Fix the allUpdatesTimeMs metric measuring in FlatMapGroupsWithStateExec

2021-06-17 Thread Venki Korukanti (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venki Korukanti updated SPARK-35799: Description: Metric {{allUpdatesTimeMs}} meant to capture the start to end walltime of the

[jira] [Created] (SPARK-35799) Fix the allUpdatesTimeMs metric measuring in FlatMapGroupsWithStateExec

2021-06-17 Thread Venki Korukanti (Jira)
Venki Korukanti created SPARK-35799: --- Summary: Fix the allUpdatesTimeMs metric measuring in FlatMapGroupsWithStateExec Key: SPARK-35799 URL: https://issues.apache.org/jira/browse/SPARK-35799 Project

[jira] [Resolved] (SPARK-34898) Send ExecutorMetricsUpdate EventLog appropriately

2021-06-17 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-34898. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 319

[jira] [Resolved] (SPARK-35789) Lateral join should only be used with subquery

2021-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35789. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32937 [https://gith

[jira] [Assigned] (SPARK-35789) Lateral join should only be used with subquery

2021-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35789: --- Assignee: Allison Wang > Lateral join should only be used with subquery > -

[jira] [Commented] (SPARK-35782) leveldbjni doesn't work in Apple Silicon on macOS

2021-06-17 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365014#comment-17365014 ] DB Tsai commented on SPARK-35782: - [~yikunkero] It will only work for Apple Silicon on L

[jira] [Commented] (SPARK-33603) Group exception messages in execution/command

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365011#comment-17365011 ] Apache Spark commented on SPARK-33603: -- User 'dgd-contributor' has created a pull r

[jira] [Commented] (SPARK-33603) Group exception messages in execution/command

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365010#comment-17365010 ] Apache Spark commented on SPARK-33603: -- User 'dgd-contributor' has created a pull r

[jira] [Assigned] (SPARK-33603) Group exception messages in execution/command

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33603: Assignee: (was: Apache Spark) > Group exception messages in execution/command > -

[jira] [Assigned] (SPARK-33603) Group exception messages in execution/command

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33603: Assignee: Apache Spark > Group exception messages in execution/command >

[jira] [Resolved] (SPARK-34054) BlockManagerDecommissioner cleanup

2021-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34054. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31102 [https://gith

[jira] [Assigned] (SPARK-34054) BlockManagerDecommissioner cleanup

2021-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34054: --- Assignee: wuyi > BlockManagerDecommissioner cleanup > -- >

[jira] [Resolved] (SPARK-35798) Fix SparkPlan.sqlContext usage

2021-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35798. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32947 [https://gith

[jira] [Assigned] (SPARK-35798) Fix SparkPlan.sqlContext usage

2021-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35798: --- Assignee: Peter Toth > Fix SparkPlan.sqlContext usage > -- > >

[jira] [Resolved] (SPARK-35792) View should not capture configs used in `RelationConversions`

2021-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35792. - Fix Version/s: 3.1.3 3.2.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-35792) View should not capture configs used in `RelationConversions`

2021-06-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35792: --- Assignee: Linhong Liu > View should not capture configs used in `RelationConversions` > ---

[jira] [Assigned] (SPARK-35726) Truncate java.time.Duration by fields of day-time interval type

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35726: Assignee: (was: Apache Spark) > Truncate java.time.Duration by fields of day-time int

[jira] [Assigned] (SPARK-35726) Truncate java.time.Duration by fields of day-time interval type

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35726: Assignee: (was: Apache Spark) > Truncate java.time.Duration by fields of day-time int

[jira] [Commented] (SPARK-35726) Truncate java.time.Duration by fields of day-time interval type

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364923#comment-17364923 ] Apache Spark commented on SPARK-35726: -- User 'AngersZh' has created a pull requ

[jira] [Commented] (SPARK-35773) Parse year-month interval literals to tightest types

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364922#comment-17364922 ] Apache Spark commented on SPARK-35773: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-35726) Truncate java.time.Duration by fields of day-time interval type

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35726: Assignee: Apache Spark > Truncate java.time.Duration by fields of day-time interval type

[jira] [Commented] (SPARK-35773) Parse year-month interval literals to tightest types

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364921#comment-17364921 ] Apache Spark commented on SPARK-35773: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-35773) Parse year-month interval literals to tightest types

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35773: Assignee: Apache Spark (was: Kousuke Saruta) > Parse year-month interval literals to tig

[jira] [Commented] (SPARK-35773) Parse year-month interval literals to tightest types

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364920#comment-17364920 ] Apache Spark commented on SPARK-35773: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-35773) Parse year-month interval literals to tightest types

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35773: Assignee: Kousuke Saruta (was: Apache Spark) > Parse year-month interval literals to tig

[jira] [Commented] (SPARK-35749) Parse unit list interval literals as year-month/day-time interval types

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364919#comment-17364919 ] Apache Spark commented on SPARK-35749: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-35749) Parse unit list interval literals as year-month/day-time interval types

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35749: Assignee: Apache Spark > Parse unit list interval literals as year-month/day-time interva

[jira] [Assigned] (SPARK-35749) Parse unit list interval literals as year-month/day-time interval types

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35749: Assignee: (was: Apache Spark) > Parse unit list interval literals as year-month/day-t

[jira] [Commented] (SPARK-35749) Parse unit list interval literals as year-month/day-time interval types

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364918#comment-17364918 ] Apache Spark commented on SPARK-35749: -- User 'sarutak' has created a pull request f

[jira] [Updated] (SPARK-35749) Parse unit list interval literals as year-month/day-time interval types

2021-06-17 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35749: --- Description: Currently, unit list interval literals like `interval '1' year '2' months` or

[jira] [Updated] (SPARK-35749) Parse unit list interval literals as year-month/day-time interval types

2021-06-17 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35749: --- Summary: Parse unit list interval literals as year-month/day-time interval types (was: Pars

[jira] [Commented] (SPARK-35796) UT `handles k8s cluster mode` fails on MacOs >= 10.15

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364909#comment-17364909 ] Apache Spark commented on SPARK-35796: -- User 'toujours33' has created a pull reques

[jira] [Assigned] (SPARK-35798) Fix SparkPlan.sqlContext usage

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35798: Assignee: (was: Apache Spark) > Fix SparkPlan.sqlContext usage >

[jira] [Commented] (SPARK-35798) Fix SparkPlan.sqlContext usage

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364903#comment-17364903 ] Apache Spark commented on SPARK-35798: -- User 'peter-toth' has created a pull reques

[jira] [Assigned] (SPARK-35798) Fix SparkPlan.sqlContext usage

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35798: Assignee: Apache Spark > Fix SparkPlan.sqlContext usage > --

[jira] [Commented] (SPARK-35798) Fix SparkPlan.sqlContext usage

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364902#comment-17364902 ] Apache Spark commented on SPARK-35798: -- User 'peter-toth' has created a pull reques

[jira] [Assigned] (SPARK-35796) UT `handles k8s cluster mode` fails on MacOs >= 10.15

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35796: Assignee: Apache Spark > UT `handles k8s cluster mode` fails on MacOs >= 10.15 >

[jira] [Assigned] (SPARK-35796) UT `handles k8s cluster mode` fails on MacOs >= 10.15

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35796: Assignee: (was: Apache Spark) > UT `handles k8s cluster mode` fails on MacOs >= 10.15

[jira] [Commented] (SPARK-35796) UT `handles k8s cluster mode` fails on MacOs >= 10.15

2021-06-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364899#comment-17364899 ] Apache Spark commented on SPARK-35796: -- User 'toujours33' has created a pull reques

[jira] [Created] (SPARK-35798) Fix SparkPlan.sqlContext usage

2021-06-17 Thread Peter Toth (Jira)
Peter Toth created SPARK-35798: -- Summary: Fix SparkPlan.sqlContext usage Key: SPARK-35798 URL: https://issues.apache.org/jira/browse/SPARK-35798 Project: Spark Issue Type: Bug Componen

  1   2   >