[jira] [Assigned] (SPARK-35722) wait until something does get queued

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35722: Assignee: (was: Apache Spark) > wait until something does get queued >

[jira] [Assigned] (SPARK-35722) wait until something does get queued

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35722: Assignee: Apache Spark > wait until something does get queued >

[jira] [Commented] (SPARK-35722) wait until something does get queued

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361416#comment-17361416 ] Apache Spark commented on SPARK-35722: -- User 'yikf' has created a pull request for this issue:

[jira] [Resolved] (SPARK-35718) Support casting of Date to timestamp without time zone type

2021-06-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35718. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32873

[jira] [Resolved] (SPARK-35640) Refactor Parquet vectorized reader to remove duplicated code paths

2021-06-10 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-35640. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32777

[jira] [Updated] (SPARK-35722) wait until something does get queued

2021-06-10 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-35722: - Summary: wait until something does get queued (was: wait until until something does get queued) > wait until

[jira] [Assigned] (SPARK-35721) Path level discover for python unittests

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35721: Assignee: Apache Spark > Path level discover for python unittests >

[jira] [Created] (SPARK-35722) wait until until something does get queued

2021-06-10 Thread yikf (Jira)
yikf created SPARK-35722: Summary: wait until until something does get queued Key: SPARK-35722 URL: https://issues.apache.org/jira/browse/SPARK-35722 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-35721) Path level discover for python unittests

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35721: Assignee: (was: Apache Spark) > Path level discover for python unittests >

[jira] [Commented] (SPARK-35721) Path level discover for python unittests

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361397#comment-17361397 ] Apache Spark commented on SPARK-35721: -- User 'Yikun' has created a pull request for this issue:

[jira] [Created] (SPARK-35721) Path level discover for python unittests

2021-06-10 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-35721: --- Summary: Path level discover for python unittests Key: SPARK-35721 URL: https://issues.apache.org/jira/browse/SPARK-35721 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-34512) Disable validate default values when parsing Avro schemas

2021-06-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34512. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32750

[jira] [Assigned] (SPARK-35703) Remove HashClusteredDistribution

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35703: Assignee: Apache Spark > Remove HashClusteredDistribution >

[jira] [Commented] (SPARK-35703) Remove HashClusteredDistribution

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361369#comment-17361369 ] Apache Spark commented on SPARK-35703: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35703) Remove HashClusteredDistribution

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35703: Assignee: (was: Apache Spark) > Remove HashClusteredDistribution >

[jira] [Assigned] (SPARK-34512) Disable validate default values when parsing Avro schemas

2021-06-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34512: - Assignee: Yuming Wang > Disable validate default values when parsing Avro schemas >

[jira] [Commented] (SPARK-35703) Remove HashClusteredDistribution

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361371#comment-17361371 ] Apache Spark commented on SPARK-35703: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-35623) Volcano resource manager for Spark on Kubernetes

2021-06-10 Thread Kevin Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361368#comment-17361368 ] Kevin Su commented on SPARK-35623: -- [~dipanjanK] Thanks for proposing this feature. I'm interested to

[jira] [Assigned] (SPARK-35699) Improve error message when creating k8s pod failed.

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35699: Assignee: (was: Apache Spark) > Improve error message when creating k8s pod failed.

[jira] [Assigned] (SPARK-35699) Improve error message when creating k8s pod failed.

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35699: Assignee: Apache Spark > Improve error message when creating k8s pod failed. >

[jira] [Commented] (SPARK-35699) Improve error message when creating k8s pod failed.

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361366#comment-17361366 ] Apache Spark commented on SPARK-35699: -- User 'pingsutw' has created a pull request for this issue:

[jira] [Commented] (SPARK-35699) Improve error message when creating k8s pod failed.

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361363#comment-17361363 ] Apache Spark commented on SPARK-35699: -- User 'pingsutw' has created a pull request for this issue:

[jira] [Commented] (SPARK-35056) Group exception messages in execution/streaming

2021-06-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361362#comment-17361362 ] jiaan.geng commented on SPARK-35056: I'm working on. > Group exception messages in

[jira] [Assigned] (SPARK-35718) Support casting of Date to timestamp without time zone type

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35718: Assignee: Gengliang Wang (was: Apache Spark) > Support casting of Date to timestamp

[jira] [Assigned] (SPARK-35718) Support casting of Date to timestamp without time zone type

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35718: Assignee: Apache Spark (was: Gengliang Wang) > Support casting of Date to timestamp

[jira] [Commented] (SPARK-35718) Support casting of Date to timestamp without time zone type

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361352#comment-17361352 ] Apache Spark commented on SPARK-35718: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-35718) Support casting of Date to timestamp without time zone type

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361353#comment-17361353 ] Apache Spark commented on SPARK-35718: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-35639) Add metrics about coalesced partitions to CustomShuffleReader in AQE

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361351#comment-17361351 ] Apache Spark commented on SPARK-35639: -- User 'ekoifman' has created a pull request for this issue:

[jira] [Created] (SPARK-35720) Support casting of String to timestamp without time zone type

2021-06-10 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35720: -- Summary: Support casting of String to timestamp without time zone type Key: SPARK-35720 URL: https://issues.apache.org/jira/browse/SPARK-35720 Project: Spark

[jira] [Created] (SPARK-35719) Support casting of timestamp to timestamp without time zone type

2021-06-10 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35719: -- Summary: Support casting of timestamp to timestamp without time zone type Key: SPARK-35719 URL: https://issues.apache.org/jira/browse/SPARK-35719 Project: Spark

[jira] [Created] (SPARK-35718) Support casting of Date to timestamp without time zone type

2021-06-10 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35718: -- Summary: Support casting of Date to timestamp without time zone type Key: SPARK-35718 URL: https://issues.apache.org/jira/browse/SPARK-35718 Project: Spark

[jira] [Commented] (SPARK-35475) Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace.

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361311#comment-17361311 ] Apache Spark commented on SPARK-35475: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-24295) Purge Structured streaming FileStreamSinkLog metadata compact file data.

2021-06-10 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361310#comment-17361310 ] Jungtaek Lim commented on SPARK-24295: -- [~parshabani] Please follow up SPARK-27188. But there're

[jira] [Commented] (SPARK-35475) Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace.

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361309#comment-17361309 ] Apache Spark commented on SPARK-35475: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35475) Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace.

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35475: Assignee: Apache Spark > Enable disallow_untyped_defs mypy check for

[jira] [Assigned] (SPARK-35475) Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace.

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35475: Assignee: (was: Apache Spark) > Enable disallow_untyped_defs mypy check for

[jira] [Resolved] (SPARK-35709) Remove links to Nomad integration in the Documentation

2021-06-10 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-35709. -- Fix Version/s: 3.2.0 Assignee: Paweł Ptaszyński Resolution: Fixed

[jira] [Commented] (SPARK-35439) Children subexpr should come first than parent subexpr in subexpression elimination

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361298#comment-17361298 ] Apache Spark commented on SPARK-35439: -- User 'viirya' has created a pull request for this issue:

[jira] [Created] (SPARK-35717) pandas_udf crashes in conjunction with .filter()

2021-06-10 Thread F. H. (Jira)
F. H. created SPARK-35717: - Summary: pandas_udf crashes in conjunction with .filter() Key: SPARK-35717 URL: https://issues.apache.org/jira/browse/SPARK-35717 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-35439) Children subexpr should come first than parent subexpr in subexpression elimination

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361297#comment-17361297 ] Apache Spark commented on SPARK-35439: -- User 'viirya' has created a pull request for this issue:

[jira] [Updated] (SPARK-35616) Make astype data-type-based

2021-06-10 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35616: - Description: There are many type checks in `astype` methods. Since `DataTypeOps` is introduced,

[jira] [Resolved] (SPARK-35593) Support shuffle data recovery on the reused PVCs

2021-06-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35593. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32730

[jira] [Assigned] (SPARK-35593) Support shuffle data recovery on the reused PVCs

2021-06-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35593: - Assignee: Dongjoon Hyun > Support shuffle data recovery on the reused PVCs >

[jira] [Comment Edited] (SPARK-24295) Purge Structured streaming FileStreamSinkLog metadata compact file data.

2021-06-10 Thread Parsa Shabani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361245#comment-17361245 ] Parsa Shabani edited comment on SPARK-24295 at 6/10/21, 10:43 PM: --

[jira] [Commented] (SPARK-24295) Purge Structured streaming FileStreamSinkLog metadata compact file data.

2021-06-10 Thread Parsa Shabani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361245#comment-17361245 ] Parsa Shabani commented on SPARK-24295: --- Hello all, Any updates on this? [~kabhwan] are you still

[jira] [Assigned] (SPARK-33350) Add support to DiskBlockManager to create merge directory and to get the local shuffle merged data

2021-06-10 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-33350: --- Assignee: Ye Zhou > Add support to DiskBlockManager to create merge

[jira] [Resolved] (SPARK-33350) Add support to DiskBlockManager to create merge directory and to get the local shuffle merged data

2021-06-10 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-33350. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Resolved] (SPARK-35692) Use int to replace long for EXECUTOR_ID_COUNTER in Kubernetes

2021-06-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35692. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32837

[jira] [Assigned] (SPARK-35692) Use int to replace long for EXECUTOR_ID_COUNTER in Kubernetes

2021-06-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35692: - Assignee: Kent Yao > Use int to replace long for EXECUTOR_ID_COUNTER in Kubernetes >

[jira] [Resolved] (SPARK-35716) Support casting of timestamp without time zone to date type

2021-06-10 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35716. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32869

[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks

2021-06-10 Thread binwei yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361175#comment-17361175 ] binwei yang commented on SPARK-24579: - We implemented the native Apache Arrow data source and Arrow

[jira] [Resolved] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

2021-06-10 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-32920. - Resolution: Fixed > Add support in Spark driver to coordinate the finalization

[jira] [Updated] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

2021-06-10 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-32920: Fix Version/s: 3.2.0 > Add support in Spark driver to coordinate the finalization

[jira] [Assigned] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

2021-06-10 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-32920: --- Assignee: Venkata krishnan Sowrirajan > Add support in Spark driver to

[jira] [Commented] (SPARK-32709) Write Hive ORC/Parquet bucketed table with hivehash (for Hive 1,2)

2021-06-10 Thread Shashank Pedamallu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361119#comment-17361119 ] Shashank Pedamallu commented on SPARK-32709: [~chengsu], thank you so much for the response.

[jira] [Commented] (SPARK-35716) Support casting of timestamp without time zone to date type

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361117#comment-17361117 ] Apache Spark commented on SPARK-35716: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-35716) Support casting of timestamp without time zone to date type

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35716: Assignee: Gengliang Wang (was: Apache Spark) > Support casting of timestamp without

[jira] [Assigned] (SPARK-35716) Support casting of timestamp without time zone to date type

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35716: Assignee: Apache Spark (was: Gengliang Wang) > Support casting of timestamp without

[jira] [Commented] (SPARK-35716) Support casting of timestamp without time zone to date type

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361116#comment-17361116 ] Apache Spark commented on SPARK-35716: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-35716) Support casting of timestamp without time zone to date type

2021-06-10 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35716: -- Summary: Support casting of timestamp without time zone to date type Key: SPARK-35716 URL: https://issues.apache.org/jira/browse/SPARK-35716 Project: Spark

[jira] [Assigned] (SPARK-35296) Dataset.observe fails with an assertion

2021-06-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35296: --- Assignee: Kousuke Saruta > Dataset.observe fails with an assertion >

[jira] [Resolved] (SPARK-35296) Dataset.observe fails with an assertion

2021-06-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35296. - Fix Version/s: 3.0.3 3.1.3 3.2.0 Resolution: Fixed

[jira] [Commented] (SPARK-34861) Support nested column in Spark vectorized readers

2021-06-10 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361090#comment-17361090 ] Chao Sun commented on SPARK-34861: -- Synced with [~chengsu] offline and I will take over this JIRA. >

[jira] [Commented] (SPARK-35715) Option "--files" with local:// prefix is not honoured for Spark on kubernetes

2021-06-10 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361086#comment-17361086 ] Erik Krogen commented on SPARK-35715: - Not sure about k8s, but at least for YARN this is expected --

[jira] [Updated] (SPARK-35715) Option "--files" with local:// prefix is not honoured for Spark on kubernetes

2021-06-10 Thread Pardhu Madipalli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pardhu Madipalli updated SPARK-35715: - Description: When we provide a local file as a dependency using "--files" option, the

[jira] [Updated] (SPARK-35715) Option "--files" with local:// prefix is not honoured for Spark on kubernetes

2021-06-10 Thread Pardhu Madipalli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pardhu Madipalli updated SPARK-35715: - Description: When we provide a local file as a dependency using "--files" option, the

[jira] [Created] (SPARK-35715) Option "--files" with local:// prefix is not honoured for Spark on kubernetes

2021-06-10 Thread Pardhu Madipalli (Jira)
Pardhu Madipalli created SPARK-35715: Summary: Option "--files" with local:// prefix is not honoured for Spark on kubernetes Key: SPARK-35715 URL: https://issues.apache.org/jira/browse/SPARK-35715

[jira] [Resolved] (SPARK-35653) [SQL] CatalystToExternalMap interpreted path fails for Map with case classes as keys or values

2021-06-10 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35653. - Fix Version/s: 3.0.3 3.1.3 3.2.0 Resolution: Fixed

[jira] [Assigned] (SPARK-35653) [SQL] CatalystToExternalMap interpreted path fails for Map with case classes as keys or values

2021-06-10 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35653: --- Assignee: Emil Ejbyfeldt > [SQL] CatalystToExternalMap interpreted path fails for Map with

[jira] [Resolved] (SPARK-35711) Support casting of timestamp without time zone to timestamp type

2021-06-10 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35711. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32864

[jira] [Commented] (SPARK-35714) Bug fix for deadlock during the executor shutdown

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360981#comment-17360981 ] Apache Spark commented on SPARK-35714: -- User 'wankunde' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35713) Bug fix for thread leak in JobCancellationSuite

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35713: Assignee: (was: Apache Spark) > Bug fix for thread leak in JobCancellationSuite >

[jira] [Assigned] (SPARK-35714) Bug fix for deadlock during the executor shutdown

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35714: Assignee: Apache Spark > Bug fix for deadlock during the executor shutdown >

[jira] [Commented] (SPARK-35714) Bug fix for deadlock during the executor shutdown

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360977#comment-17360977 ] Apache Spark commented on SPARK-35714: -- User 'wankunde' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35713) Bug fix for thread leak in JobCancellationSuite

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35713: Assignee: Apache Spark > Bug fix for thread leak in JobCancellationSuite >

[jira] [Assigned] (SPARK-35714) Bug fix for deadlock during the executor shutdown

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35714: Assignee: (was: Apache Spark) > Bug fix for deadlock during the executor shutdown >

[jira] [Commented] (SPARK-35713) Bug fix for thread leak in JobCancellationSuite

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360976#comment-17360976 ] Apache Spark commented on SPARK-35713: -- User 'wankunde' has created a pull request for this issue:

[jira] [Updated] (SPARK-35714) Bug fix for deadlock during the executor shutdown

2021-06-10 Thread Wan Kun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-35714: Description: When a executor received a TERM signal, it (the second TERM signal) will lock

[jira] [Created] (SPARK-35714) Bug fix for deadlock during the executor shutdown

2021-06-10 Thread Wan Kun (Jira)
Wan Kun created SPARK-35714: --- Summary: Bug fix for deadlock during the executor shutdown Key: SPARK-35714 URL: https://issues.apache.org/jira/browse/SPARK-35714 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35713) Bug fix for thread leak in JobCancellationSuite

2021-06-10 Thread Wan Kun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-35713: Summary: Bug fix for thread leak in JobCancellationSuite (was: Bugfix for thread leak in

[jira] [Updated] (SPARK-35713) Bugfix for thread leak in JobCancellationSuite

2021-06-10 Thread Wan Kun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-35713: Description: When we call Thread.interrupt() method, that thread's interrupt status will be set but it

[jira] [Created] (SPARK-35713) Bugfix for thread leak in JobCancellationSuite

2021-06-10 Thread Wan Kun (Jira)
Wan Kun created SPARK-35713: --- Summary: Bugfix for thread leak in JobCancellationSuite Key: SPARK-35713 URL: https://issues.apache.org/jira/browse/SPARK-35713 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-35712) Simplify ResolveAggregateFunctions

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35712: Assignee: Apache Spark > Simplify ResolveAggregateFunctions >

[jira] [Assigned] (SPARK-35712) Simplify ResolveAggregateFunctions

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35712: Assignee: (was: Apache Spark) > Simplify ResolveAggregateFunctions >

[jira] [Commented] (SPARK-35712) Simplify ResolveAggregateFunctions

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360892#comment-17360892 ] Apache Spark commented on SPARK-35712: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-35712) Simplify ResolveAggregateFunctions

2021-06-10 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-35712: --- Summary: Simplify ResolveAggregateFunctions Key: SPARK-35712 URL: https://issues.apache.org/jira/browse/SPARK-35712 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-35701) Contention on SQLConf.sqlConfEntries and SQLConf.staticConfKeys

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35701: Assignee: Haiyang Sun (was: Apache Spark) > Contention on SQLConf.sqlConfEntries and

[jira] [Assigned] (SPARK-35701) Contention on SQLConf.sqlConfEntries and SQLConf.staticConfKeys

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35701: Assignee: Apache Spark (was: Haiyang Sun) > Contention on SQLConf.sqlConfEntries and

[jira] [Commented] (SPARK-35701) Contention on SQLConf.sqlConfEntries and SQLConf.staticConfKeys

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360881#comment-17360881 ] Apache Spark commented on SPARK-35701: -- User 'haiyangsun-db' has created a pull request for this

[jira] [Assigned] (SPARK-35701) Contention on SQLConf.sqlConfEntries and SQLConf.staticConfKeys

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35701: Assignee: Apache Spark (was: Haiyang Sun) > Contention on SQLConf.sqlConfEntries and

[jira] [Commented] (SPARK-35711) Support casting of timestamp without time zone to timestamp type

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360759#comment-17360759 ] Apache Spark commented on SPARK-35711: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-35711) Support casting of timestamp without time zone to timestamp type

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360758#comment-17360758 ] Apache Spark commented on SPARK-35711: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-35711) Support casting of timestamp without time zone to timestamp type

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35711: Assignee: Apache Spark (was: Gengliang Wang) > Support casting of timestamp without

[jira] [Assigned] (SPARK-35711) Support casting of timestamp without time zone to timestamp type

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35711: Assignee: Gengliang Wang (was: Apache Spark) > Support casting of timestamp without

[jira] [Created] (SPARK-35711) Support casting of timestamp without time zone to timestamp type

2021-06-10 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35711: -- Summary: Support casting of timestamp without time zone to timestamp type Key: SPARK-35711 URL: https://issues.apache.org/jira/browse/SPARK-35711 Project: Spark

[jira] [Assigned] (SPARK-35652) Different Behaviour join vs joinWith in self joining

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35652: Assignee: (was: Apache Spark) > Different Behaviour join vs joinWith in self joining

[jira] [Commented] (SPARK-35652) Different Behaviour join vs joinWith in self joining

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360747#comment-17360747 ] Apache Spark commented on SPARK-35652: -- User 'dgd-contributor' has created a pull request for this

[jira] [Assigned] (SPARK-35652) Different Behaviour join vs joinWith in self joining

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35652: Assignee: Apache Spark > Different Behaviour join vs joinWith in self joining >

[jira] [Assigned] (SPARK-35695) QueryExecutionListener does not see any observed metrics fired before persist/cache

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35695: Assignee: (was: Apache Spark) > QueryExecutionListener does not see any observed

[jira] [Assigned] (SPARK-35695) QueryExecutionListener does not see any observed metrics fired before persist/cache

2021-06-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35695: Assignee: Apache Spark > QueryExecutionListener does not see any observed metrics fired

  1   2   >