[jira] [Commented] (SPARK-35181) Use zstd for spark.io.compression.codec by default

[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407090#comment-17407090 ] angerszhu commented on SPARK-35181: --- [~dongjoon]Thanks for your information, I will try this. > Use

[jira] [Commented] (SPARK-34801) java.lang.NoSuchMethodException: org.apache.hadoop.hive.ql.metadata.Hive.loadPartition

[ https://issues.apache.org/jira/browse/SPARK-34801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407072#comment-17407072 ] kaishen commented on SPARK-34801: - I'm using apache spark3.1.2, HDP 3.1.4，remove these config from

[jira] [Created] (SPARK-36620) Client side related push-based shuffle metrics

Thejdeep Gudivada created SPARK-36620: - Summary: Client side related push-based shuffle metrics Key: SPARK-36620 URL: https://issues.apache.org/jira/browse/SPARK-36620 Project: Spark

[jira] [Commented] (SPARK-36604) timestamp type column analyze result is wrong

[ https://issues.apache.org/jira/browse/SPARK-36604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407068#comment-17407068 ] Senthil Kumar commented on SPARK-36604: --- [~yghu] I tested this scenario in Spark2.4, but I don't

[jira] [Commented] (SPARK-35181) Use zstd for spark.io.compression.codec by default

[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407065#comment-17407065 ] Dongjoon Hyun commented on SPARK-35181: --- [~angerszhuuu], there is no problem if you upgrade your

[jira] [Updated] (SPARK-36608) Support TimestampNTZ in PySpark

[ https://issues.apache.org/jira/browse/SPARK-36608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36608: - Description: Should support in: - createDataFrame / toPandas with and without Arrow - pandas

[jira] [Commented] (SPARK-36611) Remove unused listener in HiveThriftServer2AppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-36611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407054#comment-17407054 ] Kent Yao commented on SPARK-36611: -- issue resolved by https://github.com/apache/spark/pull/33866 >

[jira] [Resolved] (SPARK-36611) Remove unused listener in HiveThriftServer2AppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-36611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-36611. -- Fix Version/s: 3.3.0 Resolution: Fixed > Remove unused listener in

[jira] [Assigned] (SPARK-36619) Iterators on prefix scan in RocksDB are not cleaned up unless aborting

[ https://issues.apache.org/jira/browse/SPARK-36619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36619: Assignee: (was: Apache Spark) > Iterators on prefix scan in RocksDB are not cleaned

[jira] [Commented] (SPARK-36619) Iterators on prefix scan in RocksDB are not cleaned up unless aborting

[ https://issues.apache.org/jira/browse/SPARK-36619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407052#comment-17407052 ] Apache Spark commented on SPARK-36619: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Assigned] (SPARK-36619) Iterators on prefix scan in RocksDB are not cleaned up unless aborting

[ https://issues.apache.org/jira/browse/SPARK-36619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36619: Assignee: Apache Spark > Iterators on prefix scan in RocksDB are not cleaned up unless

[jira] [Created] (SPARK-36619) Iterators on prefix scan in RocksDB are not cleaned up unless aborting

Jungtaek Lim created SPARK-36619: Summary: Iterators on prefix scan in RocksDB are not cleaned up unless aborting Key: SPARK-36619 URL: https://issues.apache.org/jira/browse/SPARK-36619 Project:

[jira] [Commented] (SPARK-35181) Use zstd for spark.io.compression.codec by default

[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407027#comment-17407027 ] angerszhu commented on SPARK-35181: --- Hi [~dongjoon] Can I know if I use it as default in spark-2.4,

[jira] [Commented] (SPARK-36617) Inconsistencies in approxQuantile annotations

[ https://issues.apache.org/jira/browse/SPARK-36617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407001#comment-17407001 ] Hyukjin Kwon commented on SPARK-36617: -- Thanks for cc'ing me, [~zero323]. +1 for fixing it! >

[jira] [Updated] (SPARK-36618) Support axis=0 for drop API

[ https://issues.apache.org/jira/browse/SPARK-36618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36618: - Summary: Support axis=0 for drop API (was: Support axis=0 in for drop API) > Support axis=0

[jira] [Created] (SPARK-36618) Support axis=0 in for drop API

Xinrong Meng created SPARK-36618: Summary: Support axis=0 in for drop API Key: SPARK-36618 URL: https://issues.apache.org/jira/browse/SPARK-36618 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-36617) Inconsistencies in approxQuantile annotations

[ https://issues.apache.org/jira/browse/SPARK-36617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406956#comment-17406956 ] Maciej Szymkiewicz commented on SPARK-36617: FYI [~hyukjin.kwon] > Inconsistencies in

[jira] [Created] (SPARK-36617) Inconsistencies in approxQuantile annotations

Maciej Szymkiewicz created SPARK-36617: -- Summary: Inconsistencies in approxQuantile annotations Key: SPARK-36617 URL: https://issues.apache.org/jira/browse/SPARK-36617 Project: Spark

[jira] [Updated] (SPARK-36616) Unrecognized connection property 'url' when using Presto JDBC

[ https://issues.apache.org/jira/browse/SPARK-36616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Gunasekaran updated SPARK-36616: - Description: Hi,  Here is my spark sql code, where I am trying to read a presto

[jira] [Updated] (SPARK-36616) Unrecognized connection property 'url' when using Presto JDBC

[ https://issues.apache.org/jira/browse/SPARK-36616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Gunasekaran updated SPARK-36616: - Description: Hi,  Here is my spark sql code, where I am trying to read a presto

[jira] [Updated] (SPARK-36616) Unrecognized connection property 'url' when using Presto JDBC

[ https://issues.apache.org/jira/browse/SPARK-36616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Gunasekaran updated SPARK-36616: - Description: Hi,  Here is my spark sql code, where I am trying to read a presto

[jira] [Updated] (SPARK-36616) Unrecognized connection property 'url' when using Presto JDBC

[ https://issues.apache.org/jira/browse/SPARK-36616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Gunasekaran updated SPARK-36616: - Description: Hi,  Here is my spark sql code, where I am trying to read a presto

[jira] [Created] (SPARK-36616) Unrecognized connection property 'url' when using Presto JDBC

Rajkumar Gunasekaran created SPARK-36616: Summary: Unrecognized connection property 'url' when using Presto JDBC Key: SPARK-36616 URL: https://issues.apache.org/jira/browse/SPARK-36616

[jira] [Resolved] (SPARK-36614) Executor loss reason shows "worker lost" rather "Executor decommission"

[ https://issues.apache.org/jira/browse/SPARK-36614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36614. --- Fix Version/s: 3.1.3 3.2.0 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-36614) Executor loss reason shows "worker lost" rather "Executor decommission"

[ https://issues.apache.org/jira/browse/SPARK-36614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36614: - Assignee: wuyi > Executor loss reason shows "worker lost" rather "Executor

[jira] [Commented] (SPARK-36615) SparkContext should register shutdown hook earlier

[ https://issues.apache.org/jira/browse/SPARK-36615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406727#comment-17406727 ] Apache Spark commented on SPARK-36615: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-36615) SparkContext should register shutdown hook earlier

[ https://issues.apache.org/jira/browse/SPARK-36615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36615: Assignee: Apache Spark > SparkContext should register shutdown hook earlier >

[jira] [Commented] (SPARK-36615) SparkContext should register shutdown hook earlier

[ https://issues.apache.org/jira/browse/SPARK-36615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406726#comment-17406726 ] Apache Spark commented on SPARK-36615: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-36615) SparkContext should register shutdown hook earlier

[ https://issues.apache.org/jira/browse/SPARK-36615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36615: Assignee: (was: Apache Spark) > SparkContext should register shutdown hook earlier >

[jira] [Created] (SPARK-36615) SparkContext should register shutdown hook earlier

angerszhu created SPARK-36615: - Summary: SparkContext should register shutdown hook earlier Key: SPARK-36615 URL: https://issues.apache.org/jira/browse/SPARK-36615 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-36614) Executor loss reason shows "worker lost" rather "Executor decommission"

[ https://issues.apache.org/jira/browse/SPARK-36614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406715#comment-17406715 ] Apache Spark commented on SPARK-36614: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36614) Executor loss reason shows "worker lost" rather "Executor decommission"

[ https://issues.apache.org/jira/browse/SPARK-36614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36614: Assignee: Apache Spark > Executor loss reason shows "worker lost" rather "Executor

[jira] [Assigned] (SPARK-36614) Executor loss reason shows "worker lost" rather "Executor decommission"

[ https://issues.apache.org/jira/browse/SPARK-36614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36614: Assignee: (was: Apache Spark) > Executor loss reason shows "worker lost" rather

[jira] [Commented] (SPARK-36614) Executor loss reason shows "worker lost" rather "Executor decommission"

[ https://issues.apache.org/jira/browse/SPARK-36614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406714#comment-17406714 ] Apache Spark commented on SPARK-36614: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Created] (SPARK-36614) Executor loss reason shows "worker lost" rather "Executor decommission"

wuyi created SPARK-36614: Summary: Executor loss reason shows "worker lost" rather "Executor decommission" Key: SPARK-36614 URL: https://issues.apache.org/jira/browse/SPARK-36614 Project: Spark

[jira] [Updated] (SPARK-36614) Executor loss reason shows "worker lost" rather "Executor decommission"

[ https://issues.apache.org/jira/browse/SPARK-36614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-36614: - Attachment: WeChat13c9f1345a096ff83d193e4e9853b165.png > Executor loss reason shows "worker lost" rather

[jira] [Assigned] (SPARK-36613) The return value of the Table.capabilities should use EnumSet instead of HashSet

[ https://issues.apache.org/jira/browse/SPARK-36613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36613: Assignee: (was: Apache Spark) > The return value of the Table.capabilities should

[jira] [Commented] (SPARK-36613) The return value of the Table.capabilities should use EnumSet instead of HashSet

[ https://issues.apache.org/jira/browse/SPARK-36613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406698#comment-17406698 ] Apache Spark commented on SPARK-36613: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-36613) The return value of the Table.capabilities should use EnumSet instead of HashSet

[ https://issues.apache.org/jira/browse/SPARK-36613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36613: Assignee: Apache Spark > The return value of the Table.capabilities should use EnumSet

[jira] [Commented] (SPARK-36613) The return value of the Table.capabilities should use EnumSet instead of HashSet

[ https://issues.apache.org/jira/browse/SPARK-36613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406699#comment-17406699 ] Apache Spark commented on SPARK-36613: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-36613) The return value of the Table.capabilities should use EnumSet instead of HashSet

Yang Jie created SPARK-36613: Summary: The return value of the Table.capabilities should use EnumSet instead of HashSet Key: SPARK-36613 URL: https://issues.apache.org/jira/browse/SPARK-36613 Project:

[jira] [Resolved] (SPARK-36574) pushDownPredicate=false should prevent push down filters to JDBC data source

[ https://issues.apache.org/jira/browse/SPARK-36574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36574. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33822

[jira] [Assigned] (SPARK-36574) pushDownPredicate=false should prevent push down filters to JDBC data source

[ https://issues.apache.org/jira/browse/SPARK-36574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36574: --- Assignee: jiaan.geng > pushDownPredicate=false should prevent push down filters to JDBC

[jira] [Created] (SPARK-36612) Support left outer join build left or right outer join build right in shuffled hash join

mcdull_zhang created SPARK-36612: Summary: Support left outer join build left or right outer join build right in shuffled hash join Key: SPARK-36612 URL: https://issues.apache.org/jira/browse/SPARK-36612

[jira] [Commented] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406657#comment-17406657 ] 胡振宇 commented on SPARK-36584: -  [~hyukjin.kwon]  hi, anybody can verify this question or  the driver is

[jira] [Comment Edited] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406622#comment-17406622 ] wuyi edited comment on SPARK-18105 at 8/30/21, 8:31 AM: FYI, for users who hit

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406622#comment-17406622 ] wuyi commented on SPARK-18105: -- FYI, for users who hit the "Stream is corrupted" error, please try to apply

[jira] [Commented] (SPARK-36196) Spark FetchFailedException Stream is corrupted Error

[ https://issues.apache.org/jira/browse/SPARK-36196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406620#comment-17406620 ] wuyi commented on SPARK-36196: -- Hi [~arghya18] Did you try to apply the fix of 

[jira] [Commented] (SPARK-36611) Remove unused listener in HiveThriftServer2AppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-36611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406612#comment-17406612 ] Apache Spark commented on SPARK-36611: -- User 'timothy65535' has created a pull request for this

[jira] [Commented] (SPARK-36611) Remove unused listener in HiveThriftServer2AppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-36611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406611#comment-17406611 ] Apache Spark commented on SPARK-36611: -- User 'timothy65535' has created a pull request for this

[jira] [Assigned] (SPARK-36611) Remove unused listener in HiveThriftServer2AppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-36611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36611: Assignee: Apache Spark > Remove unused listener in HiveThriftServer2AppStatusStore >

[jira] [Assigned] (SPARK-36611) Remove unused listener in HiveThriftServer2AppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-36611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36611: Assignee: (was: Apache Spark) > Remove unused listener in

[jira] [Updated] (SPARK-36611) Remove unused listener in HiveThriftServer2AppStatusStore

[ https://issues.apache.org/jira/browse/SPARK-36611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] timothy65535 updated SPARK-36611: - Summary: Remove unused listener in HiveThriftServer2AppStatusStore (was: Remove useless

[jira] [Created] (SPARK-36611) Remove useless listener in HiveThriftServer2AppStatusStore

timothy65535 created SPARK-36611: Summary: Remove useless listener in HiveThriftServer2AppStatusStore Key: SPARK-36611 URL: https://issues.apache.org/jira/browse/SPARK-36611 Project: Spark

[jira] [Created] (SPARK-36610) Add `thousands` argument to `ps.read_csv`.

Haejoon Lee created SPARK-36610: --- Summary: Add `thousands` argument to `ps.read_csv`. Key: SPARK-36610 URL: https://issues.apache.org/jira/browse/SPARK-36610 Project: Spark Issue Type:

[jira] [Created] (SPARK-36609) Add `errors` argument for `ps.to_numeric`.

Haejoon Lee created SPARK-36609: --- Summary: Add `errors` argument for `ps.to_numeric`. Key: SPARK-36609 URL: https://issues.apache.org/jira/browse/SPARK-36609 Project: Spark Issue Type: