[jira] [Updated] (SPARK-37141) WorkerSuite cannot run on Mac OS

2021-10-27 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-37141: - Description: After SPARK-35907 run `org.apache.spark.deploy.worker.WorkerSuite` on Mac os(both M1 and

[jira] [Created] (SPARK-37141) WorkerSuite cannot run on Mac OS

2021-10-27 Thread Yang Jie (Jira)
Yang Jie created SPARK-37141: Summary: WorkerSuite cannot run on Mac OS Key: SPARK-37141 URL: https://issues.apache.org/jira/browse/SPARK-37141 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-37133) Add a config to optionally enforce ANSI reserved keywords

2021-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37133. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34403

[jira] [Commented] (SPARK-37140) Inline type hints for python/pyspark/resultiterable.py

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435146#comment-17435146 ] Apache Spark commented on SPARK-37140: -- User 'dchvn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37140) Inline type hints for python/pyspark/resultiterable.py

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37140: Assignee: Apache Spark > Inline type hints for python/pyspark/resultiterable.py >

[jira] [Commented] (SPARK-37140) Inline type hints for python/pyspark/resultiterable.py

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435145#comment-17435145 ] Apache Spark commented on SPARK-37140: -- User 'dchvn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37140) Inline type hints for python/pyspark/resultiterable.py

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37140: Assignee: (was: Apache Spark) > Inline type hints for

[jira] [Commented] (SPARK-37138) Support ANSI Interval in functions that support numeric type

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435144#comment-17435144 ] Apache Spark commented on SPARK-37138: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-37138) Support ANSI Interval in functions that support numeric type

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435143#comment-17435143 ] Apache Spark commented on SPARK-37138: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37138) Support ANSI Interval in functions that support numeric type

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37138: Assignee: (was: Apache Spark) > Support ANSI Interval in functions that support

[jira] [Assigned] (SPARK-37138) Support ANSI Interval in functions that support numeric type

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37138: Assignee: Apache Spark > Support ANSI Interval in functions that support numeric type >

[jira] [Created] (SPARK-37140) Inline type hints for python/pyspark/resultiterable.py

2021-10-27 Thread dch nguyen (Jira)
dch nguyen created SPARK-37140: -- Summary: Inline type hints for python/pyspark/resultiterable.py Key: SPARK-37140 URL: https://issues.apache.org/jira/browse/SPARK-37140 Project: Spark Issue

[jira] [Created] (SPARK-37139) Inline type hints for python/pyspark/taskcontext.py and python/pyspark/version.py

2021-10-27 Thread dch nguyen (Jira)
dch nguyen created SPARK-37139: -- Summary: Inline type hints for python/pyspark/taskcontext.py and python/pyspark/version.py Key: SPARK-37139 URL: https://issues.apache.org/jira/browse/SPARK-37139

[jira] [Commented] (SPARK-37137) Inline type hints for python/pyspark/conf.py

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435142#comment-17435142 ] Apache Spark commented on SPARK-37137: -- User 'ByronHsu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37137) Inline type hints for python/pyspark/conf.py

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37137: Assignee: Apache Spark > Inline type hints for python/pyspark/conf.py >

[jira] [Assigned] (SPARK-37137) Inline type hints for python/pyspark/conf.py

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37137: Assignee: (was: Apache Spark) > Inline type hints for python/pyspark/conf.py >

[jira] [Created] (SPARK-37138) Support ANSI Interval in functions that support numeric type

2021-10-27 Thread angerszhu (Jira)
angerszhu created SPARK-37138: - Summary: Support ANSI Interval in functions that support numeric type Key: SPARK-37138 URL: https://issues.apache.org/jira/browse/SPARK-37138 Project: Spark

[jira] [Created] (SPARK-37137) Inline type hints for python/pyspark/conf.py

2021-10-27 Thread Byron Hsu (Jira)
Byron Hsu created SPARK-37137: - Summary: Inline type hints for python/pyspark/conf.py Key: SPARK-37137 URL: https://issues.apache.org/jira/browse/SPARK-37137 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-37136) Remove code about hive build in functions

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37136: Assignee: (was: Apache Spark) > Remove code about hive build in functions >

[jira] [Assigned] (SPARK-37136) Remove code about hive build in functions

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37136: Assignee: Apache Spark > Remove code about hive build in functions >

[jira] [Commented] (SPARK-37136) Remove code about hive build in functions

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435134#comment-17435134 ] Apache Spark commented on SPARK-37136: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37135) Fix some mirco-benchmarks run failed

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37135: Assignee: Apache Spark > Fix some mirco-benchmarks run failed >

[jira] [Commented] (SPARK-37135) Fix some mirco-benchmarks run failed

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435133#comment-17435133 ] Apache Spark commented on SPARK-37135: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-37135) Fix some mirco-benchmarks run failed

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37135: Assignee: Apache Spark > Fix some mirco-benchmarks run failed >

[jira] [Assigned] (SPARK-37135) Fix some mirco-benchmarks run failed

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37135: Assignee: (was: Apache Spark) > Fix some mirco-benchmarks run failed >

[jira] [Created] (SPARK-37136) Remove code about hive build in functions

2021-10-27 Thread angerszhu (Jira)
angerszhu created SPARK-37136: - Summary: Remove code about hive build in functions Key: SPARK-37136 URL: https://issues.apache.org/jira/browse/SPARK-37136 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-37135) Fix some mirco-benchmarks run failed

2021-10-27 Thread Yang Jie (Jira)
Yang Jie created SPARK-37135: Summary: Fix some mirco-benchmarks run failed Key: SPARK-37135 URL: https://issues.apache.org/jira/browse/SPARK-37135 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-37036) Add util function to raise advice warning for pandas API on Spark.

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37036. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34389

[jira] [Assigned] (SPARK-37036) Add util function to raise advice warning for pandas API on Spark.

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37036: Assignee: Haejoon Lee > Add util function to raise advice warning for pandas API on

[jira] [Updated] (SPARK-37119) parse_url can not handle `{` and `}` correctly

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37119: - Priority: Major (was: Critical) > parse_url can not handle `{` and `}` correctly >

[jira] [Resolved] (SPARK-37119) parse_url can not handle `{` and `}` correctly

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37119. -- Resolution: Invalid > parse_url can not handle `{` and `}` correctly >

[jira] [Updated] (SPARK-37117) Can't read files in one of Parquet encryption modes (external keymaterial)

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37117: - Target Version/s: (was: 3.2.1) > Can't read files in one of Parquet encryption modes

[jira] [Updated] (SPARK-37122) java.lang.IllegalArgumentException Related to Prometheus

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37122: - Priority: Major (was: Critical) > java.lang.IllegalArgumentException Related to Prometheus >

[jira] [Issue Comment Deleted] (SPARK-37095) Inline type hints for files in python/pyspark/broadcast.py

2021-10-27 Thread Byron Hsu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Byron Hsu updated SPARK-37095: -- Comment: was deleted (was: test) > Inline type hints for files in python/pyspark/broadcast.py >

[jira] [Commented] (SPARK-37128) Application has been removed by master but driver still running

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435121#comment-17435121 ] Hyukjin Kwon commented on SPARK-37128: -- Can you share the steps to reproduce the issue? which

[jira] [Commented] (SPARK-37131) Support use IN/EXISTS with subquery in Project/Aggregate

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435120#comment-17435120 ] Hyukjin Kwon commented on SPARK-37131: -- cc [~allisonwang-db] and [~cloud_fan] FYI > Support use

[jira] [Commented] (SPARK-37095) Inline type hints for files in python/pyspark/broadcast.py

2021-10-27 Thread Byron Hsu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435119#comment-17435119 ] Byron Hsu commented on SPARK-37095: --- test > Inline type hints for files in

[jira] [Resolved] (SPARK-37132) Incorrect Spark 3.2.0 package names with included Hadoop binaries

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37132. -- Resolution: Duplicate > Incorrect Spark 3.2.0 package names with included Hadoop binaries >

[jira] [Commented] (SPARK-37134) documentation - unclear "Using PySpark Native Features"

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435117#comment-17435117 ] Hyukjin Kwon commented on SPARK-37134: -- They are individual items so OR is correct. Feel free to

[jira] [Updated] (SPARK-37134) documentation - unclear "Using PySpark Native Features"

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37134: - Target Version/s: (was: 1.6.2) > documentation - unclear "Using PySpark Native Features" >

[jira] [Updated] (SPARK-37134) documentation - unclear "Using PySpark Native Features"

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37134: - Shepherd: (was: Tenovip33) > documentation - unclear "Using PySpark Native Features" >

[jira] [Updated] (SPARK-37134) documentation - unclear "Using PySpark Native Features"

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37134: - Priority: Major (was: Critical) > documentation - unclear "Using PySpark Native Features" >

[jira] [Updated] (SPARK-37134) documentation - unclear "Using PySpark Native Features"

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37134: - Environment: (was: ?) > documentation - unclear "Using PySpark Native Features" >

[jira] [Updated] (SPARK-37134) documentation - unclear "Using PySpark Native Features"

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37134: - Fix Version/s: (was: 1.6.2) > documentation - unclear "Using PySpark Native Features" >

[jira] [Resolved] (SPARK-37121) TestUtils.isPythonVersionAtLeast38 returns incorrect results

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37121. -- Fix Version/s: 3.2.1 3.3.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-37121) TestUtils.isPythonVersionAtLeast38 returns incorrect results

2021-10-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37121: Assignee: Erik Krogen > TestUtils.isPythonVersionAtLeast38 returns incorrect results >

[jira] [Commented] (SPARK-37047) Add overloads for lpad and rpad for BINARY strings

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435105#comment-17435105 ] Apache Spark commented on SPARK-37047: -- User 'mkaravel' has created a pull request for this issue:

[jira] [Updated] (SPARK-37134) documentation - unclear "Using PySpark Native Features"

2021-10-27 Thread carl rees (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] carl rees updated SPARK-37134: -- Description: sorry, no idea on the version it affects or what Shepard is? no explanation on this

[jira] [Created] (SPARK-37134) documentation - unclear "Using PySpark Native Features"

2021-10-27 Thread carl rees (Jira)
carl rees created SPARK-37134: - Summary: documentation - unclear "Using PySpark Native Features" Key: SPARK-37134 URL: https://issues.apache.org/jira/browse/SPARK-37134 Project: Spark Issue

[jira] [Commented] (SPARK-36646) Push down group by partition column for Aggregate (Min/Max/Count) for Parquet

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434995#comment-17434995 ] Apache Spark commented on SPARK-36646: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36646) Push down group by partition column for Aggregate (Min/Max/Count) for Parquet

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36646: Assignee: (was: Apache Spark) > Push down group by partition column for Aggregate

[jira] [Assigned] (SPARK-36646) Push down group by partition column for Aggregate (Min/Max/Count) for Parquet

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36646: Assignee: Apache Spark > Push down group by partition column for Aggregate

[jira] [Assigned] (SPARK-30220) Support Filter expression uses IN/EXISTS predicate sub-queries

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30220: Assignee: (was: Apache Spark) > Support Filter expression uses IN/EXISTS predicate

[jira] [Assigned] (SPARK-30220) Support Filter expression uses IN/EXISTS predicate sub-queries

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30220: Assignee: Apache Spark > Support Filter expression uses IN/EXISTS predicate sub-queries

[jira] [Commented] (SPARK-30220) Support Filter expression uses IN/EXISTS predicate sub-queries

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434897#comment-17434897 ] Apache Spark commented on SPARK-30220: -- User 'tanelk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37133) Add a config to optionally enforce ANSI reserved keywords

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37133: Assignee: Apache Spark (was: Wenchen Fan) > Add a config to optionally enforce ANSI

[jira] [Commented] (SPARK-37133) Add a config to optionally enforce ANSI reserved keywords

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434896#comment-17434896 ] Apache Spark commented on SPARK-37133: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37133) Add a config to optionally enforce ANSI reserved keywords

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37133: Assignee: Wenchen Fan (was: Apache Spark) > Add a config to optionally enforce ANSI

[jira] [Created] (SPARK-37132) Incorrect Spark 3.2.0 package names with included Hadoop binaries

2021-10-27 Thread Denis Krivenko (Jira)
Denis Krivenko created SPARK-37132: -- Summary: Incorrect Spark 3.2.0 package names with included Hadoop binaries Key: SPARK-37132 URL: https://issues.apache.org/jira/browse/SPARK-37132 Project: Spark

[jira] [Created] (SPARK-37133) Add a config to optionally enforce ANSI reserved keywords

2021-10-27 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-37133: --- Summary: Add a config to optionally enforce ANSI reserved keywords Key: SPARK-37133 URL: https://issues.apache.org/jira/browse/SPARK-37133 Project: Spark

[jira] [Resolved] (SPARK-37071) OpenHashMap should be serializable without reference tracking

2021-10-27 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-37071. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34351

[jira] [Assigned] (SPARK-37071) OpenHashMap should be serializable without reference tracking

2021-10-27 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-37071: Assignee: Emil Ejbyfeldt > OpenHashMap should be serializable without reference tracking

[jira] [Updated] (SPARK-37131) Support use IN/EXISTS with subquery in Project/Aggregate

2021-10-27 Thread Tongwei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tongwei updated SPARK-37131: Description: {code:java} CREATE TABLE tbl1 (col1 INT, col2 STRING) USING PARQUET; INSERT OVERWRITE TABLE

[jira] [Updated] (SPARK-37131) Support use IN/EXISTS with subquery in Project/Aggregate

2021-10-27 Thread Tongwei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tongwei updated SPARK-37131: Description:     {code:java} CREATE TABLE tbl1 (col1 INT, col2 STRING) USING PARQUET; INSERT OVERWRITE

[jira] [Updated] (SPARK-37131) Support use IN/EXISTS with subquery in Project/Aggregate

2021-10-27 Thread Tongwei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tongwei updated SPARK-37131: Description: CREATE TABLE tbl1 (col1 INT, col2 STRING) USING PARQUET; INSERT OVERWRITE TABLE tbl1 SELECT

[jira] [Updated] (SPARK-37131) Support use IN/EXISTS with subquery in Project/Aggregate

2021-10-27 Thread Tongwei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tongwei updated SPARK-37131: Description: ``` CREATE TABLE tbl1 (col1 INT, col2 STRING) USING PARQUET; INSERT OVERWRITE TABLE tbl1

[jira] [Created] (SPARK-37131) Support use IN/EXISTS with subquery in Project/Aggregate

2021-10-27 Thread Tongwei (Jira)
Tongwei created SPARK-37131: --- Summary: Support use IN/EXISTS with subquery in Project/Aggregate Key: SPARK-37131 URL: https://issues.apache.org/jira/browse/SPARK-37131 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37130) why spark-X.X.X-bin-without-hadoop.tgz does not provide spark-hive_X.jar (and spark-hive-thriftserver_X.jar)

2021-10-27 Thread Patrice DUROUX (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434848#comment-17434848 ] Patrice DUROUX commented on SPARK-37130: ps: a diff output {{$ diff spark-without.lst

[jira] [Created] (SPARK-37130) why spark-X.X.X-bin-without-hadoop.tgz does not provide spark-hive_X.jar (and spark-hive-thriftserver_X.jar)

2021-10-27 Thread Patrice DUROUX (Jira)
Patrice DUROUX created SPARK-37130: -- Summary: why spark-X.X.X-bin-without-hadoop.tgz does not provide spark-hive_X.jar (and spark-hive-thriftserver_X.jar) Key: SPARK-37130 URL:

[jira] [Commented] (SPARK-30537) toPandas gets wrong dtypes when applied on empty DF when Arrow enabled

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434840#comment-17434840 ] Apache Spark commented on SPARK-30537: -- User 'pralabhkumar' has created a pull request for this

[jira] [Assigned] (SPARK-30537) toPandas gets wrong dtypes when applied on empty DF when Arrow enabled

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30537: Assignee: Apache Spark > toPandas gets wrong dtypes when applied on empty DF when Arrow

[jira] [Assigned] (SPARK-30537) toPandas gets wrong dtypes when applied on empty DF when Arrow enabled

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30537: Assignee: (was: Apache Spark) > toPandas gets wrong dtypes when applied on empty DF

[jira] [Commented] (SPARK-30537) toPandas gets wrong dtypes when applied on empty DF when Arrow enabled

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434839#comment-17434839 ] Apache Spark commented on SPARK-30537: -- User 'pralabhkumar' has created a pull request for this

[jira] [Commented] (SPARK-16280) Implement histogram_numeric SQL function

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434821#comment-17434821 ] Apache Spark commented on SPARK-16280: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-16280) Implement histogram_numeric SQL function

2021-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-16280: --- Assignee: angerszhu > Implement histogram_numeric SQL function >

[jira] [Commented] (SPARK-16280) Implement histogram_numeric SQL function

2021-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434812#comment-17434812 ] Wenchen Fan commented on SPARK-16280: - resolved by https://github.com/apache/spark/pull/34380 >

[jira] [Resolved] (SPARK-37082) Implement histogram_numeric aggregate function in spark

2021-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37082. - Resolution: Duplicate > Implement histogram_numeric aggregate function in spark >

[jira] [Commented] (SPARK-36975) Refactor HiveClientImpl collect hive client call logic

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434790#comment-17434790 ] Apache Spark commented on SPARK-36975: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-36975) Refactor HiveClientImpl collect hive client call logic

2021-10-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434789#comment-17434789 ] Apache Spark commented on SPARK-36975: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-37129) Supplement all micro benchmark results use to Java 17

2021-10-27 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434762#comment-17434762 ] Yang Jie commented on SPARK-37129: -- I try to run all benchmarks on GA now > Supplement all micro

[jira] [Commented] (SPARK-35260) DataSourceV2 Function Catalog implementation

2021-10-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434754#comment-17434754 ] Dongjoon Hyun commented on SPARK-35260: --- I assigned this umbrella issue to, [~csun]. >

[jira] [Assigned] (SPARK-35260) DataSourceV2 Function Catalog implementation

2021-10-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35260: - Assignee: Chao Sun > DataSourceV2 Function Catalog implementation >

[jira] [Issue Comment Deleted] (SPARK-37129) Supplement all micro benchmark results use to Java 17

2021-10-27 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-37129: - Comment: was deleted (was: working on this) > Supplement all micro benchmark results use to Java 17 >

[jira] [Commented] (SPARK-37129) Supplement all micro benchmark results use to Java 17

2021-10-27 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434711#comment-17434711 ] Yang Jie commented on SPARK-37129: -- working on this > Supplement all micro benchmark results use to

[jira] [Created] (SPARK-37129) Supplement all micro benchmark results use to Java 17

2021-10-27 Thread Yang Jie (Jira)
Yang Jie created SPARK-37129: Summary: Supplement all micro benchmark results use to Java 17 Key: SPARK-37129 URL: https://issues.apache.org/jira/browse/SPARK-37129 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-37115) Replace HiveClient call with hive shim

2021-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37115: --- Assignee: angerszhu > Replace HiveClient call with hive shim >

[jira] [Resolved] (SPARK-37115) Replace HiveClient call with hive shim

2021-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37115. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34388

[jira] [Resolved] (SPARK-37125) Support AnsiInterval radix sort

2021-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37125. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34398

[jira] [Assigned] (SPARK-37125) Support AnsiInterval radix sort

2021-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37125: --- Assignee: XiDuo You > Support AnsiInterval radix sort > --- >

[jira] [Created] (SPARK-37128) Application has been removed by master but driver still running

2021-10-27 Thread JacobZheng (Jira)
JacobZheng created SPARK-37128: -- Summary: Application has been removed by master but driver still running Key: SPARK-37128 URL: https://issues.apache.org/jira/browse/SPARK-37128 Project: Spark

[jira] [Resolved] (SPARK-36647) Push down filter by partition column for Aggregate (Min/Max/Count) for Parquet

2021-10-27 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-36647. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34248

[jira] [Assigned] (SPARK-36647) Push down filter by partition column for Aggregate (Min/Max/Count) for Parquet

2021-10-27 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-36647: --- Assignee: Huaxin Gao > Push down filter by partition column for Aggregate (Min/Max/Count)