[jira] [Assigned] (SPARK-37332) Check adding of ANSI interval columns to v1/v2 tables

2021-11-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-37332: Assignee: Max Gekk > Check adding of ANSI interval columns to v1/v2 tables >

[jira] [Resolved] (SPARK-37332) Check adding of ANSI interval columns to v1/v2 tables

2021-11-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-37332. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34600

[jira] [Assigned] (SPARK-37277) Support DayTimeIntervalType in Arrow

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37277: Assignee: (was: Apache Spark) > Support DayTimeIntervalType in Arrow >

[jira] [Assigned] (SPARK-37277) Support DayTimeIntervalType in Arrow

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37277: Assignee: Apache Spark > Support DayTimeIntervalType in Arrow >

[jira] [Commented] (SPARK-37277) Support DayTimeIntervalType in Arrow

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444258#comment-17444258 ] Apache Spark commented on SPARK-37277: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444256#comment-17444256 ] pralabhkumar commented on SPARK-37181: -- [~yikunkero] [~chconnell] . I'll work on this and will

[jira] [Assigned] (SPARK-37337) Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37337: Assignee: Apache Spark > Improve the API of Spark DataFrame to pandas-on-Spark DataFrame

[jira] [Commented] (SPARK-37337) Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444247#comment-17444247 ] Apache Spark commented on SPARK-37337: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-37337) Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37337: Assignee: (was: Apache Spark) > Improve the API of Spark DataFrame to

[jira] [Updated] (SPARK-37337) Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion

2021-11-15 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-37337: - Description: - Undeprecate (Spark)DataFrame.to_koalasĀ  - Deprecate

[jira] [Resolved] (SPARK-37338) Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api

2021-11-15 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-37338. -- Resolution: Duplicate > Rename (Spark)DataFrame.to_pandas_on_spark to

[jira] [Comment Edited] (SPARK-37344) split function behave differently between spark 2.3 and spark 3.2

2021-11-15 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444225#comment-17444225 ] angerszhu edited comment on SPARK-37344 at 11/16/21, 2:51 AM: -- In latest

[jira] [Commented] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444235#comment-17444235 ] Yikun Jiang commented on SPARK-37181: - Actully, lating-1 is also do same covert in Python internal

[jira] [Comment Edited] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444235#comment-17444235 ] Yikun Jiang edited comment on SPARK-37181 at 11/16/21, 2:41 AM: Agree,

[jira] [Commented] (SPARK-37344) split function behave differently between spark 2.3 and spark 3.2

2021-11-15 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444225#comment-17444225 ] angerszhu commented on SPARK-37344: --- Work on this > split function behave differently between spark

[jira] [Updated] (SPARK-37344) split function behave differently between spark 2.3 and spark 3.2

2021-11-15 Thread ocean (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ocean updated SPARK-37344: -- Labels: incorrect (was: ) > split function behave differently between spark 2.3 and spark 3.2 >

[jira] [Created] (SPARK-37344) split function behave differently between spark 2.3 and spark 3.2

2021-11-15 Thread ocean (Jira)
ocean created SPARK-37344: - Summary: split function behave differently between spark 2.3 and spark 3.2 Key: SPARK-37344 URL: https://issues.apache.org/jira/browse/SPARK-37344 Project: Spark Issue

[jira] [Commented] (SPARK-37343) Implement createIndex and IndexExists in JDBC (Postgres dialect)

2021-11-15 Thread dch nguyen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444210#comment-17444210 ] dch nguyen commented on SPARK-37343: I'm working on this. > Implement createIndex and IndexExists

[jira] [Created] (SPARK-37343) Implement createIndex and IndexExists in JDBC (Postgres dialect)

2021-11-15 Thread dch nguyen (Jira)
dch nguyen created SPARK-37343: -- Summary: Implement createIndex and IndexExists in JDBC (Postgres dialect) Key: SPARK-37343 URL: https://issues.apache.org/jira/browse/SPARK-37343 Project: Spark

[jira] [Assigned] (SPARK-37335) Clarify output of FPGrowth

2021-11-15 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-37335: Assignee: Nicholas Chammas > Clarify output of FPGrowth > -- > >

[jira] [Resolved] (SPARK-37335) Clarify output of FPGrowth

2021-11-15 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-37335. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34605

[jira] [Commented] (SPARK-37342) Upgrade Apache Arrow to 6.0.0

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444195#comment-17444195 ] Apache Spark commented on SPARK-37342: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37342) Upgrade Apache Arrow to 6.0.0

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37342: Assignee: (was: Apache Spark) > Upgrade Apache Arrow to 6.0.0 >

[jira] [Commented] (SPARK-37342) Upgrade Apache Arrow to 6.0.0

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444194#comment-17444194 ] Apache Spark commented on SPARK-37342: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37342) Upgrade Apache Arrow to 6.0.0

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37342: Assignee: Apache Spark > Upgrade Apache Arrow to 6.0.0 > - >

[jira] [Updated] (SPARK-37342) Upgrade Apache Arrow to 6.0.0

2021-11-15 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-37342: - Component/s: Build (was: Spark Core) > Upgrade Apache Arrow to 6.0.0 >

[jira] [Created] (SPARK-37342) Upgrade Apache Arrow to 6.0.0

2021-11-15 Thread Chao Sun (Jira)
Chao Sun created SPARK-37342: Summary: Upgrade Apache Arrow to 6.0.0 Key: SPARK-37342 URL: https://issues.apache.org/jira/browse/SPARK-37342 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-37341) Avoid unnecessary buffer and copy in full outer sort merge join

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444190#comment-17444190 ] Apache Spark commented on SPARK-37341: -- User 'c21' has created a pull request for this issue:

[jira] [Commented] (SPARK-37341) Avoid unnecessary buffer and copy in full outer sort merge join

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444188#comment-17444188 ] Apache Spark commented on SPARK-37341: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37341) Avoid unnecessary buffer and copy in full outer sort merge join

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37341: Assignee: Apache Spark > Avoid unnecessary buffer and copy in full outer sort merge join

[jira] [Assigned] (SPARK-37341) Avoid unnecessary buffer and copy in full outer sort merge join

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37341: Assignee: (was: Apache Spark) > Avoid unnecessary buffer and copy in full outer sort

[jira] [Created] (SPARK-37341) Avoid unnecessary buffer and copy in full outer sort merge join

2021-11-15 Thread Cheng Su (Jira)
Cheng Su created SPARK-37341: Summary: Avoid unnecessary buffer and copy in full outer sort merge join Key: SPARK-37341 URL: https://issues.apache.org/jira/browse/SPARK-37341 Project: Spark

[jira] [Comment Edited] (SPARK-31646) Remove unused registeredConnections counter from ShuffleMetrics

2021-11-15 Thread Yongjun Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444173#comment-17444173 ] Yongjun Zhang edited comment on SPARK-31646 at 11/15/21, 11:17 PM: --- HI

[jira] [Commented] (SPARK-31646) Remove unused registeredConnections counter from ShuffleMetrics

2021-11-15 Thread Yongjun Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444173#comment-17444173 ] Yongjun Zhang commented on SPARK-31646: --- HI [~mauzhang] , wonder if you have been monitoring the

[jira] [Commented] (SPARK-37340) Display StageIds in Operators for SQL UI

2021-11-15 Thread Yian Liou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444168#comment-17444168 ] Yian Liou commented on SPARK-37340: --- Will be working on this issue and opening pull request. >

[jira] [Created] (SPARK-37340) Display StageIds in Operators for SQL UI

2021-11-15 Thread Yian Liou (Jira)
Yian Liou created SPARK-37340: - Summary: Display StageIds in Operators for SQL UI Key: SPARK-37340 URL: https://issues.apache.org/jira/browse/SPARK-37340 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-37339) Add `spark-version` label to driver and executor pods

2021-11-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37339. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34609

[jira] [Assigned] (SPARK-37339) Add `spark-version` label to driver and executor pods

2021-11-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37339: - Assignee: Dongjoon Hyun > Add `spark-version` label to driver and executor pods >

[jira] [Assigned] (SPARK-35867) Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35867: Assignee: Apache Spark > Enable vectorized read for

[jira] [Commented] (SPARK-35867) Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444135#comment-17444135 ] Apache Spark commented on SPARK-35867: -- User 'kazuyukitanimura' has created a pull request for this

[jira] [Assigned] (SPARK-35867) Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35867: Assignee: (was: Apache Spark) > Enable vectorized read for

[jira] [Commented] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread Chuck Connell (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444117#comment-17444117 ] Chuck Connell commented on SPARK-37181: --- That would be a good solution, just convert latin-1

[jira] [Assigned] (SPARK-34332) Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34332: Assignee: Max Gekk (was: Apache Spark) > Unify v1 and v2 ALTER TABLE .. SET LOCATION

[jira] [Assigned] (SPARK-34332) Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34332: Assignee: Apache Spark (was: Max Gekk) > Unify v1 and v2 ALTER TABLE .. SET LOCATION

[jira] [Commented] (SPARK-34332) Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444095#comment-17444095 ] Apache Spark commented on SPARK-34332: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37339) Add `spark-version` label to driver and executor pods

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37339: Assignee: Apache Spark > Add `spark-version` label to driver and executor pods >

[jira] [Commented] (SPARK-37339) Add `spark-version` label to driver and executor pods

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444087#comment-17444087 ] Apache Spark commented on SPARK-37339: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-37324) Support Decimal RoundingMode.UP, DOWN, HALF_DOWN

2021-11-15 Thread Sathiya Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sathiya Kumar updated SPARK-37324: -- Description: Currently we support only Decimal RoundingModes : HALF_UP (round) and HALF_EVEN

[jira] [Assigned] (SPARK-37339) Add `spark-version` label to driver and executor pods

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37339: Assignee: (was: Apache Spark) > Add `spark-version` label to driver and executor

[jira] [Created] (SPARK-37339) Add `spark-version` label to driver and executor pods

2021-11-15 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-37339: - Summary: Add `spark-version` label to driver and executor pods Key: SPARK-37339 URL: https://issues.apache.org/jira/browse/SPARK-37339 Project: Spark

[jira] [Assigned] (SPARK-37338) Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37338: Assignee: (was: Apache Spark) > Rename (Spark)DataFrame.to_pandas_on_spark to

[jira] [Assigned] (SPARK-37338) Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37338: Assignee: Apache Spark > Rename (Spark)DataFrame.to_pandas_on_spark to

[jira] [Commented] (SPARK-37338) Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444081#comment-17444081 ] Apache Spark commented on SPARK-37338: -- User 'xinrong-databricks' has created a pull request for

[jira] [Updated] (SPARK-37338) Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api

2021-11-15 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-37338: - Summary: Rename (Spark)DataFrame.to_pandas_on_spark to (Spark)DataFrame.pandas_api (was:

[jira] [Updated] (SPARK-37338) Rename to_pandas_on_spark to pandas_api

2021-11-15 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-37338: - Description: Currently, (Spark)DataFrame.to_pandas_on_spark is too long to memorize and

[jira] [Created] (SPARK-37338) Rename to_pandas_on_spark to pandas_api

2021-11-15 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-37338: Summary: Rename to_pandas_on_spark to pandas_api Key: SPARK-37338 URL: https://issues.apache.org/jira/browse/SPARK-37338 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-37337) Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion

2021-11-15 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-37337: Summary: Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion Key: SPARK-37337 URL: https://issues.apache.org/jira/browse/SPARK-37337 Project:

[jira] [Commented] (SPARK-36038) Basic speculation metrics at stage level

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444046#comment-17444046 ] Apache Spark commented on SPARK-36038: -- User 'thejdeep' has created a pull request for this issue:

[jira] [Updated] (SPARK-37320) Delete py_container_checks.zip after the test in DepsTestsSuite finishes

2021-11-15 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-37320: - Component/s: Kubernetes (was: k8) > Delete py_container_checks.zip after

[jira] [Commented] (SPARK-37336) Migrate _java2py to SparkSession

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443991#comment-17443991 ] Apache Spark commented on SPARK-37336: -- User 'nchammas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37336) Migrate _java2py to SparkSession

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37336: Assignee: (was: Apache Spark) > Migrate _java2py to SparkSession >

[jira] [Assigned] (SPARK-37336) Migrate _java2py to SparkSession

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37336: Assignee: Apache Spark > Migrate _java2py to SparkSession >

[jira] [Commented] (SPARK-37336) Migrate _java2py to SparkSession

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443990#comment-17443990 ] Apache Spark commented on SPARK-37336: -- User 'nchammas' has created a pull request for this issue:

[jira] [Updated] (SPARK-37336) Migrate _java2py to SparkSession

2021-11-15 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-37336: - Summary: Migrate _java2py to SparkSession (was: Migrate common ML utils to

[jira] [Created] (SPARK-37336) Migrate common ML utils to SparkSession

2021-11-15 Thread Nicholas Chammas (Jira)
Nicholas Chammas created SPARK-37336: Summary: Migrate common ML utils to SparkSession Key: SPARK-37336 URL: https://issues.apache.org/jira/browse/SPARK-37336 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37335) Clarify output of FPGrowth

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443973#comment-17443973 ] Apache Spark commented on SPARK-37335: -- User 'nchammas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37335) Clarify output of FPGrowth

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37335: Assignee: Apache Spark > Clarify output of FPGrowth > -- > >

[jira] [Commented] (SPARK-37335) Clarify output of FPGrowth

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443970#comment-17443970 ] Apache Spark commented on SPARK-37335: -- User 'nchammas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37335) Clarify output of FPGrowth

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37335: Assignee: (was: Apache Spark) > Clarify output of FPGrowth >

[jira] [Updated] (SPARK-37335) Clarify output of FPGrowth

2021-11-15 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-37335: - Description: The association rules returned by FPGrow include more columns than are

[jira] [Created] (SPARK-37335) Clarify output of FPGrowth

2021-11-15 Thread Nicholas Chammas (Jira)
Nicholas Chammas created SPARK-37335: Summary: Clarify output of FPGrowth Key: SPARK-37335 URL: https://issues.apache.org/jira/browse/SPARK-37335 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-37329) File system delegation tokens are leaked

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37329: Assignee: Apache Spark > File system delegation tokens are leaked >

[jira] [Assigned] (SPARK-37329) File system delegation tokens are leaked

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37329: Assignee: (was: Apache Spark) > File system delegation tokens are leaked >

[jira] [Commented] (SPARK-37329) File system delegation tokens are leaked

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443890#comment-17443890 ] Apache Spark commented on SPARK-37329: -- User 'jojochuang' has created a pull request for this

[jira] [Comment Edited] (SPARK-37329) File system delegation tokens are leaked

2021-11-15 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443618#comment-17443618 ] Wei-Chiu Chuang edited comment on SPARK-37329 at 11/15/21, 2:57 PM:

[jira] [Resolved] (SPARK-37266) View text can only be SELECT queries

2021-11-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37266. - Fix Version/s: 3.3.0 Assignee: jiaan.geng Resolution: Fixed > View text can

[jira] [Comment Edited] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443764#comment-17443764 ] pralabhkumar edited comment on SPARK-37181 at 11/15/21, 2:10 PM: -

[jira] [Comment Edited] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443764#comment-17443764 ] pralabhkumar edited comment on SPARK-37181 at 11/15/21, 2:10 PM: -

[jira] [Created] (SPARK-37334) pandas `convert_dtypes` method support

2021-11-15 Thread Ali Amin-Nejad (Jira)
Ali Amin-Nejad created SPARK-37334: -- Summary: pandas `convert_dtypes` method support Key: SPARK-37334 URL: https://issues.apache.org/jira/browse/SPARK-37334 Project: Spark Issue Type: New

[jira] [Assigned] (SPARK-37328) SPARK-33832 brings the bug that OptimizeSkewedJoin may not work since it was applied on whole plan innstead of new stage plan

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37328: Assignee: (was: Apache Spark) > SPARK-33832 brings the bug that OptimizeSkewedJoin

[jira] [Assigned] (SPARK-37328) SPARK-33832 brings the bug that OptimizeSkewedJoin may not work since it was applied on whole plan innstead of new stage plan

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37328: Assignee: Apache Spark > SPARK-33832 brings the bug that OptimizeSkewedJoin may not work

[jira] [Commented] (SPARK-37328) SPARK-33832 brings the bug that OptimizeSkewedJoin may not work since it was applied on whole plan innstead of new stage plan

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443801#comment-17443801 ] Apache Spark commented on SPARK-37328: -- User 'Liulietong' has created a pull request for this

[jira] [Created] (SPARK-37333) Specify the required distribution at V1Write

2021-11-15 Thread XiDuo You (Jira)
XiDuo You created SPARK-37333: - Summary: Specify the required distribution at V1Write Key: SPARK-37333 URL: https://issues.apache.org/jira/browse/SPARK-37333 Project: Spark Issue Type:

[jira] [Updated] (SPARK-37328) SPARK-33832 brings the bug that OptimizeSkewedJoin may not work since it was applied on whole plan innstead of new stage plan

2021-11-15 Thread Lietong Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lietong Liu updated SPARK-37328: Summary: SPARK-33832 brings the bug that OptimizeSkewedJoin may not work since it was applied on

[jira] [Commented] (SPARK-37316) Add code-gen for existence sort merge join

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443794#comment-17443794 ] Apache Spark commented on SPARK-37316: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37316) Add code-gen for existence sort merge join

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37316: Assignee: (was: Apache Spark) > Add code-gen for existence sort merge join >

[jira] [Assigned] (SPARK-37316) Add code-gen for existence sort merge join

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37316: Assignee: Apache Spark > Add code-gen for existence sort merge join >

[jira] [Commented] (SPARK-37316) Add code-gen for existence sort merge join

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443793#comment-17443793 ] Apache Spark commented on SPARK-37316: -- User 'c21' has created a pull request for this issue:

[jira] [Resolved] (SPARK-35352) Add code-gen for full outer sort merge join

2021-11-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35352. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34581

[jira] [Assigned] (SPARK-35352) Add code-gen for full outer sort merge join

2021-11-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35352: --- Assignee: Cheng Su > Add code-gen for full outer sort merge join >

[jira] [Assigned] (SPARK-37332) Check adding of ANSI interval columns to v1/v2 tables

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37332: Assignee: (was: Apache Spark) > Check adding of ANSI interval columns to v1/v2

[jira] [Commented] (SPARK-37332) Check adding of ANSI interval columns to v1/v2 tables

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443768#comment-17443768 ] Apache Spark commented on SPARK-37332: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37332) Check adding of ANSI interval columns to v1/v2 tables

2021-11-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37332: Assignee: Apache Spark > Check adding of ANSI interval columns to v1/v2 tables >

[jira] [Commented] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443764#comment-17443764 ] pralabhkumar commented on SPARK-37181: -- However from users point of view , if user mention latin-1

[jira] [Comment Edited] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443720#comment-17443720 ] pralabhkumar edited comment on SPARK-37181 at 11/15/21, 10:32 AM: -- from

[jira] [Commented] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443720#comment-17443720 ] pralabhkumar commented on SPARK-37181: -- from pyspark import pandas as ps latin-1 encoding is same

[jira] [Commented] (SPARK-37332) Check adding of ANSI interval columns to v1/v2 tables

2021-11-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443715#comment-17443715 ] Max Gekk commented on SPARK-37332: -- I am working on this. > Check adding of ANSI interval columns to

[jira] [Updated] (SPARK-37332) Check adding of ANSI interval columns to v1/v2 tables

2021-11-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37332: - Summary: Check adding of ANSI interval columns to v1/v2 tables (was: Check adding of ANSI interval

[jira] [Created] (SPARK-37332) Check adding of ANSI interval columns

2021-11-15 Thread Max Gekk (Jira)
Max Gekk created SPARK-37332: Summary: Check adding of ANSI interval columns Key: SPARK-37332 URL: https://issues.apache.org/jira/browse/SPARK-37332 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-37283) Don't try to store a V1 table which contains ANSI intervals in Hive compatible format

2021-11-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37283. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34551

  1   2   >