[jira] [Assigned] (SPARK-31317) Add withField method to Column class

2020-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31317: --- Assignee: fqaiser94 > Add withField method to Column class >

[jira] [Created] (SPARK-32219) Add SHOW CACHED TABLES Command

2020-07-07 Thread ulysses you (Jira)
ulysses you created SPARK-32219: --- Summary: Add SHOW CACHED TABLES Command Key: SPARK-32219 URL: https://issues.apache.org/jira/browse/SPARK-32219 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-32159) New udaf(Aggregator) has an integration bug with UnresolvedMapObjects serialization

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32159: -- Fix Version/s: (was: 3.0.1) Target Version/s: 3.0.1 > New udaf(Aggregator) has an

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

2020-07-07 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153131#comment-17153131 ] JinxinTang edited comment on SPARK-32205 at 7/8/20, 5:10 AM: -

[jira] [Created] (SPARK-32218) spark-ml must support one hot encoded output labels for classification

2020-07-07 Thread Raghuvarran V H (Jira)
Raghuvarran V H created SPARK-32218: --- Summary: spark-ml must support one hot encoded output labels for classification Key: SPARK-32218 URL: https://issues.apache.org/jira/browse/SPARK-32218

[jira] [Comment Edited] (SPARK-32213) saveAsTable deletes all files in path

2020-07-07 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153200#comment-17153200 ] angerszhu edited comment on SPARK-32213 at 7/8/20, 3:13 AM:

[jira] [Commented] (SPARK-32213) saveAsTable deletes all files in path

2020-07-07 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153200#comment-17153200 ] angerszhu commented on SPARK-32213: --- https://issues.apache.org/jira/browse/SPARK-

[jira] [Commented] (SPARK-32163) Nested pruning should still work for nested column extractors of attributes with cosmetic variations

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153180#comment-17153180 ] Dongjoon Hyun commented on SPARK-32163: --- This lands at branch-3.0 via

[jira] [Updated] (SPARK-32163) Nested pruning should still work for nested column extractors of attributes with cosmetic variations

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32163: -- Fix Version/s: 3.0.1 > Nested pruning should still work for nested column extractors of

[jira] [Commented] (SPARK-30703) Add a documentation page for ANSI mode

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153177#comment-17153177 ] Apache Spark commented on SPARK-30703: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-20680) Spark-sql do not support for void column datatype of view

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20680: Assignee: (was: Apache Spark) > Spark-sql do not support for void column datatype of

[jira] [Resolved] (SPARK-20680) Spark-sql do not support for void column datatype of view

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-20680. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28833

[jira] [Assigned] (SPARK-20680) Spark-sql do not support for void column datatype of view

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-20680: - Assignee: Lantao Jin > Spark-sql do not support for void column datatype of view >

[jira] [Assigned] (SPARK-20680) Spark-sql do not support for void column datatype of view

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20680: Assignee: Apache Spark > Spark-sql do not support for void column datatype of view >

[jira] [Updated] (SPARK-20680) Spark-sql do not support for void column datatype of view

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-20680: -- Affects Version/s: 2.4.6 3.0.0 > Spark-sql do not support for void

[jira] [Reopened] (SPARK-20680) Spark-sql do not support for void column datatype of view

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-20680: --- > Spark-sql do not support for void column datatype of view >

[jira] [Updated] (SPARK-20680) Spark-sql do not support for void column datatype of view

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-20680: -- Labels: (was: bulk-closed) > Spark-sql do not support for void column datatype of view >

[jira] [Commented] (SPARK-29038) SPIP: Support Spark Materialized View

2020-07-07 Thread AidenZhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153170#comment-17153170 ] AidenZhang commented on SPARK-29038: Hi [~cltlfcjin],Thanks for you reply The situation is that

[jira] [Commented] (SPARK-31760) Simplification Based on Containment

2020-07-07 Thread Nikita Glashenko (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153161#comment-17153161 ] Nikita Glashenko commented on SPARK-31760: -- Hi, [~yumwang]. I'd like to work on this. >

[jira] [Assigned] (SPARK-32217) Track whether the worker is also being decommissioned along with an executor

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32217: Assignee: Apache Spark > Track whether the worker is also being decommissioned along

[jira] [Assigned] (SPARK-32217) Track whether the worker is also being decommissioned along with an executor

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32217: Assignee: (was: Apache Spark) > Track whether the worker is also being

[jira] [Commented] (SPARK-32217) Track whether the worker is also being decommissioned along with an executor

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153157#comment-17153157 ] Apache Spark commented on SPARK-32217: -- User 'agrawaldevesh' has created a pull request for this

[jira] [Created] (SPARK-32217) Track whether the worker is also being decommissioned along with an executor

2020-07-07 Thread Devesh Agrawal (Jira)
Devesh Agrawal created SPARK-32217: -- Summary: Track whether the worker is also being decommissioned along with an executor Key: SPARK-32217 URL: https://issues.apache.org/jira/browse/SPARK-32217

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

2020-07-07 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153131#comment-17153131 ] JinxinTang edited comment on SPARK-32205 at 7/8/20, 12:41 AM: --

[jira] [Resolved] (SPARK-32057) SparkExecuteStatementOperation does not set CANCELED state correctly

2020-07-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32057. -- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-32057) SparkExecuteStatementOperation does not set CANCELED state correctly

2020-07-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32057: Assignee: Ali Smesseim > SparkExecuteStatementOperation does not set CANCELED state

[jira] [Assigned] (SPARK-32216) Remove redundant ProjectExec

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32216: Assignee: (was: Apache Spark) > Remove redundant ProjectExec >

[jira] [Commented] (SPARK-32216) Remove redundant ProjectExec

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153136#comment-17153136 ] Apache Spark commented on SPARK-32216: -- User 'allisonwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-32216) Remove redundant ProjectExec

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32216: Assignee: Apache Spark > Remove redundant ProjectExec > > >

[jira] [Created] (SPARK-32216) Remove redundant ProjectExec

2020-07-07 Thread Allison Wang (Jira)
Allison Wang created SPARK-32216: Summary: Remove redundant ProjectExec Key: SPARK-32216 URL: https://issues.apache.org/jira/browse/SPARK-32216 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

2020-07-07 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153131#comment-17153131 ] JinxinTang edited comment on SPARK-32205 at 7/7/20, 11:31 PM: --

[jira] [Commented] (SPARK-32205) Writing timestamp in mysql gets fails

2020-07-07 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153131#comment-17153131 ] JinxinTang commented on SPARK-32205: [~nileshr.patil] Seems a issue, we can insert timestamp to

[jira] [Commented] (SPARK-32215) Expose end point on Master so that it can be informed about decommissioned workers out of band

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153127#comment-17153127 ] Apache Spark commented on SPARK-32215: -- User 'agrawaldevesh' has created a pull request for this

[jira] [Commented] (SPARK-32215) Expose end point on Master so that it can be informed about decommissioned workers out of band

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153126#comment-17153126 ] Apache Spark commented on SPARK-32215: -- User 'agrawaldevesh' has created a pull request for this

[jira] [Assigned] (SPARK-32215) Expose end point on Master so that it can be informed about decommissioned workers out of band

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32215: Assignee: Apache Spark > Expose end point on Master so that it can be informed about

[jira] [Assigned] (SPARK-32215) Expose end point on Master so that it can be informed about decommissioned workers out of band

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32215: Assignee: (was: Apache Spark) > Expose end point on Master so that it can be

[jira] [Updated] (SPARK-32215) Expose end point on Master so that it can be informed about decommissioned workers out of band

2020-07-07 Thread Devesh Agrawal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devesh Agrawal updated SPARK-32215: --- Description: The use case here is to allow some external entity that has made a

[jira] [Created] (SPARK-32215) Expose end point on Master so that it can be informed about decommissioned workers out of band

2020-07-07 Thread Devesh Agrawal (Jira)
Devesh Agrawal created SPARK-32215: -- Summary: Expose end point on Master so that it can be informed about decommissioned workers out of band Key: SPARK-32215 URL:

[jira] [Commented] (SPARK-32093) Add hadoop-ozone-filesystem jar to ozone profile

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153123#comment-17153123 ] Apache Spark commented on SPARK-32093: -- User 'bharatviswa504' has created a pull request for this

[jira] [Assigned] (SPARK-32093) Add hadoop-ozone-filesystem jar to ozone profile

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32093: Assignee: Apache Spark > Add hadoop-ozone-filesystem jar to ozone profile >

[jira] [Assigned] (SPARK-32093) Add hadoop-ozone-filesystem jar to ozone profile

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32093: Assignee: (was: Apache Spark) > Add hadoop-ozone-filesystem jar to ozone profile >

[jira] [Updated] (SPARK-32159) New udaf(Aggregator) has an integration bug with UnresolvedMapObjects serialization

2020-07-07 Thread Erik Erlandson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson updated SPARK-32159: --- Fix Version/s: 3.0.1 > New udaf(Aggregator) has an integration bug with

[jira] [Assigned] (SPARK-32214) The type conversion function generated in makeFromJava for "other" type uses a wrong variable.

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32214: Assignee: Kousuke Saruta (was: Apache Spark) > The type conversion function generated

[jira] [Commented] (SPARK-32214) The type conversion function generated in makeFromJava for "other" type uses a wrong variable.

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153118#comment-17153118 ] Apache Spark commented on SPARK-32214: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-32214) The type conversion function generated in makeFromJava for "other" type uses a wrong variable.

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153117#comment-17153117 ] Apache Spark commented on SPARK-32214: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32214) The type conversion function generated in makeFromJava for "other" type uses a wrong variable.

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32214: Assignee: Apache Spark (was: Kousuke Saruta) > The type conversion function generated

[jira] [Created] (SPARK-32214) The type conversion function generated in makeFromJava for "other" type uses a wrong variable.

2020-07-07 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-32214: -- Summary: The type conversion function generated in makeFromJava for "other" type uses a wrong variable. Key: SPARK-32214 URL:

[jira] [Created] (SPARK-32213) saveAsTable deletes all files in path

2020-07-07 Thread Yuval Rochman (Jira)
Yuval Rochman created SPARK-32213: - Summary: saveAsTable deletes all files in path Key: SPARK-32213 URL: https://issues.apache.org/jira/browse/SPARK-32213 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-31625) Unregister application from YARN resource manager outside the shutdown hook

2020-07-07 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31625. -- Resolution: Not A Problem > Unregister application from YARN resource manager outside the

[jira] [Commented] (SPARK-32212) RDD.takeOrdered can choose to merge intermediate results in executor or driver

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153060#comment-17153060 ] Apache Spark commented on SPARK-32212: -- User 'izchen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32212) RDD.takeOrdered can choose to merge intermediate results in executor or driver

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32212: Assignee: (was: Apache Spark) > RDD.takeOrdered can choose to merge intermediate

[jira] [Commented] (SPARK-32212) RDD.takeOrdered can choose to merge intermediate results in executor or driver

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153059#comment-17153059 ] Apache Spark commented on SPARK-32212: -- User 'izchen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32212) RDD.takeOrdered can choose to merge intermediate results in executor or driver

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32212: Assignee: Apache Spark > RDD.takeOrdered can choose to merge intermediate results in

[jira] [Updated] (SPARK-32212) RDD.takeOrdered can choose to merge intermediate results in executor or driver

2020-07-07 Thread Chen Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Zhang updated SPARK-32212: --- Summary: RDD.takeOrdered can choose to merge intermediate results in executor or driver (was:

[jira] [Commented] (SPARK-32163) Nested pruning should still work for nested column extractors of attributes with cosmetic variations

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152977#comment-17152977 ] Apache Spark commented on SPARK-32163: -- User 'viirya' has created a pull request for this issue:

[jira] [Resolved] (SPARK-32163) Nested pruning should still work for nested column extractors of attributes with cosmetic variations

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32163. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28988

[jira] [Created] (SPARK-32212) RDD.takeOrdered merge intermediate results can be configured in driver or executor

2020-07-07 Thread Chen Zhang (Jira)
Chen Zhang created SPARK-32212: -- Summary: RDD.takeOrdered merge intermediate results can be configured in driver or executor Key: SPARK-32212 URL: https://issues.apache.org/jira/browse/SPARK-32212

[jira] [Commented] (SPARK-32174) toPandas attempted Arrow optimization but has reached an error and can not continue

2020-07-07 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152958#comment-17152958 ] Bryan Cutler commented on SPARK-32174: -- >From the stacktrace, it looks like you are using JDK9 or

[jira] [Updated] (SPARK-32163) Nested pruning should still work for nested column extractors of attributes with cosmetic variations

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32163: -- Description: If the expressions extracting nested fields have cosmetic variations like

[jira] [Commented] (SPARK-31317) Add withField method to Column class

2020-07-07 Thread fqaiser94 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152894#comment-17152894 ] fqaiser94 commented on SPARK-31317: --- Done.  > Add withField method to Column class >

[jira] [Updated] (SPARK-32211) Pin mariadb-plugin-gssapi-server version to fix MariaDBKrbIntegrationSuite

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32211: -- Summary: Pin mariadb-plugin-gssapi-server version to fix MariaDBKrbIntegrationSuite (was:

[jira] [Assigned] (SPARK-32211) MariaDBKrbIntegrationSuite fails because of unwanted server upgrade

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32211: - Assignee: Gabor Somogyi > MariaDBKrbIntegrationSuite fails because of unwanted server

[jira] [Resolved] (SPARK-32211) MariaDBKrbIntegrationSuite fails because of unwanted server upgrade

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32211. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29025

[jira] [Resolved] (SPARK-31317) Add withField method to Column class

2020-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31317. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27066

[jira] [Commented] (SPARK-32018) Fix UnsafeRow set overflowed decimal

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152871#comment-17152871 ] Apache Spark commented on SPARK-32018: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32018) Fix UnsafeRow set overflowed decimal

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32018: Assignee: (was: Apache Spark) > Fix UnsafeRow set overflowed decimal >

[jira] [Assigned] (SPARK-32018) Fix UnsafeRow set overflowed decimal

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32018: Assignee: Apache Spark > Fix UnsafeRow set overflowed decimal >

[jira] [Commented] (SPARK-32018) Fix UnsafeRow set overflowed decimal

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152869#comment-17152869 ] Apache Spark commented on SPARK-32018: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-28067) Incorrect results in decimal aggregation with whole-stage code gen enabled

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152867#comment-17152867 ] Apache Spark commented on SPARK-28067: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Updated] (SPARK-32163) Nested pruning should still work for nested column extractors of attributes with cosmetic variations

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32163: -- Description: Note that this is just an optimization issue and not a regression. The newly

[jira] [Resolved] (SPARK-32164) GeneralizedLinearRegressionSummary optimization

2020-07-07 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-32164. Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28990

[jira] [Assigned] (SPARK-32164) GeneralizedLinearRegressionSummary optimization

2020-07-07 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao reassigned SPARK-32164: -- Assignee: zhengruifeng > GeneralizedLinearRegressionSummary optimization >

[jira] [Updated] (SPARK-30985) Propagate SPARK_CONF_DIR files to driver and exec pods.

2020-07-07 Thread Prashant Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-30985: Description: SPARK_CONF_DIR hosts configuration files like, 1) spark-defaults.conf -

[jira] [Commented] (SPARK-32047) Add provider disable possibility just like in delegation token provider

2020-07-07 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152784#comment-17152784 ] Gabor Somogyi commented on SPARK-32047: --- Started to work on this. > Add provider disable

[jira] [Resolved] (SPARK-32209) Re-use GetTimestamp in ParseToDate

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32209. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28999

[jira] [Assigned] (SPARK-32209) Re-use GetTimestamp in ParseToDate

2020-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32209: - Assignee: Maxim Gekk > Re-use GetTimestamp in ParseToDate >

[jira] [Updated] (SPARK-32211) MariaDBKrbIntegrationSuite fails because of unwanted server upgrade

2020-07-07 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-32211: -- Description: The test fails with the following error: {code:java} 2020-07-07 12:43:13 8

[jira] [Commented] (SPARK-32211) MariaDBKrbIntegrationSuite fails because of unwanted server upgrade

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152763#comment-17152763 ] Apache Spark commented on SPARK-32211: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Assigned] (SPARK-32211) MariaDBKrbIntegrationSuite fails because of unwanted server upgrade

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32211: Assignee: (was: Apache Spark) > MariaDBKrbIntegrationSuite fails because of unwanted

[jira] [Assigned] (SPARK-32211) MariaDBKrbIntegrationSuite fails because of unwanted server upgrade

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32211: Assignee: Apache Spark > MariaDBKrbIntegrationSuite fails because of unwanted server

[jira] [Created] (SPARK-32211) MariaDBKrbIntegrationSuite fails because of unwanted server upgrade

2020-07-07 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-32211: - Summary: MariaDBKrbIntegrationSuite fails because of unwanted server upgrade Key: SPARK-32211 URL: https://issues.apache.org/jira/browse/SPARK-32211 Project: Spark

[jira] [Created] (SPARK-32210) Failed to serialize large MapStatuses

2020-07-07 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-32210: --- Summary: Failed to serialize large MapStatuses Key: SPARK-32210 URL: https://issues.apache.org/jira/browse/SPARK-32210 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-31975) Throw user facing error when use WindowFunction directly

2020-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31975: --- Assignee: ulysses you > Throw user facing error when use WindowFunction directly >

[jira] [Resolved] (SPARK-31975) Throw user facing error when use WindowFunction directly

2020-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31975. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28808

[jira] [Comment Edited] (SPARK-29038) SPIP: Support Spark Materialized View

2020-07-07 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152738#comment-17152738 ] Lantao Jin edited comment on SPARK-29038 at 7/7/20, 1:14 PM: - Hi

[jira] [Commented] (SPARK-29038) SPIP: Support Spark Materialized View

2020-07-07 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152738#comment-17152738 ] Lantao Jin commented on SPARK-29038: Hi [~AidenZhang], my focusings of MV in recent months are two

[jira] [Assigned] (SPARK-32001) Create Kerberos authentication provider API in JDBC connector

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32001: Assignee: Apache Spark > Create Kerberos authentication provider API in JDBC connector >

[jira] [Assigned] (SPARK-32001) Create Kerberos authentication provider API in JDBC connector

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32001: Assignee: (was: Apache Spark) > Create Kerberos authentication provider API in JDBC

[jira] [Commented] (SPARK-32001) Create Kerberos authentication provider API in JDBC connector

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152725#comment-17152725 ] Apache Spark commented on SPARK-32001: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Commented] (SPARK-32205) Writing timestamp in mysql gets fails

2020-07-07 Thread Nilesh Patil (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152683#comment-17152683 ] Nilesh Patil commented on SPARK-32205: -- [~JinxinTang] In mysql there is having DATETIME datatype

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

2020-07-07 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152679#comment-17152679 ] JinxinTang edited comment on SPARK-32205 at 7/7/20, 11:44 AM: --

[jira] [Commented] (SPARK-32205) Writing timestamp in mysql gets fails

2020-07-07 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152679#comment-17152679 ] JinxinTang commented on SPARK-32205: Sure, this problem seems in mysql side, because we cannot

[jira] [Commented] (SPARK-31635) Spark SQL Sort fails when sorting big data points

2020-07-07 Thread George George (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152674#comment-17152674 ] George George commented on SPARK-31635: --- Hello [~Chen Zhang],  Thanks a lot for getting back on

[jira] [Assigned] (SPARK-32209) Re-use GetTimestamp in ParseToDate

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32209: Assignee: Apache Spark > Re-use GetTimestamp in ParseToDate >

[jira] [Assigned] (SPARK-32209) Re-use GetTimestamp in ParseToDate

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32209: Assignee: (was: Apache Spark) > Re-use GetTimestamp in ParseToDate >

[jira] [Commented] (SPARK-32209) Re-use GetTimestamp in ParseToDate

2020-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152641#comment-17152641 ] Apache Spark commented on SPARK-32209: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Created] (SPARK-32209) Re-use GetTimestamp in ParseToDate

2020-07-07 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-32209: -- Summary: Re-use GetTimestamp in ParseToDate Key: SPARK-32209 URL: https://issues.apache.org/jira/browse/SPARK-32209 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-31635) Spark SQL Sort fails when sorting big data points

2020-07-07 Thread Chen Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147653#comment-17147653 ] Chen Zhang edited comment on SPARK-31635 at 7/7/20, 10:14 AM: -- In fact, the

[jira] [Commented] (SPARK-31635) Spark SQL Sort fails when sorting big data points

2020-07-07 Thread Chen Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152618#comment-17152618 ] Chen Zhang commented on SPARK-31635: This problem is not a bug, but I think it is necessary to

[jira] [Comment Edited] (SPARK-31635) Spark SQL Sort fails when sorting big data points

2020-07-07 Thread Chen Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147653#comment-17147653 ] Chen Zhang edited comment on SPARK-31635 at 7/7/20, 10:12 AM: -- In fact, the

  1   2   >