[jira] [Commented] (SPARK-28605) Performance regression in SS's foreach

[ https://issues.apache.org/jira/browse/SPARK-28605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899790#comment-16899790 ] Dongjoon Hyun commented on SPARK-28605: --- Oh, thank you, [~zsxwing]! > Performance regression in

[jira] [Commented] (SPARK-28605) Performance regression in SS's foreach

[ https://issues.apache.org/jira/browse/SPARK-28605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899789#comment-16899789 ] Shixiong Zhu commented on SPARK-28605: -- By the way, this is not a critical regression. It's not

[jira] [Commented] (SPARK-28422) GROUPED_AGG pandas_udf doesn't with spark.sql() without group by clause

[ https://issues.apache.org/jira/browse/SPARK-28422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899788#comment-16899788 ] Liang-Chi Hsieh commented on SPARK-28422: - Thanks [~dongjoon]! > GROUPED_AGG pandas_udf

[jira] [Commented] (SPARK-28605) Performance regression in SS's foreach

[ https://issues.apache.org/jira/browse/SPARK-28605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899787#comment-16899787 ] Shixiong Zhu commented on SPARK-28605: -- This is a regression at all 2.4 branches. It's caused by 

[jira] [Updated] (SPARK-28605) Performance regression in SS's foreach

[ https://issues.apache.org/jira/browse/SPARK-28605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28605: - Affects Version/s: 2.4.0 2.4.1 2.4.2 >

[jira] [Commented] (SPARK-28605) Performance regression in SS's foreach

[ https://issues.apache.org/jira/browse/SPARK-28605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899784#comment-16899784 ] Dongjoon Hyun commented on SPARK-28605: --- Hi, [~zsxwing]. Is this a regression at 2.4.3? >

[jira] [Commented] (SPARK-28422) GROUPED_AGG pandas_udf doesn't with spark.sql() without group by clause

[ https://issues.apache.org/jira/browse/SPARK-28422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899781#comment-16899781 ] Dongjoon Hyun commented on SPARK-28422: --- Thank you for reporting, [~icexelloss]. And, thank you

[jira] [Updated] (SPARK-28422) GROUPED_AGG pandas_udf doesn't with spark.sql() without group by clause

[ https://issues.apache.org/jira/browse/SPARK-28422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28422: -- Affects Version/s: 2.4.0 > GROUPED_AGG pandas_udf doesn't with spark.sql() without group by

[jira] [Updated] (SPARK-28422) GROUPED_AGG pandas_udf doesn't with spark.sql() without group by clause

[ https://issues.apache.org/jira/browse/SPARK-28422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28422: -- Description:   {code:python} from pyspark.sql.functions import pandas_udf, PandasUDFType

[jira] [Updated] (SPARK-28422) GROUPED_AGG pandas_udf doesn't with spark.sql() without group by clause

[ https://issues.apache.org/jira/browse/SPARK-28422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28422: -- Affects Version/s: 2.4.1 > GROUPED_AGG pandas_udf doesn't with spark.sql() without group by

[jira] [Updated] (SPARK-28422) GROUPED_AGG pandas_udf doesn't with spark.sql() without group by clause

[ https://issues.apache.org/jira/browse/SPARK-28422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28422: -- Affects Version/s: 2.4.2 > GROUPED_AGG pandas_udf doesn't with spark.sql() without group by

[jira] [Updated] (SPARK-28422) GROUPED_AGG pandas_udf doesn't with spark.sql() without group by clause

[ https://issues.apache.org/jira/browse/SPARK-28422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28422: -- Description:   {code:java} @pandas_udf('double', PandasUDFType.GROUPED_AGG) def max_udf(v):

[jira] [Comment Edited] (SPARK-28580) ANSI SQL: unique predicate

[ https://issues.apache.org/jira/browse/SPARK-28580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899773#comment-16899773 ] Dongjoon Hyun edited comment on SPARK-28580 at 8/5/19 5:08 AM: --- Hi,

[jira] [Commented] (SPARK-28580) ANSI SQL: unique predicate

[ https://issues.apache.org/jira/browse/SPARK-28580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899773#comment-16899773 ] Dongjoon Hyun commented on SPARK-28580: --- Please make a PR with a working implementation. > ANSI

[jira] [Resolved] (SPARK-27661) Add SupportsNamespaces interface for v2 catalogs

[ https://issues.apache.org/jira/browse/SPARK-27661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-27661. - Resolution: Fixed Fix Version/s: 3.0.0 Resolved by 

[jira] [Assigned] (SPARK-27661) Add SupportsNamespaces interface for v2 catalogs

[ https://issues.apache.org/jira/browse/SPARK-27661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz reassigned SPARK-27661: --- Assignee: Ryan Blue > Add SupportsNamespaces interface for v2 catalogs >

[jira] [Assigned] (SPARK-28616) Improve merge-spark-pr script to warn WIP PRs and strip trailing dots

[ https://issues.apache.org/jira/browse/SPARK-28616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28616: - Assignee: Dongjoon Hyun > Improve merge-spark-pr script to warn WIP PRs and strip

[jira] [Resolved] (SPARK-28616) Improve merge-spark-pr script to warn WIP PRs and strip trailing dots

[ https://issues.apache.org/jira/browse/SPARK-28616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28616. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25356

[jira] [Created] (SPARK-28617) Completely remove comments from the gold result file

Yuming Wang created SPARK-28617: --- Summary: Completely remove comments from the gold result file Key: SPARK-28617 URL: https://issues.apache.org/jira/browse/SPARK-28617 Project: Spark Issue

[jira] [Created] (SPARK-28616) Improve merge-spark-pr script to warn WIP PRs and strip trailing dots

Dongjoon Hyun created SPARK-28616: - Summary: Improve merge-spark-pr script to warn WIP PRs and strip trailing dots Key: SPARK-28616 URL: https://issues.apache.org/jira/browse/SPARK-28616 Project:

[jira] [Created] (SPARK-28615) Add a guide line for dataframe functions to say column signature function is by default

Weichen Xu created SPARK-28615: -- Summary: Add a guide line for dataframe functions to say column signature function is by default Key: SPARK-28615 URL: https://issues.apache.org/jira/browse/SPARK-28615

[jira] [Resolved] (SPARK-28614) Do not remove leading write space in the golden result file

[ https://issues.apache.org/jira/browse/SPARK-28614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28614. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25351

[jira] [Assigned] (SPARK-28614) Do not remove leading write space in the golden result file

[ https://issues.apache.org/jira/browse/SPARK-28614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28614: - Assignee: Yuming Wang > Do not remove leading write space in the golden result file >

[jira] [Resolved] (SPARK-28604) Use log1p(x) instead of log(1+x) and expm1(x) instead of exp(x)-1

[ https://issues.apache.org/jira/browse/SPARK-28604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28604. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25337

[jira] [Created] (SPARK-28614) Do not remove leading write space in the golden result file

Yuming Wang created SPARK-28614: --- Summary: Do not remove leading write space in the golden result file Key: SPARK-28614 URL: https://issues.apache.org/jira/browse/SPARK-28614 Project: Spark

[jira] [Comment Edited] (SPARK-28527) Build a Test Framework for Thriftserver

[ https://issues.apache.org/jira/browse/SPARK-28527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899368#comment-16899368 ] Yuming Wang edited comment on SPARK-28527 at 8/4/19 5:23 PM: - When building

[jira] [Updated] (SPARK-28527) Build a Test Framework for Thriftserver

[ https://issues.apache.org/jira/browse/SPARK-28527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28527: Attachment: (was: image-2019-08-03-13-58-06-106.png) > Build a Test Framework for

[jira] [Commented] (SPARK-28471) Formatting dates with negative years

[ https://issues.apache.org/jira/browse/SPARK-28471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899657#comment-16899657 ] Yuming Wang commented on SPARK-28471: - Thank you [~maxgekk] I see. > Formatting dates with negative

[jira] [Resolved] (SPARK-28609) Fix broken styles/links and make up-to-date

[ https://issues.apache.org/jira/browse/SPARK-28609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28609. --- Resolution: Fixed Fix Version/s: 2.4.4 3.0.0 Issue resolved by

[jira] [Assigned] (SPARK-28609) Fix broken styles/links and make up-to-date

[ https://issues.apache.org/jira/browse/SPARK-28609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28609: - Assignee: Dongjoon Hyun > Fix broken styles/links and make up-to-date >

[jira] [Comment Edited] (SPARK-28471) Formatting dates with negative years

[ https://issues.apache.org/jira/browse/SPARK-28471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899633#comment-16899633 ] Maxim Gekk edited comment on SPARK-28471 at 8/4/19 1:53 PM: [~yumwang] There

[jira] [Commented] (SPARK-28471) Formatting dates with negative years

[ https://issues.apache.org/jira/browse/SPARK-28471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899633#comment-16899633 ] Maxim Gekk commented on SPARK-28471: [~yumwang] There are 2 equal methods to output negative years

[jira] [Commented] (SPARK-28471) Formatting dates with negative years

[ https://issues.apache.org/jira/browse/SPARK-28471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899630#comment-16899630 ] Yuming Wang commented on SPARK-28471: - Hi [~maxgekk], It seems the issue still exist: {code:scala}

[jira] [Updated] (SPARK-28613) Spark SQL action collect just judge size of compressed RDD's size, not accurate enough

[ https://issues.apache.org/jira/browse/SPARK-28613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-28613: -- Description: When we run action DataFrame.collect() , for the configuration 

[jira] [Updated] (SPARK-28613) Spark SQL action collect just judge size of compressed RDD's size, not accurate enough

[ https://issues.apache.org/jira/browse/SPARK-28613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-28613: -- Description: When we run action DataFrame.collect() , for the configuration 

[jira] [Created] (SPARK-28613) Spark SQL action collect just judge size of compressed RDD's size, not accurate enough

angerszhu created SPARK-28613: - Summary: Spark SQL action collect just judge size of compressed RDD's size, not accurate enough Key: SPARK-28613 URL: https://issues.apache.org/jira/browse/SPARK-28613

[jira] [Commented] (SPARK-28611) Histogram's height is diffrent

[ https://issues.apache.org/jira/browse/SPARK-28611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899585#comment-16899585 ] Marco Gaido commented on SPARK-28611: - Mmmhthat's weird! How can you get a different result than

[jira] [Commented] (SPARK-24091) Internally used ConfigMap prevents use of user-specified ConfigMaps carrying Spark configs files

[ https://issues.apache.org/jira/browse/SPARK-24091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899578#comment-16899578 ] Michael Gendelman commented on SPARK-24091: --- [~tmckay] We are experiencing the same issue and

[jira] [Updated] (SPARK-28411) insertInto with overwrite inconsistent behaviour Python/Scala

[ https://issues.apache.org/jira/browse/SPARK-28411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-28411: Labels: release-notes (was: ) > insertInto with overwrite inconsistent behaviour Python/Scala >