[jira] [Created] (SPARK-28550) Unset SPARK_HOME environment variable in K8S integration preparation

Hyukjin Kwon created SPARK-28550: Summary: Unset SPARK_HOME environment variable in K8S integration preparation Key: SPARK-28550 URL: https://issues.apache.org/jira/browse/SPARK-28550 Project: Spark

[jira] [Commented] (SPARK-28522) Pass dynamic parameters to custom file input format

[ https://issues.apache.org/jira/browse/SPARK-28522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894942#comment-16894942 ] Ayan Mukherjee commented on SPARK-28522: Thanks for the response but I am trying

[jira] [Updated] (SPARK-23160) Port window.sql

[ https://issues.apache.org/jira/browse/SPARK-23160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23160: -- Summary: Port window.sql (was: Add window.sql) > Port window.sql > --- > >

[jira] [Commented] (SPARK-28086) Adds `random()` sql function

[ https://issues.apache.org/jira/browse/SPARK-28086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894922#comment-16894922 ] Dongjoon Hyun commented on SPARK-28086: --- This issue is reported twice at [~DylanGu

[jira] [Commented] (SPARK-28522) Pass dynamic parameters to custom file input format

[ https://issues.apache.org/jira/browse/SPARK-28522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894909#comment-16894909 ] Hyukjin Kwon commented on SPARK-28522: -- {{sc.hadoopConfiguration.set("my.mapreduce.

[jira] [Resolved] (SPARK-28471) Formatting dates with negative years

[ https://issues.apache.org/jira/browse/SPARK-28471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28471. --- Resolution: Fixed Fix Version/s: 3.0.0 This is resolved via https://github.com/apache

[jira] [Commented] (SPARK-28546) Why does the File Sink operation of Spark 2.4 Structured Streaming include double-level version validation?

[ https://issues.apache.org/jira/browse/SPARK-28546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894900#comment-16894900 ] Hyukjin Kwon commented on SPARK-28546: -- [~yy3b2007com], questions should better go

[jira] [Resolved] (SPARK-28546) Why does the File Sink operation of Spark 2.4 Structured Streaming include double-level version validation?

[ https://issues.apache.org/jira/browse/SPARK-28546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28546. -- Resolution: Invalid > Why does the File Sink operation of Spark 2.4 Structured Streaming inclu

[jira] [Resolved] (SPARK-28549) Use `text.StringEscapeUtils` instead `lang3.StringEscapeUtils`

[ https://issues.apache.org/jira/browse/SPARK-28549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28549. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25281 [https://gi

[jira] [Assigned] (SPARK-28549) Use `text.StringEscapeUtils` instead `lang3.StringEscapeUtils`

[ https://issues.apache.org/jira/browse/SPARK-28549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28549: Assignee: Dongjoon Hyun > Use `text.StringEscapeUtils` instead `lang3.StringEscapeUtils`

[jira] [Commented] (SPARK-28519) Tests failed on aarch64 due the value of math.log and power function is different

[ https://issues.apache.org/jira/browse/SPARK-28519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894876#comment-16894876 ] huangtianhua commented on SPARK-28519: -- Sorry, I didn't see you have proposed pr, t

[jira] [Commented] (SPARK-28519) Tests failed on aarch64 due the value of math.log and power function is different

[ https://issues.apache.org/jira/browse/SPARK-28519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894854#comment-16894854 ] huangtianhua commented on SPARK-28519: -- Thank you all. I will test with modificatio

[jira] [Comment Edited] (SPARK-28519) Tests failed on aarch64 due the value of math.log and power function is different

[ https://issues.apache.org/jira/browse/SPARK-28519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894854#comment-16894854 ] huangtianhua edited comment on SPARK-28519 at 7/29/19 1:40 AM: ---

[jira] [Resolved] (SPARK-28547) Make it work for wide (> 10K columns data)

[ https://issues.apache.org/jira/browse/SPARK-28547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28547. -- Resolution: Invalid > Make it work for wide (> 10K columns data) > ---

[jira] [Commented] (SPARK-28547) Make it work for wide (> 10K columns data)

[ https://issues.apache.org/jira/browse/SPARK-28547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894839#comment-16894839 ] Takeshi Yamamuro commented on SPARK-28547: -- You need to ask in the dev mailingl

[jira] [Resolved] (SPARK-28520) WholeStageCodegen does not work property for LocalTableScanExec

[ https://issues.apache.org/jira/browse/SPARK-28520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28520. -- Resolution: Fixed Fix Version/s: 3.0.0 Target Version/s: (was: 3.0.0

[jira] [Assigned] (SPARK-25474) Support `spark.sql.statistics.fallBackToHdfs` in data source tables

[ https://issues.apache.org/jira/browse/SPARK-25474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25474: - Assignee: shahid > Support `spark.sql.statistics.fallBackToHdfs` in data source tables

[jira] [Resolved] (SPARK-25474) Support `spark.sql.statistics.fallBackToHdfs` in data source tables

[ https://issues.apache.org/jira/browse/SPARK-25474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25474. --- Resolution: Fixed Fix Version/s: 3.0.0 This is resolved via https://github.com/apache

[jira] [Updated] (SPARK-25474) Support `spark.sql.statistics.fallBackToHdfs` in data source tables

[ https://issues.apache.org/jira/browse/SPARK-25474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25474: -- Summary: Support `spark.sql.statistics.fallBackToHdfs` in data source tables (was: Size in by

[jira] [Updated] (SPARK-28306) Once optimizer rule NormalizeFloatingNumbers is not idempotent

[ https://issues.apache.org/jira/browse/SPARK-28306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-28306: Issue Type: Sub-task (was: Improvement) Parent: SPARK-28528 > Once optimizer rule NormalizeFloati

[jira] [Updated] (SPARK-28237) Idempotence checker for Idempotent batches in RuleExecutors

[ https://issues.apache.org/jira/browse/SPARK-28237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-28237: Issue Type: Sub-task (was: Improvement) Parent: SPARK-28528 > Idempotence checker for Idempotent

[jira] [Commented] (SPARK-28377) Fully support correlation names in the FROM clause

[ https://issues.apache.org/jira/browse/SPARK-28377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894788#comment-16894788 ] Dongjoon Hyun commented on SPARK-28377: --- You can increase the priority if you want

[jira] [Updated] (SPARK-28549) Use `text.StringEscapeUtils` instead `lang3.StringEscapeUtils`

[ https://issues.apache.org/jira/browse/SPARK-28549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28549: -- Description: `org.apache.commons.lang3.StringEscapeUtils` is deprecated over two years ago at

[jira] [Created] (SPARK-28549) Use `text.StringEscapeUtils` instead `lang3.StringEscapeUtils`

Dongjoon Hyun created SPARK-28549: - Summary: Use `text.StringEscapeUtils` instead `lang3.StringEscapeUtils` Key: SPARK-28549 URL: https://issues.apache.org/jira/browse/SPARK-28549 Project: Spark

[jira] [Updated] (SPARK-28549) Use `text.StringEscapeUtils` instead `lang3.StringEscapeUtils`

[ https://issues.apache.org/jira/browse/SPARK-28549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28549: -- Component/s: Build > Use `text.StringEscapeUtils` instead `lang3.StringEscapeUtils` >

[jira] [Created] (SPARK-28548) explain() shows wrong result for persisted DataFrames after some operations

Kousuke Saruta created SPARK-28548: -- Summary: explain() shows wrong result for persisted DataFrames after some operations Key: SPARK-28548 URL: https://issues.apache.org/jira/browse/SPARK-28548 Proje

[jira] [Comment Edited] (SPARK-28036) Built-in udf left/right has inconsistent behavior

[ https://issues.apache.org/jira/browse/SPARK-28036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16877765#comment-16877765 ] ShuMing Li edited comment on SPARK-28036 at 7/28/19 2:28 PM: -

[jira] [Assigned] (SPARK-21481) Add indexOf method in ml.feature.HashingTF similar to mllib.feature.HashingTF

[ https://issues.apache.org/jira/browse/SPARK-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21481: - Assignee: Huaxin Gao > Add indexOf method in ml.feature.HashingTF similar to mllib.feature.Hash

[jira] [Resolved] (SPARK-21481) Add indexOf method in ml.feature.HashingTF similar to mllib.feature.HashingTF

[ https://issues.apache.org/jira/browse/SPARK-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21481. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25250 [https://github.c

[jira] [Updated] (SPARK-28547) Make it work for wide (> 10K columns data)

[ https://issues.apache.org/jira/browse/SPARK-28547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] antonkulaga updated SPARK-28547: Description: Spark is super-slow for all wide data (when there are >15kb columns and >15kb rows).

[jira] [Updated] (SPARK-28547) Make it work for wide (> 10K columns data)

[ https://issues.apache.org/jira/browse/SPARK-28547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] antonkulaga updated SPARK-28547: Description: Spark is super-slow for all wide data (when there are >15kb columns and >15kb rows).

[jira] [Created] (SPARK-28547) Make it work for wide (> 10K columns data)

antonkulaga created SPARK-28547: --- Summary: Make it work for wide (> 10K columns data) Key: SPARK-28547 URL: https://issues.apache.org/jira/browse/SPARK-28547 Project: Spark Issue Type: Improvem