[jira] [Resolved] (SPARK-31326) create Function docs structure for SQL Reference

2020-04-02 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-31326. -- Fix Version/s: 3.0.0 Assignee: Huaxin Gao Resolution: Fixed Resolved

[jira] [Resolved] (SPARK-31328) Incorrect timestamps rebasing on autumn daylight saving time

2020-04-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31328. - Resolution: Fixed Issue resolved by pull request 28101

[jira] [Resolved] (SPARK-31325) Control a plan explain mode in the events of SQL listeners via SQLConf

2020-04-02 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-31325. Assignee: Takeshi Yamamuro Resolution: Fixed This issue is resolved in

[jira] [Commented] (SPARK-31330) Automatically label PRs based on the paths they touch

2020-04-02 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17074217#comment-17074217 ] Nicholas Chammas commented on SPARK-31330: -- Hmm, I didn't see anything from you on the mailing

[jira] [Created] (SPARK-31333) Document Join Hints

2020-04-02 Thread Xiao Li (Jira)
Xiao Li created SPARK-31333: --- Summary: Document Join Hints Key: SPARK-31333 URL: https://issues.apache.org/jira/browse/SPARK-31333 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Commented] (SPARK-31330) Automatically label PRs based on the paths they touch

2020-04-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-31330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17074142#comment-17074142 ] Ismaël Mejía commented on SPARK-31330: -- What about the approach I suggested in the ML? The

[jira] [Commented] (SPARK-31330) Automatically label PRs based on the paths they touch

2020-04-02 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17074124#comment-17074124 ] Nicholas Chammas commented on SPARK-31330: -- Unfortunately, it seems I jumped the gun on sending

[jira] [Updated] (SPARK-31332) Proposal to add Proximity Measure in Random Forest

2020-04-02 Thread Stanley Poon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanley Poon updated SPARK-31332: - Description: h3. Background The RandomForest model does not provide proximity measure as

[jira] [Updated] (SPARK-31332) Proposal to add Proximity Measure in Random Forest

2020-04-02 Thread Stanley Poon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanley Poon updated SPARK-31332: - Description: h3. Background The RandomForest model does not provide proximity measure as

[jira] [Updated] (SPARK-31332) Proposal to add Proximity Measure in Random Forest

2020-04-02 Thread Stanley Poon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanley Poon updated SPARK-31332: - Description: h3. Background The RandomForest model does not provide proximity measure as

[jira] [Created] (SPARK-31332) Proposal to add Proximity Measure in Random Forest

2020-04-02 Thread Stanley Poon (Jira)
Stanley Poon created SPARK-31332: Summary: Proposal to add Proximity Measure in Random Forest Key: SPARK-31332 URL: https://issues.apache.org/jira/browse/SPARK-31332 Project: Spark Issue

[jira] [Created] (SPARK-31331) Document Spark integration with Hive UDFs/UDAFs/UDTFs

2020-04-02 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-31331: -- Summary: Document Spark integration with Hive UDFs/UDAFs/UDTFs Key: SPARK-31331 URL: https://issues.apache.org/jira/browse/SPARK-31331 Project: Spark Issue

[jira] [Created] (SPARK-31330) Automatically label PRs based on the paths they touch

2020-04-02 Thread Nicholas Chammas (Jira)
Nicholas Chammas created SPARK-31330: Summary: Automatically label PRs based on the paths they touch Key: SPARK-31330 URL: https://issues.apache.org/jira/browse/SPARK-31330 Project: Spark

[jira] [Created] (SPARK-31329) Modify executor monitor to allow for moving shuffle blocks

2020-04-02 Thread Holden Karau (Jira)
Holden Karau created SPARK-31329: Summary: Modify executor monitor to allow for moving shuffle blocks Key: SPARK-31329 URL: https://issues.apache.org/jira/browse/SPARK-31329 Project: Spark

[jira] [Commented] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-04-02 Thread Michael Armbrust (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073951#comment-17073951 ] Michael Armbrust commented on SPARK-29358: -- Sure, but it is very easy to make this not a

[jira] [Commented] (SPARK-27913) Spark SQL's native ORC reader implements its own schema evolution

2020-04-02 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073928#comment-17073928 ] L. C. Hsieh commented on SPARK-27913: - As we support schema merging in ORC by SPARK-11412, is this

[jira] [Updated] (SPARK-31328) Incorrect timestamps rebasing on autumn daylight saving time

2020-04-02 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-31328: --- Description: Run the following code in the *America/Los_Angeles* time zone: {code:scala}

[jira] [Updated] (SPARK-31328) Incorrect timestamps rebasing on autumn daylight saving time

2020-04-02 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-31328: --- Description: Run the following code in the *America/Los_Angeles* time zone: {code:scala}

[jira] [Created] (SPARK-31328) Incorrect timestamps rebasing on autumn daylight saving time

2020-04-02 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-31328: -- Summary: Incorrect timestamps rebasing on autumn daylight saving time Key: SPARK-31328 URL: https://issues.apache.org/jira/browse/SPARK-31328 Project: Spark

[jira] [Created] (SPARK-31327) write spark version to avro file metadata

2020-04-02 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-31327: --- Summary: write spark version to avro file metadata Key: SPARK-31327 URL: https://issues.apache.org/jira/browse/SPARK-31327 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-29153) ResourceProfile conflict resolution stage level scheduling

2020-04-02 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-29153. --- Fix Version/s: 3.1.0 Assignee: Thomas Graves Resolution: Fixed >

[jira] [Resolved] (SPARK-31179) Fast fail the connection while last shuffle connection failed in the last retry IO wait

2020-04-02 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-31179. --- Fix Version/s: 3.1.0 Assignee: feiwang Resolution: Fixed > Fast fail the

[jira] [Resolved] (SPARK-31315) SQLQueryTestSuite: Display the total compile time for generated java code.

2020-04-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31315. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28081

[jira] [Assigned] (SPARK-31315) SQLQueryTestSuite: Display the total compile time for generated java code.

2020-04-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31315: --- Assignee: jiaan.geng > SQLQueryTestSuite: Display the total compile time for generated

[jira] [Resolved] (SPARK-30839) Add version information for Spark configuration

2020-04-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30839. -- Resolution: Done Thanks, [~beliefer] for working on this. > Add version information for

[jira] [Assigned] (SPARK-31321) Remove SaveMode check in v2 FileWriteBuilder

2020-04-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31321: --- Assignee: Kent Yao > Remove SaveMode check in v2 FileWriteBuilder >

[jira] [Resolved] (SPARK-31321) Remove SaveMode check in v2 FileWriteBuilder

2020-04-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31321. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28090

[jira] [Commented] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

2020-04-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073463#comment-17073463 ] Wenchen Fan commented on SPARK-30951: - Theoretically, Parquet spec implicitly requires Gregorian

[jira] [Created] (SPARK-31326) create Function docs structure for SQL Reference

2020-04-02 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-31326: -- Summary: create Function docs structure for SQL Reference Key: SPARK-31326 URL: https://issues.apache.org/jira/browse/SPARK-31326 Project: Spark Issue Type:

[jira] [Commented] (SPARK-31299) Pyspark.ml.clustering illegalArgumentException with dataframe created from rows

2020-04-02 Thread Lukas Thaler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073422#comment-17073422 ] Lukas Thaler commented on SPARK-31299: -- Oh dear. Now, that's embarrassing. Thank you for pointing

[jira] [Commented] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-04-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073409#comment-17073409 ] Dongjoon Hyun commented on SPARK-31312: --- Since I saw your opinion, I'll not ping you about that

[jira] [Commented] (SPARK-31312) Transforming Hive simple UDF (using JAR) expression may incur CNFE in later evaluation

2020-04-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073407#comment-17073407 ] Dongjoon Hyun commented on SPARK-31312: --- It's for informimg the users (and the downstream