[jira] [Created] (SPARK-29073) Add GitHub Action to branch-2.4 for `Scala-2.11 / Scala-2.12` build

2019-09-12 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-29073: - Summary: Add GitHub Action to branch-2.4 for `Scala-2.11 / Scala-2.12` build Key: SPARK-29073 URL: https://issues.apache.org/jira/browse/SPARK-29073 Project: Spark

[jira] [Commented] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928995#comment-16928995 ] Dongjoon Hyun commented on SPARK-29066: --- Oops. One last piece~ -

[jira] [Comment Edited] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928995#comment-16928995 ] Dongjoon Hyun edited comment on SPARK-29066 at 9/13/19 4:52 AM: Oops.

[jira] [Commented] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928994#comment-16928994 ] Dongjoon Hyun commented on SPARK-29066: --- Thanks! :) > Remove old Jenkins jobs for EOL versions or

[jira] [Commented] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread shane knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928991#comment-16928991 ] shane knapp commented on SPARK-29066: - done > Remove old Jenkins jobs for EOL versions or obsolete

[jira] [Updated] (SPARK-29072) Properly track shuffle write time with refactor

2019-09-12 Thread Matt Cheah (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-29072: --- Description: From SPARK-28209, SPARK-28570, and SPARK-28571, we used the new shuffle writer plugin

[jira] [Updated] (SPARK-29072) Properly track shuffle write time with refactor

2019-09-12 Thread Matt Cheah (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-29072: --- Description: >From SPARK-28209, SPARK-28570, and SPARK-28571, we used the new shuffle writer

[jira] [Updated] (SPARK-29072) Properly track shuffle write time with refactor

2019-09-12 Thread Matt Cheah (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-29072: --- Description: >From SPARK-28209, SPARK-28570, and SPARK-28571, we used the new shuffle writer

[jira] [Created] (SPARK-29072) Properly track shuffle write time with refactor

2019-09-12 Thread Matt Cheah (Jira)
Matt Cheah created SPARK-29072: -- Summary: Properly track shuffle write time with refactor Key: SPARK-29072 URL: https://issues.apache.org/jira/browse/SPARK-29072 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-29048) Query optimizer slow when using Column.isInCollection() with a large size collection

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-29048. - Fix Version/s: 3.0.0 Resolution: Fixed > Query optimizer slow when using Column.isInCollection()

[jira] [Resolved] (SPARK-27149) Support LOCALTIMESTAMP when ANSI mode enabled

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27149. - Resolution: Won't Fix > Support LOCALTIMESTAMP when ANSI mode enabled >

[jira] [Resolved] (SPARK-28453) Support recursive view syntax

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28453. - Resolution: Won't Fix > Support recursive view syntax > - > >

[jira] [Commented] (SPARK-28453) Support recursive view syntax

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928947#comment-16928947 ] Xiao Li commented on SPARK-28453: - This is a very low priority task. Let us do not spend time on it.  >

[jira] [Closed] (SPARK-28298) Fully support char and varchar types

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-28298. --- > Fully support char and varchar types > > > Key:

[jira] [Closed] (SPARK-28448) Implement ILIKE operator

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-28448. --- > Implement ILIKE operator > > > Key: SPARK-28448 >

[jira] [Resolved] (SPARK-28298) Fully support char and varchar types

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28298. - Resolution: Won't Fix > Fully support char and varchar types > > >

[jira] [Resolved] (SPARK-28448) Implement ILIKE operator

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28448. - Resolution: Won't Fix > Implement ILIKE operator > > > Key:

[jira] [Closed] (SPARK-27877) ANSI SQL: LATERAL derived table(T491)

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-27877. --- > ANSI SQL: LATERAL derived table(T491) > - > > Key:

[jira] [Resolved] (SPARK-27877) ANSI SQL: LATERAL derived table(T491)

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27877. - Resolution: Won't Fix > ANSI SQL: LATERAL derived table(T491) > - >

[jira] [Closed] (SPARK-27980) Ordered-Set Aggregate Functions

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-27980. --- > Ordered-Set Aggregate Functions > --- > > Key: SPARK-27980 >

[jira] [Resolved] (SPARK-28027) Missing some mathematical operators

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28027. - Resolution: Won't Fix > Missing some mathematical operators > --- > >

[jira] [Resolved] (SPARK-27980) Ordered-Set Aggregate Functions

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27980. - Resolution: Won't Fix > Ordered-Set Aggregate Functions > --- > >

[jira] [Commented] (SPARK-28027) Missing some mathematical operators

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928940#comment-16928940 ] Xiao Li commented on SPARK-28027: - Here, we will only support ^.  > Missing some mathematical operators

[jira] [Commented] (SPARK-28027) Missing some mathematical operators

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928942#comment-16928942 ] Xiao Li commented on SPARK-28027: - OK, since ^ has different semantics. Let us close it.  > Missing

[jira] [Commented] (SPARK-23945) Column.isin() should accept a single-column DataFrame as input

2019-09-12 Thread Jean-Marc Spaggiari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928938#comment-16928938 ] Jean-Marc Spaggiari commented on SPARK-23945: - It is what I ended up doing (with the

[jira] [Commented] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928920#comment-16928920 ] Dongjoon Hyun commented on SPARK-29066: --- Thank you so much. It's nice! Could you remove the

[jira] [Updated] (SPARK-27468) "Storage Level" in "RDD Storage Page" is not correct

2019-09-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27468: -- Affects Version/s: 3.0.0 2.4.2 2.4.3

[jira] [Commented] (SPARK-23945) Column.isin() should accept a single-column DataFrame as input

2019-09-12 Thread Herman van Hovell (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928869#comment-16928869 ] Herman van Hovell commented on SPARK-23945: --- You can use a left semi join, e.g.: {code} SELECT

[jira] [Resolved] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread shane knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp resolved SPARK-29066. - Resolution: Fixed > Remove old Jenkins jobs for EOL versions or obsolete combinations >

[jira] [Commented] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread shane knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928824#comment-16928824 ] shane knapp commented on SPARK-29066: - done! that was quite satisfying. :) > Remove old Jenkins

[jira] [Commented] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread shane knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928816#comment-16928816 ] shane knapp commented on SPARK-29066: - i am currently deleting old jobs w/absolute relish. > Remove

[jira] [Resolved] (SPARK-29071) Upgrade Scala to 2.12.10

2019-09-12 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-29071. - Resolution: Duplicate > Upgrade Scala to 2.12.10 > > > Key:

[jira] [Commented] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928806#comment-16928806 ] Dongjoon Hyun commented on SPARK-29066: --- Oops. Sorry for the duplication. Thank you,

[jira] [Commented] (SPARK-23945) Column.isin() should accept a single-column DataFrame as input

2019-09-12 Thread Jean-Marc Spaggiari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928799#comment-16928799 ] Jean-Marc Spaggiari commented on SPARK-23945: - What's about a query like: {code:java} SELECT

[jira] [Created] (SPARK-29071) Upgrade Scala to 2.12.10

2019-09-12 Thread Xiao Li (Jira)
Xiao Li created SPARK-29071: --- Summary: Upgrade Scala to 2.12.10 Key: SPARK-29071 URL: https://issues.apache.org/jira/browse/SPARK-29071 Project: Spark Issue Type: Improvement Components:

[jira] [Updated] (SPARK-29070) Allow SparkLauncher to return full spark-submit command line

2019-09-12 Thread Jeff Evans (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Evans updated SPARK-29070: --- Priority: Minor (was: Major) > Allow SparkLauncher to return full spark-submit command line >

[jira] [Commented] (SPARK-22796) Add multiple column support to PySpark QuantileDiscretizer

2019-09-12 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928704#comment-16928704 ] Huaxin Gao commented on SPARK-22796: [~podongfeng] Could you please reopen your PR

[jira] [Comment Edited] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928596#comment-16928596 ] feiwang edited comment on SPARK-29037 at 9/12/19 4:17 PM: -- [~advancedxy] 1.

[jira] [Issue Comment Deleted] (SPARK-29068) CSV read reports incorrect row count

2019-09-12 Thread HondaWei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HondaWei updated SPARK-29068: - Comment: was deleted (was: [~tdiesler], May you provide your test CSV data?  thanks!) > CSV read

[jira] [Commented] (SPARK-29067) divide function does not throw an error, if a number is not passed to it

2019-09-12 Thread HondaWei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928688#comment-16928688 ] HondaWei commented on SPARK-29067: -- I'll try to figure out this issue. > divide function does not

[jira] [Created] (SPARK-29070) Allow SparkLauncher to return full spark-submit command line

2019-09-12 Thread Jeff Evans (Jira)
Jeff Evans created SPARK-29070: -- Summary: Allow SparkLauncher to return full spark-submit command line Key: SPARK-29070 URL: https://issues.apache.org/jira/browse/SPARK-29070 Project: Spark

[jira] [Commented] (SPARK-29068) CSV read reports incorrect row count

2019-09-12 Thread HondaWei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928686#comment-16928686 ] HondaWei commented on SPARK-29068: -- [~tdiesler], May you provide your test CSV data?  thanks! > CSV

[jira] [Updated] (SPARK-29015) Can't use 'add jar' jar's class as create table serde class on JDK 11

2019-09-12 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29015: -- Summary: Can't use 'add jar' jar's class as create table serde class on JDK 11 (was: Can not support

[jira] [Comment Edited] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928674#comment-16928674 ] feiwang edited comment on SPARK-29037 at 9/12/19 3:58 PM: -- [~advancedxy] Thanks

[jira] [Comment Edited] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928674#comment-16928674 ] feiwang edited comment on SPARK-29037 at 9/12/19 3:49 PM: -- [~advancedxy] Thanks

[jira] [Comment Edited] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928674#comment-16928674 ] feiwang edited comment on SPARK-29037 at 9/12/19 3:48 PM: -- [~advancedxy] I just

[jira] [Comment Edited] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928674#comment-16928674 ] feiwang edited comment on SPARK-29037 at 9/12/19 3:46 PM: -- [~advancedxy] I just

[jira] [Commented] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928674#comment-16928674 ] feiwang commented on SPARK-29037: - [~advancedxy] I just checked the code, as shown below.

[jira] [Issue Comment Deleted] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29037: Comment: was deleted (was: In detail, I think we need change the logic of

[jira] [Commented] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928656#comment-16928656 ] feiwang commented on SPARK-29037: - In detail, I think we need change the logic of

[jira] [Commented] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread Xianjin YE (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928650#comment-16928650 ] Xianjin YE commented on SPARK-29037: > About output check, I think it is not appropriate, because

[jira] [Commented] (SPARK-26205) Optimize InSet expression for bytes, shorts, ints, dates

2019-09-12 Thread Liang-Chi Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928649#comment-16928649 ] Liang-Chi Hsieh commented on SPARK-26205: - Yeah, I will look at it. > Optimize InSet expression

[jira] [Commented] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread shane knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928620#comment-16928620 ] shane knapp commented on SPARK-29066: - yep, i can take care of this pretty easily. > Remove old

[jira] [Assigned] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread shane knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp reassigned SPARK-29066: --- Assignee: shane knapp > Remove old Jenkins jobs for EOL versions or obsolete combinations

[jira] [Updated] (SPARK-28948) Support passing all Table metadata in TableProvider

2019-09-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-28948: Summary: Support passing all Table metadata in TableProvider (was: support data source v2 in

[jira] [Commented] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928598#comment-16928598 ] feiwang commented on SPARK-29037: - The implementation of InsertIntoHiveTable prevent reuse same

[jira] [Comment Edited] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928596#comment-16928596 ] feiwang edited comment on SPARK-29037 at 9/12/19 2:39 PM: -- [~advancedxy] 1.

[jira] [Commented] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928596#comment-16928596 ] feiwang commented on SPARK-29037: - [~advancedxy] 1. We re-submit the same application again. We meet

[jira] [Updated] (SPARK-29069) ResolveInsertInto should not do table look up

2019-09-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-29069: Summary: ResolveInsertInto should not do table look up (was: merge UnresolvedRelation resolution

[jira] [Commented] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread Xianjin YE (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928561#comment-16928561 ] Xianjin YE commented on SPARK-29037: [~hzfeiwang] by rerun the application, do you mean re-submit

[jira] [Created] (SPARK-29069) merge UnresolvedRelation resolution logic for v1 and v2 tables

2019-09-12 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-29069: --- Summary: merge UnresolvedRelation resolution logic for v1 and v2 tables Key: SPARK-29069 URL: https://issues.apache.org/jira/browse/SPARK-29069 Project: Spark

[jira] [Resolved] (SPARK-29060) Add tree traversal helper for adaptive spark plans

2019-09-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29060. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25764

[jira] [Assigned] (SPARK-29060) Add tree traversal helper for adaptive spark plans

2019-09-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29060: --- Assignee: Maryann Xue > Add tree traversal helper for adaptive spark plans >

[jira] [Commented] (SPARK-22796) Add multiple column support to PySpark QuantileDiscretizer

2019-09-12 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928541#comment-16928541 ] Sean Owen commented on SPARK-22796: --- Yes, but you can see that it was reverted. There are pointers to

[jira] [Resolved] (SPARK-26731) remove EOLed spark jobs from jenkins

2019-09-12 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26731. --- Target Version/s: (was: 3.0.0) Resolution: Duplicate > remove EOLed spark jobs from

[jira] [Commented] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928539#comment-16928539 ] Sean Owen commented on SPARK-29066: --- Sounds good. This is the same as

[jira] [Commented] (SPARK-26205) Optimize InSet expression for bytes, shorts, ints, dates

2019-09-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928504#comment-16928504 ] Wenchen Fan commented on SPARK-26205: - [~viirya] can you take a look at the issues I mentioned

[jira] [Commented] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928503#comment-16928503 ] Wenchen Fan commented on SPARK-29037: - [~advancedxy] can you take a look? > [Core] Spark gives

[jira] [Resolved] (SPARK-29065) Extend `EXTRACT` benchmark

2019-09-12 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-29065. -- Fix Version/s: 3.0.0 Assignee: Maxim Gekk Resolution: Fixed Resolved

[jira] [Resolved] (SPARK-28998) reorganize the packages of DS v2 interfaces/classes

2019-09-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28998. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25700

[jira] [Commented] (SPARK-29027) KafkaDelegationTokenSuite fails

2019-09-12 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928466#comment-16928466 ] Gabor Somogyi commented on SPARK-29027: --- [~koert] I see your concern and sending a mail... >

[jira] [Resolved] (SPARK-28954) For SparkCLI, start up with conf of HIVEAUXJARS, we add jar with SessionStateResourceLoader's addJar() API

2019-09-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-28954. - Resolution: Duplicate > For SparkCLI, start up with conf of HIVEAUXJARS, we add jar with >

[jira] [Resolved] (SPARK-28840) --jar argument with spark-sql failed to load the jars to driver classpath

2019-09-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-28840. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25542

[jira] [Assigned] (SPARK-28840) --jar argument with spark-sql failed to load the jars to driver classpath

2019-09-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-28840: --- Assignee: Sandeep Katta > --jar argument with spark-sql failed to load the jars to driver

[jira] [Created] (SPARK-29068) CSV read reports incorrect row count

2019-09-12 Thread Thomas Diesler (Jira)
Thomas Diesler created SPARK-29068: -- Summary: CSV read reports incorrect row count Key: SPARK-29068 URL: https://issues.apache.org/jira/browse/SPARK-29068 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-29015) Can not support "add jar" on JDK 11

2019-09-12 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928429#comment-16928429 ] angerszhu commented on SPARK-29015: --- [~oceaneast]  Hi , have you find the root cause of this problem?

[jira] [Commented] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928394#comment-16928394 ] feiwang commented on SPARK-29037: - But for the version 2, it may produce partial result when we kill an

[jira] [Updated] (SPARK-29067) divide function does not throw an error, if a number is not passed to it

2019-09-12 Thread Mangesh Rananavare (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mangesh Rananavare updated SPARK-29067: --- Description: Ex.  dataset.where(col("col1").divide("col2")).$greater(10)).show();

[jira] [Updated] (SPARK-29067) divide function does not throw an error, if a number is not passed to it

2019-09-12 Thread Mangesh Rananavare (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mangesh Rananavare updated SPARK-29067: --- Description: Ex.  dataset.where(col("col1").divide("col2")).$greater(10)).show();

[jira] [Created] (SPARK-29067) divide function does not throw an error, if a number is not passed to it

2019-09-12 Thread Mangesh Rananavare (Jira)
Mangesh Rananavare created SPARK-29067: -- Summary: divide function does not throw an error, if a number is not passed to it Key: SPARK-29067 URL: https://issues.apache.org/jira/browse/SPARK-29067

[jira] [Commented] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928375#comment-16928375 ] feiwang commented on SPARK-29037: - If we set

[jira] [Issue Comment Deleted] (SPARK-29015) Can not support "add jar" on JDK 11

2019-09-12 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29015: -- Comment: was deleted (was: I have tried in Java8, the same error, maybe not jdk's problem.) > Can

[jira] [Updated] (SPARK-29003) Spark history server startup hang due to deadlock

2019-09-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29003: -- Affects Version/s: 2.3.4 > Spark history server startup hang due to deadlock >

[jira] [Updated] (SPARK-28588) Build a SQL reference doc

2019-09-12 Thread Dilip Biswal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Biswal updated SPARK-28588: - Description: Spark SQL requires a SQL reference doc for end users, like all the other query

[jira] [Commented] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928303#comment-16928303 ] Dongjoon Hyun commented on SPARK-29066: --- Hi, [~srowen] and [~shaneknapp]. How do you think about

[jira] [Created] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-29066: - Summary: Remove old Jenkins jobs for EOL versions or obsolete combinations Key: SPARK-29066 URL: https://issues.apache.org/jira/browse/SPARK-29066 Project: Spark

[jira] [Updated] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29066: -- Description: This issue aims to remove the old Jenkins jobs for EOL versions (1.6 ~ 2.3) and

[jira] [Created] (SPARK-29065) Extend `EXTRACT` benchmark

2019-09-12 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-29065: -- Summary: Extend `EXTRACT` benchmark Key: SPARK-29065 URL: https://issues.apache.org/jira/browse/SPARK-29065 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-29042) Sampling-based RDD with unordered input should be INDETERMINATE

2019-09-12 Thread Felix Cheung (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-29042: - Labels: correctness (was: ) > Sampling-based RDD with unordered input should be INDETERMINATE

[jira] [Resolved] (SPARK-29045) Test failed due to table already exists in SQLMetricsSuite

2019-09-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-29045. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25752

[jira] [Assigned] (SPARK-29045) Test failed due to table already exists in SQLMetricsSuite

2019-09-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-29045: --- Assignee: Lantao Jin > Test failed due to table already exists in SQLMetricsSuite >