[jira] [Resolved] (SPARK-27322) DataSourceV2 table relation

2019-06-13 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27322. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24741 [https://gith

[jira] [Updated] (SPARK-28016) Spark hangs when an execution plan has many projections on nested structs

2019-06-13 Thread Ruslan Yushchenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Yushchenko updated SPARK-28016: -- Attachment: SparkApp1IssueSelfContained.scala > Spark hangs when an execution plan has

[jira] [Updated] (SPARK-28024) Incorrect numeric values when out of range

2019-06-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-28024: Priority: Critical (was: Major) > Incorrect numeric values when out of range > --

[jira] [Updated] (SPARK-28024) Incorrect numeric values when out of range

2019-06-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-28024: Labels: correctness (was: ) > Incorrect numeric values when out of range > --

[jira] [Updated] (SPARK-28024) Incorrect numeric values when out of range

2019-06-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-28024: Target Version/s: 3.0.0 > Incorrect numeric values when out of range > ---

[jira] [Comment Edited] (SPARK-13882) Remove org.apache.spark.sql.execution.local

2019-06-13 Thread Lai Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862641#comment-16862641 ] Lai Zhou edited comment on SPARK-13882 at 6/13/19 8:07 AM: --- hi

[jira] [Commented] (SPARK-27966) input_file_name empty when listing files in parallel

2019-06-13 Thread Christian Homberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862822#comment-16862822 ] Christian Homberg commented on SPARK-27966: --- This is the truncated output I ge

[jira] [Commented] (SPARK-27463) Support Dataframe Cogroup via Pandas UDFs

2019-06-13 Thread Chris Martin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862828#comment-16862828 ] Chris Martin commented on SPARK-27463: -- Hi [~hyukjin.kwon] Ah I see your concern n

[jira] [Created] (SPARK-28033) String concatenation low priority than other arithmeticBinary

2019-06-13 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28033: --- Summary: String concatenation low priority than other arithmeticBinary Key: SPARK-28033 URL: https://issues.apache.org/jira/browse/SPARK-28033 Project: Spark

[jira] [Commented] (SPARK-28033) String concatenation low priority than other arithmeticBinary

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862844#comment-16862844 ] Yuming Wang commented on SPARK-28033: - I'm working on. > String concatenation low p

[jira] [Updated] (SPARK-28033) String concatenation low priority than other arithmeticBinary

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28033: Affects Version/s: (was: 3.0.0) 2.3.3 > String concatenation low priori

[jira] [Assigned] (SPARK-28033) String concatenation low priority than other arithmeticBinary

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28033: Assignee: Apache Spark > String concatenation low priority than other arithmeticBinary >

[jira] [Assigned] (SPARK-28033) String concatenation low priority than other arithmeticBinary

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28033: Assignee: (was: Apache Spark) > String concatenation low priority than other arithmet

[jira] [Created] (SPARK-28034) Add with.sql

2019-06-13 Thread Peter Toth (JIRA)
Peter Toth created SPARK-28034: -- Summary: Add with.sql Key: SPARK-28034 URL: https://issues.apache.org/jira/browse/SPARK-28034 Project: Spark Issue Type: Sub-task Components: SQL A

[jira] [Assigned] (SPARK-28034) Add with.sql

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28034: Assignee: (was: Apache Spark) > Add with.sql > > > Key:

[jira] [Assigned] (SPARK-28034) Add with.sql

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28034: Assignee: Apache Spark > Add with.sql > > > Key: SPARK-28034

[jira] [Commented] (SPARK-28016) Spark hangs when an execution plan has many projections on nested structs

2019-06-13 Thread Ruslan Yushchenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862898#comment-16862898 ] Ruslan Yushchenko commented on SPARK-28016: --- Attached a self-contained example

[jira] [Updated] (SPARK-27930) List all built-in UDFs have different names

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27930: Description: This ticket list all built-in UDFs have different names:  |PostgreSQL|Spark SQL|Note|

[jira] [Updated] (SPARK-27930) List all built-in UDFs have different names

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27930: Description: This ticket list all built-in UDFs have different names:  |PostgreSQL|Spark SQL|Note|

[jira] [Commented] (SPARK-28025) HDFSBackedStateStoreProvider should not leak .crc files

2019-06-13 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862908#comment-16862908 ] Stavros Kontopoulos commented on SPARK-28025: - I just found out that the fol

[jira] [Reopened] (SPARK-27546) Should repalce DateTimeUtils#defaultTimeZoneuse with sessionLocalTimeZone

2019-06-13 Thread Jiatao Tao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao reopened SPARK-27546: Hi, I update the comment, could someone take a look? > Should repalce DateTimeUtils#defaultTimeZoneus

[jira] [Created] (SPARK-28035) Test JoinSuite."equi-join is hash-join" is incompatible with its title.

2019-06-13 Thread Jiatao Tao (JIRA)
Jiatao Tao created SPARK-28035: -- Summary: Test JoinSuite."equi-join is hash-join" is incompatible with its title. Key: SPARK-28035 URL: https://issues.apache.org/jira/browse/SPARK-28035 Project: Spark

[jira] [Updated] (SPARK-28035) Test JoinSuite."equi-join is hash-join" is incompatible with its title.

2019-06-13 Thread Jiatao Tao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao updated SPARK-28035: --- Attachment: image-2019-06-13-10-32-06-759.png > Test JoinSuite."equi-join is hash-join" is incompati

[jira] [Comment Edited] (SPARK-28025) HDFSBackedStateStoreProvider should not leak .crc files

2019-06-13 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862908#comment-16862908 ] Stavros Kontopoulos edited comment on SPARK-28025 at 6/13/19 10:35 AM: ---

[jira] [Comment Edited] (SPARK-28025) HDFSBackedStateStoreProvider should not leak .crc files

2019-06-13 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862908#comment-16862908 ] Stavros Kontopoulos edited comment on SPARK-28025 at 6/13/19 10:36 AM: ---

[jira] [Commented] (SPARK-28035) Test JoinSuite."equi-join is hash-join" is incompatible with its title.

2019-06-13 Thread Jiatao Tao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862927#comment-16862927 ] Jiatao Tao commented on SPARK-28035: The title means hash-join, but when I debug, fo

[jira] [Comment Edited] (SPARK-28025) HDFSBackedStateStoreProvider should not leak .crc files

2019-06-13 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862908#comment-16862908 ] Stavros Kontopoulos edited comment on SPARK-28025 at 6/13/19 10:38 AM: ---

[jira] [Commented] (SPARK-28035) Test JoinSuite."equi-join is hash-join" is incompatible with its title.

2019-06-13 Thread Jiatao Tao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862928#comment-16862928 ] Jiatao Tao commented on SPARK-28035: this test "multiple-key equi-join is hash-join"

[jira] [Created] (SPARK-28036) Built-in udf left/right has inconsistent behavior

2019-06-13 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28036: --- Summary: Built-in udf left/right has inconsistent behavior Key: SPARK-28036 URL: https://issues.apache.org/jira/browse/SPARK-28036 Project: Spark Issue Type: S

[jira] [Comment Edited] (SPARK-28025) HDFSBackedStateStoreProvider should not leak .crc files

2019-06-13 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862908#comment-16862908 ] Stavros Kontopoulos edited comment on SPARK-28025 at 6/13/19 10:51 AM: ---

[jira] [Comment Edited] (SPARK-28025) HDFSBackedStateStoreProvider should not leak .crc files

2019-06-13 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862908#comment-16862908 ] Stavros Kontopoulos edited comment on SPARK-28025 at 6/13/19 10:53 AM: ---

[jira] [Updated] (SPARK-28033) String concatenation low priority than other operators

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28033: Summary: String concatenation low priority than other operators (was: String concatenation low pr

[jira] [Issue Comment Deleted] (SPARK-28033) String concatenation low priority than other operators

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28033: Comment: was deleted (was: I'm working on.) > String concatenation low priority than other operat

[jira] [Created] (SPARK-28037) Add built-in String Functions: quote_literal

2019-06-13 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28037: --- Summary: Add built-in String Functions: quote_literal Key: SPARK-28037 URL: https://issues.apache.org/jira/browse/SPARK-28037 Project: Spark Issue Type: Sub-ta

[jira] [Created] (SPARK-28038) Add text.sql

2019-06-13 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28038: --- Summary: Add text.sql Key: SPARK-28038 URL: https://issues.apache.org/jira/browse/SPARK-28038 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Commented] (SPARK-27966) input_file_name empty when listing files in parallel

2019-06-13 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863030#comment-16863030 ] Liang-Chi Hsieh commented on SPARK-27966: - I can't see where input_file_name is,

[jira] [Assigned] (SPARK-28038) Add text.sql

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28038: Assignee: (was: Apache Spark) > Add text.sql > > > Key:

[jira] [Assigned] (SPARK-28038) Add text.sql

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28038: Assignee: Apache Spark > Add text.sql > > > Key: SPARK-28038

[jira] [Updated] (SPARK-28033) String concatenation should low priority than other operators

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28033: Summary: String concatenation should low priority than other operators (was: String concatenation

[jira] [Resolved] (SPARK-16692) multilabel classification to DataFrame, ML

2019-06-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16692. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24777 [https://github.c

[jira] [Assigned] (SPARK-16692) multilabel classification to DataFrame, ML

2019-06-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-16692: - Assignee: zhengruifeng > multilabel classification to DataFrame, ML >

[jira] [Created] (SPARK-28039) Add float4.sql

2019-06-13 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28039: --- Summary: Add float4.sql Key: SPARK-28039 URL: https://issues.apache.org/jira/browse/SPARK-28039 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Commented] (SPARK-27463) Support Dataframe Cogroup via Pandas UDFs

2019-06-13 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863177#comment-16863177 ] Li Jin commented on SPARK-27463: Yeah I think the exact spelling of the API can go eithe

[jira] [Commented] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

2019-06-13 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863180#comment-16863180 ] Liang-Chi Hsieh commented on SPARK-28006: - I'm curious about two questions: Can

[jira] [Comment Edited] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

2019-06-13 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863186#comment-16863186 ] Li Jin edited comment on SPARK-28006 at 6/13/19 3:36 PM: - Hi [~v

[jira] [Commented] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

2019-06-13 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863186#comment-16863186 ] Li Jin commented on SPARK-28006: Hi [~viirya] good questions: >> Can we use pandas agg

[jira] [Created] (SPARK-28040) sql() fails to process output of glue::glue_data()

2019-06-13 Thread Michael Chirico (JIRA)
Michael Chirico created SPARK-28040: --- Summary: sql() fails to process output of glue::glue_data() Key: SPARK-28040 URL: https://issues.apache.org/jira/browse/SPARK-28040 Project: Spark Issu

[jira] [Commented] (SPARK-14864) [MLLIB] Implement Doc2Vec

2019-06-13 Thread Ayush Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863246#comment-16863246 ] Ayush Singh commented on SPARK-14864: - [~michelle] any reason why this issue has bee

[jira] [Resolved] (SPARK-27578) Support INTERVAL ... HOUR TO SECOND syntax

2019-06-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27578. --- Resolution: Fixed Assignee: Zhu, Lipeng Fix Version/s: 3.0.0 This is resolve

[jira] [Created] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-13 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-28041: Summary: Increase the minimum pandas version to 0.23.2 Key: SPARK-28041 URL: https://issues.apache.org/jira/browse/SPARK-28041 Project: Spark Issue Type: Imp

[jira] [Updated] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-06-13 Thread Parth Chandra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated SPARK-27100: -- Attachment: SPARK-27100-Overflow.txt > dag-scheduler-event-loop" java.lang.StackOverflowError

[jira] [Commented] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-06-13 Thread Parth Chandra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863375#comment-16863375 ] Parth Chandra commented on SPARK-27100: --- The stack overflow is due to serializatio

[jira] [Assigned] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27100: Assignee: (was: Apache Spark) > dag-scheduler-event-loop" java.lang.StackOverflowErro

[jira] [Assigned] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27100: Assignee: Apache Spark > dag-scheduler-event-loop" java.lang.StackOverflowError > ---

[jira] [Commented] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-06-13 Thread Parth Chandra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863381#comment-16863381 ] Parth Chandra commented on SPARK-27100: --- Opened a PR with a fix and a test to repr

[jira] [Created] (SPARK-28042) Support mapping spark.local.dir to hostPath volume

2019-06-13 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-28042: - Summary: Support mapping spark.local.dir to hostPath volume Key: SPARK-28042 URL: https://issues.apache.org/jira/browse/SPARK-28042 Project: Spark Issue Ty

[jira] [Created] (SPARK-28043) Reading json with duplicate columns drops the first column value

2019-06-13 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-28043: Summary: Reading json with duplicate columns drops the first column value Key: SPARK-28043 URL: https://issues.apache.org/jira/browse/SPARK-28043 Project: Spark

[jira] [Updated] (SPARK-28043) Reading json with duplicate columns drops the first column value

2019-06-13 Thread Mukul Murthy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Murthy updated SPARK-28043: - Description: When reading a JSON blob with duplicate fields, Spark appears to ignore the value

[jira] [Commented] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-13 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863565#comment-16863565 ] Hyukjin Kwon commented on SPARK-28041: -- [~bryanc], BTW, can we quickly discuss this

[jira] [Assigned] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28041: Assignee: Apache Spark > Increase the minimum pandas version to 0.23.2 >

[jira] [Assigned] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28041: Assignee: (was: Apache Spark) > Increase the minimum pandas version to 0.23.2 > -

[jira] [Commented] (SPARK-27463) Support Dataframe Cogroup via Pandas UDFs

2019-06-13 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863567#comment-16863567 ] Hyukjin Kwon commented on SPARK-27463: -- Yea I think it'd be easier to discuss about

[jira] [Commented] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-13 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863570#comment-16863570 ] Bryan Cutler commented on SPARK-28041: -- Yes, definitely. I made a quick PR, but we

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

2019-06-13 Thread HonglunChen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863598#comment-16863598 ] HonglunChen commented on SPARK-18112: - [~dongjoon] Thank you, I get it. > Spark2.x

[jira] [Created] (SPARK-28044) MulticlassClassificationEvaluator support more metrics

2019-06-13 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-28044: Summary: MulticlassClassificationEvaluator support more metrics Key: SPARK-28044 URL: https://issues.apache.org/jira/browse/SPARK-28044 Project: Spark Issue

[jira] [Assigned] (SPARK-28044) MulticlassClassificationEvaluator support more metrics

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28044: Assignee: Apache Spark > MulticlassClassificationEvaluator support more metrics > ---

[jira] [Assigned] (SPARK-28044) MulticlassClassificationEvaluator support more metrics

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28044: Assignee: (was: Apache Spark) > MulticlassClassificationEvaluator support more metric

[jira] [Created] (SPARK-28045) add missing RankingEvaluator

2019-06-13 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-28045: Summary: add missing RankingEvaluator Key: SPARK-28045 URL: https://issues.apache.org/jira/browse/SPARK-28045 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-28046) OOM caused by building hash table when the compressed ratio of small table is normal

2019-06-13 Thread Ke Jia (JIRA)
Ke Jia created SPARK-28046: -- Summary: OOM caused by building hash table when the compressed ratio of small table is normal Key: SPARK-28046 URL: https://issues.apache.org/jira/browse/SPARK-28046 Project: Spa

[jira] [Updated] (SPARK-28046) OOM caused by building hash table when the compressed ratio of small table is normal

2019-06-13 Thread Ke Jia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Jia updated SPARK-28046: --- Attachment: image-2019-06-14-10-34-53-379.png > OOM caused by building hash table when the compressed ratio

[jira] [Updated] (SPARK-28046) OOM caused by building hash table when the compressed ratio of small table is normal

2019-06-13 Thread Ke Jia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Jia updated SPARK-28046: --- Description: Currently, spark will convert the sort merge join to broadcast hash join when the small table

[jira] [Assigned] (SPARK-28045) add missing RankingEvaluator

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28045: Assignee: Apache Spark > add missing RankingEvaluator > > >

[jira] [Assigned] (SPARK-28045) add missing RankingEvaluator

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28045: Assignee: (was: Apache Spark) > add missing RankingEvaluator > --

[jira] [Resolved] (SPARK-27925) Better control numBins of curves in BinaryClassificationMetrics

2019-06-13 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-27925. -- Resolution: Not A Problem > Better control numBins of curves in BinaryClassificationMetrics >

[jira] [Commented] (SPARK-28023) Trim the string when cast string type to other types

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863643#comment-16863643 ] Yuming Wang commented on SPARK-28023: - I'm working on. > Trim the string when cast

[jira] [Commented] (SPARK-28021) A unappropriate exception in StaticMemoryManager.getMaxExecutionMemory

2019-06-13 Thread child2d (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863648#comment-16863648 ] child2d commented on SPARK-28021: - Thanks for reminding. I will close the issue. > A un

[jira] [Updated] (SPARK-28022) k8s pod affinity to achieve cloud native friendly autoscaling

2019-06-13 Thread Henry Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henry Yu updated SPARK-28022: - Summary: k8s pod affinity to achieve cloud native friendly autoscaling (was: k8s pod affinity achieve

[jira] [Closed] (SPARK-28021) A unappropriate exception in StaticMemoryManager.getMaxExecutionMemory

2019-06-13 Thread child2d (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] child2d closed SPARK-28021. --- > A unappropriate exception in StaticMemoryManager.getMaxExecutionMemory > -

[jira] [Updated] (SPARK-27018) Checkpointed RDD deleted prematurely when using GBTClassifier

2019-06-13 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-27018: - Component/s: Spark Core > Checkpointed RDD deleted prematurely when using GBTClassifier > --

[jira] [Assigned] (SPARK-27018) Checkpointed RDD deleted prematurely when using GBTClassifier

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27018: Assignee: (was: Apache Spark) > Checkpointed RDD deleted prematurely when using GBTCl

[jira] [Assigned] (SPARK-27018) Checkpointed RDD deleted prematurely when using GBTClassifier

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27018: Assignee: Apache Spark > Checkpointed RDD deleted prematurely when using GBTClassifier >

[jira] [Created] (SPARK-28047) [UI] Little bug in executorspage.js

2019-06-13 Thread feiwang (JIRA)
feiwang created SPARK-28047: --- Summary: [UI] Little bug in executorspage.js Key: SPARK-28047 URL: https://issues.apache.org/jira/browse/SPARK-28047 Project: Spark Issue Type: Bug Componen

[jira] [Commented] (SPARK-28043) Reading json with duplicate columns drops the first column value

2019-06-13 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863680#comment-16863680 ] Liang-Chi Hsieh commented on SPARK-28043: - I tried to look around that, like ht

[jira] [Created] (SPARK-28048) pyspark.sql.functions.explode will abondon the row which has a empty list column when applied to the column

2019-06-13 Thread Ma Xinmin (JIRA)
Ma Xinmin created SPARK-28048: - Summary: pyspark.sql.functions.explode will abondon the row which has a empty list column when applied to the column Key: SPARK-28048 URL: https://issues.apache.org/jira/browse/SPARK-28

[jira] [Updated] (SPARK-28048) pyspark.sql.functions.explode will abondon the row which has a empty list column when applied to the column

2019-06-13 Thread Ma Xinmin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ma Xinmin updated SPARK-28048: -- Shepherd: Dongjoon Hyun > pyspark.sql.functions.explode will abondon the row which has a empty list >

[jira] [Resolved] (SPARK-28047) [UI] Little bug in executorspage.js

2019-06-13 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang resolved SPARK-28047. - Resolution: Not A Problem > [UI] Little bug in executorspage.js >

[jira] [Created] (SPARK-28049) i want to a first ticket in zira

2019-06-13 Thread sanjeet (JIRA)
sanjeet created SPARK-28049: --- Summary: i want to a first ticket in zira Key: SPARK-28049 URL: https://issues.apache.org/jira/browse/SPARK-28049 Project: Spark Issue Type: Test Components:

[jira] [Commented] (SPARK-28049) i want to a first ticket in zira

2019-06-13 Thread sanjeet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863742#comment-16863742 ] sanjeet commented on SPARK-28049: - 2 nd comment > i want to a first ticket in zira > --

[jira] [Resolved] (SPARK-28049) i want to a first ticket in zira

2019-06-13 Thread sanjeet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sanjeet resolved SPARK-28049. - Resolution: Fixed code change has been delivered > i want to a first ticket in zira > -

[jira] [Updated] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-06-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27100: -- Component/s: (was: MLlib) SQL > dag-scheduler-event-loop" java.lang.Stack

[jira] [Created] (SPARK-28050) DataFrameWriter support insertInto a specific table partition

2019-06-13 Thread Leanken.Lin (JIRA)
Leanken.Lin created SPARK-28050: --- Summary: DataFrameWriter support insertInto a specific table partition Key: SPARK-28050 URL: https://issues.apache.org/jira/browse/SPARK-28050 Project: Spark

[jira] [Updated] (SPARK-28050) DataFrameWriter support insertInto a specific table partition

2019-06-13 Thread Leanken.Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-28050: Description: ``` val ptTableName = "mc_test_pt_table" sql(s"CREATE TABLE ${ptTableName} (name STRI

[jira] [Updated] (SPARK-28050) DataFrameWriter support insertInto a specific table partition

2019-06-13 Thread Leanken.Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-28050: Description: {code:java} // Some comments here val ptTableName = "mc_test_pt_table" sql(s"CREATE T

[jira] [Created] (SPARK-28051) Exposing JIRA issue component types at GitHub PRs

2019-06-13 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-28051: - Summary: Exposing JIRA issue component types at GitHub PRs Key: SPARK-28051 URL: https://issues.apache.org/jira/browse/SPARK-28051 Project: Spark Issue Typ