[jira] [Resolved] (SPARK-28047) [UI] Little bug in executorspage.js

2019-06-13 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang resolved SPARK-28047. - Resolution: Not A Problem > [UI] Little bug in executorspage.js >

[jira] [Updated] (SPARK-28048) pyspark.sql.functions.explode will abondon the row which has a empty list column when applied to the column

2019-06-13 Thread Ma Xinmin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ma Xinmin updated SPARK-28048: -- Shepherd: Dongjoon Hyun > pyspark.sql.functions.explode will abondon the row which has a empty list

[jira] [Created] (SPARK-28048) pyspark.sql.functions.explode will abondon the row which has a empty list column when applied to the column

2019-06-13 Thread Ma Xinmin (JIRA)
Ma Xinmin created SPARK-28048: - Summary: pyspark.sql.functions.explode will abondon the row which has a empty list column when applied to the column Key: SPARK-28048 URL:

[jira] [Commented] (SPARK-28043) Reading json with duplicate columns drops the first column value

2019-06-13 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863680#comment-16863680 ] Liang-Chi Hsieh commented on SPARK-28043: - I tried to look around that, like

[jira] [Created] (SPARK-28047) [UI] Little bug in executorspage.js

2019-06-13 Thread feiwang (JIRA)
feiwang created SPARK-28047: --- Summary: [UI] Little bug in executorspage.js Key: SPARK-28047 URL: https://issues.apache.org/jira/browse/SPARK-28047 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-27018) Checkpointed RDD deleted prematurely when using GBTClassifier

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27018: Assignee: (was: Apache Spark) > Checkpointed RDD deleted prematurely when using

[jira] [Assigned] (SPARK-27018) Checkpointed RDD deleted prematurely when using GBTClassifier

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27018: Assignee: Apache Spark > Checkpointed RDD deleted prematurely when using GBTClassifier >

[jira] [Updated] (SPARK-27018) Checkpointed RDD deleted prematurely when using GBTClassifier

2019-06-13 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-27018: - Component/s: Spark Core > Checkpointed RDD deleted prematurely when using GBTClassifier >

[jira] [Updated] (SPARK-28022) k8s pod affinity to achieve cloud native friendly autoscaling

2019-06-13 Thread Henry Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henry Yu updated SPARK-28022: - Summary: k8s pod affinity to achieve cloud native friendly autoscaling (was: k8s pod affinity achieve

[jira] [Closed] (SPARK-28021) A unappropriate exception in StaticMemoryManager.getMaxExecutionMemory

2019-06-13 Thread child2d (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] child2d closed SPARK-28021. --- > A unappropriate exception in StaticMemoryManager.getMaxExecutionMemory >

[jira] [Commented] (SPARK-28021) A unappropriate exception in StaticMemoryManager.getMaxExecutionMemory

2019-06-13 Thread child2d (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863648#comment-16863648 ] child2d commented on SPARK-28021: - Thanks for reminding. I will close the issue. > A unappropriate

[jira] [Commented] (SPARK-28023) Trim the string when cast string type to other types

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863643#comment-16863643 ] Yuming Wang commented on SPARK-28023: - I'm working on. > Trim the string when cast string type to

[jira] [Resolved] (SPARK-27925) Better control numBins of curves in BinaryClassificationMetrics

2019-06-13 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-27925. -- Resolution: Not A Problem > Better control numBins of curves in BinaryClassificationMetrics >

[jira] [Assigned] (SPARK-28045) add missing RankingEvaluator

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28045: Assignee: (was: Apache Spark) > add missing RankingEvaluator >

[jira] [Assigned] (SPARK-28045) add missing RankingEvaluator

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28045: Assignee: Apache Spark > add missing RankingEvaluator > > >

[jira] [Updated] (SPARK-28046) OOM caused by building hash table when the compressed ratio of small table is normal

2019-06-13 Thread Ke Jia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Jia updated SPARK-28046: --- Attachment: image-2019-06-14-10-34-53-379.png > OOM caused by building hash table when the compressed ratio

[jira] [Updated] (SPARK-28046) OOM caused by building hash table when the compressed ratio of small table is normal

2019-06-13 Thread Ke Jia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Jia updated SPARK-28046: --- Description: Currently, spark will convert the sort merge join to broadcast hash join when the small table

[jira] [Created] (SPARK-28046) OOM caused by building hash table when the compressed ratio of small table is normal

2019-06-13 Thread Ke Jia (JIRA)
Ke Jia created SPARK-28046: -- Summary: OOM caused by building hash table when the compressed ratio of small table is normal Key: SPARK-28046 URL: https://issues.apache.org/jira/browse/SPARK-28046 Project:

[jira] [Created] (SPARK-28045) add missing RankingEvaluator

2019-06-13 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-28045: Summary: add missing RankingEvaluator Key: SPARK-28045 URL: https://issues.apache.org/jira/browse/SPARK-28045 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-28044) MulticlassClassificationEvaluator support more metrics

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28044: Assignee: Apache Spark > MulticlassClassificationEvaluator support more metrics >

[jira] [Assigned] (SPARK-28044) MulticlassClassificationEvaluator support more metrics

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28044: Assignee: (was: Apache Spark) > MulticlassClassificationEvaluator support more

[jira] [Created] (SPARK-28044) MulticlassClassificationEvaluator support more metrics

2019-06-13 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-28044: Summary: MulticlassClassificationEvaluator support more metrics Key: SPARK-28044 URL: https://issues.apache.org/jira/browse/SPARK-28044 Project: Spark Issue

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

2019-06-13 Thread HonglunChen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863598#comment-16863598 ] HonglunChen commented on SPARK-18112: - [~dongjoon] Thank you, I get it. > Spark2.x does not support

[jira] [Commented] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-13 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863570#comment-16863570 ] Bryan Cutler commented on SPARK-28041: -- Yes, definitely. I made a quick PR, but we should run it

[jira] [Commented] (SPARK-27463) Support Dataframe Cogroup via Pandas UDFs

2019-06-13 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863567#comment-16863567 ] Hyukjin Kwon commented on SPARK-27463: -- Yea I think it'd be easier to discuss about this with a Pr

[jira] [Assigned] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28041: Assignee: Apache Spark > Increase the minimum pandas version to 0.23.2 >

[jira] [Assigned] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28041: Assignee: (was: Apache Spark) > Increase the minimum pandas version to 0.23.2 >

[jira] [Commented] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-13 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863565#comment-16863565 ] Hyukjin Kwon commented on SPARK-28041: -- [~bryanc], BTW, can we quickly discuss this one in dev

[jira] [Updated] (SPARK-28043) Reading json with duplicate columns drops the first column value

2019-06-13 Thread Mukul Murthy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Murthy updated SPARK-28043: - Description: When reading a JSON blob with duplicate fields, Spark appears to ignore the value

[jira] [Created] (SPARK-28043) Reading json with duplicate columns drops the first column value

2019-06-13 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-28043: Summary: Reading json with duplicate columns drops the first column value Key: SPARK-28043 URL: https://issues.apache.org/jira/browse/SPARK-28043 Project: Spark

[jira] [Created] (SPARK-28042) Support mapping spark.local.dir to hostPath volume

2019-06-13 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-28042: - Summary: Support mapping spark.local.dir to hostPath volume Key: SPARK-28042 URL: https://issues.apache.org/jira/browse/SPARK-28042 Project: Spark Issue

[jira] [Commented] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-06-13 Thread Parth Chandra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863381#comment-16863381 ] Parth Chandra commented on SPARK-27100: --- Opened a PR with a fix and a test to reproduce the issue. 

[jira] [Assigned] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27100: Assignee: Apache Spark > dag-scheduler-event-loop" java.lang.StackOverflowError >

[jira] [Assigned] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27100: Assignee: (was: Apache Spark) > dag-scheduler-event-loop"

[jira] [Updated] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-06-13 Thread Parth Chandra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated SPARK-27100: -- Attachment: SPARK-27100-Overflow.txt > dag-scheduler-event-loop" java.lang.StackOverflowError

[jira] [Commented] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-06-13 Thread Parth Chandra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863375#comment-16863375 ] Parth Chandra commented on SPARK-27100: --- The stack overflow is due to serialization of a

[jira] [Created] (SPARK-28041) Increase the minimum pandas version to 0.23.2

2019-06-13 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-28041: Summary: Increase the minimum pandas version to 0.23.2 Key: SPARK-28041 URL: https://issues.apache.org/jira/browse/SPARK-28041 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-27578) Support INTERVAL ... HOUR TO SECOND syntax

2019-06-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27578. --- Resolution: Fixed Assignee: Zhu, Lipeng Fix Version/s: 3.0.0 This is

[jira] [Commented] (SPARK-14864) [MLLIB] Implement Doc2Vec

2019-06-13 Thread Ayush Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863246#comment-16863246 ] Ayush Singh commented on SPARK-14864: - [~michelle] any reason why this issue has been marked

[jira] [Created] (SPARK-28040) sql() fails to process output of glue::glue_data()

2019-06-13 Thread Michael Chirico (JIRA)
Michael Chirico created SPARK-28040: --- Summary: sql() fails to process output of glue::glue_data() Key: SPARK-28040 URL: https://issues.apache.org/jira/browse/SPARK-28040 Project: Spark

[jira] [Comment Edited] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

2019-06-13 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863186#comment-16863186 ] Li Jin edited comment on SPARK-28006 at 6/13/19 3:36 PM: - Hi [~viirya] good

[jira] [Commented] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

2019-06-13 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863186#comment-16863186 ] Li Jin commented on SPARK-28006: Hi [~viirya] good questions: >> Can we use pandas agg udfs as window

[jira] [Commented] (SPARK-28006) User-defined grouped transform pandas_udf for window operations

2019-06-13 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863180#comment-16863180 ] Liang-Chi Hsieh commented on SPARK-28006: - I'm curious about two questions: Can we use pandas

[jira] [Commented] (SPARK-27463) Support Dataframe Cogroup via Pandas UDFs

2019-06-13 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863177#comment-16863177 ] Li Jin commented on SPARK-27463: Yeah I think the exact spelling of the API can go either way. I think

[jira] [Created] (SPARK-28039) Add float4.sql

2019-06-13 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28039: --- Summary: Add float4.sql Key: SPARK-28039 URL: https://issues.apache.org/jira/browse/SPARK-28039 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Assigned] (SPARK-16692) multilabel classification to DataFrame, ML

2019-06-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-16692: - Assignee: zhengruifeng > multilabel classification to DataFrame, ML >

[jira] [Resolved] (SPARK-16692) multilabel classification to DataFrame, ML

2019-06-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16692. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24777

[jira] [Updated] (SPARK-28033) String concatenation should low priority than other operators

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28033: Summary: String concatenation should low priority than other operators (was: String

[jira] [Assigned] (SPARK-28038) Add text.sql

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28038: Assignee: Apache Spark > Add text.sql > > > Key:

[jira] [Assigned] (SPARK-28038) Add text.sql

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28038: Assignee: (was: Apache Spark) > Add text.sql > > > Key:

[jira] [Commented] (SPARK-27966) input_file_name empty when listing files in parallel

2019-06-13 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863030#comment-16863030 ] Liang-Chi Hsieh commented on SPARK-27966: - I can't see where input_file_name is, from the

[jira] [Created] (SPARK-28038) Add text.sql

2019-06-13 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28038: --- Summary: Add text.sql Key: SPARK-28038 URL: https://issues.apache.org/jira/browse/SPARK-28038 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Created] (SPARK-28037) Add built-in String Functions: quote_literal

2019-06-13 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28037: --- Summary: Add built-in String Functions: quote_literal Key: SPARK-28037 URL: https://issues.apache.org/jira/browse/SPARK-28037 Project: Spark Issue Type:

[jira] [Issue Comment Deleted] (SPARK-28033) String concatenation low priority than other operators

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28033: Comment: was deleted (was: I'm working on.) > String concatenation low priority than other

[jira] [Updated] (SPARK-28033) String concatenation low priority than other operators

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28033: Summary: String concatenation low priority than other operators (was: String concatenation low

[jira] [Comment Edited] (SPARK-28025) HDFSBackedStateStoreProvider should not leak .crc files

2019-06-13 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862908#comment-16862908 ] Stavros Kontopoulos edited comment on SPARK-28025 at 6/13/19 10:53 AM:

[jira] [Comment Edited] (SPARK-28025) HDFSBackedStateStoreProvider should not leak .crc files

2019-06-13 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862908#comment-16862908 ] Stavros Kontopoulos edited comment on SPARK-28025 at 6/13/19 10:51 AM:

[jira] [Created] (SPARK-28036) Built-in udf left/right has inconsistent behavior

2019-06-13 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28036: --- Summary: Built-in udf left/right has inconsistent behavior Key: SPARK-28036 URL: https://issues.apache.org/jira/browse/SPARK-28036 Project: Spark Issue Type:

[jira] [Commented] (SPARK-28035) Test JoinSuite."equi-join is hash-join" is incompatible with its title.

2019-06-13 Thread Jiatao Tao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862928#comment-16862928 ] Jiatao Tao commented on SPARK-28035: this test "multiple-key equi-join is hash-join" also the same

[jira] [Comment Edited] (SPARK-28025) HDFSBackedStateStoreProvider should not leak .crc files

2019-06-13 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862908#comment-16862908 ] Stavros Kontopoulos edited comment on SPARK-28025 at 6/13/19 10:38 AM:

[jira] [Commented] (SPARK-28035) Test JoinSuite."equi-join is hash-join" is incompatible with its title.

2019-06-13 Thread Jiatao Tao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862927#comment-16862927 ] Jiatao Tao commented on SPARK-28035: The title means hash-join, but when I debug, found it is a

[jira] [Comment Edited] (SPARK-28025) HDFSBackedStateStoreProvider should not leak .crc files

2019-06-13 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862908#comment-16862908 ] Stavros Kontopoulos edited comment on SPARK-28025 at 6/13/19 10:36 AM:

[jira] [Comment Edited] (SPARK-28025) HDFSBackedStateStoreProvider should not leak .crc files

2019-06-13 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862908#comment-16862908 ] Stavros Kontopoulos edited comment on SPARK-28025 at 6/13/19 10:35 AM:

[jira] [Updated] (SPARK-28035) Test JoinSuite."equi-join is hash-join" is incompatible with its title.

2019-06-13 Thread Jiatao Tao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao updated SPARK-28035: --- Attachment: image-2019-06-13-10-32-06-759.png > Test JoinSuite."equi-join is hash-join" is

[jira] [Created] (SPARK-28035) Test JoinSuite."equi-join is hash-join" is incompatible with its title.

2019-06-13 Thread Jiatao Tao (JIRA)
Jiatao Tao created SPARK-28035: -- Summary: Test JoinSuite."equi-join is hash-join" is incompatible with its title. Key: SPARK-28035 URL: https://issues.apache.org/jira/browse/SPARK-28035 Project: Spark

[jira] [Reopened] (SPARK-27546) Should repalce DateTimeUtils#defaultTimeZoneuse with sessionLocalTimeZone

2019-06-13 Thread Jiatao Tao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao reopened SPARK-27546: Hi, I update the comment, could someone take a look? > Should repalce

[jira] [Commented] (SPARK-28025) HDFSBackedStateStoreProvider should not leak .crc files

2019-06-13 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862908#comment-16862908 ] Stavros Kontopoulos commented on SPARK-28025: - I just found out that the following config

[jira] [Updated] (SPARK-27930) List all built-in UDFs have different names

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27930: Description: This ticket list all built-in UDFs have different names:  |PostgreSQL|Spark

[jira] [Updated] (SPARK-27930) List all built-in UDFs have different names

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27930: Description: This ticket list all built-in UDFs have different names:  |PostgreSQL|Spark

[jira] [Commented] (SPARK-28016) Spark hangs when an execution plan has many projections on nested structs

2019-06-13 Thread Ruslan Yushchenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862898#comment-16862898 ] Ruslan Yushchenko commented on SPARK-28016: --- Attached a self-contained example

[jira] [Assigned] (SPARK-28034) Add with.sql

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28034: Assignee: (was: Apache Spark) > Add with.sql > > > Key:

[jira] [Assigned] (SPARK-28034) Add with.sql

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28034: Assignee: Apache Spark > Add with.sql > > > Key:

[jira] [Created] (SPARK-28034) Add with.sql

2019-06-13 Thread Peter Toth (JIRA)
Peter Toth created SPARK-28034: -- Summary: Add with.sql Key: SPARK-28034 URL: https://issues.apache.org/jira/browse/SPARK-28034 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Assigned] (SPARK-28033) String concatenation low priority than other arithmeticBinary

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28033: Assignee: (was: Apache Spark) > String concatenation low priority than other

[jira] [Assigned] (SPARK-28033) String concatenation low priority than other arithmeticBinary

2019-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28033: Assignee: Apache Spark > String concatenation low priority than other arithmeticBinary >

[jira] [Updated] (SPARK-28033) String concatenation low priority than other arithmeticBinary

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28033: Affects Version/s: (was: 3.0.0) 2.3.3 > String concatenation low

[jira] [Commented] (SPARK-28033) String concatenation low priority than other arithmeticBinary

2019-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862844#comment-16862844 ] Yuming Wang commented on SPARK-28033: - I'm working on. > String concatenation low priority than

[jira] [Created] (SPARK-28033) String concatenation low priority than other arithmeticBinary

2019-06-13 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28033: --- Summary: String concatenation low priority than other arithmeticBinary Key: SPARK-28033 URL: https://issues.apache.org/jira/browse/SPARK-28033 Project: Spark

[jira] [Commented] (SPARK-27463) Support Dataframe Cogroup via Pandas UDFs

2019-06-13 Thread Chris Martin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862828#comment-16862828 ] Chris Martin commented on SPARK-27463: -- Hi [~hyukjin.kwon] Ah I see your concern now.  It think

[jira] [Commented] (SPARK-27966) input_file_name empty when listing files in parallel

2019-06-13 Thread Christian Homberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862822#comment-16862822 ] Christian Homberg commented on SPARK-27966: --- This is the truncated output I get from

[jira] [Comment Edited] (SPARK-13882) Remove org.apache.spark.sql.execution.local

2019-06-13 Thread Lai Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862641#comment-16862641 ] Lai Zhou edited comment on SPARK-13882 at 6/13/19 8:07 AM: --- hi,[~rxin], is 

[jira] [Updated] (SPARK-28024) Incorrect numeric values when out of range

2019-06-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-28024: Priority: Critical (was: Major) > Incorrect numeric values when out of range >

[jira] [Updated] (SPARK-28024) Incorrect numeric values when out of range

2019-06-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-28024: Labels: correctness (was: ) > Incorrect numeric values when out of range >

[jira] [Updated] (SPARK-28024) Incorrect numeric values when out of range

2019-06-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-28024: Target Version/s: 3.0.0 > Incorrect numeric values when out of range >

[jira] [Updated] (SPARK-28016) Spark hangs when an execution plan has many projections on nested structs

2019-06-13 Thread Ruslan Yushchenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Yushchenko updated SPARK-28016: -- Attachment: SparkApp1IssueSelfContained.scala > Spark hangs when an execution plan

[jira] [Resolved] (SPARK-27322) DataSourceV2 table relation

2019-06-13 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27322. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24741

[jira] [Updated] (SPARK-25299) Use remote storage for persisting shuffle data

2019-06-13 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-25299: Labels: SPIP (was: ) > Use remote storage for persisting shuffle data >

[jira] [Commented] (SPARK-28016) Spark hangs when an execution plan has many projections on nested structs

2019-06-13 Thread Ruslan Yushchenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862752#comment-16862752 ] Ruslan Yushchenko commented on SPARK-28016: --- Thank you for your quick response. * Will make

[jira] [Commented] (SPARK-28024) Incorrect numeric values when out of range

2019-06-13 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862747#comment-16862747 ] Wenchen Fan commented on SPARK-28024: - AFAIK other databases would throw overflow exception. We may

[jira] [Assigned] (SPARK-27322) DataSourceV2 table relation

2019-06-13 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27322: --- Assignee: John Zhuge > DataSourceV2 table relation > --- > >

[jira] [Commented] (SPARK-27842) Inconsistent results of Statistics.corr() and PearsonCorrelation.computeCorrelationMatrix()

2019-06-13 Thread Peter Nijem (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862739#comment-16862739 ] Peter Nijem commented on SPARK-27842: - [~hyukjin.kwon] do you need another input from my side? If