[jira] [Created] (SPARK-27070) DefaultPartitionCoalescer can lock up driver for hours

2019-03-06 Thread Yuli Fiterman (JIRA)
Yuli Fiterman created SPARK-27070: - Summary: DefaultPartitionCoalescer can lock up driver for hours Key: SPARK-27070 URL: https://issues.apache.org/jira/browse/SPARK-27070 Project: Spark Issu

[jira] [Closed] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

2019-03-06 Thread Andy A (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy A closed SPARK-27059. -- > spark-submit on kubernetes cluster does not recognise k8s --master property > --

[jira] [Commented] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

2019-03-06 Thread Andy A (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785434#comment-16785434 ] Andy A commented on SPARK-27059: That was it - thanks. > spark-submit on kubernetes clu

[jira] [Commented] (SPARK-27068) Support failed jobs ui and completed jobs ui use different queue

2019-03-06 Thread Shahid K I (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785427#comment-16785427 ] Shahid K I commented on SPARK-27068: Hi, In the jobs page, both completed and failed

[jira] [Assigned] (SPARK-27070) DefaultPartitionCoalescer can lock up driver for hours

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27070: Assignee: (was: Apache Spark) > DefaultPartitionCoalescer can lock up driver for hour

[jira] [Assigned] (SPARK-27070) DefaultPartitionCoalescer can lock up driver for hours

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27070: Assignee: Apache Spark > DefaultPartitionCoalescer can lock up driver for hours > ---

[jira] [Updated] (SPARK-27070) DefaultPartitionCoalescer can lock up driver for hours

2019-03-06 Thread Yuli Fiterman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuli Fiterman updated SPARK-27070: -- Patch:https://github.com/apache/spark/pull/23986 > DefaultPartitionCoalescer can lock up driver f

[jira] [Commented] (SPARK-27068) Support failed jobs ui and completed jobs ui use different queue

2019-03-06 Thread Shahid K I (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785451#comment-16785451 ] Shahid K I commented on SPARK-27068: Okay. This is configuration issue. This can be

[jira] [Comment Edited] (SPARK-27068) Support failed jobs ui and completed jobs ui use different queue

2019-03-06 Thread Shahid K I (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785451#comment-16785451 ] Shahid K I edited comment on SPARK-27068 at 3/6/19 9:39 AM:

[jira] [Commented] (SPARK-26330) Duplicate query execution events generated for SQL commands

2019-03-06 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785466#comment-16785466 ] Sandeep Katta commented on SPARK-26330: --- [~vanzin] I will work on this issue, and

[jira] [Commented] (SPARK-27055) Update Structured Streaming documentation because of DSv2 changes

2019-03-06 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785469#comment-16785469 ] Sandeep Katta commented on SPARK-27055: --- [~gsomogyi] thank you for reporting this,

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785575#comment-16785575 ] Stavros Kontopoulos commented on SPARK-26742: - We need to backport it to 2.4

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785575#comment-16785575 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 12:35 PM:

[jira] [Commented] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785581#comment-16785581 ] Stavros Kontopoulos commented on SPARK-26997: - [~shaneknapp] are the test no

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785575#comment-16785575 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 12:37 PM:

[jira] [Commented] (SPARK-24621) WebUI - application 'name' urls point to http instead of https (even when ssl enabled)

2019-03-06 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785583#comment-16785583 ] Gabor Somogyi commented on SPARK-24621: --- I've analyzed through this issue with the

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785575#comment-16785575 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 12:34 PM:

[jira] [Comment Edited] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785581#comment-16785581 ] Stavros Kontopoulos edited comment on SPARK-26997 at 3/6/19 12:37 PM:

[jira] [Issue Comment Deleted] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-26997: Comment: was deleted (was: [~shaneknapp] are the test nodes upgraded so we can cre

[jira] [Created] (SPARK-27071) Expose additional metrics in status.api.v1.StageData

2019-03-06 Thread Tom van Bussel (JIRA)
Tom van Bussel created SPARK-27071: -- Summary: Expose additional metrics in status.api.v1.StageData Key: SPARK-27071 URL: https://issues.apache.org/jira/browse/SPARK-27071 Project: Spark Issu

[jira] [Assigned] (SPARK-26509) Parquet DELTA_BYTE_ARRAY is not supported in Spark 2.x's Vectorized Reader

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26509: Assignee: Apache Spark > Parquet DELTA_BYTE_ARRAY is not supported in Spark 2.x's Vectori

[jira] [Assigned] (SPARK-26509) Parquet DELTA_BYTE_ARRAY is not supported in Spark 2.x's Vectorized Reader

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26509: Assignee: (was: Apache Spark) > Parquet DELTA_BYTE_ARRAY is not supported in Spark 2.

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785575#comment-16785575 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 1:34 PM: -

[jira] [Commented] (SPARK-27017) Creating orc table with special symbols in column name via spark.sql

2019-03-06 Thread Chakravarthi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785640#comment-16785640 ] Chakravarthi commented on SPARK-27017: -- Hi [~uNxe] ,can you please provide which hi

[jira] [Commented] (SPARK-27017) Creating orc table with special symbols in column name via spark.sql

2019-03-06 Thread Henryk Cesnolovic (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785647#comment-16785647 ] Henryk Cesnolovic commented on SPARK-27017: --- Hive 1.2.1000.2.6.5.0-292 > Crea

[jira] [Updated] (SPARK-27072) Changing the parameter value of completedJob.sort to X prints stacktrace in sparkWebUI

2019-03-06 Thread Haripriya (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haripriya updated SPARK-27072: -- Summary: Changing the parameter value of completedJob.sort to X prints stacktrace in sparkWebUI (was:

[jira] [Created] (SPARK-27072) Changing the parameter value of completedStage.sort to X prints stacktrace in sparkWebUI

2019-03-06 Thread Haripriya (JIRA)
Haripriya created SPARK-27072: - Summary: Changing the parameter value of completedStage.sort to X prints stacktrace in sparkWebUI Key: SPARK-27072 URL: https://issues.apache.org/jira/browse/SPARK-27072 Pr

[jira] [Assigned] (SPARK-27031) Avoid double formatting in timestampToString

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27031: - Assignee: Maxim Gekk > Avoid double formatting in timestampToString > -

[jira] [Resolved] (SPARK-27031) Avoid double formatting in timestampToString

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27031. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23936 [https://github.c

[jira] [Assigned] (SPARK-26981) Add 'Recall_at_k' metric to RankingMetrics

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26981: - Assignee: Masahiro Kazama > Add 'Recall_at_k' metric to RankingMetrics > --

[jira] [Updated] (SPARK-27035) Current time with microsecond resolution

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-27035: -- Docs Text: In Spark version 2.4 and earlier, the `current_timestamp` function returns a timestamp with

[jira] [Resolved] (SPARK-27035) Current time with microsecond resolution

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27035. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23945 [https://github.c

[jira] [Assigned] (SPARK-27035) Current time with microsecond resolution

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27035: - Assignee: Maxim Gekk > Current time with microsecond resolution > -

[jira] [Resolved] (SPARK-26981) Add 'Recall_at_k' metric to RankingMetrics

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26981. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23881 [https://github.c

[jira] [Created] (SPARK-27073) Handling of IdleStateEvent causes the normal connection to close

2019-03-06 Thread dzcxzl (JIRA)
dzcxzl created SPARK-27073: -- Summary: Handling of IdleStateEvent causes the normal connection to close Key: SPARK-27073 URL: https://issues.apache.org/jira/browse/SPARK-27073 Project: Spark Issue T

[jira] [Assigned] (SPARK-27073) Handling of IdleStateEvent causes the normal connection to close

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27073: Assignee: Apache Spark > Handling of IdleStateEvent causes the normal connection to close

[jira] [Resolved] (SPARK-26602) Subsequent queries are failing after querying the UDF which is loaded with wrong hdfs path

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26602. --- Resolution: Not A Problem > Subsequent queries are failing after querying the UDF which is loaded wi

[jira] [Assigned] (SPARK-27073) Handling of IdleStateEvent causes the normal connection to close

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27073: Assignee: (was: Apache Spark) > Handling of IdleStateEvent causes the normal connecti

[jira] [Commented] (SPARK-27072) Changing the parameter value of completedJob.sort to X prints stacktrace in sparkWebUI

2019-03-06 Thread Chakravarthi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785713#comment-16785713 ] Chakravarthi commented on SPARK-27072: -- Hi [~vanzin] , Please confirm, is it ok to

[jira] [Commented] (SPARK-27061) Expose Driver UI port on driver service to access UI using service

2019-03-06 Thread Chandu Kavar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785724#comment-16785724 ] Chandu Kavar commented on SPARK-27061: -- I started working on it.  > Expose Driver

[jira] [Assigned] (SPARK-27047) Document stop-slave.sh in spark-standalone

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27047: - Assignee: Ajith S > Document stop-slave.sh in spark-standalone > --

[jira] [Updated] (SPARK-27061) Expose Driver UI port on driver service to access UI using service

2019-03-06 Thread Chandu Kavar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandu Kavar updated SPARK-27061: - Summary: Expose Driver UI port on driver service to access UI using service (was: Expose 4040 p

[jira] [Updated] (SPARK-26016) Document that UTF-8 is required in text data source

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26016: -- Issue Type: Improvement (was: Bug) Summary: Document that UTF-8 is required in text data source

[jira] [Resolved] (SPARK-27047) Document stop-slave.sh in spark-standalone

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27047. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23960 [https://github.c

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785734#comment-16785734 ] Sean Owen commented on SPARK-26742: --- The problem is that that's a major dependency upg

[jira] [Assigned] (SPARK-27061) Expose Driver UI port on driver service to access UI using service

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27061: Assignee: Apache Spark > Expose Driver UI port on driver service to access UI using servi

[jira] [Assigned] (SPARK-27061) Expose Driver UI port on driver service to access UI using service

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27061: Assignee: (was: Apache Spark) > Expose Driver UI port on driver service to access UI

[jira] [Created] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27074: --- Summary: Refactor HiveClientImpl runHive Key: SPARK-27074 URL: https://issues.apache.org/jira/browse/SPARK-27074 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785829#comment-16785829 ] shane knapp commented on SPARK-26742: - [~skonto] – the nodes all have the latest min

[jira] [Assigned] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27074: Assignee: (was: Apache Spark) > Refactor HiveClientImpl runHive > ---

[jira] [Assigned] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27074: Assignee: Apache Spark > Refactor HiveClientImpl runHive > --

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785842#comment-16785842 ] Sean Owen commented on SPARK-26742: --- [~shaneknapp] ping me when ready and I can try th

[jira] [Updated] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27074: Description: Hive 3.1.1's {{CommandProcessor}} have 2 changes: # HIVE-17626(Hive 3.0.0) add ReExe

[jira] [Updated] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27074: Description: Hive 3.1.1's {{CommandProcessor}} have 2 changes: # HIVE-17626(Hive 3.0.0) add ReExe

[jira] [Commented] (SPARK-27072) Changing the parameter value of completedJob.sort to X prints stacktrace in sparkWebUI

2019-03-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785889#comment-16785889 ] Marcelo Vanzin commented on SPARK-27072: I'd expect an error 401 here, but you g

[jira] [Commented] (SPARK-24669) Managed table was not cleared of path after drop database cascade

2019-03-06 Thread Udbhav Agrawal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785900#comment-16785900 ] Udbhav Agrawal commented on SPARK-24669: Thankyou [~dongjoon] > Managed table w

[jira] [Commented] (SPARK-24624) Can not mix vectorized and non-vectorized UDFs

2019-03-06 Thread peay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785920#comment-16785920 ] peay commented on SPARK-24624: -- Are there plans to support something similar for aggregatio

[jira] [Resolved] (SPARK-24669) Managed table was not cleared of path after drop database cascade

2019-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-24669. --- Resolution: Fixed Assignee: Udbhav Agrawal Fix Version/s: 3.0.0

[jira] [Assigned] (SPARK-25250) Race condition with tasks running when new attempt for same stage is created leads to other task in the next attempt running on the same partition id retry multiple tim

2019-03-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-25250: Assignee: Parth Gandhi > Race condition with tasks running when new attempt for same stag

[jira] [Updated] (SPARK-25250) Race condition with tasks running when new attempt for same stage is created leads to other task in the next attempt running on the same partition id retry multiple time

2019-03-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25250: - Fix Version/s: 3.0.0 2.4.1 > Race condition with tasks running when new attem

[jira] [Resolved] (SPARK-25250) Race condition with tasks running when new attempt for same stage is created leads to other task in the next attempt running on the same partition id retry multiple tim

2019-03-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-25250. -- Resolution: Fixed > Race condition with tasks running when new attempt for same stage is creat

[jira] [Resolved] (SPARK-27065) avoid more than one active task set managers for a stage

2019-03-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-27065. -- Resolution: Fixed Fix Version/s: 2.3.4 2.4.1 3.0.0

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785970#comment-16785970 ] shane knapp commented on SPARK-26742: - you can submit a PR now. it will fail on the

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785962#comment-16785962 ] Stavros Kontopoulos commented on SPARK-26742: - I created one for the 2.4 bra

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785962#comment-16785962 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 6:14 PM: -

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785983#comment-16785983 ] shane knapp commented on SPARK-26742: - please link to the PRs (master + 2.4) here an

[jira] [Updated] (SPARK-27045) SQL tab in UI shows callsite instead of actual SQL

2019-03-06 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27045: Issue Type: Improvement (was: Bug) > SQL tab in UI shows callsite instead of actual SQL > ---

[jira] [Updated] (SPARK-27045) SQL tab in UI shows actual SQL instead of callsite

2019-03-06 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27045: Summary: SQL tab in UI shows actual SQL instead of callsite (was: SQL tab in UI shows callsite instead of

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785987#comment-16785987 ] shane knapp commented on SPARK-26742: - ok, i found the 2.4 PR: https://github.com/a

[jira] [Commented] (SPARK-27006) SPIP: .NET bindings for Apache Spark

2019-03-06 Thread Terry Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786001#comment-16786001 ] Terry Kim commented on SPARK-27006: --- Thanks for the feedback. We are happy to work on

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786019#comment-16786019 ] Stavros Kontopoulos commented on SPARK-26742: - Cool! The question is does it

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786019#comment-16786019 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 6:58 PM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786019#comment-16786019 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 6:58 PM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786019#comment-16786019 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 6:59 PM: -

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786012#comment-16786012 ] shane knapp commented on SPARK-26742: - 2.4 k8s integration tests pass w/the client u

[jira] [Commented] (SPARK-18748) UDF multiple evaluations causes very poor performance

2019-03-06 Thread Qingbo Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786029#comment-16786029 ] Qingbo Hu commented on SPARK-18748: --- We have the same problem when using Spark structu

[jira] [Commented] (SPARK-27006) SPIP: .NET bindings for Apache Spark

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786030#comment-16786030 ] Sean Owen commented on SPARK-27006: --- You can use the Apache license, and follow Apache

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786023#comment-16786023 ] shane knapp commented on SPARK-26742: - i think you mean: s/minikube/k8s anyways, e

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786019#comment-16786019 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 6:59 PM: -

[jira] [Resolved] (SPARK-27023) Kubernetes client timeouts should be configurable

2019-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27023. --- Resolution: Fixed Assignee: Onur Satici Fix Version/s: 3.0.0 This is resolve

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786047#comment-16786047 ] shane knapp edited comment on SPARK-26742 at 3/6/19 7:17 PM: -

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786047#comment-16786047 ] shane knapp commented on SPARK-26742: - works for me against k8s v1.11.7: {noformat

[jira] [Commented] (SPARK-27063) Spark on K8S Integration Tests timeouts are too short for some test clusters

2019-03-06 Thread Rob Vesse (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786051#comment-16786051 ] Rob Vesse commented on SPARK-27063: --- [~skonto] Yes we have experienced the same proble

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786060#comment-16786060 ] shane knapp commented on SPARK-26742: - i was unable to get 1.11.7 to work until i di

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786056#comment-16786056 ] Stavros Kontopoulos commented on SPARK-26742: - I mean `minikube` binary beca

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786060#comment-16786060 ] shane knapp edited comment on SPARK-26742 at 3/6/19 7:33 PM: -

[jira] [Updated] (SPARK-27023) Kubernetes client timeouts should be configurable

2019-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27023: -- Issue Type: Improvement (was: New Feature) > Kubernetes client timeouts should be configurabl

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786067#comment-16786067 ] Stavros Kontopoulos commented on SPARK-26742: - I did all that plus rm /etc/k

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786067#comment-16786067 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 7:40 PM: -

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786068#comment-16786068 ] shane knapp commented on SPARK-26742: - 1.12.6 passes for me as well: {noformat} [I

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786069#comment-16786069 ] Stavros Kontopoulos commented on SPARK-26742: - That is awesome! > Bump Kube

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786074#comment-16786074 ] shane knapp commented on SPARK-26742: - [~skonto] the only problems i've had are when

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786097#comment-16786097 ] shane knapp commented on SPARK-26742: - [~skonto] -- i was able to downgrade k8s succ

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786102#comment-16786102 ] Stavros Kontopoulos commented on SPARK-26742: - Yeah I was able to do so than

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786102#comment-16786102 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 8:29 PM: -

[jira] [Created] (SPARK-27075) Sorting table column in SQL WEBUI page throws 'IllegalArgumentException'

2019-03-06 Thread Shahid K I (JIRA)
Shahid K I created SPARK-27075: -- Summary: Sorting table column in SQL WEBUI page throws 'IllegalArgumentException' Key: SPARK-27075 URL: https://issues.apache.org/jira/browse/SPARK-27075 Project: Spark

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786129#comment-16786129 ] shane knapp commented on SPARK-26742: - alright, last thing before these two PRs are

[jira] [Assigned] (SPARK-27075) Sorting table column in SQL WEBUI page throws 'IllegalArgumentException'

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27075: Assignee: Apache Spark > Sorting table column in SQL WEBUI page throws 'IllegalArgumentEx

[jira] [Assigned] (SPARK-27075) Sorting table column in SQL WEBUI page throws 'IllegalArgumentException'

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27075: Assignee: (was: Apache Spark) > Sorting table column in SQL WEBUI page throws 'Illega

[jira] [Commented] (SPARK-27075) Sorting table column in SQL WEBUI page throws 'IllegalArgumentException'

2019-03-06 Thread Shahid K I (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786122#comment-16786122 ] Shahid K I commented on SPARK-27075: I will raise a PR > Sorting table column in SQ

  1   2   >