[jira] [Updated] (SPARK-25603) Nested Column Pruning

[ https://issues.apache.org/jira/browse/SPARK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25603: Summary: Nested Column Pruning (was: Project Pushdown for Nested fields through various operators) > Nes

[jira] [Commented] (SPARK-27026) Upgrade Docker image for release build to Ubuntu 18.04

[ https://issues.apache.org/jira/browse/SPARK-27026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792390#comment-16792390 ] DB Tsai commented on SPARK-27026: - Issue resolved by pull request 23932 https://github.c

[jira] [Resolved] (SPARK-27026) Upgrade Docker image for release build to Ubuntu 18.04

[ https://issues.apache.org/jira/browse/SPARK-27026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-27026. - Resolution: Resolved > Upgrade Docker image for release build to Ubuntu 18.04 >

[jira] [Assigned] (SPARK-27026) Upgrade Docker image for release build to Ubuntu 18.04

[ https://issues.apache.org/jira/browse/SPARK-27026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-27026: --- Assignee: DB Tsai > Upgrade Docker image for release build to Ubuntu 18.04 > --

[jira] [Commented] (SPARK-27122) YARN test failures in Java 9+

[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792355#comment-16792355 ] Ajith S commented on SPARK-27122: - ping [~srowen] [~dongjoon] [~Gengliang.Wang] > YARN

[jira] [Commented] (SPARK-27156) why is the "http://:18080/static" browse able?

[ https://issues.apache.org/jira/browse/SPARK-27156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792331#comment-16792331 ] Shivu Sondur commented on SPARK-27156: -- i am working on it > why is the "http://:1

[jira] [Commented] (SPARK-27157) Add Executor metrics to monitoring doc

[ https://issues.apache.org/jira/browse/SPARK-27157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792327#comment-16792327 ] Lantao Jin commented on SPARK-27157: I am going to prepare a PR > Add Executor metr

[jira] [Updated] (SPARK-27157) Add Executor metrics to monitoring doc

[ https://issues.apache.org/jira/browse/SPARK-27157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-27157: --- Issue Type: Sub-task (was: Documentation) Parent: SPARK-23206 > Add Executor metrics to mon

[jira] [Created] (SPARK-27157) Add Executor metrics to monitoring doc

Lantao Jin created SPARK-27157: -- Summary: Add Executor metrics to monitoring doc Key: SPARK-27157 URL: https://issues.apache.org/jira/browse/SPARK-27157 Project: Spark Issue Type: Documentation

[jira] [Comment Edited] (SPARK-27122) YARN test failures in Java 9+

[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792318#comment-16792318 ] Ajith S edited comment on SPARK-27122 at 3/14/19 4:15 AM: -- The

[jira] [Comment Edited] (SPARK-27122) YARN test failures in Java 9+

[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792318#comment-16792318 ] Ajith S edited comment on SPARK-27122 at 3/14/19 4:14 AM: -- The

[jira] [Comment Edited] (SPARK-27122) YARN test failures in Java 9+

[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792318#comment-16792318 ] Ajith S edited comment on SPARK-27122 at 3/14/19 4:07 AM: -- The

[jira] [Comment Edited] (SPARK-27122) YARN test failures in Java 9+

[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792318#comment-16792318 ] Ajith S edited comment on SPARK-27122 at 3/14/19 4:06 AM: -- The

[jira] [Updated] (SPARK-27122) YARN test failures in Java 9+

[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27122: Attachment: image-2019-03-14-09-35-23-046.png > YARN test failures in Java 9+ > --

[jira] [Commented] (SPARK-27122) YARN test failures in Java 9+

[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792318#comment-16792318 ] Ajith S commented on SPARK-27122: - The problem seems to be shading of jetty package. Wh

[jira] [Updated] (SPARK-27122) YARN test failures in Java 9+

[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27122: Attachment: image-2019-03-14-09-34-20-592.png > YARN test failures in Java 9+ > --

[jira] [Updated] (SPARK-27156) why is the "http://:18080/static" browse able?

[ https://issues.apache.org/jira/browse/SPARK-27156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27156: - Description: I would like to know is there a way to disable spark history server /static folder

[jira] [Updated] (SPARK-27156) why is the "http://:18080/static" browse able?

[ https://issues.apache.org/jira/browse/SPARK-27156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27156: - Attachment: Screen Shot 2019-03-14 at 11.46.31 AM.png > why is the "http://:18080/static" browse

[jira] [Updated] (SPARK-27156) why is the "http://:18080/static" browse able?

[ https://issues.apache.org/jira/browse/SPARK-27156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27156: - Description: I would like to know is there a way to disable spark history server /static folder

[jira] [Updated] (SPARK-27156) why is the "http://:18080/static" browse able?

[ https://issues.apache.org/jira/browse/SPARK-27156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27156: - Description: I would like to know is there a way to disable spark history server /static folder

[jira] [Created] (SPARK-27156) why is the "http://:18080/static" browse able?

Jerry Garcia created SPARK-27156: Summary: why is the "http://:18080/static" browse able? Key: SPARK-27156 URL: https://issues.apache.org/jira/browse/SPARK-27156 Project: Spark Issue Type: Q

[jira] [Commented] (SPARK-27122) YARN test failures in Java 9+

[ https://issues.apache.org/jira/browse/SPARK-27122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792299#comment-16792299 ] Ajith S commented on SPARK-27122: - I can reproduce this issue even in Java8. I would lik

[jira] [Assigned] (SPARK-27155) Docker Oracle XE image docker image has been removed by DockerHub

[ https://issues.apache.org/jira/browse/SPARK-27155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27155: Assignee: (was: Apache Spark) > Docker Oracle XE image docker image has been removed

[jira] [Updated] (SPARK-27155) Docker Oracle XE image docker image has been removed by DockerHub

[ https://issues.apache.org/jira/browse/SPARK-27155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu, Lipeng updated SPARK-27155: Description: Since 2019-Feb-13(the Valentine's day eve) this docker image has been removed by Doc

[jira] [Commented] (SPARK-27155) Docker Oracle XE image docker image has been removed by DockerHub

[ https://issues.apache.org/jira/browse/SPARK-27155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792292#comment-16792292 ] Apache Spark commented on SPARK-27155: -- User 'lipzhu' has created a pull request fo

[jira] [Commented] (SPARK-27155) Docker Oracle XE image docker image has been removed by DockerHub

[ https://issues.apache.org/jira/browse/SPARK-27155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792293#comment-16792293 ] Apache Spark commented on SPARK-27155: -- User 'lipzhu' has created a pull request fo

[jira] [Assigned] (SPARK-27155) Docker Oracle XE image docker image has been removed by DockerHub

[ https://issues.apache.org/jira/browse/SPARK-27155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27155: Assignee: Apache Spark > Docker Oracle XE image docker image has been removed by DockerHu

[jira] [Updated] (SPARK-27155) Docker Oracle XE image docker image has been removed by DockerHub

[ https://issues.apache.org/jira/browse/SPARK-27155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu, Lipeng updated SPARK-27155: Attachment: image-2019-03-14-11-00-05-498.png > Docker Oracle XE image docker image has been remov

[jira] [Created] (SPARK-27155) Docker Oracle XE image docker image has been removed by DockerHub

Zhu, Lipeng created SPARK-27155: --- Summary: Docker Oracle XE image docker image has been removed by DockerHub Key: SPARK-27155 URL: https://issues.apache.org/jira/browse/SPARK-27155 Project: Spark

[jira] [Assigned] (SPARK-26555) Thread safety issue causes createDataset to fail with misleading errors

[ https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26555: Assignee: Apache Spark > Thread safety issue causes createDataset to fail with misleading

[jira] [Assigned] (SPARK-26555) Thread safety issue causes createDataset to fail with misleading errors

[ https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26555: Assignee: (was: Apache Spark) > Thread safety issue causes createDataset to fail with

[jira] [Updated] (SPARK-27151) ClearCacheCommand extends IgnoreCahedData to avoid plan node copys

[ https://issues.apache.org/jira/browse/SPARK-27151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-27151: - Description: In SPARK-27011, we introduced IgnoreCahedData to avoid plan node copys in

[jira] [Updated] (SPARK-27151) ClearCacheCommand extends IgnoreCahedData to avoid plan node copys

[ https://issues.apache.org/jira/browse/SPARK-27151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-27151: - Summary: ClearCacheCommand extends IgnoreCahedData to avoid plan node copys (was: Clear

[jira] [Commented] (SPARK-27152) Column equality does not work for aliased columns.

[ https://issues.apache.org/jira/browse/SPARK-27152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792248#comment-16792248 ] Hyukjin Kwon commented on SPARK-27152: -- What does this matter? > Column equality d

[jira] [Created] (SPARK-27154) Incomplete Execution for Spark/dev/run-test-jenkins.py

Vaibhavd created SPARK-27154: Summary: Incomplete Execution for Spark/dev/run-test-jenkins.py Key: SPARK-27154 URL: https://issues.apache.org/jira/browse/SPARK-27154 Project: Spark Issue Type: Qu

[jira] [Resolved] (SPARK-23264) Support interval values without INTERVAL clauses

[ https://issues.apache.org/jira/browse/SPARK-23264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-23264. -- Resolution: Fixed Fix Version/s: 3.0.0 Resolved by https://github.com/apache/sp

[jira] [Commented] (SPARK-27093) Honor ParseMode in AvroFileFormat

[ https://issues.apache.org/jira/browse/SPARK-27093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792246#comment-16792246 ] Hyukjin Kwon commented on SPARK-27093: -- Yea, for second case, it sounds {{ignoreCor

[jira] [Commented] (SPARK-26555) Thread safety issue causes createDataset to fail with misleading errors

[ https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792240#comment-16792240 ] Martin Loncaric commented on SPARK-26555: - This is an existing issue with scala:

[jira] [Commented] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection issue

[ https://issues.apache.org/jira/browse/SPARK-25694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792230#comment-16792230 ] Eugene commented on SPARK-25694: [~howardatwork],   I have hit this problem as well, s

[jira] [Commented] (SPARK-27141) Use ConfigEntry for hardcoded configs Yarn

[ https://issues.apache.org/jira/browse/SPARK-27141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792223#comment-16792223 ] wangjiaochun commented on SPARK-27141: -- I'm sorry,it‘s something mistake close this

[jira] [Resolved] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26742. Resolution: Fixed Issue resolved by pull request 24002 [https://github.com/apache/spark/pu

[jira] [Assigned] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26742: -- Assignee: Jiaxin Shan > Bump Kubernetes Client Version to 4.1.2 > ---

[jira] [Closed] (SPARK-15251) Cannot apply PythonUDF to aggregated column

[ https://issues.apache.org/jira/browse/SPARK-15251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Livesey closed SPARK-15251. --- Cannot be reproduced on master, and is now years old. Closing as no longer relevant. > Cannot

[jira] [Commented] (SPARK-26910) Re-release SparkR to CRAN

[ https://issues.apache.org/jira/browse/SPARK-26910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792120#comment-16792120 ] Felix Cheung commented on SPARK-26910: -- 2.3.3 failed. we are waiting for 2.4.1 to b

[jira] [Comment Edited] (SPARK-26555) Thread safety issue causes createDataset to fail with misleading errors

[ https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792086#comment-16792086 ] Martin Loncaric edited comment on SPARK-26555 at 3/13/19 8:26 PM:

[jira] [Commented] (SPARK-26555) Thread safety issue causes createDataset to fail with misleading errors

[ https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792086#comment-16792086 ] Martin Loncaric commented on SPARK-26555: - Update: I have been able to replicate

[jira] [Commented] (SPARK-27123) Improve CollapseProject to handle projects cross limit/repartition/sample

[ https://issues.apache.org/jira/browse/SPARK-27123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791991#comment-16791991 ] Apache Spark commented on SPARK-27123: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-25449) Don't send zero accumulators in heartbeats

[ https://issues.apache.org/jira/browse/SPARK-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791960#comment-16791960 ] Shixiong Zhu commented on SPARK-25449: -- I think this patch actually fixed a bug int

[jira] [Assigned] (SPARK-27153) add weightCol in python RegressionEvaluator

[ https://issues.apache.org/jira/browse/SPARK-27153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27153: Assignee: Apache Spark > add weightCol in python RegressionEvaluator > --

[jira] [Assigned] (SPARK-27153) add weightCol in python RegressionEvaluator

[ https://issues.apache.org/jira/browse/SPARK-27153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27153: Assignee: (was: Apache Spark) > add weightCol in python RegressionEvaluator > ---

[jira] [Commented] (SPARK-20082) Incremental update of LDA model, by adding initialModel as start point

[ https://issues.apache.org/jira/browse/SPARK-20082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791938#comment-16791938 ] yuhao yang commented on SPARK-20082: Yuhao is taking family bonding leave from March

[jira] [Created] (SPARK-27153) add weightCol in python RegressionEvaluator

Huaxin Gao created SPARK-27153: -- Summary: add weightCol in python RegressionEvaluator Key: SPARK-27153 URL: https://issues.apache.org/jira/browse/SPARK-27153 Project: Spark Issue Type: Improveme

[jira] [Assigned] (SPARK-24432) Add support for dynamic resource allocation

[ https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24432: Assignee: (was: Apache Spark) > Add support for dynamic resource allocation > ---

[jira] [Assigned] (SPARK-24432) Add support for dynamic resource allocation

[ https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24432: Assignee: Apache Spark > Add support for dynamic resource allocation > --

[jira] [Commented] (SPARK-20082) Incremental update of LDA model, by adding initialModel as start point

[ https://issues.apache.org/jira/browse/SPARK-20082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791919#comment-16791919 ] Marcellus de Castro Tavares commented on SPARK-20082: - Hi, is this f

[jira] [Resolved] (SPARK-27106) merge CaseInsensitiveStringMap and DataSourceOptions

[ https://issues.apache.org/jira/browse/SPARK-27106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27106. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24025 [https://gith

[jira] [Commented] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791907#comment-16791907 ] Thincrs commented on SPARK-27060: - A user of thincrs has selected this issue. Deadline:

[jira] [Resolved] (SPARK-26103) OutOfMemory error with large query plans

[ https://issues.apache.org/jira/browse/SPARK-26103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26103. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23169 [https:

[jira] [Assigned] (SPARK-26103) OutOfMemory error with large query plans

[ https://issues.apache.org/jira/browse/SPARK-26103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26103: -- Assignee: Dave DeCaprio > OutOfMemory error with large query plans >

[jira] [Created] (SPARK-27152) Column equality does not work for aliased columns.

Ryan Radtke created SPARK-27152: --- Summary: Column equality does not work for aliased columns. Key: SPARK-27152 URL: https://issues.apache.org/jira/browse/SPARK-27152 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791751#comment-16791751 ] Ajith S edited comment on SPARK-26961 at 3/13/19 2:37 PM: -- 1) Y

[jira] [Updated] (SPARK-27142) Provide REST API for SQL level information

[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27142: Description: Currently for Monitoring Spark application SQL information is not available from REST but on

[jira] [Comment Edited] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791751#comment-16791751 ] Ajith S edited comment on SPARK-26961 at 3/13/19 2:32 PM: -- 1) Y

[jira] [Commented] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791739#comment-16791739 ] Sean Owen commented on SPARK-26961: --- When I run your class and print the result of reg

[jira] [Commented] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791751#comment-16791751 ] Ajith S commented on SPARK-26961: - Yes, the registerAsParallelCapable will return true,

[jira] [Updated] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-26961: Attachment: image-2019-03-13-19-53-52-390.png > Found Java-level deadlock in Spark Driver > --

[jira] [Commented] (SPARK-27137) Spark captured variable is null if the code is pasted via :paste

[ https://issues.apache.org/jira/browse/SPARK-27137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791744#comment-16791744 ] Osira Ben commented on SPARK-27137: --- Indeed it works in the current master! Thank you!

[jira] [Updated] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-26961: Attachment: (was: image-2019-03-13-19-51-38-708.png) > Found Java-level deadlock in Spark Driver > ---

[jira] [Updated] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-26961: Attachment: image-2019-03-13-19-51-38-708.png > Found Java-level deadlock in Spark Driver > --

[jira] [Updated] (SPARK-27142) Provide REST API for SQL level information

[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27142: Attachment: image-2019-03-13-19-29-26-896.png > Provide REST API for SQL level information > -

[jira] [Commented] (SPARK-27142) Provide REST API for SQL level information

[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791728#comment-16791728 ] Ajith S commented on SPARK-27142: - Ok apologies for being abstract about this requiremen

[jira] [Updated] (SPARK-27142) Provide REST API for SQL level information

[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27142: Attachment: (was: image-2019-03-13-19-19-27-831.png) > Provide REST API for SQL level information > --

[jira] [Updated] (SPARK-27142) Provide REST API for SQL level information

[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27142: Attachment: (was: image-2019-03-13-19-19-24-951.png) > Provide REST API for SQL level information > --

[jira] [Updated] (SPARK-27142) Provide REST API for SQL level information

[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27142: Attachment: image-2019-03-13-19-19-24-951.png > Provide REST API for SQL level information > -

[jira] [Updated] (SPARK-27142) Provide REST API for SQL level information

[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27142: Attachment: image-2019-03-13-19-19-27-831.png > Provide REST API for SQL level information > -

[jira] [Updated] (SPARK-27150) Scheduling Within an Application : Spark SQL randomly failed on UDF

[ https://issues.apache.org/jira/browse/SPARK-27150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Sean updated SPARK-27150: -- Description: I run this (reduced) following code multiples times under the same exact input files : 

[jira] [Commented] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791712#comment-16791712 ] Ajith S commented on SPARK-26961: - [~srowen] That too will not work Here is my custom c

[jira] [Updated] (SPARK-27150) Scheduling Within an Application : Spark SQL randomly failed on UDF

[ https://issues.apache.org/jira/browse/SPARK-27150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Sean updated SPARK-27150: -- Description: I run this (reduced) following code multiples times under the same exact input files : 

[jira] [Updated] (SPARK-27151) ClearCacheCommand should be case-object to avoid copys

[ https://issues.apache.org/jira/browse/SPARK-27151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-27151: - Priority: Trivial (was: Major) > ClearCacheCommand should be case-object to avoid copys

[jira] [Assigned] (SPARK-27151) ClearCacheCommand should be case-object to avoid copys

[ https://issues.apache.org/jira/browse/SPARK-27151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27151: Assignee: (was: Apache Spark) > ClearCacheCommand should be case-object to avoid copy

[jira] [Assigned] (SPARK-27151) ClearCacheCommand should be case-object to avoid copys

[ https://issues.apache.org/jira/browse/SPARK-27151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27151: Assignee: Apache Spark > ClearCacheCommand should be case-object to avoid copys > ---

[jira] [Updated] (SPARK-27151) ClearCacheCommand should be case-object to avoid copys

[ https://issues.apache.org/jira/browse/SPARK-27151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-27151: - Summary: ClearCacheCommand should be case-object to avoid copys (was: Makes ClearCacheC

[jira] [Created] (SPARK-27151) Makes ClearCacheCommand case-object

Takeshi Yamamuro created SPARK-27151: Summary: Makes ClearCacheCommand case-object Key: SPARK-27151 URL: https://issues.apache.org/jira/browse/SPARK-27151 Project: Spark Issue Type: Impro

[jira] [Comment Edited] (SPARK-11284) ALS produces predictions as floats and should be double

[ https://issues.apache.org/jira/browse/SPARK-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791671#comment-16791671 ] Alex Combessie edited comment on SPARK-11284 at 3/13/19 1:00 PM: -

[jira] [Commented] (SPARK-11284) ALS produces predictions as floats and should be double

[ https://issues.apache.org/jira/browse/SPARK-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791671#comment-16791671 ] Alex Combessie commented on SPARK-11284: Hello everyone, I am still getting a T

[jira] [Updated] (SPARK-27150) Scheduling Within an Application : Spark SQL randomly failed on UDF

[ https://issues.apache.org/jira/browse/SPARK-27150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Sean updated SPARK-27150: -- Affects Version/s: 2.3.2 2.3.3 > Scheduling Within an Application : Spark SQL r

[jira] [Updated] (SPARK-27150) Scheduling Within an Application : Spark SQL randomly failed on UDF

[ https://issues.apache.org/jira/browse/SPARK-27150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Sean updated SPARK-27150: -- Affects Version/s: 2.4.0 > Scheduling Within an Application : Spark SQL randomly failed on UDF > -

[jira] [Commented] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791634#comment-16791634 ] Sean Owen commented on SPARK-26961: --- [~ajithshetty] I see, OK. It could happen in the

[jira] [Updated] (SPARK-27145) Close store after test, in the SQLAppStatusListenerSuite

[ https://issues.apache.org/jira/browse/SPARK-27145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-27145: -- Issue Type: Improvement (was: Bug) > Close store after test, in the SQLAppStatusListenerSuite > -

[jira] [Resolved] (SPARK-27064) create StreamingWrite at the begining of streaming execution

[ https://issues.apache.org/jira/browse/SPARK-27064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27064. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23981 [https://gith

[jira] [Commented] (SPARK-27142) Provide REST API for SQL level information

[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791614#comment-16791614 ] Sean Owen commented on SPARK-27142: --- We have a metrics system for metrics and a SQL ta

[jira] [Commented] (SPARK-27143) Provide REST API for JDBC/ODBC level information

[ https://issues.apache.org/jira/browse/SPARK-27143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791615#comment-16791615 ] Sean Owen commented on SPARK-27143: --- Likewise here, we have a metrics system already.

[jira] [Updated] (SPARK-27150) Scheduling Within an Application : Spark SQL randomly failed on UDF

[ https://issues.apache.org/jira/browse/SPARK-27150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Sean updated SPARK-27150: -- Description: I run this (reduced) following code multiples times under the same exact input files : 

[jira] [Updated] (SPARK-27150) Scheduling Within an Application : Spark SQL randomly failed on UDF

[ https://issues.apache.org/jira/browse/SPARK-27150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Sean updated SPARK-27150: -- Description: I run this (reduced) following code multiples times under the same exact input files : 

[jira] [Updated] (SPARK-27150) Scheduling Within an Application : Spark SQL randomly failed on UDF

[ https://issues.apache.org/jira/browse/SPARK-27150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Sean updated SPARK-27150: -- Description: I run this (reduced) following code multiples times under the same exact input files : 

[jira] [Updated] (SPARK-27144) Explode with structType may throw NPE when the first column's nullable is false while the second column's nullable is true

[ https://issues.apache.org/jira/browse/SPARK-27144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27144: - Description:  Create a dataFrame containing two columns names [weight, animal], the weight's nu

[jira] [Updated] (SPARK-27150) Scheduling Within an Application : Spark SQL randomly failed on UDF

[ https://issues.apache.org/jira/browse/SPARK-27150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Sean updated SPARK-27150: -- Description:   I run this (reduced) following code multiples times under the same exact input files

[jira] [Commented] (SPARK-27146) Add two Yarn Configs according to spark home page configuration Instructions document

[ https://issues.apache.org/jira/browse/SPARK-27146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791587#comment-16791587 ] Hyukjin Kwon commented on SPARK-27146: -- It doesn't necessarily document all the con

[jira] [Updated] (SPARK-27150) Scheduling Within an Application : Spark SQL randomly failed on UDF

[ https://issues.apache.org/jira/browse/SPARK-27150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Sean updated SPARK-27150: -- Description:   I run this (reduced) following code multiples times under the same exact input files

[jira] [Resolved] (SPARK-27144) Explode with structType may throw NPE when the first column's nullable is false while the second column's nullable is true

[ https://issues.apache.org/jira/browse/SPARK-27144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27144. -- Resolution: Cannot Reproduce > Explode with structType may throw NPE when the first column's n

  1   2   >