[jira] [Commented] (SPARK-22974) CountVectorModel does not attach attributes to output column

2018-01-18 Thread Xiayun Sun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330235#comment-16330235 ] Xiayun Sun commented on SPARK-22974: do we define the expected behavior here as Inter

[jira] [Updated] (SPARK-23144) Add console sink for continuous queries

2018-01-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-23144: -- Issue Type: Improvement (was: Bug) > Add console sink for continuous queries > ---

[jira] [Commented] (SPARK-23130) Spark Thrift does not clean-up temporary files (/tmp/*_resources and /tmp/hive/*.pipeout)

2018-01-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330269#comment-16330269 ] Marco Gaido commented on SPARK-23130: - [~seano] there is no JIRA for the pipeout issu

[jira] [Created] (SPARK-23144) Add console sink for continuous queries

2018-01-18 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-23144: - Summary: Add console sink for continuous queries Key: SPARK-23144 URL: https://issues.apache.org/jira/browse/SPARK-23144 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-23144) Add console sink for continuous queries

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23144: Assignee: Apache Spark (was: Tathagata Das) > Add console sink for continuous queries > -

[jira] [Commented] (SPARK-23144) Add console sink for continuous queries

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330276#comment-16330276 ] Apache Spark commented on SPARK-23144: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-23144) Add console sink for continuous queries

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23144: Assignee: Tathagata Das (was: Apache Spark) > Add console sink for continuous queries > -

[jira] [Assigned] (SPARK-22974) CountVectorModel does not attach attributes to output column

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22974: Assignee: Apache Spark > CountVectorModel does not attach attributes to output column > --

[jira] [Assigned] (SPARK-22974) CountVectorModel does not attach attributes to output column

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22974: Assignee: (was: Apache Spark) > CountVectorModel does not attach attributes to output

[jira] [Commented] (SPARK-22974) CountVectorModel does not attach attributes to output column

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330292#comment-16330292 ] Apache Spark commented on SPARK-22974: -- User 'viirya' has created a pull request for

[jira] [Commented] (SPARK-23016) Spark UI access and documentation

2018-01-18 Thread Eric Charles (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330309#comment-16330309 ] Eric Charles commented on SPARK-23016: -- Side note : the Spark UI can not be browsed

[jira] [Created] (SPARK-23145) How to set Apache Spark Executor memory

2018-01-18 Thread Azharuddin (JIRA)
Azharuddin created SPARK-23145: -- Summary: How to set Apache Spark Executor memory Key: SPARK-23145 URL: https://issues.apache.org/jira/browse/SPARK-23145 Project: Spark Issue Type: Question

[jira] [Updated] (SPARK-23145) How to set Apache Spark Executor memory

2018-01-18 Thread Azharuddin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Azharuddin updated SPARK-23145: --- Description: How can I increase the memory available for Apache spark executor nodes? I have a 2 GB

[jira] [Updated] (SPARK-23145) How to set Apache Spark Executor memory

2018-01-18 Thread Azharuddin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Azharuddin updated SPARK-23145: --- Description: How can I increase the memory available for Apache spark executor nodes? I have a 2 GB

[jira] [Resolved] (SPARK-23145) How to set Apache Spark Executor memory

2018-01-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23145. --- Resolution: Invalid Fix Version/s: (was: 2.1.1) Please post these to the mailing list or S

[jira] [Commented] (SPARK-23104) Document that kubernetes is still "experimental"

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330381#comment-16330381 ] Apache Spark commented on SPARK-23104: -- User 'foxish' has created a pull request for

[jira] [Assigned] (SPARK-23104) Document that kubernetes is still "experimental"

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23104: Assignee: Anirudh Ramanathan (was: Apache Spark) > Document that kubernetes is still "exp

[jira] [Assigned] (SPARK-23104) Document that kubernetes is still "experimental"

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23104: Assignee: Apache Spark (was: Anirudh Ramanathan) > Document that kubernetes is still "exp

[jira] [Created] (SPARK-23146) Support client mode for Kubernetes cluster backend

2018-01-18 Thread Anirudh Ramanathan (JIRA)
Anirudh Ramanathan created SPARK-23146: -- Summary: Support client mode for Kubernetes cluster backend Key: SPARK-23146 URL: https://issues.apache.org/jira/browse/SPARK-23146 Project: Spark

[jira] [Updated] (SPARK-23146) Support client mode for Kubernetes cluster backend

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-23146: --- Issue Type: New Feature (was: Improvement) > Support client mode for Kubernetes clus

[jira] [Commented] (SPARK-23016) Spark UI access and documentation

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330389#comment-16330389 ] Anirudh Ramanathan commented on SPARK-23016: Good point - yeah, we don't reco

[jira] [Assigned] (SPARK-23140) DataSourceV2Strategy is missing in HiveSessionStateBuilder

2018-01-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23140: --- Assignee: Saisai Shao > DataSourceV2Strategy is missing in HiveSessionStateBuilder > ---

[jira] [Resolved] (SPARK-23140) DataSourceV2Strategy is missing in HiveSessionStateBuilder

2018-01-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23140. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20305 [https://githu

[jira] [Updated] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23147: Attachment: Screen Shot 2018-01-18 at 8.50.08 PM.png > Stage page will throw exception when there's

[jira] [Created] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-23147: --- Summary: Stage page will throw exception when there's no complete tasks Key: SPARK-23147 URL: https://issues.apache.org/jira/browse/SPARK-23147 Project: Spark

[jira] [Updated] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23147: Description: Current Stage page's task table will throw an exception when there's no complete task

[jira] [Created] (SPARK-23148) spark.read.csv with multiline=true gives FileNotFoundException if path contains spaces

2018-01-18 Thread Bogdan Raducanu (JIRA)
Bogdan Raducanu created SPARK-23148: --- Summary: spark.read.csv with multiline=true gives FileNotFoundException if path contains spaces Key: SPARK-23148 URL: https://issues.apache.org/jira/browse/SPARK-23148

[jira] [Assigned] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23147: Assignee: (was: Apache Spark) > Stage page will throw exception when there's no comple

[jira] [Assigned] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23147: Assignee: Apache Spark > Stage page will throw exception when there's no complete tasks >

[jira] [Commented] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330496#comment-16330496 ] Apache Spark commented on SPARK-23147: -- User 'jerryshao' has created a pull request

[jira] [Assigned] (SPARK-22036) BigDecimal multiplication sometimes returns null

2018-01-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22036: --- Assignee: Marco Gaido > BigDecimal multiplication sometimes returns null > -

[jira] [Resolved] (SPARK-22036) BigDecimal multiplication sometimes returns null

2018-01-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22036. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20023 [https://githu

[jira] [Assigned] (SPARK-23141) Support data type string as a returnType for registerJavaFunction.

2018-01-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23141: Assignee: Takuya Ueshin > Support data type string as a returnType for registerJavaFunctio

[jira] [Resolved] (SPARK-23141) Support data type string as a returnType for registerJavaFunction.

2018-01-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23141. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20307 [https://git

[jira] [Commented] (SPARK-19280) Failed Recovery from checkpoint caused by the multi-threads issue in Spark Streaming scheduler

2018-01-18 Thread Riccardo Vincelli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330558#comment-16330558 ] Riccardo Vincelli commented on SPARK-19280: --- What is the exception you are gett

[jira] [Comment Edited] (SPARK-19280) Failed Recovery from checkpoint caused by the multi-threads issue in Spark Streaming scheduler

2018-01-18 Thread Riccardo Vincelli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330558#comment-16330558 ] Riccardo Vincelli edited comment on SPARK-19280 at 1/18/18 2:20 PM: ---

[jira] [Created] (SPARK-23149) polish ColumnarBatch

2018-01-18 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-23149: --- Summary: polish ColumnarBatch Key: SPARK-23149 URL: https://issues.apache.org/jira/browse/SPARK-23149 Project: Spark Issue Type: Improvement Componen

[jira] [Commented] (SPARK-23149) polish ColumnarBatch

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330610#comment-16330610 ] Apache Spark commented on SPARK-23149: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-23149) polish ColumnarBatch

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23149: Assignee: Apache Spark (was: Wenchen Fan) > polish ColumnarBatch > >

[jira] [Assigned] (SPARK-23149) polish ColumnarBatch

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23149: Assignee: Wenchen Fan (was: Apache Spark) > polish ColumnarBatch > >

[jira] [Created] (SPARK-23150) kafka job failing assertion failed: Beginning offset 5451 is after the ending offset 5435 for topic

2018-01-18 Thread Balu (JIRA)
Balu created SPARK-23150: Summary: kafka job failing assertion failed: Beginning offset 5451 is after the ending offset 5435 for topic Key: SPARK-23150 URL: https://issues.apache.org/jira/browse/SPARK-23150 P

[jira] [Resolved] (SPARK-23150) kafka job failing assertion failed: Beginning offset 5451 is after the ending offset 5435 for topic

2018-01-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23150. --- Resolution: Invalid > kafka job failing assertion failed: Beginning offset 5451 is after the ending

[jira] [Created] (SPARK-23151) Provide a distribution Spark with Hadoop 3.0

2018-01-18 Thread Louis Burke (JIRA)
Louis Burke created SPARK-23151: --- Summary: Provide a distribution Spark with Hadoop 3.0 Key: SPARK-23151 URL: https://issues.apache.org/jira/browse/SPARK-23151 Project: Spark Issue Type: Depend

[jira] [Updated] (SPARK-23151) Provide a distribution of Spark with Hadoop 3.0

2018-01-18 Thread Louis Burke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Louis Burke updated SPARK-23151: Summary: Provide a distribution of Spark with Hadoop 3.0 (was: Provide a distribution Spark with H

[jira] [Resolved] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23147. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20315 [https:/

[jira] [Assigned] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23147: -- Assignee: Saisai Shao > Stage page will throw exception when there's no complete tasks

[jira] [Commented] (SPARK-22982) Remove unsafe asynchronous close() call from FileDownloadChannel

2018-01-18 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330943#comment-16330943 ] Andrew Ash commented on SPARK-22982: [~joshrosen] do you have some example stacktrace

[jira] [Issue Comment Deleted] (SPARK-13572) HiveContext reads avro Hive tables incorrectly

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-13572: --- Comment: was deleted (was: Guten Tag, Bonjour, Buongiorno Besten Dank für Ihr E-Mail. Ich bin

[jira] [Resolved] (SPARK-23029) Doc spark.shuffle.file.buffer units are kb when no units specified

2018-01-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23029. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20269 [https://github.co

[jira] [Assigned] (SPARK-23029) Doc spark.shuffle.file.buffer units are kb when no units specified

2018-01-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-23029: - Assignee: Fernando Pereira > Doc spark.shuffle.file.buffer units are kb when no units specified

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends "org.apache.spark.ml.classification.Clas

[jira] [Created] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
Matthew Tovbin created SPARK-23152: -- Summary: Invalid guard condition in org.apache.spark.ml.classification.Classifier Key: SPARK-23152 URL: https://issues.apache.org/jira/browse/SPARK-23152 Project:

[jira] [Commented] (SPARK-20928) SPIP: Continuous Processing Mode for Structured Streaming

2018-01-18 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331045#comment-16331045 ] Thomas Graves commented on SPARK-20928: --- what is status of this, it looks like subt

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends "org.apache.spark.ml.classification.Clas

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends "org.apache.spark.ml.classification.Clas

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends "org.apache.spark.ml.classification.Clas

[jira] [Commented] (SPARK-23133) Spark options are not passed to the Executor in Docker context

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331074#comment-16331074 ] Anirudh Ramanathan commented on SPARK-23133: Thanks for submitting the PR fix

[jira] [Commented] (SPARK-23082) Allow separate node selectors for driver and executors in Kubernetes

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331081#comment-16331081 ] Anirudh Ramanathan commented on SPARK-23082: This is an interesting feature r

[jira] [Comment Edited] (SPARK-23082) Allow separate node selectors for driver and executors in Kubernetes

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331081#comment-16331081 ] Anirudh Ramanathan edited comment on SPARK-23082 at 1/18/18 7:52 PM: --

[jira] [Resolved] (SPARK-23016) Spark UI access and documentation

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan resolved SPARK-23016. Resolution: Fixed This is resolved and we've verified it for 2.3.0. > Spark UI acc

[jira] [Commented] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331085#comment-16331085 ] Anirudh Ramanathan commented on SPARK-22962: This is the resource staging ser

[jira] [Updated] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-22962: --- Affects Version/s: (was: 2.3.0) 2.4.0 > Kubernetes app fai

[jira] [Updated] (SPARK-23146) Support client mode for Kubernetes cluster backend

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-23146: --- Affects Version/s: (was: 2.4.0) 2.3.0 > Support client mod

[jira] [Updated] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-22962: --- Affects Version/s: (was: 2.4.0) 2.3.0 > Kubernetes app fai

[jira] [Updated] (SPARK-23146) Support client mode for Kubernetes cluster backend

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-23146: --- Target Version/s: 2.4.0 > Support client mode for Kubernetes cluster backend > --

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends "org.apache.spark.ml.classification.Clas

[jira] [Commented] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331132#comment-16331132 ] Yinan Li commented on SPARK-22962: -- I agree that before we upstream the staging server,

[jira] [Commented] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331140#comment-16331140 ] Marcelo Vanzin commented on SPARK-22962: Yes send a PR. > Kubernetes app fails i

[jira] [Commented] (SPARK-20928) SPIP: Continuous Processing Mode for Structured Streaming

2018-01-18 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331141#comment-16331141 ] Michael Armbrust commented on SPARK-20928: -- There is more work to do so I might

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends "org.apache.spark.ml.classification.Clas

[jira] [Resolved] (SPARK-23143) Add Python support for continuous trigger

2018-01-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23143. --- Resolution: Fixed Fix Version/s: 2.3.0 3.0.0 Issue resolved by pull

[jira] [Commented] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331153#comment-16331153 ] Anirudh Ramanathan commented on SPARK-22962: I think this isn't super critica

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends "org.apache.spark.ml.classification.Clas

[jira] [Commented] (SPARK-22884) ML test for StructuredStreaming: spark.ml.clustering

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331157#comment-16331157 ] Apache Spark commented on SPARK-22884: -- User 'smurakozi' has created a pull request

[jira] [Assigned] (SPARK-22884) ML test for StructuredStreaming: spark.ml.clustering

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22884: Assignee: Apache Spark > ML test for StructuredStreaming: spark.ml.clustering > --

[jira] [Assigned] (SPARK-22884) ML test for StructuredStreaming: spark.ml.clustering

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22884: Assignee: (was: Apache Spark) > ML test for StructuredStreaming: spark.ml.clustering >

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends "org.apache.spark.ml.classification.Clas

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends "org.apache.spark.ml.classification.Clas

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends "org.apache.spark.ml.classification.Clas

[jira] [Resolved] (SPARK-23144) Add console sink for continuous queries

2018-01-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23144. --- Resolution: Fixed Fix Version/s: 2.3.0 3.0.0 Issue resolved by pull

[jira] [Commented] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331185#comment-16331185 ] Apache Spark commented on SPARK-22962: -- User 'liyinan926' has created a pull request

[jira] [Assigned] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22962: Assignee: Apache Spark > Kubernetes app fails if local files are used > --

[jira] [Assigned] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22962: Assignee: (was: Apache Spark) > Kubernetes app fails if local files are used > ---

[jira] [Commented] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331194#comment-16331194 ] Apache Spark commented on SPARK-23152: -- User 'tovbinm' has created a pull request fo

[jira] [Assigned] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23152: Assignee: (was: Apache Spark) > Invalid guard condition in org.apache.spark.ml.classif

[jira] [Assigned] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23152: Assignee: Apache Spark > Invalid guard condition in org.apache.spark.ml.classification.Cla

[jira] [Assigned] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-19185: -- Assignee: (was: Marcelo Vanzin) > ConcurrentModificationExceptions with CachedKafk

[jira] [Assigned] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-19185: -- Assignee: Marcelo Vanzin > ConcurrentModificationExceptions with CachedKafkaConsumers

[jira] [Commented] (SPARK-23133) Spark options are not passed to the Executor in Docker context

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331241#comment-16331241 ] Apache Spark commented on SPARK-23133: -- User 'foxish' has created a pull request for

[jira] [Created] (SPARK-23153) Support application dependencies in submission client's local file system

2018-01-18 Thread Yinan Li (JIRA)
Yinan Li created SPARK-23153: Summary: Support application dependencies in submission client's local file system Key: SPARK-23153 URL: https://issues.apache.org/jira/browse/SPARK-23153 Project: Spark

[jira] [Resolved] (SPARK-23133) Spark options are not passed to the Executor in Docker context

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23133. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20322 [https:/

[jira] [Assigned] (SPARK-23133) Spark options are not passed to the Executor in Docker context

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23133: -- Assignee: Andrew Korzhuev > Spark options are not passed to the Executor in Docker con

[jira] [Updated] (SPARK-22216) Improving PySpark/Pandas interoperability

2018-01-18 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-22216: --- Description: This is an umbrella ticket tracking the general effort to improve performance and interoperabili

[jira] [Assigned] (SPARK-23094) Json Readers choose wrong encoding when bad records are present and fail

2018-01-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23094: Assignee: Burak Yavuz > Json Readers choose wrong encoding when bad records are present an

[jira] [Resolved] (SPARK-23094) Json Readers choose wrong encoding when bad records are present and fail

2018-01-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23094. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20302 [https://git

[jira] [Resolved] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22962. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20320 [https:/

[jira] [Assigned] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-22962: -- Assignee: Yinan Li > Kubernetes app fails if local files are used > --

[jira] [Resolved] (SPARK-23142) Add documentation for Continuous Processing

2018-01-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23142. --- Resolution: Fixed Fix Version/s: 2.3.0 3.0.0 Issue resolved by pull

[jira] [Assigned] (SPARK-23091) Incorrect unit test for approxQuantile

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23091: Assignee: (was: Apache Spark) > Incorrect unit test for approxQuantile > -

[jira] [Assigned] (SPARK-23091) Incorrect unit test for approxQuantile

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23091: Assignee: Apache Spark > Incorrect unit test for approxQuantile >

  1   2   >