[jira] [Commented] (SPARK-18584) multiple Spark Thrift Servers running in the same machine throws org.apache.hadoop.security.AccessControlException

[ https://issues.apache.org/jira/browse/SPARK-18584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15697237#comment-15697237 ] tanxinz commented on SPARK-18584: - Two STS ran diffent Queue on yarn etl Spark Thrift Ser

[jira] [Resolved] (SPARK-18583) Fix nullability of InputFileName.

[ https://issues.apache.org/jira/browse/SPARK-18583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18583. - Resolution: Fixed Assignee: Takuya Ueshin Fix Version/s: 2.1.0 > Fix nullability

[jira] [Commented] (SPARK-18584) multiple Spark Thrift Servers running in the same machine throws org.apache.hadoop.security.AccessControlException

[ https://issues.apache.org/jira/browse/SPARK-18584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15697216#comment-15697216 ] tanxinz commented on SPARK-18584: - Different users have different authorizations to acces

[jira] [Commented] (SPARK-18502) Spark does not handle columns that contain backquote (`)

[ https://issues.apache.org/jira/browse/SPARK-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15697210#comment-15697210 ] Takeshi Yamamuro commented on SPARK-18502: -- Please give us a simple query to rep

[jira] [Commented] (SPARK-17251) "ClassCastException: OuterReference cannot be cast to NamedExpression" for correlated subquery on the RHS of an IN operator

[ https://issues.apache.org/jira/browse/SPARK-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15697176#comment-15697176 ] Apache Spark commented on SPARK-17251: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Commented] (SPARK-18591) Replace hash-based aggregates with sort-based ones if inputs already sorted

[ https://issues.apache.org/jira/browse/SPARK-18591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15697111#comment-15697111 ] Takeshi Yamamuro commented on SPARK-18591: -- If it's worth trying this, I'll do.

[jira] [Created] (SPARK-18591) Replace hash-based aggregates with sort-based ones if inputs already sorted

Takeshi Yamamuro created SPARK-18591: Summary: Replace hash-based aggregates with sort-based ones if inputs already sorted Key: SPARK-18591 URL: https://issues.apache.org/jira/browse/SPARK-18591 P

[jira] [Comment Edited] (SPARK-18405) Add yarn-cluster mode support to Spark Thrift Server

[ https://issues.apache.org/jira/browse/SPARK-18405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696978#comment-15696978 ] Jeff Zhang edited comment on SPARK-18405 at 11/26/16 1:01 AM: -

[jira] [Commented] (SPARK-18405) Add yarn-cluster mode support to Spark Thrift Server

[ https://issues.apache.org/jira/browse/SPARK-18405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696978#comment-15696978 ] Jeff Zhang commented on SPARK-18405: I think he mean to launch multiple spark thrift

[jira] [Assigned] (SPARK-18590) R - Include package vignettes and help pages, build source package in Spark distribution

[ https://issues.apache.org/jira/browse/SPARK-18590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18590: Assignee: Felix Cheung (was: Apache Spark) > R - Include package vignettes and help pages

[jira] [Commented] (SPARK-18590) R - Include package vignettes and help pages, build source package in Spark distribution

[ https://issues.apache.org/jira/browse/SPARK-18590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696848#comment-15696848 ] Apache Spark commented on SPARK-18590: -- User 'felixcheung' has created a pull reques

[jira] [Assigned] (SPARK-18590) R - Include package vignettes and help pages, build source package in Spark distribution

[ https://issues.apache.org/jira/browse/SPARK-18590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18590: Assignee: Apache Spark (was: Felix Cheung) > R - Include package vignettes and help pages

[jira] [Updated] (SPARK-18590) R - Include package vignettes and help pages, build source package in Spark distribution

[ https://issues.apache.org/jira/browse/SPARK-18590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-18590: - Description: We should include in Spark distribution the built source package for SparkR. This w

[jira] [Created] (SPARK-18590) R - Include package vignettes and help pages, build source package in Spark distribution

Felix Cheung created SPARK-18590: Summary: R - Include package vignettes and help pages, build source package in Spark distribution Key: SPARK-18590 URL: https://issues.apache.org/jira/browse/SPARK-18590

[jira] [Commented] (SPARK-18589) persist() resolves "java.lang.RuntimeException: Invalid PythonUDF (...), requires attributes from more than one child"

[ https://issues.apache.org/jira/browse/SPARK-18589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696717#comment-15696717 ] Nicholas Chammas commented on SPARK-18589: -- cc [~davies] [~hvanhovell] > persis

[jira] [Created] (SPARK-18589) persist() resolves "java.lang.RuntimeException: Invalid PythonUDF (...), requires attributes from more than one child"

Nicholas Chammas created SPARK-18589: Summary: persist() resolves "java.lang.RuntimeException: Invalid PythonUDF (...), requires attributes from more than one child" Key: SPARK-18589 URL: https://issues.apache

[jira] [Resolved] (SPARK-18436) isin causing SQL syntax error with JDBC

[ https://issues.apache.org/jira/browse/SPARK-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18436. --- Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-18527) UDAFPercentile (bigint, array) needs explicity cast to double

[ https://issues.apache.org/jira/browse/SPARK-18527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696598#comment-15696598 ] Thomas Sebastian commented on SPARK-18527: -- I am interested to work on this. >

[jira] [Commented] (SPARK-18543) SaveAsTable(CTAS) using overwrite could change table definition

[ https://issues.apache.org/jira/browse/SPARK-18543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696586#comment-15696586 ] Thomas Sebastian commented on SPARK-18543: -- I would like to take a look at this

[jira] [Commented] (SPARK-18220) ClassCastException occurs when using select query on ORC file

[ https://issues.apache.org/jira/browse/SPARK-18220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696541#comment-15696541 ] Herman van Hovell commented on SPARK-18220: --- I tried reproducing this but to no

[jira] [Commented] (SPARK-18487) Add task completion listener to HashAggregate to avoid memory leak

[ https://issues.apache.org/jira/browse/SPARK-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696458#comment-15696458 ] Reynold Xin commented on SPARK-18487: - As discussed on the pull request, this is not

[jira] [Closed] (SPARK-18487) Add task completion listener to HashAggregate to avoid memory leak

[ https://issues.apache.org/jira/browse/SPARK-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-18487. --- Resolution: Not A Problem > Add task completion listener to HashAggregate to avoid memory leak >

[jira] [Commented] (SPARK-3359) `sbt/sbt unidoc` doesn't work with Java 8

[ https://issues.apache.org/jira/browse/SPARK-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696373#comment-15696373 ] Apache Spark commented on SPARK-3359: - User 'HyukjinKwon' has created a pull request f

[jira] [Commented] (SPARK-18407) Inferred partition columns cause assertion error

[ https://issues.apache.org/jira/browse/SPARK-18407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696337#comment-15696337 ] Burak Yavuz commented on SPARK-18407: - This is also resolved as part of https://issu

[jira] [Comment Edited] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696321#comment-15696321 ] Herman van Hovell edited comment on SPARK-17788 at 11/25/16 5:09 PM: --

[jira] [Commented] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696321#comment-15696321 ] Herman van Hovell commented on SPARK-17788: --- That is fair. The solution is not

[jira] [Comment Edited] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696134#comment-15696134 ] Herman van Hovell edited comment on SPARK-17788 at 11/25/16 4:56 PM: --

[jira] [Comment Edited] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696134#comment-15696134 ] Herman van Hovell edited comment on SPARK-17788 at 11/25/16 4:10 PM: --

[jira] [Reopened] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reopened SPARK-17788: - This is somewhat distinct from the join case, but certainly related. > RangePartitioner results in few very

[jira] [Commented] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696188#comment-15696188 ] holdenk commented on SPARK-17788: - I don't think this is a duplicate - its related but a

[jira] [Updated] (SPARK-18220) ClassCastException occurs when using select query on ORC file

[ https://issues.apache.org/jira/browse/SPARK-18220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18220: -- Description: Error message is below. {noformat} ===

[jira] [Commented] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696154#comment-15696154 ] Herman van Hovell commented on SPARK-17788: --- I am closing this one as a duplica

[jira] [Closed] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-17788. - Resolution: Duplicate > RangePartitioner results in few very large tasks and many small t

[jira] [Commented] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696134#comment-15696134 ] Herman van Hovell commented on SPARK-17788: --- Spark makes a sketch of your data

[jira] [Commented] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696056#comment-15696056 ] Hao Ren commented on SPARK-18581: - Thank you for the clarification. I totally missed that

[jira] [Commented] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696030#comment-15696030 ] Sean Owen commented on SPARK-18581: --- The PDF largest at the mean, and it can be > 1 if

[jira] [Closed] (SPARK-6522) Standardize Random Number Generation

[ https://issues.apache.org/jira/browse/SPARK-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk closed SPARK-6522. -- Resolution: Fixed Fix Version/s: 1.1.0 > Standardize Random Number Generation > -

[jira] [Commented] (SPARK-6522) Standardize Random Number Generation

[ https://issues.apache.org/jira/browse/SPARK-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696024#comment-15696024 ] holdenk commented on SPARK-6522: We have a standardized RDD generator in MLlib (see the Ra

[jira] [Commented] (SPARK-5997) Increase partition count without performing a shuffle

[ https://issues.apache.org/jira/browse/SPARK-5997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696016#comment-15696016 ] holdenk commented on SPARK-5997: That could work, although we'd probably want a different

[jira] [Resolved] (SPARK-3348) Support user-defined SparkListeners properly

[ https://issues.apache.org/jira/browse/SPARK-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-3348. Resolution: Duplicate > Support user-defined SparkListeners properly > -

[jira] [Commented] (SPARK-5190) Allow spark listeners to be added before spark context gets initialized.

[ https://issues.apache.org/jira/browse/SPARK-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696003#comment-15696003 ] holdenk commented on SPARK-5190: This seems to be fixed, but we forgot to close (cc [~josh

[jira] [Created] (SPARK-18588) KafkaSourceStressForDontFailOnDataLossSuite is flaky

Herman van Hovell created SPARK-18588: - Summary: KafkaSourceStressForDontFailOnDataLossSuite is flaky Key: SPARK-18588 URL: https://issues.apache.org/jira/browse/SPARK-18588 Project: Spark

[jira] [Commented] (SPARK-18588) KafkaSourceStressForDontFailOnDataLossSuite is flaky

[ https://issues.apache.org/jira/browse/SPARK-18588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695975#comment-15695975 ] Herman van Hovell commented on SPARK-18588: --- cc [~zsxwing] > KafkaSourceStress

[jira] [Commented] (SPARK-636) Add mechanism to run system management/configuration tasks on all workers

[ https://issues.apache.org/jira/browse/SPARK-636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695967#comment-15695967 ] holdenk commented on SPARK-636: --- If you have a logging system you want to initialize wouldn't

[jira] [Commented] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695956#comment-15695956 ] holdenk commented on SPARK-17788: - This is semi-expected behaviour of the range partition

[jira] [Updated] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-17788: Target Version/s: (was: 2.1.0) > RangePartitioner results in few very large tasks and many small to empty

[jira] [Comment Edited] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695942#comment-15695942 ] Hao Ren edited comment on SPARK-18581 at 11/25/16 2:14 PM: --- Aft

[jira] [Comment Edited] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695942#comment-15695942 ] Hao Ren edited comment on SPARK-18581 at 11/25/16 2:09 PM: --- Aft

[jira] [Commented] (SPARK-18554) leader master lost the leadership, when the slave become master, the perivious app's state display as waitting

[ https://issues.apache.org/jira/browse/SPARK-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695947#comment-15695947 ] Saisai Shao commented on SPARK-18554: - Nothing blocked actually, just no one reviewed

[jira] [Comment Edited] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695942#comment-15695942 ] Hao Ren edited comment on SPARK-18581 at 11/25/16 2:08 PM: --- Aft

[jira] [Comment Edited] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695942#comment-15695942 ] Hao Ren edited comment on SPARK-18581 at 11/25/16 2:07 PM: --- Aft

[jira] [Comment Edited] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695942#comment-15695942 ] Hao Ren edited comment on SPARK-18581 at 11/25/16 2:06 PM: --- Aft

[jira] [Updated] (SPARK-18108) Partition discovery fails with explicitly written long partitions

[ https://issues.apache.org/jira/browse/SPARK-18108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-18108: Component/s: (was: Spark Core) SQL > Partition discovery fails with explicitly written

[jira] [Comment Edited] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695942#comment-15695942 ] Hao Ren edited comment on SPARK-18581 at 11/25/16 2:05 PM: --- Aft

[jira] [Comment Edited] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695942#comment-15695942 ] Hao Ren edited comment on SPARK-18581 at 11/25/16 2:04 PM: --- Aft

[jira] [Commented] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695942#comment-15695942 ] Hao Ren commented on SPARK-18581: - After reading the code comments, I find it takes into

[jira] [Commented] (SPARK-18128) Add support for publishing to PyPI

[ https://issues.apache.org/jira/browse/SPARK-18128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695941#comment-15695941 ] holdenk commented on SPARK-18128: - Thanks! :) I'll start working on this issue once we st

[jira] [Commented] (SPARK-18128) Add support for publishing to PyPI

[ https://issues.apache.org/jira/browse/SPARK-18128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695939#comment-15695939 ] holdenk commented on SPARK-18128: - Thanks! :) I'll start working on this issue once we st

[jira] [Commented] (SPARK-18405) Add yarn-cluster mode support to Spark Thrift Server

[ https://issues.apache.org/jira/browse/SPARK-18405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695936#comment-15695936 ] holdenk commented on SPARK-18405: - Even in cluster mode you could overwhelm the node runn

[jira] [Updated] (SPARK-18502) Spark does not handle columns that contain backquote (`)

[ https://issues.apache.org/jira/browse/SPARK-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-18502: Component/s: (was: Spark Core) SQL > Spark does not handle columns that contain backqu

[jira] [Updated] (SPARK-18532) Code generation memory issue

[ https://issues.apache.org/jira/browse/SPARK-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-18532: Component/s: (was: Spark Core) SQL > Code generation memory issue > --

[jira] [Commented] (SPARK-18541) Add pyspark.sql.Column.aliasWithMetadata to allow dynamic metadata management in pyspark SQL API

[ https://issues.apache.org/jira/browse/SPARK-18541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695922#comment-15695922 ] holdenk commented on SPARK-18541: - Making it easier for PySpark SQL users to specify meta

[jira] [Updated] (SPARK-18538) Concurrent Fetching DataFrameReader JDBC APIs Do Not Work

[ https://issues.apache.org/jira/browse/SPARK-18538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18538: -- Priority: Blocker (was: Critical) > Concurrent Fetching DataFrameReader JDBC APIs Do N

[jira] [Updated] (SPARK-18356) Issue + Resolution: Kmeans Spark Performances (ML package)

[ https://issues.apache.org/jira/browse/SPARK-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18356: -- Assignee: zakaria hili > Issue + Resolution: Kmeans Spark Performances (ML package) > -

[jira] [Resolved] (SPARK-18356) Issue + Resolution: Kmeans Spark Performances (ML package)

[ https://issues.apache.org/jira/browse/SPARK-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18356. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 15965 [https://github.co

[jira] [Resolved] (SPARK-18565) subtractByKey modifes values in the source RDD

[ https://issues.apache.org/jira/browse/SPARK-18565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Dzhus resolved SPARK-18565. -- Resolution: Invalid > subtractByKey modifes values in the source RDD >

[jira] [Commented] (SPARK-18565) subtractByKey modifes values in the source RDD

[ https://issues.apache.org/jira/browse/SPARK-18565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695809#comment-15695809 ] Dmitry Dzhus commented on SPARK-18565: -- The problem was that I assumed that caching

[jira] [Commented] (SPARK-18512) FileNotFoundException on _temporary directory with Spark Streaming 2.0.1 and S3A

[ https://issues.apache.org/jira/browse/SPARK-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695800#comment-15695800 ] Genmao Yu commented on SPARK-18512: --- [~giuseppe.bonaccorso] How can i reproduce this fa

[jira] [Commented] (SPARK-17251) "ClassCastException: OuterReference cannot be cast to NamedExpression" for correlated subquery on the RHS of an IN operator

[ https://issues.apache.org/jira/browse/SPARK-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695645#comment-15695645 ] Apache Spark commented on SPARK-17251: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-17251) "ClassCastException: OuterReference cannot be cast to NamedExpression" for correlated subquery on the RHS of an IN operator

[ https://issues.apache.org/jira/browse/SPARK-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17251: Assignee: Apache Spark > "ClassCastException: OuterReference cannot be cast to NamedExpres

[jira] [Assigned] (SPARK-17251) "ClassCastException: OuterReference cannot be cast to NamedExpression" for correlated subquery on the RHS of an IN operator

[ https://issues.apache.org/jira/browse/SPARK-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17251: Assignee: (was: Apache Spark) > "ClassCastException: OuterReference cannot be cast to

[jira] [Updated] (SPARK-18584) multiple Spark Thrift Servers running in the same machine throws org.apache.hadoop.security.AccessControlException

[ https://issues.apache.org/jira/browse/SPARK-18584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18584: -- Target Version/s: (was: 2.0.2) Fix Version/s: (was: 2.0.2) > multiple Spark Thrift Servers

[jira] [Updated] (SPARK-3359) `sbt/sbt unidoc` doesn't work with Java 8

[ https://issues.apache.org/jira/browse/SPARK-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3359: - Assignee: Hyukjin Kwon > `sbt/sbt unidoc` doesn't work with Java 8 > -

[jira] [Commented] (SPARK-16151) Make generated params non-final

[ https://issues.apache.org/jira/browse/SPARK-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695624#comment-15695624 ] zhengruifeng commented on SPARK-16151: -- I suggest that we can add a option to make g

[jira] [Commented] (SPARK-16151) Make generated params non-final

[ https://issues.apache.org/jira/browse/SPARK-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695614#comment-15695614 ] zhengruifeng commented on SPARK-16151: -- Some param shoud be made non-final : such as

[jira] [Commented] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695552#comment-15695552 ] Sean Owen commented on SPARK-18581: --- Yes, but it need not be invertible, for the reason

[jira] [Commented] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695481#comment-15695481 ] Hao Ren commented on SPARK-18581: - [~srowen] I have updated the description. The problem

[jira] [Updated] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Ren updated SPARK-18581: Description: When training GaussianMixtureModel, I found some probability much larger than 1. That leads m

[jira] [Commented] (SPARK-18318) ML, Graph 2.1 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-18318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695473#comment-15695473 ] Yanbo Liang commented on SPARK-18318: - Finished reviewing for all classes which were

[jira] [Assigned] (SPARK-18587) Remove handleInvalid from QuantileDiscretizer

[ https://issues.apache.org/jira/browse/SPARK-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang reassigned SPARK-18587: --- Assignee: Yanbo Liang > Remove handleInvalid from QuantileDiscretizer >

[jira] [Updated] (SPARK-18481) ML 2.1 QA: Remove deprecated methods for ML

[ https://issues.apache.org/jira/browse/SPARK-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-18481: Description: Remove deprecated methods for ML. We removed the following public APIs in this JIRA:

[jira] [Updated] (SPARK-18481) ML 2.1 QA: Remove deprecated methods for ML

[ https://issues.apache.org/jira/browse/SPARK-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-18481: Description: Remove deprecated methods for ML. We removed the following public APIs: org.apache.sp

[jira] [Updated] (SPARK-18481) ML 2.1 QA: Remove deprecated methods for ML

[ https://issues.apache.org/jira/browse/SPARK-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-18481: Priority: Major (was: Minor) > ML 2.1 QA: Remove deprecated methods for ML >

[jira] [Updated] (SPARK-18481) ML 2.1 QA: Remove deprecated methods for ML

[ https://issues.apache.org/jira/browse/SPARK-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-18481: Priority: Minor (was: Major) > ML 2.1 QA: Remove deprecated methods for ML >

[jira] [Updated] (SPARK-18481) ML 2.1 QA: Remove deprecated methods for ML

[ https://issues.apache.org/jira/browse/SPARK-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-18481: Description: Remove deprecated methods for ML. We removed was:Remove deprecated methods for ML

[jira] [Resolved] (SPARK-18548) OnlineLDAOptimizer reads the same broadcast data after deletion

[ https://issues.apache.org/jira/browse/SPARK-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18548. --- Resolution: Duplicate > OnlineLDAOptimizer reads the same broadcast data after deletion > ---

[jira] [Updated] (SPARK-18586) netty-3.8.0.Final.jar has vulnerability CVE-2014-3488 and CVE-2014-0193

[ https://issues.apache.org/jira/browse/SPARK-18586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18586: -- Priority: Major (was: Critical) Spark doesn't use netty 3, but it is pulled in as a transitive depende

[jira] [Updated] (SPARK-18119) Namenode safemode check is only performed on one namenode which can stuck the startup of SparkHistory server

[ https://issues.apache.org/jira/browse/SPARK-18119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18119: -- Assignee: Nicolas Fraison > Namenode safemode check is only performed on one namenode which can stuck t

[jira] [Resolved] (SPARK-18119) Namenode safemode check is only performed on one namenode which can stuck the startup of SparkHistory server

[ https://issues.apache.org/jira/browse/SPARK-18119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18119. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15648 [https://github.co

[jira] [Assigned] (SPARK-18324) ML, Graph 2.1 QA: Programming guide update and migration guide

[ https://issues.apache.org/jira/browse/SPARK-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang reassigned SPARK-18324: --- Assignee: Yanbo Liang > ML, Graph 2.1 QA: Programming guide update and migration guide > ---

[jira] [Resolved] (SPARK-18554) leader master lost the leadership, when the slave become master, the perivious app's state display as waitting

[ https://issues.apache.org/jira/browse/SPARK-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18554. --- Resolution: Duplicate [~jerryshao] is anything blocking you from proceeding on that PR? > leader mas

[jira] [Commented] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695359#comment-15695359 ] Sean Owen commented on SPARK-18581: --- It sounds like there's definitely a problem here b

[jira] [Updated] (SPARK-18575) Keep same style: adjust the position of driver log links

[ https://issues.apache.org/jira/browse/SPARK-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18575: -- Assignee: Genmao Yu Target Version/s: (was: 2.0.3, 2.1.0) Priority: Trivial (

[jira] [Resolved] (SPARK-18575) Keep same style: adjust the position of driver log links

[ https://issues.apache.org/jira/browse/SPARK-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18575. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 16001 [https://github.co

[jira] [Updated] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Ren updated SPARK-18581: Description: When training GaussianMixtureModel, I found some probability much larger than 1. That leads m

[jira] [Updated] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Ren updated SPARK-18581: Description: When training GaussianMixtureModel, I found some probability much larger than 1. That leads m

[jira] [Updated] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Ren updated SPARK-18581: Description: When training GaussianMixtureModel, I found some probability much larger than 1. That leads m

[jira] [Updated] (SPARK-18581) MultivariateGaussian does not check if covariance matrix is invertible

[ https://issues.apache.org/jira/browse/SPARK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Ren updated SPARK-18581: Summary: MultivariateGaussian does not check if covariance matrix is invertible (was: MultivariateGaussian

[jira] [Assigned] (SPARK-18587) Remove handleInvalid from QuantileDiscretizer

[ https://issues.apache.org/jira/browse/SPARK-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18587: Assignee: Apache Spark > Remove handleInvalid from QuantileDiscretizer > -

[jira] [Commented] (SPARK-18587) Remove handleInvalid from QuantileDiscretizer

[ https://issues.apache.org/jira/browse/SPARK-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695291#comment-15695291 ] Apache Spark commented on SPARK-18587: -- User 'yanboliang' has created a pull request

  1   2   >