[jira] [Commented] (SPARK-5791) [Spark SQL] show poor performance when multiple table do join operation

[ https://issues.apache.org/jira/browse/SPARK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493350#comment-14493350 ] Yi Zhou commented on SPARK-5791: [~yhuai], yes, Both used Parquet. [Spark SQL] show poor

[jira] [Assigned] (SPARK-4638) Spark's MLlib SVM classification to include Kernels like Gaussian / (RBF) to find non linear boundaries

[ https://issues.apache.org/jira/browse/SPARK-4638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4638: --- Assignee: Apache Spark Spark's MLlib SVM classification to include Kernels like Gaussian /

[jira] [Commented] (SPARK-4638) Spark's MLlib SVM classification to include Kernels like Gaussian / (RBF) to find non linear boundaries

[ https://issues.apache.org/jira/browse/SPARK-4638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493347#comment-14493347 ] Apache Spark commented on SPARK-4638: - User 'mandar2812' has created a pull request

[jira] [Assigned] (SPARK-4638) Spark's MLlib SVM classification to include Kernels like Gaussian / (RBF) to find non linear boundaries

[ https://issues.apache.org/jira/browse/SPARK-4638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4638: --- Assignee: (was: Apache Spark) Spark's MLlib SVM classification to include Kernels like

[jira] [Updated] (SPARK-4766) ML Estimator Params should subclass Transformer Params

[ https://issues.apache.org/jira/browse/SPARK-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-4766: - Description: Currently, in spark.ml, both Transformers and Estimators extend the same

[jira] [Created] (SPARK-6892) Recovery from checkpoint will also reuse the application id when write eventLog in yarn-cluster mode

yangping wu created SPARK-6892: -- Summary: Recovery from checkpoint will also reuse the application id when write eventLog in yarn-cluster mode Key: SPARK-6892 URL: https://issues.apache.org/jira/browse/SPARK-6892

[jira] [Created] (SPARK-6893) Better handling of pipeline parameters in PySpark

Xiangrui Meng created SPARK-6893: Summary: Better handling of pipeline parameters in PySpark Key: SPARK-6893 URL: https://issues.apache.org/jira/browse/SPARK-6893 Project: Spark Issue Type:

[jira] [Commented] (SPARK-6511) Publish hadoop provided build with instructions for different distros

[ https://issues.apache.org/jira/browse/SPARK-6511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493585#comment-14493585 ] Kannan Rajah commented on SPARK-6511: - As requested by Patrick, here is an example of

[jira] [Commented] (SPARK-6511) Publish hadoop provided build with instructions for different distros

[ https://issues.apache.org/jira/browse/SPARK-6511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493592#comment-14493592 ] Kannan Rajah commented on SPARK-6511: - [~pwendell] Just wanted to let you know that we

[jira] [Assigned] (SPARK-5924) Add the ability to specify withMean or withStd parameters with StandarScaler

[ https://issues.apache.org/jira/browse/SPARK-5924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5924: --- Assignee: (was: Apache Spark) Add the ability to specify withMean or withStd parameters

[jira] [Commented] (SPARK-6847) Stack overflow on updateStateByKey which followed by a dstream with checkpoint set

[ https://issues.apache.org/jira/browse/SPARK-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493517#comment-14493517 ] Jack Hu commented on SPARK-6847: Here is the part of the stack (Full stack at:

[jira] [Updated] (SPARK-6892) Recovery from checkpoint will also reuse the application id when write eventLog in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangping wu updated SPARK-6892: --- Description: When I recovery from checkpoint in yarn-cluster mode using Spark Streaming, I found

[jira] [Resolved] (SPARK-5957) Better handling of default parameter values.

[ https://issues.apache.org/jira/browse/SPARK-5957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5957. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5431

[jira] [Commented] (SPARK-6884) random forest predict probabilities functionality (like in sklearn)

[ https://issues.apache.org/jira/browse/SPARK-6884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492868#comment-14492868 ] Max Kaznady commented on SPARK-6884: Implemented a prototype, testing mapReduce code.

[jira] [Commented] (SPARK-3727) DecisionTree, RandomForest: More prediction functionality

[ https://issues.apache.org/jira/browse/SPARK-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492871#comment-14492871 ] Max Kaznady commented on SPARK-3727: I thought it would be more fitting to separate

[jira] [Updated] (SPARK-6884) Random forest: predict class probabilities

[ https://issues.apache.org/jira/browse/SPARK-6884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6884: - Summary: Random forest: predict class probabilities (was: random forest predict

[jira] [Updated] (SPARK-6884) random forest predict probabilities functionality (like in sklearn)

[ https://issues.apache.org/jira/browse/SPARK-6884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6884: - Issue Type: Sub-task (was: New Feature) Parent: SPARK-3727 random forest

[jira] [Created] (SPARK-6883) Fork pyspark's cloudpickle as a separate dependency

Kyle Kelley created SPARK-6883: -- Summary: Fork pyspark's cloudpickle as a separate dependency Key: SPARK-6883 URL: https://issues.apache.org/jira/browse/SPARK-6883 Project: Spark Issue Type:

[jira] [Updated] (SPARK-6352) Supporting non-default OutputCommitter when using saveAsParquetFile

[ https://issues.apache.org/jira/browse/SPARK-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-6352: -- Assignee: Pei-Lun Lee Supporting non-default OutputCommitter when using saveAsParquetFile

[jira] [Commented] (SPARK-6352) Supporting non-default OutputCommitter when using saveAsParquetFile

[ https://issues.apache.org/jira/browse/SPARK-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492699#comment-14492699 ] Josh Rosen commented on SPARK-6352: --- [~lian cheng], we can only assign tickets to users

[jira] [Assigned] (SPARK-5888) Add OneHotEncoder as a Transformer

[ https://issues.apache.org/jira/browse/SPARK-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reassigned SPARK-5888: - Assignee: Sandy Ryza Add OneHotEncoder as a Transformer --

[jira] [Resolved] (SPARK-6849) The constructor of GradientDescent should be public

[ https://issues.apache.org/jira/browse/SPARK-6849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6849. -- Resolution: Duplicate Yes, I think this is a subset of opening up optimization APIs The constructor

[jira] [Updated] (SPARK-5632) not able to resolve dot('.') in field name

[ https://issues.apache.org/jira/browse/SPARK-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5632: --- Description: My cassandra table task_trace has a field sm.result which contains dot in the name. So

[jira] [Commented] (SPARK-6847) Stack overflow on updateStateByKey which followed by a dstream with checkpoint set

[ https://issues.apache.org/jira/browse/SPARK-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492126#comment-14492126 ] Sean Owen commented on SPARK-6847: -- Can you provide (the top part of) the stack overflow

[jira] [Updated] (SPARK-6303) Remove unnecessary Average in GeneratedAggregate

[ https://issues.apache.org/jira/browse/SPARK-6303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-6303: --- Summary: Remove unnecessary Average in GeneratedAggregate (was: Average should be in

[jira] [Updated] (SPARK-6303) Average should be in canBeCodeGened list

[ https://issues.apache.org/jira/browse/SPARK-6303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-6303: --- Issue Type: Improvement (was: Bug) Average should be in canBeCodeGened list

[jira] [Updated] (SPARK-6303) Average should be in canBeCodeGened list

[ https://issues.apache.org/jira/browse/SPARK-6303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-6303: --- Priority: Minor (was: Major) Average should be in canBeCodeGened list

[jira] [Commented] (SPARK-6877) Add code generation support for Min

[ https://issues.apache.org/jira/browse/SPARK-6877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492041#comment-14492041 ] Apache Spark commented on SPARK-6877: - User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-6877) Add code generation support for Min

[ https://issues.apache.org/jira/browse/SPARK-6877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6877: --- Assignee: (was: Apache Spark) Add code generation support for Min

[jira] [Created] (SPARK-6877) Add code generation support for Min

Liang-Chi Hsieh created SPARK-6877: -- Summary: Add code generation support for Min Key: SPARK-6877 URL: https://issues.apache.org/jira/browse/SPARK-6877 Project: Spark Issue Type: New

[jira] [Assigned] (SPARK-6877) Add code generation support for Min

[ https://issues.apache.org/jira/browse/SPARK-6877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6877: --- Assignee: Apache Spark Add code generation support for Min

[jira] [Updated] (SPARK-6303) Remove unnecessary Average in GeneratedAggregate

[ https://issues.apache.org/jira/browse/SPARK-6303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-6303: --- Description: Because {{Average}} is a {{PartialAggregate}}, we never get a {{Average}} node

[jira] [Commented] (SPARK-4783) System.exit() calls in SparkContext disrupt applications embedding Spark

[ https://issues.apache.org/jira/browse/SPARK-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491993#comment-14491993 ] Alberto commented on SPARK-4783: Does it mean that you guys are going to create a PR with

[jira] [Assigned] (SPARK-4961) Put HadoopRDD.getPartitions forward to reduce DAGScheduler.JobSubmitted processing time

[ https://issues.apache.org/jira/browse/SPARK-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4961: --- Assignee: (was: Apache Spark) Put HadoopRDD.getPartitions forward to reduce

[jira] [Assigned] (SPARK-4961) Put HadoopRDD.getPartitions forward to reduce DAGScheduler.JobSubmitted processing time

[ https://issues.apache.org/jira/browse/SPARK-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4961: --- Assignee: Apache Spark Put HadoopRDD.getPartitions forward to reduce

[jira] [Updated] (SPARK-6562) DataFrame.na.replace value support in Scala/Java

[ https://issues.apache.org/jira/browse/SPARK-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6562: --- Summary: DataFrame.na.replace value support in Scala/Java (was: DataFrame.na.replace value support)

[jira] [Commented] (SPARK-6682) Deprecate static train and use builder instead for Scala/Java

[ https://issues.apache.org/jira/browse/SPARK-6682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491965#comment-14491965 ] Yu Ishikawa commented on SPARK-6682: [~josephkb] sorry, one more question. Are we

[jira] [Resolved] (SPARK-6868) Container link broken on Spark UI Executors page when YARN is set to HTTPS_ONLY

[ https://issues.apache.org/jira/browse/SPARK-6868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6868. -- Resolution: Fixed Fix Version/s: 1.4.0 1.3.2 Assignee: Dean Chen

[jira] [Updated] (SPARK-6868) Container link broken on Spark UI Executors page when YARN is set to HTTPS_ONLY

[ https://issues.apache.org/jira/browse/SPARK-6868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6868: - Priority: Minor (was: Major) Container link broken on Spark UI Executors page when YARN is set to

[jira] [Updated] (SPARK-6860) Fix the possible inconsistency of StreamingPage

[ https://issues.apache.org/jira/browse/SPARK-6860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6860: - Priority: Minor (was: Major) Assignee: Shixiong Zhu Fix the possible inconsistency of StreamingPage

[jira] [Updated] (SPARK-6870) Catch InterruptedException when yarn application state monitor thread been interrupted

[ https://issues.apache.org/jira/browse/SPARK-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6870: - Priority: Trivial (was: Minor) Assignee: Weizhong Catch InterruptedException when yarn application

[jira] [Commented] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492215#comment-14492215 ] Sean Owen commented on SPARK-1529: -- (Sorry if this double-posts.) Is there a good way to

[jira] [Created] (SPARK-6878) Sum on empty RDD fails with exception

Erik van Oosten created SPARK-6878: -- Summary: Sum on empty RDD fails with exception Key: SPARK-6878 URL: https://issues.apache.org/jira/browse/SPARK-6878 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-6762) Fix potential resource leaks in CheckPoint CheckpointWriter and CheckpointReader

[ https://issues.apache.org/jira/browse/SPARK-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6762. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5407

[jira] [Commented] (SPARK-6878) Sum on empty RDD fails with exception

[ https://issues.apache.org/jira/browse/SPARK-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492302#comment-14492302 ] Erik van Oosten commented on SPARK-6878: Ah, yes. I now see that fold also first

[jira] [Updated] (SPARK-6440) ipv6 URI for HttpServer

[ https://issues.apache.org/jira/browse/SPARK-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6440: - Assignee: Arsenii Krasikov ipv6 URI for HttpServer --- Key:

[jira] [Resolved] (SPARK-6440) ipv6 URI for HttpServer

[ https://issues.apache.org/jira/browse/SPARK-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6440. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5424

[jira] [Resolved] (SPARK-6870) Catch InterruptedException when yarn application state monitor thread been interrupted

[ https://issues.apache.org/jira/browse/SPARK-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6870. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5479

[jira] [Resolved] (SPARK-6738) EstimateSize is difference with spill file size

[ https://issues.apache.org/jira/browse/SPARK-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6738. -- Resolution: Not A Problem We can reopen if there is more detail, but the problem report is focusing on

[jira] [Updated] (SPARK-6762) Fix potential resource leaks in CheckPoint CheckpointWriter and CheckpointReader

[ https://issues.apache.org/jira/browse/SPARK-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6762: - Assignee: zhichao-li Fix potential resource leaks in CheckPoint CheckpointWriter and CheckpointReader

[jira] [Commented] (SPARK-5281) Registering table on RDD is giving MissingRequirementError

[ https://issues.apache.org/jira/browse/SPARK-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492246#comment-14492246 ] Yajun Dong commented on SPARK-5281: --- I also have this isssue with Eclipse Luna and spark

[jira] [Assigned] (SPARK-6800) Reading from JDBC with SQLContext, using lower/upper bounds and numPartitions gives incorrect results.

[ https://issues.apache.org/jira/browse/SPARK-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6800: --- Assignee: Apache Spark Reading from JDBC with SQLContext, using lower/upper bounds and

[jira] [Commented] (SPARK-6800) Reading from JDBC with SQLContext, using lower/upper bounds and numPartitions gives incorrect results.

[ https://issues.apache.org/jira/browse/SPARK-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492244#comment-14492244 ] Apache Spark commented on SPARK-6800: - User 'viirya' has created a pull request for

[jira] [Resolved] (SPARK-6860) Fix the possible inconsistency of StreamingPage

[ https://issues.apache.org/jira/browse/SPARK-6860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6860. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5470

[jira] [Commented] (SPARK-6878) Sum on empty RDD fails with exception

[ https://issues.apache.org/jira/browse/SPARK-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492284#comment-14492284 ] Sean Owen commented on SPARK-6878: -- Yes, and I think it could even be a little simpler by

[jira] [Assigned] (SPARK-6878) Sum on empty RDD fails with exception

[ https://issues.apache.org/jira/browse/SPARK-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6878: --- Assignee: Apache Spark Sum on empty RDD fails with exception

[jira] [Commented] (SPARK-6878) Sum on empty RDD fails with exception

[ https://issues.apache.org/jira/browse/SPARK-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492336#comment-14492336 ] Erik van Oosten commented on SPARK-6878: Pull request:

[jira] [Commented] (SPARK-6878) Sum on empty RDD fails with exception

[ https://issues.apache.org/jira/browse/SPARK-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492271#comment-14492271 ] Sean Owen commented on SPARK-6878: -- Interesting question -- what's the expected sum of

[jira] [Commented] (SPARK-6878) Sum on empty RDD fails with exception

[ https://issues.apache.org/jira/browse/SPARK-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492282#comment-14492282 ] Erik van Oosten commented on SPARK-6878: The answer is only defined because the

[jira] [Assigned] (SPARK-6800) Reading from JDBC with SQLContext, using lower/upper bounds and numPartitions gives incorrect results.

[ https://issues.apache.org/jira/browse/SPARK-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6800: --- Assignee: (was: Apache Spark) Reading from JDBC with SQLContext, using lower/upper

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492303#comment-14492303 ] Steve Loughran commented on SPARK-1537: --- HADOOP-11826 patches the hadoop

[jira] [Resolved] (SPARK-6671) Add status command for spark daemons

[ https://issues.apache.org/jira/browse/SPARK-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6671. -- Resolution: Fixed Issue resolved by pull request 5327 [https://github.com/apache/spark/pull/5327] Add

[jira] [Updated] (SPARK-6671) Add status command for spark daemons

[ https://issues.apache.org/jira/browse/SPARK-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6671: - Priority: Minor (was: Major) Assignee: PRADEEP CHANUMOLU Add status command for spark daemons

[jira] [Created] (SPARK-6879) Check if the app is completed before clean it up

Tao Wang created SPARK-6879: --- Summary: Check if the app is completed before clean it up Key: SPARK-6879 URL: https://issues.apache.org/jira/browse/SPARK-6879 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-6878) Sum on empty RDD fails with exception

[ https://issues.apache.org/jira/browse/SPARK-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik van Oosten updated SPARK-6878: --- Flags: Patch Sum on empty RDD fails with exception -

[jira] [Assigned] (SPARK-6878) Sum on empty RDD fails with exception

[ https://issues.apache.org/jira/browse/SPARK-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6878: --- Assignee: (was: Apache Spark) Sum on empty RDD fails with exception

[jira] [Commented] (SPARK-6878) Sum on empty RDD fails with exception

[ https://issues.apache.org/jira/browse/SPARK-6878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492335#comment-14492335 ] Apache Spark commented on SPARK-6878: - User 'erikvanoosten' has created a pull request

[jira] [Updated] (SPARK-6875) Add support for Joda-time types

[ https://issues.apache.org/jira/browse/SPARK-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Grandjean updated SPARK-6875: - Description: The need comes from the following use case: val objs: RDD[MyClass] = [...]

[jira] [Resolved] (SPARK-6352) Supporting non-default OutputCommitter when using saveAsParquetFile

[ https://issues.apache.org/jira/browse/SPARK-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-6352. --- Resolution: Fixed Fix Version/s: 1.4.0 Target Version/s: 1.4.0 Resolved by

[jira] [Updated] (SPARK-6875) Add support for Joda-time types

[ https://issues.apache.org/jira/browse/SPARK-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Grandjean updated SPARK-6875: - Description: The need comes from the following use case: val objs: RDD[MyClass] = [...]

[jira] [Resolved] (SPARK-6207) YARN secure cluster mode doesn't obtain a hive-metastore token

[ https://issues.apache.org/jira/browse/SPARK-6207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-6207. -- Resolution: Fixed Fix Version/s: 1.4.0 YARN secure cluster mode doesn't obtain a

[jira] [Assigned] (SPARK-5689) Document what can be run in different YARN modes

[ https://issues.apache.org/jira/browse/SPARK-5689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5689: --- Assignee: (was: Apache Spark) Document what can be run in different YARN modes

[jira] [Commented] (SPARK-6879) Check if the app is completed before clean it up

[ https://issues.apache.org/jira/browse/SPARK-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492348#comment-14492348 ] Apache Spark commented on SPARK-6879: - User 'WangTaoTheTonic' has created a pull

[jira] [Assigned] (SPARK-6879) Check if the app is completed before clean it up

[ https://issues.apache.org/jira/browse/SPARK-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6879: --- Assignee: Apache Spark Check if the app is completed before clean it up

[jira] [Assigned] (SPARK-6879) Check if the app is completed before clean it up

[ https://issues.apache.org/jira/browse/SPARK-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6879: --- Assignee: (was: Apache Spark) Check if the app is completed before clean it up

[jira] [Commented] (SPARK-5689) Document what can be run in different YARN modes

[ https://issues.apache.org/jira/browse/SPARK-5689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492349#comment-14492349 ] Apache Spark commented on SPARK-5689: - User 'Sephiroth-Lin' has created a pull request

[jira] [Assigned] (SPARK-5689) Document what can be run in different YARN modes

[ https://issues.apache.org/jira/browse/SPARK-5689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5689: --- Assignee: Apache Spark Document what can be run in different YARN modes

[jira] [Commented] (SPARK-6800) Reading from JDBC with SQLContext, using lower/upper bounds and numPartitions gives incorrect results.

[ https://issues.apache.org/jira/browse/SPARK-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492364#comment-14492364 ] Micael Capitão commented on SPARK-6800: --- The above pull request seem to only fix the

[jira] [Commented] (SPARK-4783) System.exit() calls in SparkContext disrupt applications embedding Spark

[ https://issues.apache.org/jira/browse/SPARK-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492346#comment-14492346 ] Sean Owen commented on SPARK-4783: -- I have a PR ready, but am testing it. I am seeing

[jira] [Assigned] (SPARK-4783) System.exit() calls in SparkContext disrupt applications embedding Spark

[ https://issues.apache.org/jira/browse/SPARK-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4783: --- Assignee: Apache Spark System.exit() calls in SparkContext disrupt applications embedding

[jira] [Assigned] (SPARK-4783) System.exit() calls in SparkContext disrupt applications embedding Spark

[ https://issues.apache.org/jira/browse/SPARK-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4783: --- Assignee: (was: Apache Spark) System.exit() calls in SparkContext disrupt applications

[jira] [Commented] (SPARK-4783) System.exit() calls in SparkContext disrupt applications embedding Spark

[ https://issues.apache.org/jira/browse/SPARK-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492393#comment-14492393 ] Apache Spark commented on SPARK-4783: - User 'srowen' has created a pull request for

[jira] [Commented] (SPARK-5791) [Spark SQL] show poor performance when multiple table do join operation

[ https://issues.apache.org/jira/browse/SPARK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492618#comment-14492618 ] Yin Huai commented on SPARK-5791: - [~jameszhouyi] Thank you for the update :) For Hive, it

[jira] [Commented] (SPARK-6880) Spark Shutdowns with NoSuchElementException when running parallel collect on cachedRDD

[ https://issues.apache.org/jira/browse/SPARK-6880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492622#comment-14492622 ] Apache Spark commented on SPARK-6880: - User 'pankajarora12' has created a pull request

[jira] [Created] (SPARK-6882) Spark ThriftServer2 Kerberos failed encountering java.lang.IllegalArgumentException: Unknown auth type: null Allowed values are: [auth-int, auth-conf, auth]

Andrew Lee created SPARK-6882: - Summary: Spark ThriftServer2 Kerberos failed encountering java.lang.IllegalArgumentException: Unknown auth type: null Allowed values are: [auth-int, auth-conf, auth] Key: SPARK-6882

[jira] [Assigned] (SPARK-6880) Spark Shutdowns with NoSuchElementException when running parallel collect on cachedRDD

[ https://issues.apache.org/jira/browse/SPARK-6880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6880: --- Assignee: Apache Spark Spark Shutdowns with NoSuchElementException when running parallel

[jira] [Assigned] (SPARK-6880) Spark Shutdowns with NoSuchElementException when running parallel collect on cachedRDD

[ https://issues.apache.org/jira/browse/SPARK-6880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6880: --- Assignee: (was: Apache Spark) Spark Shutdowns with NoSuchElementException when running

[jira] [Updated] (SPARK-6880) Spark Shutdowns with NoSuchElementException when running parallel collect on cachedRDD

[ https://issues.apache.org/jira/browse/SPARK-6880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pankaj arora updated SPARK-6880: Description: Spark Shutdowns with NoSuchElementException when running parallel collect on

[jira] [Commented] (SPARK-6880) Spark Shutdowns with NoSuchElementException when running parallel collect on cachedRDD

[ https://issues.apache.org/jira/browse/SPARK-6880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492642#comment-14492642 ] pankaj arora commented on SPARK-6880: - Sean, Sorry for missing stack trace. Added that

[jira] [Commented] (SPARK-6823) Add a model.matrix like capability to DataFrames (modelDataFrame)

[ https://issues.apache.org/jira/browse/SPARK-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492661#comment-14492661 ] Shivaram Venkataraman commented on SPARK-6823: -- I think the goal of the

[jira] [Commented] (SPARK-6859) Parquet File Binary column statistics error when reuse byte[] among rows

[ https://issues.apache.org/jira/browse/SPARK-6859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492667#comment-14492667 ] Cheng Lian commented on SPARK-6859: --- [~rdblue] pointed out 1 fact that I missed in

[jira] [Updated] (SPARK-6882) Spark ThriftServer2 Kerberos failed encountering java.lang.IllegalArgumentException: Unknown auth type: null Allowed values are: [auth-int, auth-conf, auth]

[ https://issues.apache.org/jira/browse/SPARK-6882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6882: - Component/s: SQL Spark ThriftServer2 Kerberos failed encountering java.lang.IllegalArgumentException:

[jira] [Resolved] (SPARK-6765) Turn scalastyle on for test code

[ https://issues.apache.org/jira/browse/SPARK-6765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6765. Resolution: Fixed Fix Version/s: 1.4.0 Turn scalastyle on for test code

[jira] [Created] (SPARK-6880) Spark Shutdowns with NoSuchElementException when running parallel collect on cachedRDD

pankaj arora created SPARK-6880: --- Summary: Spark Shutdowns with NoSuchElementException when running parallel collect on cachedRDD Key: SPARK-6880 URL: https://issues.apache.org/jira/browse/SPARK-6880

[jira] [Created] (SPARK-6881) Change the checkpoint directory name from checkpoints to checkpoint

Hao created SPARK-6881: -- Summary: Change the checkpoint directory name from checkpoints to checkpoint Key: SPARK-6881 URL: https://issues.apache.org/jira/browse/SPARK-6881 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-6881) Change the checkpoint directory name from checkpoints to checkpoint

[ https://issues.apache.org/jira/browse/SPARK-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6881: --- Assignee: Apache Spark Change the checkpoint directory name from checkpoints to checkpoint

[jira] [Commented] (SPARK-6881) Change the checkpoint directory name from checkpoints to checkpoint

[ https://issues.apache.org/jira/browse/SPARK-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492548#comment-14492548 ] Apache Spark commented on SPARK-6881: - User 'hlin09' has created a pull request for

[jira] [Assigned] (SPARK-6881) Change the checkpoint directory name from checkpoints to checkpoint

[ https://issues.apache.org/jira/browse/SPARK-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6881: --- Assignee: (was: Apache Spark) Change the checkpoint directory name from checkpoints to

[jira] [Updated] (SPARK-6880) Spark Shutdowns with NoSuchElementException when running parallel collect on cachedRDD

[ https://issues.apache.org/jira/browse/SPARK-6880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6880: - Target Version/s: (was: 1.3.2) Fix Version/s: (was: 1.3.2) (Don't assign Target / Fix

<    1   2