[jira] [Commented] (SPARK-19823) Support Gang Distribution of Task

[ https://issues.apache.org/jira/browse/SPARK-19823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896107#comment-15896107 ] DjvuLee commented on SPARK-19823: - [~zsxwing] Can you have a look at? > Support Gang Dis

[jira] [Comment Edited] (SPARK-19823) Support Gang Distribution of Task

[ https://issues.apache.org/jira/browse/SPARK-19823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896096#comment-15896096 ] DjvuLee edited comment on SPARK-19823 at 3/5/17 7:19 AM: - When Sp

[jira] [Comment Edited] (SPARK-19823) Support Gang Distribution of Task

[ https://issues.apache.org/jira/browse/SPARK-19823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896096#comment-15896096 ] DjvuLee edited comment on SPARK-19823 at 3/5/17 7:10 AM: - When Sp

[jira] [Comment Edited] (SPARK-19823) Support Gang Distribution of Task

[ https://issues.apache.org/jira/browse/SPARK-19823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896096#comment-15896096 ] DjvuLee edited comment on SPARK-19823 at 3/5/17 7:10 AM: - When Sp

[jira] [Commented] (SPARK-13156) JDBC using multiple partitions creates additional tasks but only executes on one

[ https://issues.apache.org/jira/browse/SPARK-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896100#comment-15896100 ] zhuo bao commented on SPARK-13156: -- I had the same problem, but I found that it is the p

[jira] [Comment Edited] (SPARK-19823) Support Gang Distribution of Task

[ https://issues.apache.org/jira/browse/SPARK-19823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896096#comment-15896096 ] DjvuLee edited comment on SPARK-19823 at 3/5/17 7:10 AM: - When Sp

[jira] [Commented] (SPARK-19823) Support Gang Distribution of Task

[ https://issues.apache.org/jira/browse/SPARK-19823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896096#comment-15896096 ] DjvuLee commented on SPARK-19823: - When Spark distributes tasks to Executors, it uses a

[jira] [Commented] (SPARK-19823) Support Gang Distribution of Task

[ https://issues.apache.org/jira/browse/SPARK-19823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896097#comment-15896097 ] DjvuLee commented on SPARK-19823: - If this is a good advice, I will give a Pull Request.

[jira] [Created] (SPARK-19823) Support Gang Distribution of Task

DjvuLee created SPARK-19823: --- Summary: Support Gang Distribution of Task Key: SPARK-19823 URL: https://issues.apache.org/jira/browse/SPARK-19823 Project: Spark Issue Type: Improvement Co

[jira] [Updated] (SPARK-19798) Query returns stale results when tables are modified on other sessions

[ https://issues.apache.org/jira/browse/SPARK-19798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19798: - Component/s: (was: Spark Core) SQL > Query returns stale results when tables

[jira] [Updated] (SPARK-19821) Throw out the Read-only disk information when create file for Shuffle

[ https://issues.apache.org/jira/browse/SPARK-19821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19821: - Priority: Minor (was: Major) > Throw out the Read-only disk information when create file for Shu

[jira] [Commented] (SPARK-19821) Throw out the Read-only disk information when create file for Shuffle

[ https://issues.apache.org/jira/browse/SPARK-19821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896079#comment-15896079 ] Shixiong Zhu commented on SPARK-19821: -- This is more like a Java issue. > Throw out

[jira] [Assigned] (SPARK-19822) CheckpointSuite.testCheckpointedOperation: should not check checkpointFilesOfLatestTime by the PATH string.

[ https://issues.apache.org/jira/browse/SPARK-19822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19822: Assignee: Apache Spark > CheckpointSuite.testCheckpointedOperation: should not check > ch

[jira] [Assigned] (SPARK-19822) CheckpointSuite.testCheckpointedOperation: should not check checkpointFilesOfLatestTime by the PATH string.

[ https://issues.apache.org/jira/browse/SPARK-19822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19822: Assignee: (was: Apache Spark) > CheckpointSuite.testCheckpointedOperation: should not

[jira] [Commented] (SPARK-19822) CheckpointSuite.testCheckpointedOperation: should not check checkpointFilesOfLatestTime by the PATH string.

[ https://issues.apache.org/jira/browse/SPARK-19822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896078#comment-15896078 ] Apache Spark commented on SPARK-19822: -- User 'uncleGen' has created a pull request f

[jira] [Created] (SPARK-19822) CheckpointSuite.testCheckpointedOperation: should not check checkpointFilesOfLatestTime by the PATH string.

Genmao Yu created SPARK-19822: - Summary: CheckpointSuite.testCheckpointedOperation: should not check checkpointFilesOfLatestTime by the PATH string. Key: SPARK-19822 URL: https://issues.apache.org/jira/browse/SPARK-19

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896070#comment-15896070 ] DjvuLee commented on SPARK-18085: - [~vanzin] Thanks for your reply! Does your new soluti

[jira] [Commented] (SPARK-19821) Throw out the Read-only disk information when create file for Shuffle

[ https://issues.apache.org/jira/browse/SPARK-19821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896067#comment-15896067 ] DjvuLee commented on SPARK-19821: - Currently, when the disk is just read-only, we will ju

[jira] [Updated] (SPARK-19821) Throw out the Read-only disk information when create file for Shuffle

[ https://issues.apache.org/jira/browse/SPARK-19821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DjvuLee updated SPARK-19821: Description: java.io.FileNotFoundException: /data01/yarn/nmdata/usercache/tiger/appcache/application_14863

[jira] [Created] (SPARK-19821) Throw out the Read-only disk information when create file for Shuffle

DjvuLee created SPARK-19821: --- Summary: Throw out the Read-only disk information when create file for Shuffle Key: SPARK-19821 URL: https://issues.apache.org/jira/browse/SPARK-19821 Project: Spark

[jira] [Commented] (SPARK-19145) Timestamp to String casting is slowing the query significantly

[ https://issues.apache.org/jira/browse/SPARK-19145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896061#comment-15896061 ] gagan taneja commented on SPARK-19145: -- I am suggesting following changes introduce

[jira] [Commented] (SPARK-19145) Timestamp to String casting is slowing the query significantly

[ https://issues.apache.org/jira/browse/SPARK-19145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896046#comment-15896046 ] gagan taneja commented on SPARK-19145: -- 17/03/04 19:05:32 TRACE HiveSessionState$$an

[jira] [Commented] (SPARK-19145) Timestamp to String casting is slowing the query significantly

[ https://issues.apache.org/jira/browse/SPARK-19145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896040#comment-15896040 ] gagan taneja commented on SPARK-19145: -- Code responsible for this belongs to class a

[jira] [Commented] (SPARK-6407) Streaming ALS for Collaborative Filtering

[ https://issues.apache.org/jira/browse/SPARK-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896033#comment-15896033 ] Daniel Li commented on SPARK-6407: -- Appreciate the quick reply, [~srowen]. Yeah, we'd be

[jira] [Commented] (SPARK-19541) High Availability support for ThriftServer

[ https://issues.apache.org/jira/browse/SPARK-19541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896032#comment-15896032 ] gagan taneja commented on SPARK-19541: -- This would a great improvement as we are als

[jira] [Updated] (SPARK-19705) Preferred location supporting HDFS Cache for FileScanRDD

[ https://issues.apache.org/jira/browse/SPARK-19705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gagan taneja updated SPARK-19705: - Shepherd: Herman van Hovell > Preferred location supporting HDFS Cache for FileScanRDD >

[jira] [Commented] (SPARK-19705) Preferred location supporting HDFS Cache for FileScanRDD

[ https://issues.apache.org/jira/browse/SPARK-19705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896026#comment-15896026 ] gagan taneja commented on SPARK-19705: -- Herman, Can you help me with this enhancemen

[jira] [Comment Edited] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895998#comment-15895998 ] Eric Maynard edited comment on SPARK-19656 at 3/5/17 12:58 AM:

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895998#comment-15895998 ] Eric Maynard commented on SPARK-19656: -- Normally after getting the `datum` you shoul

[jira] [Commented] (SPARK-19713) saveAsTable

[ https://issues.apache.org/jira/browse/SPARK-19713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895991#comment-15895991 ] Eric Maynard commented on SPARK-19713: -- In general instead of using `DataFrameWriter

[jira] [Assigned] (SPARK-19820) Allow reason to be specified for task kill

[ https://issues.apache.org/jira/browse/SPARK-19820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19820: Assignee: (was: Apache Spark) > Allow reason to be specified for task kill > -

[jira] [Commented] (SPARK-19820) Allow reason to be specified for task kill

[ https://issues.apache.org/jira/browse/SPARK-19820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895954#comment-15895954 ] Apache Spark commented on SPARK-19820: -- User 'ericl' has created a pull request for

[jira] [Assigned] (SPARK-19820) Allow reason to be specified for task kill

[ https://issues.apache.org/jira/browse/SPARK-19820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19820: Assignee: Apache Spark > Allow reason to be specified for task kill >

[jira] [Created] (SPARK-19820) Allow reason to be specified for task kill

Eric Liang created SPARK-19820: -- Summary: Allow reason to be specified for task kill Key: SPARK-19820 URL: https://issues.apache.org/jira/browse/SPARK-19820 Project: Spark Issue Type: Improvemen

[jira] [Commented] (SPARK-8556) Beeline script throws ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-8556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895898#comment-15895898 ] Arvind Surve commented on SPARK-8556: - Hi Cheng, Would you mind sharing configuration

[jira] [Commented] (SPARK-16844) Generate code for sort based aggregation

[ https://issues.apache.org/jira/browse/SPARK-16844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895762#comment-15895762 ] Apache Spark commented on SPARK-16844: -- User 'maropu' has created a pull request for

[jira] [Commented] (SPARK-16617) Upgrade to Avro 1.8.x

[ https://issues.apache.org/jira/browse/SPARK-16617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895756#comment-15895756 ] Apache Spark commented on SPARK-16617: -- User 'srowen' has created a pull request for

[jira] [Commented] (SPARK-19503) Execution Plan Optimizer: avoid sort or shuffle when it does not change end result such as df.sort(...).count()

[ https://issues.apache.org/jira/browse/SPARK-19503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895740#comment-15895740 ] Kazuaki Ishizaki commented on SPARK-19503: -- Is it better to control whether we p

[jira] [Commented] (SPARK-19550) Remove reflection, docs, build elements related to Java 7

[ https://issues.apache.org/jira/browse/SPARK-19550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895677#comment-15895677 ] Apache Spark commented on SPARK-19550: -- User 'wangyum' has created a pull request fo

[jira] [Commented] (SPARK-7146) Should ML sharedParams be a public API?

[ https://issues.apache.org/jira/browse/SPARK-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895629#comment-15895629 ] Nick Pentreath commented on SPARK-7146: --- Personally I support developer API - these

[jira] [Commented] (SPARK-14409) Investigate adding a RankingEvaluator to ML

[ https://issues.apache.org/jira/browse/SPARK-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895619#comment-15895619 ] Danilo Ascione commented on SPARK-14409: I can help with both PR. Please consider

[jira] [Resolved] (SPARK-14273) Add FileFormat.isSplittable to indicate whether a format is splittable

[ https://issues.apache.org/jira/browse/SPARK-14273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14273. --- Resolution: Duplicate > Add FileFormat.isSplittable to indicate whether a format is splittable >

[jira] [Commented] (SPARK-6407) Streaming ALS for Collaborative Filtering

[ https://issues.apache.org/jira/browse/SPARK-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895605#comment-15895605 ] Sean Owen commented on SPARK-6407: -- How is it different from recomputing all of U and V?

[jira] [Assigned] (SPARK-19819) Use concrete data in SparkR DataFrame examples

[ https://issues.apache.org/jira/browse/SPARK-19819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19819: Assignee: Apache Spark > Use concrete data in SparkR DataFrame examples > ---

[jira] [Assigned] (SPARK-19819) Use concrete data in SparkR DataFrame examples

[ https://issues.apache.org/jira/browse/SPARK-19819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19819: Assignee: (was: Apache Spark) > Use concrete data in SparkR DataFrame examples >

[jira] [Commented] (SPARK-19819) Use concrete data in SparkR DataFrame examples

[ https://issues.apache.org/jira/browse/SPARK-19819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895599#comment-15895599 ] Apache Spark commented on SPARK-19819: -- User 'actuaryzhang' has created a pull reque

[jira] [Created] (SPARK-19819) Use concrete data in SparkR DataFrame examples

Wayne Zhang created SPARK-19819: --- Summary: Use concrete data in SparkR DataFrame examples Key: SPARK-19819 URL: https://issues.apache.org/jira/browse/SPARK-19819 Project: Spark Issue Type: Imp

[jira] [Commented] (SPARK-6407) Streaming ALS for Collaborative Filtering

[ https://issues.apache.org/jira/browse/SPARK-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895589#comment-15895589 ] Daniel Li commented on SPARK-6407: -- Reviving this thread since I'm interested in implemen

[jira] [Commented] (SPARK-19792) In the Master Page,the column named “Memory per Node” ,I think it is not all right

[ https://issues.apache.org/jira/browse/SPARK-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895576#comment-15895576 ] liuxian commented on SPARK-19792: - I think it refers to the memory allocated to each exec