[jira] [Created] (SPARK-3027) Tighten the visibility of fields in TaskContext and provide Java friendly callback API

2014-08-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3027: -- Summary: Tighten the visibility of fields in TaskContext and provide Java friendly callback API Key: SPARK-3027 URL: https://issues.apache.org/jira/browse/SPARK-3027

[jira] [Commented] (SPARK-3027) Tighten visibility and provide Java friendly callback API in TaskContext

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096632#comment-14096632 ] Apache Spark commented on SPARK-3027: - User 'rxin' has created a pull request for this

[jira] [Updated] (SPARK-3027) Tighten visibility and provide Java friendly callback API in TaskContext

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3027: --- Summary: Tighten visibility and provide Java friendly callback API in TaskContext (was: Tighten the

[jira] [Commented] (SPARK-2356) Exception: Could not locate executable null\bin\winutils.exe in the Hadoop

2014-08-14 Thread Kostiantyn Kudriavtsev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096636#comment-14096636 ] Kostiantyn Kudriavtsev commented on SPARK-2356: --- Guoqiang, Spark works not

[jira] [Commented] (SPARK-2356) Exception: Could not locate executable null\bin\winutils.exe in the Hadoop

2014-08-14 Thread Tarek Nabil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096635#comment-14096635 ] Tarek Nabil commented on SPARK-2356: Yes, but the whole point is that you should do

[jira] [Created] (SPARK-3028) sparkEventToJson should support SparkListenerExecutorMetricsUpdate

2014-08-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3028: -- Summary: sparkEventToJson should support SparkListenerExecutorMetricsUpdate Key: SPARK-3028 URL: https://issues.apache.org/jira/browse/SPARK-3028 Project: Spark

[jira] [Commented] (SPARK-3028) sparkEventToJson should support SparkListenerExecutorMetricsUpdate

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096645#comment-14096645 ] Reynold Xin commented on SPARK-3028: [~sandyr] [~sandyryza] Would you be able to do

[jira] [Updated] (SPARK-2456) Scheduler refactoring

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2456: --- Component/s: Spark Core Scheduler refactoring - Key:

[jira] [Created] (SPARK-3029) Disable local execution of Spark jobs by default

2014-08-14 Thread Aaron Davidson (JIRA)
Aaron Davidson created SPARK-3029: - Summary: Disable local execution of Spark jobs by default Key: SPARK-3029 URL: https://issues.apache.org/jira/browse/SPARK-3029 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3019) Pluggable block transfer (data plane communication) interface

2014-08-14 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096654#comment-14096654 ] Hari Shreedharan commented on SPARK-3019: - Why specifically MapR FS? You could use

[jira] [Created] (SPARK-3030) reuse python worker

2014-08-14 Thread Davies Liu (JIRA)
Davies Liu created SPARK-3030: - Summary: reuse python worker Key: SPARK-3030 URL: https://issues.apache.org/jira/browse/SPARK-3030 Project: Spark Issue Type: Improvement Components:

[jira] [Commented] (SPARK-3019) Pluggable block transfer (data plane communication) interface

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096658#comment-14096658 ] Reynold Xin commented on SPARK-3019: Possibly, although I think MapR FS is more

[jira] [Resolved] (SPARK-1170) Add histogram() to PySpark

2014-08-14 Thread Chandan Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan Kumar resolved SPARK-1170. -- Resolution: Duplicate Davies is working on this. Add histogram() to PySpark

[jira] [Commented] (SPARK-3031) Create JsonSerializable and move JSON serialization from JsonProtocol into each class

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096664#comment-14096664 ] Reynold Xin commented on SPARK-3031: cc [~andrewor14] [~adav] Create

[jira] [Updated] (SPARK-3029) Disable local execution of Spark jobs by default

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3029: --- Priority: Blocker (was: Major) Target Version/s: 1.1.0 Disable local execution of

[jira] [Updated] (SPARK-3031) Create JsonSerializable and move JSON serialization from JsonProtocol into each class

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3031: --- Component/s: Spark Core Create JsonSerializable and move JSON serialization from JsonProtocol into

[jira] [Created] (SPARK-3031) Create JsonSerializable and move JSON serialization from JsonProtocol into each class

2014-08-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3031: -- Summary: Create JsonSerializable and move JSON serialization from JsonProtocol into each class Key: SPARK-3031 URL: https://issues.apache.org/jira/browse/SPARK-3031

[jira] [Resolved] (SPARK-2995) Allow to set storage level for intermediate RDDs in ALS

2014-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2995. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1913

[jira] [Updated] (SPARK-2893) Should not swallow exception when cannot find custom Kryo registrator

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2893: --- Priority: Blocker (was: Major) Target Version/s: 1.1.0 Should not swallow exception

[jira] [Updated] (SPARK-2893) Should not swallow exception when cannot find custom Kryo registrator

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2893: --- Assignee: Graham Dennis Should not swallow exception when cannot find custom Kryo registrator

[jira] [Updated] (SPARK-2893) Should not swallow exception when cannot find custom Kryo registrator

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2893: --- Target Version/s: 1.1.0, 1.0.3 (was: 1.1.0) Should not swallow exception when cannot find custom

[jira] [Updated] (SPARK-2878) Inconsistent Kryo serialisation with custom Kryo Registrator

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2878: --- Assignee: Graham Dennis Inconsistent Kryo serialisation with custom Kryo Registrator

[jira] [Comment Edited] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

2014-08-14 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096608#comment-14096608 ] Saisai Shao edited comment on SPARK-2926 at 8/14/14 7:12 AM: -

[jira] [Updated] (SPARK-2878) Inconsistent Kryo serialisation with custom Kryo Registrator

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2878: --- Target Version/s: 1.1.0, 1.0.3 Inconsistent Kryo serialisation with custom Kryo Registrator

[jira] [Commented] (SPARK-3028) sparkEventToJson should support SparkListenerExecutorMetricsUpdate

2014-08-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096689#comment-14096689 ] Patrick Wendell commented on SPARK-3028: I think we intentionally do not intend to

[jira] [Commented] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

2014-08-14 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096688#comment-14096688 ] Saisai Shao commented on SPARK-2926: I think this prototype can easily offer the

[jira] [Comment Edited] (SPARK-3028) sparkEventToJson should support SparkListenerExecutorMetricsUpdate

2014-08-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096689#comment-14096689 ] Patrick Wendell edited comment on SPARK-3028 at 8/14/14 7:20 AM:

[jira] [Commented] (SPARK-3028) sparkEventToJson should support SparkListenerExecutorMetricsUpdate

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096696#comment-14096696 ] Reynold Xin commented on SPARK-3028: Ok in that case, you suggestion makes sense

[jira] [Issue Comment Deleted] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

2014-08-14 Thread Xu Zhongxing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Zhongxing updated SPARK-3005: Comment: was deleted (was: A related question: why does fined-grain mode and coarse-grained mode

[jira] [Updated] (SPARK-2878) Inconsistent Kryo serialisation with custom Kryo Registrator

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2878: --- Priority: Critical (was: Major) Inconsistent Kryo serialisation with custom Kryo Registrator

[jira] [Comment Edited] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

2014-08-14 Thread Xu Zhongxing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095288#comment-14095288 ] Xu Zhongxing edited comment on SPARK-3005 at 8/14/14 7:36 AM: --

[jira] [Comment Edited] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

2014-08-14 Thread Xu Zhongxing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096539#comment-14096539 ] Xu Zhongxing edited comment on SPARK-3005 at 8/14/14 7:37 AM: --

[jira] [Created] (SPARK-3032) Potential bug when running sort-based shuffle with sorting using TimSort

2014-08-14 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-3032: -- Summary: Potential bug when running sort-based shuffle with sorting using TimSort Key: SPARK-3032 URL: https://issues.apache.org/jira/browse/SPARK-3032 Project: Spark

[jira] [Commented] (SPARK-3005) Spark with Mesos fine-grained mode throws UnsupportedOperationException in MesosSchedulerBackend.killTask()

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096725#comment-14096725 ] Apache Spark commented on SPARK-3005: - User 'xuzhongxing' has created a pull request

[jira] [Resolved] (SPARK-3029) Disable local execution of Spark jobs by default

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3029. Resolution: Fixed Fix Version/s: 1.1.0 Disable local execution of Spark jobs by default

[jira] [Updated] (SPARK-3033) java.math.BigDecimal cannot be cast to org.apache.hadoop.hive.common.type.HiveDecimal

2014-08-14 Thread pengyanhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pengyanhong updated SPARK-3033: --- Description: run a complex HiveQL via yarn-cluster, got error as below: {quote} 14/08/14 15:05:24

[jira] [Created] (SPARK-3034) java.sql.Date cannot be cast to java.sql.Timestamp

2014-08-14 Thread pengyanhong (JIRA)
pengyanhong created SPARK-3034: -- Summary: java.sql.Date cannot be cast to java.sql.Timestamp Key: SPARK-3034 URL: https://issues.apache.org/jira/browse/SPARK-3034 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-3034) java.sql.Date cannot be cast to java.sql.Timestamp

2014-08-14 Thread pengyanhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pengyanhong updated SPARK-3034: --- Description: run a simple HiveQL via yarn-cluster, got error as below: {quote} Exception in thread

[jira] [Updated] (SPARK-3034) java.sql.Date cannot be cast to java.sql.Timestamp

2014-08-14 Thread pengyanhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pengyanhong updated SPARK-3034: --- Description: run a simple HiveQL via yarn-cluster, got error as below: {quote} Exception in thread

[jira] [Created] (SPARK-3035) Wrong example with SparkContext.addFile

2014-08-14 Thread Daehan Kim (JIRA)
Daehan Kim created SPARK-3035: - Summary: Wrong example with SparkContext.addFile Key: SPARK-3035 URL: https://issues.apache.org/jira/browse/SPARK-3035 Project: Spark Issue Type: Documentation

[jira] [Resolved] (SPARK-2893) Should not swallow exception when cannot find custom Kryo registrator

2014-08-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2893. Resolution: Fixed Fix Version/s: 1.0.3 1.1.0 Should not swallow

[jira] [Created] (SPARK-3036) Add MapType containing null value support to Parquet.

2014-08-14 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-3036: Summary: Add MapType containing null value support to Parquet. Key: SPARK-3036 URL: https://issues.apache.org/jira/browse/SPARK-3036 Project: Spark Issue

[jira] [Created] (SPARK-3038) delete history server logs when there are too many logs

2014-08-14 Thread wangfei (JIRA)
wangfei created SPARK-3038: -- Summary: delete history server logs when there are too many logs Key: SPARK-3038 URL: https://issues.apache.org/jira/browse/SPARK-3038 Project: Spark Issue Type:

[jira] [Updated] (SPARK-3038) delete history server logs when there are too many logs

2014-08-14 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei updated SPARK-3038: --- Description: enhance history server to delete logs automatically 1 use spark.history.deletelogs.enable to

[jira] [Created] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

2014-08-14 Thread Bertrand Bossy (JIRA)
Bertrand Bossy created SPARK-3039: - Summary: Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API Key: SPARK-3039 URL: https://issues.apache.org/jira/browse/SPARK-3039

[jira] [Commented] (SPARK-3026) Provide a good error message if JDBC server is used but Spark is not compiled with -Pthriftserver

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096961#comment-14096961 ] Apache Spark commented on SPARK-3026: - User 'liancheng' has created a pull request for

[jira] [Updated] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

2014-08-14 Thread Bertrand Bossy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bertrand Bossy updated SPARK-3039: -- Affects Version/s: 1.1.0 Spark assembly for new hadoop API (hadoop 2) contains avro-mapred

[jira] [Updated] (SPARK-3034) [HIve] java.sql.Date cannot be cast to java.sql.Timestamp

2014-08-14 Thread pengyanhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pengyanhong updated SPARK-3034: --- Summary: [HIve] java.sql.Date cannot be cast to java.sql.Timestamp (was: java.sql.Date cannot be

[jira] [Updated] (SPARK-3033) [Hive] java.math.BigDecimal cannot be cast to org.apache.hadoop.hive.common.type.HiveDecimal

2014-08-14 Thread pengyanhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pengyanhong updated SPARK-3033: --- Summary: [Hive] java.math.BigDecimal cannot be cast to

[jira] [Commented] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14096973#comment-14096973 ] Apache Spark commented on SPARK-3039: - User 'bbossy' has created a pull request for

[jira] [Created] (SPARK-3040) pick up a more proper local ip address for Utils.findLocalIpAddress method

2014-08-14 Thread Ye Xianjin (JIRA)
Ye Xianjin created SPARK-3040: - Summary: pick up a more proper local ip address for Utils.findLocalIpAddress method Key: SPARK-3040 URL: https://issues.apache.org/jira/browse/SPARK-3040 Project: Spark

[jira] [Commented] (SPARK-3040) pick up a more proper local ip address for Utils.findLocalIpAddress method

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097115#comment-14097115 ] Apache Spark commented on SPARK-3040: - User 'advancedxy' has created a pull request

[jira] [Commented] (SPARK-3028) sparkEventToJson should support SparkListenerExecutorMetricsUpdate

2014-08-14 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097193#comment-14097193 ] Sandy Ryza commented on SPARK-3028: --- +1 to what Patrick said. I'll post a patch along

[jira] [Commented] (SPARK-3009) ApplicationInfo doesn't get initialised after deserialisation during recovery

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097268#comment-14097268 ] Apache Spark commented on SPARK-3009: - User 'jacek-lewandowski' has created a pull

[jira] [Resolved] (SPARK-2927) Add a conf to configure if we always read Binary columns stored in Parquet as String columns

2014-08-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2927. - Resolution: Fixed Fix Version/s: 1.1.0 Add a conf to configure if we always read

[jira] [Resolved] (SPARK-3011) _temporary directory should be filtered out by sqlContext.parquetFile

2014-08-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3011. - Resolution: Fixed Fix Version/s: 1.1.0 _temporary directory should be filtered

[jira] [Created] (SPARK-3041) DecisionTree: isSampleValid indexing incorrect

2014-08-14 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-3041: Summary: DecisionTree: isSampleValid indexing incorrect Key: SPARK-3041 URL: https://issues.apache.org/jira/browse/SPARK-3041 Project: Spark Issue

[jira] [Created] (SPARK-3043) DecisionTree aggregation is inefficient

2014-08-14 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-3043: Summary: DecisionTree aggregation is inefficient Key: SPARK-3043 URL: https://issues.apache.org/jira/browse/SPARK-3043 Project: Spark Issue Type:

[jira] [Updated] (SPARK-3042) DecisionTree filtering is very inefficient

2014-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3042: - Assignee: Joseph K. Bradley DecisionTree filtering is very inefficient

[jira] [Updated] (SPARK-3043) DecisionTree aggregation is inefficient

2014-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3043: - Assignee: Joseph K. Bradley DecisionTree aggregation is inefficient

[jira] [Updated] (SPARK-3042) DecisionTree filtering is very inefficient

2014-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3042: - Target Version/s: 1.1.0 DecisionTree filtering is very inefficient

[jira] [Updated] (SPARK-3041) DecisionTree: isSampleValid indexing incorrect

2014-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3041: - Assignee: Joseph K. Bradley DecisionTree: isSampleValid indexing incorrect

[jira] [Updated] (SPARK-3043) DecisionTree aggregation is inefficient

2014-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3043: - Target Version/s: 1.1.0 Affects Version/s: 1.1.0 DecisionTree aggregation is inefficient

[jira] [Updated] (SPARK-3041) DecisionTree: isSampleValid indexing incorrect

2014-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3041: - Target Version/s: 1.1.0 Affects Version/s: 1.1.0 DecisionTree: isSampleValid indexing

[jira] [Created] (SPARK-3044) Create RSS feed for Spark News

2014-08-14 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3044: --- Summary: Create RSS feed for Spark News Key: SPARK-3044 URL: https://issues.apache.org/jira/browse/SPARK-3044 Project: Spark Issue Type: Documentation

[jira] [Updated] (SPARK-2979) Improve the convergence rate by minimizing the condition number in LOR with LBFGS

2014-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2979: - Assignee: DB Tsai Improve the convergence rate by minimizing the condition number in LOR with

[jira] [Resolved] (SPARK-2979) Improve the convergence rate by minimizing the condition number in LOR with LBFGS

2014-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2979. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1897

[jira] [Created] (SPARK-3046) Set executor's class loader as the default serializer class loader

2014-08-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3046: -- Summary: Set executor's class loader as the default serializer class loader Key: SPARK-3046 URL: https://issues.apache.org/jira/browse/SPARK-3046 Project: Spark

[jira] [Created] (SPARK-3045) Make Serializer interface Java friendly

2014-08-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3045: -- Summary: Make Serializer interface Java friendly Key: SPARK-3045 URL: https://issues.apache.org/jira/browse/SPARK-3045 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3045) Make Serializer interface Java friendly

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097503#comment-14097503 ] Apache Spark commented on SPARK-3045: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-3046) Set executor's class loader as the default serializer class loader

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097504#comment-14097504 ] Apache Spark commented on SPARK-3046: - User 'rxin' has created a pull request for this

[jira] [Created] (SPARK-3047) Use utf-8 for textFile() by default

2014-08-14 Thread Davies Liu (JIRA)
Davies Liu created SPARK-3047: - Summary: Use utf-8 for textFile() by default Key: SPARK-3047 URL: https://issues.apache.org/jira/browse/SPARK-3047 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3011) _temporary directory should be filtered out by sqlContext.parquetFile

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097537#comment-14097537 ] Apache Spark commented on SPARK-3011: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-3041) DecisionTree: isSampleValid indexing incorrect

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097564#comment-14097564 ] Apache Spark commented on SPARK-3041: - User 'jkbradley' has created a pull request for

[jira] [Commented] (SPARK-3022) FindBinsForLevel in decision tree should call findBin only once for each feature

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097563#comment-14097563 ] Apache Spark commented on SPARK-3022: - User 'jkbradley' has created a pull request for

[jira] [Updated] (SPARK-3047) add an option to use str in textFileRDD()

2014-08-14 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-3047: -- Summary: add an option to use str in textFileRDD() (was: Use utf-8 for textFile() by default) add

[jira] [Commented] (SPARK-3047) add an option to use str in textFileRDD()

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097639#comment-14097639 ] Apache Spark commented on SPARK-3047: - User 'davies' has created a pull request for

[jira] [Updated] (SPARK-2736) Create PySpark RDD from Apache Avro File

2014-08-14 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-2736: - Priority: Major (was: Minor) Create PySpark RDD from Apache Avro File

[jira] [Commented] (SPARK-2736) Create PySpark RDD from Apache Avro File

2014-08-14 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097701#comment-14097701 ] Matei Zaharia commented on SPARK-2736: -- I bumped this up to Major because the PR also

[jira] [Created] (SPARK-3048) Make LabeledPointParser public

2014-08-14 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-3048: Summary: Make LabeledPointParser public Key: SPARK-3048 URL: https://issues.apache.org/jira/browse/SPARK-3048 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-3049) Make sure client doesn't block when server/connection has error(s)

2014-08-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-3049: -- Summary: Make sure client doesn't block when server/connection has error(s) Key: SPARK-3049 URL: https://issues.apache.org/jira/browse/SPARK-3049 Project: Spark

[jira] [Updated] (SPARK-3048) Make LabeledPointParser public

2014-08-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3048: - Description: `LabeledPointParser` is used in `MLUtils.loadLabeledPoint`. Making it public may be

[jira] [Created] (SPARK-3050) Spark program running with 1.0.2 jar cannot run against a 1.0.1 cluster

2014-08-14 Thread Mingyu Kim (JIRA)
Mingyu Kim created SPARK-3050: - Summary: Spark program running with 1.0.2 jar cannot run against a 1.0.1 cluster Key: SPARK-3050 URL: https://issues.apache.org/jira/browse/SPARK-3050 Project: Spark

[jira] [Commented] (SPARK-3048) Make LabeledPointParser public

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097792#comment-14097792 ] Apache Spark commented on SPARK-3048: - User 'mengxr' has created a pull request for

[jira] [Commented] (SPARK-1284) pyspark hangs after IOError on Executor

2014-08-14 Thread Jim Blomo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097808#comment-14097808 ] Jim Blomo commented on SPARK-1284: -- Hi, having trouble compiling either master or

[jira] [Commented] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

2014-08-14 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097826#comment-14097826 ] Sandy Ryza commented on SPARK-2089: --- H, it's true that my suggestion would require

[jira] [Comment Edited] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

2014-08-14 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097826#comment-14097826 ] Sandy Ryza edited comment on SPARK-2089 at 8/14/14 10:41 PM: -

[jira] [Created] (SPARK-3051) Support looking-up named accumulators in a registry

2014-08-14 Thread Neil Ferguson (JIRA)
Neil Ferguson created SPARK-3051: Summary: Support looking-up named accumulators in a registry Key: SPARK-3051 URL: https://issues.apache.org/jira/browse/SPARK-3051 Project: Spark Issue

[jira] [Commented] (SPARK-3051) Support looking-up named accumulators in a registry

2014-08-14 Thread Neil Ferguson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097893#comment-14097893 ] Neil Ferguson commented on SPARK-3051: -- I've done an initial prototype of this in the

[jira] [Created] (SPARK-3052) Misleading and spurious FileSystem closed errors whenever a job fails while reading from Hadoop

2014-08-14 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-3052: - Summary: Misleading and spurious FileSystem closed errors whenever a job fails while reading from Hadoop Key: SPARK-3052 URL: https://issues.apache.org/jira/browse/SPARK-3052

[jira] [Commented] (SPARK-3052) Misleading and spurious FileSystem closed errors whenever a job fails while reading from Hadoop

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097948#comment-14097948 ] Apache Spark commented on SPARK-3052: - User 'sryza' has created a pull request for

[jira] [Commented] (SPARK-3050) Spark program running with 1.0.2 jar cannot run against a 1.0.1 cluster

2014-08-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097947#comment-14097947 ] Patrick Wendell commented on SPARK-3050: Hi [~mkim] - when you launch jobs in

[jira] [Resolved] (SPARK-3050) Spark program running with 1.0.2 jar cannot run against a 1.0.1 cluster

2014-08-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3050. Resolution: Not a Problem I think the issue here is just needing to use the newer version

[jira] [Created] (SPARK-3053) Reconcile spark.files.userClassPathFirst with spark.yarn.user.classpath.first

2014-08-14 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-3053: - Summary: Reconcile spark.files.userClassPathFirst with spark.yarn.user.classpath.first Key: SPARK-3053 URL: https://issues.apache.org/jira/browse/SPARK-3053 Project: Spark

[jira] [Updated] (SPARK-3050) Spark program running with 1.0.2 jar cannot run against a 1.0.1 cluster

2014-08-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3050: --- Priority: Major (was: Critical) Spark program running with 1.0.2 jar cannot run against a

[jira] [Created] (SPARK-3054) Add tests for SparkSink

2014-08-14 Thread Hari Shreedharan (JIRA)
Hari Shreedharan created SPARK-3054: --- Summary: Add tests for SparkSink Key: SPARK-3054 URL: https://issues.apache.org/jira/browse/SPARK-3054 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3054) Add tests for SparkSink

2014-08-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097973#comment-14097973 ] Apache Spark commented on SPARK-3054: - User 'harishreedharan' has created a pull

[jira] [Commented] (SPARK-2213) Sort Merge Join

2014-08-14 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097995#comment-14097995 ] Cheng Hao commented on SPARK-2213: -- Sort Merge Join depends on the reduce side sort

[jira] [Created] (SPARK-3055) Stack trace logged in driver on job failure is usually uninformative

2014-08-14 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-3055: - Summary: Stack trace logged in driver on job failure is usually uninformative Key: SPARK-3055 URL: https://issues.apache.org/jira/browse/SPARK-3055 Project: Spark

[jira] [Created] (SPARK-3056) Sort-based Aggregation

2014-08-14 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-3056: Summary: Sort-based Aggregation Key: SPARK-3056 URL: https://issues.apache.org/jira/browse/SPARK-3056 Project: Spark Issue Type: Improvement Components:

  1   2   >