[jira] [Commented] (SPARK-10100) AggregateFunction2's Max is slower than AggregateExpression1's MaxFunction

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704418#comment-14704418 ] Apache Spark commented on SPARK-10100: -- User 'rxin' has created a pull request for

[jira] [Created] (SPARK-10133) loadLibSVMFile fails to detect zero-based lines

2015-08-20 Thread Xusen Yin (JIRA)
Xusen Yin created SPARK-10133: - Summary: loadLibSVMFile fails to detect zero-based lines Key: SPARK-10133 URL: https://issues.apache.org/jira/browse/SPARK-10133 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-9107) Include memory usage for each job stage

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9107: --- Assignee: Apache Spark Include memory usage for each job stage

[jira] [Commented] (SPARK-9106) Log the memory usage info into history server

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704511#comment-14704511 ] Apache Spark commented on SPARK-9106: - User 'liyezhang556520' has created a pull

[jira] [Commented] (SPARK-9105) Add an additional WebUI Tab for Memory Usage

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704510#comment-14704510 ] Apache Spark commented on SPARK-9105: - User 'liyezhang556520' has created a pull

[jira] [Commented] (SPARK-9107) Include memory usage for each job stage

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704512#comment-14704512 ] Apache Spark commented on SPARK-9107: - User 'liyezhang556520' has created a pull

[jira] [Assigned] (SPARK-9107) Include memory usage for each job stage

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9107: --- Assignee: (was: Apache Spark) Include memory usage for each job stage

[jira] [Assigned] (SPARK-9106) Log the memory usage info into history server

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9106: --- Assignee: Apache Spark Log the memory usage info into history server

[jira] [Assigned] (SPARK-9105) Add an additional WebUI Tab for Memory Usage

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9105: --- Assignee: (was: Apache Spark) Add an additional WebUI Tab for Memory Usage

[jira] [Assigned] (SPARK-9105) Add an additional WebUI Tab for Memory Usage

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9105: --- Assignee: Apache Spark Add an additional WebUI Tab for Memory Usage

[jira] [Assigned] (SPARK-9106) Log the memory usage info into history server

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9106: --- Assignee: (was: Apache Spark) Log the memory usage info into history server

[jira] [Commented] (SPARK-10133) loadLibSVMFile fails to detect zero-based lines

2015-08-20 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704521#comment-14704521 ] Xusen Yin commented on SPARK-10133: --- Thanks. I ignored the above line. loadLibSVMFile

[jira] [Assigned] (SPARK-9089) Failing to run simple job on Spark Standalone Cluster

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9089: --- Assignee: Apache Spark Failing to run simple job on Spark Standalone Cluster

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2015-08-20 Thread Silas Davis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705278#comment-14705278 ] Silas Davis commented on SPARK-3533: I've looked at various solutions, and have

[jira] [Commented] (SPARK-9944) hive.metastore.warehouse.dir is not respected

2015-08-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705315#comment-14705315 ] Yin Huai commented on SPARK-9944: - OK. I guess {{/user/ec2-user/warehouse}} is not the one

[jira] [Resolved] (SPARK-9982) SparkR DataFrame fail to return data of Decimal type

2015-08-20 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-9982. -- Resolution: Fixed Fix Version/s: 1.5.0 SparkR DataFrame fail to return

[jira] [Updated] (SPARK-9982) SparkR DataFrame fail to return data of Decimal type

2015-08-20 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-9982: - Assignee: Alex Shkurenko SparkR DataFrame fail to return data of Decimal type

[jira] [Commented] (SPARK-9982) SparkR DataFrame fail to return data of Decimal type

2015-08-20 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705328#comment-14705328 ] Shivaram Venkataraman commented on SPARK-9982: -- Resolved by

[jira] [Commented] (SPARK-7544) pyspark.sql.types.Row should implement __getitem__

2015-08-20 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704430#comment-14704430 ] Yanbo Liang commented on SPARK-7544: I can work on it. pyspark.sql.types.Row should

[jira] [Resolved] (SPARK-10131) running spark job in docker by mesos-slave

2015-08-20 Thread Sean Owen (JIRA)
: Stream Liu I try to running spark job in docker by mesos-slave. by i always get ERROR in mesos-slave E0820 07:46:08.780293 9 slave.cpp:1643] Failed to update resources for container f2aeb5ee-2419-430c-be7d-8276947b909a of executor '20150820-064813-1684252864-5050-1-S0' of framework 20150820

[jira] [Assigned] (SPARK-10134) Improve the performance of Binary Comparison

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10134: Assignee: Apache Spark Improve the performance of Binary Comparison

[jira] [Commented] (SPARK-10134) Improve the performance of Binary Comparison

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704487#comment-14704487 ] Apache Spark commented on SPARK-10134: -- User 'chenghao-intel' has created a pull

[jira] [Assigned] (SPARK-10134) Improve the performance of Binary Comparison

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10134: Assignee: (was: Apache Spark) Improve the performance of Binary Comparison

[jira] [Commented] (SPARK-10130) type coercion for IF should have children resolved first

2015-08-20 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704513#comment-14704513 ] Cheng Hao commented on SPARK-10130: --- Can you change the fix version to 1.5? Lots of

[jira] [Commented] (SPARK-10100) AggregateFunction2's Max is slower than AggregateExpression1's MaxFunction

2015-08-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704351#comment-14704351 ] Yin Huai commented on SPARK-10100: -- How about we leave these functions as is for now

[jira] [Commented] (SPARK-9686) Spark hive jdbc client cannot get table from metadata store

2015-08-20 Thread pin_zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704374#comment-14704374 ] pin_zhang commented on SPARK-9686: -- What's the status of this bug? will it be fixed in

[jira] [Created] (SPARK-10130) type coercion for IF should have children resolved first

2015-08-20 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-10130: --- Summary: type coercion for IF should have children resolved first Key: SPARK-10130 URL: https://issues.apache.org/jira/browse/SPARK-10130 Project: Spark Issue

[jira] [Commented] (SPARK-9040) StructField datatype Conversion Error

2015-08-20 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704393#comment-14704393 ] Yanbo Liang commented on SPARK-9040: [~vnayak053] The code work well on Spark 1.4. Do

[jira] [Resolved] (SPARK-9098) Inconsistent Dense Vectors hashing between PySpark and Scala

2015-08-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9098. -- Resolution: Duplicate Target Version/s: (was: 1.6.0) I agree, I think this is a subset of

[jira] [Issue Comment Deleted] (SPARK-10067) Long delay (16 seconds) when running local session on offline machine

2015-08-20 Thread Daniel Pinyol (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Pinyol updated SPARK-10067: -- Comment: was deleted (was: Fixed after upgrading to JDK 1.8.0._60. Probably due to

[jira] [Updated] (SPARK-10132) daemon crash caused by memory leak

2015-08-20 Thread ZemingZhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZemingZhao updated SPARK-10132: --- Attachment: xqjmap.live xqjmap.all oracle_gclog attach the gclog and

[jira] [Commented] (SPARK-9089) Failing to run simple job on Spark Standalone Cluster

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704540#comment-14704540 ] Apache Spark commented on SPARK-9089: - User 'yanboliang' has created a pull request

[jira] [Assigned] (SPARK-9089) Failing to run simple job on Spark Standalone Cluster

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9089: --- Assignee: (was: Apache Spark) Failing to run simple job on Spark Standalone Cluster

[jira] [Comment Edited] (SPARK-9089) Failing to run simple job on Spark Standalone Cluster

2015-08-20 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704559#comment-14704559 ] Yanbo Liang edited comment on SPARK-9089 at 8/20/15 9:18 AM: -

[jira] [Commented] (SPARK-8805) Spark shell not working

2015-08-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704497#comment-14704497 ] Sean Owen commented on SPARK-8805: -- Yeah, I think the problem is your version of bash is

[jira] [Resolved] (SPARK-10122) AttributeError: 'RDD' object has no attribute 'offsetRanges'

2015-08-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10122. --- Resolution: Not A Problem I don't think this is a bug. Yes, only the initial RDD is actually a Kafka

[jira] [Commented] (SPARK-10092) Multi-DB support follow up

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704516#comment-14704516 ] Apache Spark commented on SPARK-10092: -- User 'liancheng' has created a pull request

[jira] [Updated] (SPARK-10130) type coercion for IF should have children resolved first

2015-08-20 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-10130: Fix Version/s: 1.5.0 type coercion for IF should have children resolved first

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-08-20 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704534#comment-14704534 ] Littlestar commented on SPARK-2883: --- spark 1.4.1: The orc file writer relies on

[jira] [Created] (SPARK-10132) daemon crash caused by memory leak

2015-08-20 Thread ZemingZhao (JIRA)
ZemingZhao created SPARK-10132: -- Summary: daemon crash caused by memory leak Key: SPARK-10132 URL: https://issues.apache.org/jira/browse/SPARK-10132 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-10134) Improve the performance of Binary Comparison

2015-08-20 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-10134: - Summary: Improve the performance of Binary Comparison Key: SPARK-10134 URL: https://issues.apache.org/jira/browse/SPARK-10134 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-10132) daemon crash caused by memory leak

2015-08-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10132. --- Resolution: Invalid I don't think any of this suggests a problem in Spark though, right? You just

[jira] [Updated] (SPARK-9089) Failing to run simple job on Spark Standalone Cluster

2015-08-20 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-9089: --- Component/s: (was: PySpark) Spark Core Failing to run simple job on Spark

[jira] [Resolved] (SPARK-10133) loadLibSVMFile fails to detect zero-based lines

2015-08-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10133. --- Resolution: Not A Problem No, because the indices have already had 1 subtracted from them.

[jira] [Commented] (SPARK-9089) Failing to run simple job on Spark Standalone Cluster

2015-08-20 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704559#comment-14704559 ] Yanbo Liang commented on SPARK-9089: I think this issue is due to failure of

[jira] [Resolved] (SPARK-8854) Documentation for Association Rules

2015-08-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-8854. -- Resolution: Duplicate Documentation for Association Rules ---

[jira] [Updated] (SPARK-10143) Parquet changed the behavior of calculating splits

2015-08-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10143: - Component/s: SQL Parquet changed the behavior of calculating splits

[jira] [Commented] (SPARK-10146) Have an easy way to set data source reader/writer specific confs

2015-08-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706205#comment-14706205 ] Yin Huai commented on SPARK-10146: -- One possible way to do it is that every data source

[jira] [Updated] (SPARK-10146) Have an easy way to set data source reader/writer specific confs

2015-08-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10146: - Issue Type: Improvement (was: Bug) Have an easy way to set data source reader/writer specific confs

[jira] [Created] (SPARK-10146) Have an easy way to set data source reader/writer specific confs

2015-08-20 Thread Yin Huai (JIRA)
Yin Huai created SPARK-10146: Summary: Have an easy way to set data source reader/writer specific confs Key: SPARK-10146 URL: https://issues.apache.org/jira/browse/SPARK-10146 Project: Spark

[jira] [Updated] (SPARK-10147) App shouldn't show in HistoryServer web when the event file has been deleted on hdfs

2015-08-20 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] meiyoula updated SPARK-10147: - Description: Phenomenon:App still shows in HistoryServer web when the event file has been deleted on

[jira] [Updated] (SPARK-10147) App shouldn't show in HistoryServer web when the event file has been deleted on hdfs

2015-08-20 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] meiyoula updated SPARK-10147: - Summary: App shouldn't show in HistoryServer web when the event file has been deleted on hdfs (was: App

[jira] [Updated] (SPARK-9983) Local physical operators for query execution

2015-08-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9983: --- Description: In distributed query execution, there are two kinds of operators: (1) operators that

[jira] [Commented] (SPARK-10122) AttributeError: 'RDD' object has no attribute 'offsetRanges'

2015-08-20 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706186#comment-14706186 ] Saisai Shao commented on SPARK-10122: - Hi [~aramesh], thanks a lot for pointing this

[jira] [Commented] (SPARK-10122) AttributeError: 'RDD' object has no attribute 'offsetRanges'

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706202#comment-14706202 ] Apache Spark commented on SPARK-10122: -- User 'jerryshao' has created a pull request

[jira] [Comment Edited] (SPARK-10146) Have an easy way to set data source reader/writer specific confs

2015-08-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706205#comment-14706205 ] Yin Huai edited comment on SPARK-10146 at 8/21/15 3:42 AM: --- One

[jira] [Created] (SPARK-10147) App still shows in HistoryServer web when the event file has been deleted on hdfs

2015-08-20 Thread meiyoula (JIRA)
meiyoula created SPARK-10147: Summary: App still shows in HistoryServer web when the event file has been deleted on hdfs Key: SPARK-10147 URL: https://issues.apache.org/jira/browse/SPARK-10147 Project:

[jira] [Commented] (SPARK-8467) Add LDAModel.describeTopics() in Python

2015-08-20 Thread Hrishikesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706257#comment-14706257 ] Hrishikesh commented on SPARK-8467: --- [~yuu.ishik...@gmail.com], are you still working on

[jira] [Assigned] (SPARK-9669) Support PySpark with Mesos Cluster mode

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9669: --- Assignee: (was: Apache Spark) Support PySpark with Mesos Cluster mode

[jira] [Assigned] (SPARK-9669) Support PySpark with Mesos Cluster mode

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9669: --- Assignee: Apache Spark Support PySpark with Mesos Cluster mode

[jira] [Commented] (SPARK-9669) Support PySpark with Mesos Cluster mode

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706262#comment-14706262 ] Apache Spark commented on SPARK-9669: - User 'tnachen' has created a pull request for

[jira] [Commented] (SPARK-9848) Add @Since annotation to new public APIs in 1.5

2015-08-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706178#comment-14706178 ] Xiangrui Meng commented on SPARK-9848: -- No, that would be too much for this release.

[jira] [Commented] (SPARK-8400) ml.ALS doesn't handle -1 block size

2015-08-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706182#comment-14706182 ] Xiangrui Meng commented on SPARK-8400: -- Sorry for my late reply! We check numBlocks

[jira] [Updated] (SPARK-10137) Avoid to restart receivers if scheduleReceivers returns balanced results

2015-08-20 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-10137: -- Assignee: Shixiong Zhu Avoid to restart receivers if scheduleReceivers returns balanced

[jira] [Updated] (SPARK-10137) Avoid to restart receivers if scheduleReceivers returns balanced results

2015-08-20 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-10137: -- Priority: Critical (was: Major) Avoid to restart receivers if scheduleReceivers returns

[jira] [Comment Edited] (SPARK-10145) Executor exit without useful messages when spark runs in spark-streaming

2015-08-20 Thread Baogang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706139#comment-14706139 ] Baogang Wang edited comment on SPARK-10145 at 8/21/15 3:27 AM:

[jira] [Assigned] (SPARK-10147) App shouldn't show in HistoryServer web when the event file has been deleted on hdfs

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10147: Assignee: (was: Apache Spark) App shouldn't show in HistoryServer web when the event

[jira] [Commented] (SPARK-10147) App shouldn't show in HistoryServer web when the event file has been deleted on hdfs

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706217#comment-14706217 ] Apache Spark commented on SPARK-10147: -- User 'XuTingjun' has created a pull request

[jira] [Assigned] (SPARK-10147) App shouldn't show in HistoryServer web when the event file has been deleted on hdfs

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10147: Assignee: Apache Spark App shouldn't show in HistoryServer web when the event file has

[jira] [Commented] (SPARK-9999) RDD-like API on top of Catalyst/DataFrame

2015-08-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706244#comment-14706244 ] Reynold Xin commented on SPARK-: This needs to be designed first. I'm not sure if

[jira] [Created] (SPARK-10142) Python checkpoint recovery does not work with non-local file path

2015-08-20 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-10142: - Summary: Python checkpoint recovery does not work with non-local file path Key: SPARK-10142 URL: https://issues.apache.org/jira/browse/SPARK-10142 Project: Spark

[jira] [Updated] (SPARK-10144) Actually show peak execution memory on UI by default

2015-08-20 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10144: -- Summary: Actually show peak execution memory on UI by default (was: Actually show peak execution

[jira] [Created] (SPARK-10144) Actually show peak execution memory by default

2015-08-20 Thread Andrew Or (JIRA)
Andrew Or created SPARK-10144: - Summary: Actually show peak execution memory by default Key: SPARK-10144 URL: https://issues.apache.org/jira/browse/SPARK-10144 Project: Spark Issue Type: Bug

[jira] [Reopened] (SPARK-10122) AttributeError: 'RDD' object has no attribute 'offsetRanges'

2015-08-20 Thread Amit Ramesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Ramesh reopened SPARK-10122: - AttributeError: 'RDD' object has no attribute 'offsetRanges'

[jira] [Comment Edited] (SPARK-10122) AttributeError: 'RDD' object has no attribute 'offsetRanges'

2015-08-20 Thread Amit Ramesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704674#comment-14704674 ] Amit Ramesh edited comment on SPARK-10122 at 8/20/15 10:51 AM:

[jira] [Updated] (SPARK-10122) AttributeError: 'RDD' object has no attribute 'offsetRanges'

2015-08-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10122: -- Priority: Major (was: Critical) Ah I see now, you are not operating on the transformed stream

[jira] [Commented] (SPARK-10122) AttributeError: 'RDD' object has no attribute 'offsetRanges'

2015-08-20 Thread Amit Ramesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704674#comment-14704674 ] Amit Ramesh commented on SPARK-10122: - [~srowen] as you can see in the example,

[jira] [Commented] (SPARK-10015) ML model broadcasts should be stored in private vars: spark.ml tree ensembles

2015-08-20 Thread Sameer Abhyankar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704740#comment-14704740 ] Sameer Abhyankar commented on SPARK-10015: -- [~josephkb] I have created a common

[jira] [Resolved] (SPARK-10092) Multi-DB support follow up

2015-08-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-10092. Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8336

[jira] [Commented] (SPARK-8436) Inconsistent behavior when converting a Timestamp column to Integer/Long and then convert back to Timestamp

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704712#comment-14704712 ] Apache Spark commented on SPARK-8436: - User 'x1-' has created a pull request for this

[jira] [Assigned] (SPARK-8436) Inconsistent behavior when converting a Timestamp column to Integer/Long and then convert back to Timestamp

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8436: --- Assignee: (was: Apache Spark) Inconsistent behavior when converting a Timestamp column

[jira] [Commented] (SPARK-10100) AggregateFunction2's Max is slower than AggregateExpression1's MaxFunction

2015-08-20 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704742#comment-14704742 ] Herman van Hovell commented on SPARK-10100: --- Lets leave it for 1.6.

[jira] [Commented] (SPARK-6196) Add MAPR 4.0.2 support to the build

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704704#comment-14704704 ] Apache Spark commented on SPARK-6196: - User 'srowen' has created a pull request for

[jira] [Commented] (SPARK-10109) NPE when saving Parquet To HDFS

2015-08-20 Thread Virgil Palanciuc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704710#comment-14704710 ] Virgil Palanciuc commented on SPARK-10109: -- I think I know what caused this -

[jira] [Assigned] (SPARK-8436) Inconsistent behavior when converting a Timestamp column to Integer/Long and then convert back to Timestamp

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8436: --- Assignee: Apache Spark Inconsistent behavior when converting a Timestamp column to

[jira] [Created] (SPARK-10135) Percent of pruned partitions is shown wrong

2015-08-20 Thread Romi Kuntsman (JIRA)
Romi Kuntsman created SPARK-10135: - Summary: Percent of pruned partitions is shown wrong Key: SPARK-10135 URL: https://issues.apache.org/jira/browse/SPARK-10135 Project: Spark Issue Type:

[jira] [Commented] (SPARK-8580) Add Parquet files generated by different systems to test interoperability and compatibility

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706041#comment-14706041 ] Apache Spark commented on SPARK-8580: - User 'liancheng' has created a pull request for

[jira] [Assigned] (SPARK-8580) Add Parquet files generated by different systems to test interoperability and compatibility

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8580: --- Assignee: Cheng Lian (was: Apache Spark) Add Parquet files generated by different systems

[jira] [Assigned] (SPARK-8580) Add Parquet files generated by different systems to test interoperability and compatibility

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8580: --- Assignee: Apache Spark (was: Cheng Lian) Add Parquet files generated by different systems

[jira] [Assigned] (SPARK-10143) Parquet changed the behavior of calculating splits

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10143: Assignee: (was: Apache Spark) Parquet changed the behavior of calculating splits

[jira] [Commented] (SPARK-10143) Parquet changed the behavior of calculating splits

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706045#comment-14706045 ] Apache Spark commented on SPARK-10143: -- User 'yhuai' has created a pull request for

[jira] [Assigned] (SPARK-10143) Parquet changed the behavior of calculating splits

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10143: Assignee: Apache Spark Parquet changed the behavior of calculating splits

[jira] [Issue Comment Deleted] (SPARK-10143) Parquet changed the behavior of calculating splits

2015-08-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10143: - Comment: was deleted (was: For something quick, we can use the row group size set in hadoop conf to set

[jira] [Commented] (SPARK-10143) Parquet changed the behavior of calculating splits

2015-08-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705988#comment-14705988 ] Yin Huai commented on SPARK-10143: -- [~rdblue] Can you confirm the behavior change of

[jira] [Assigned] (SPARK-10144) Actually show peak execution memory on UI by default

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10144: Assignee: Apache Spark (was: Andrew Or) Actually show peak execution memory on UI by

[jira] [Commented] (SPARK-10144) Actually show peak execution memory on UI by default

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706033#comment-14706033 ] Apache Spark commented on SPARK-10144: -- User 'andrewor14' has created a pull request

[jira] [Assigned] (SPARK-10144) Actually show peak execution memory on UI by default

2015-08-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10144: Assignee: Andrew Or (was: Apache Spark) Actually show peak execution memory on UI by

[jira] [Commented] (SPARK-10145) Executor exit without useful messages when spark runs in spark-streaming

2015-08-20 Thread Baogang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706139#comment-14706139 ] Baogang Wang commented on SPARK-10145: -- the spark-defaults.conf is as follows:

[jira] [Comment Edited] (SPARK-10145) Executor exit without useful messages when spark runs in spark-streaming

2015-08-20 Thread Baogang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706139#comment-14706139 ] Baogang Wang edited comment on SPARK-10145 at 8/21/15 2:34 AM:

[jira] [Resolved] (SPARK-10140) Add target fields to @Since annotation

2015-08-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-10140. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8344

  1   2   3   >