[jira] [Created] (SPARK-6802) User Defined Aggregate Function Refactoring

2015-04-09 Thread cynepia (JIRA)
cynepia created SPARK-6802: -- Summary: User Defined Aggregate Function Refactoring Key: SPARK-6802 URL: https://issues.apache.org/jira/browse/SPARK-6802 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-6028) Provide an alternative RPC implementation based on the network transport module

2015-04-09 Thread Andrea Peruffo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487329#comment-14487329 ] Andrea Peruffo commented on SPARK-6028: --- The main blocking problem in the

[jira] [Updated] (SPARK-6799) Add dataframe examples for SparkR

2015-04-09 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-6799: - Target Version/s: 1.4.0 Add dataframe examples for SparkR

[jira] [Commented] (SPARK-6772) spark sql error when running code on large number of records

2015-04-09 Thread Aditya Parmar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486853#comment-14486853 ] Aditya Parmar commented on SPARK-6772: -- Please find the code below import

[jira] [Assigned] (SPARK-2352) [MLLIB] Add Artificial Neural Network (ANN) to Spark

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-2352: --- Assignee: Apache Spark (was: Bert Greevenbosch) [MLLIB] Add Artificial Neural Network

[jira] [Commented] (SPARK-765) Test suite should run Spark example programs

2015-04-09 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487060#comment-14487060 ] Yu Ishikawa commented on SPARK-765: --- I'm very sorry. I could run a test in

[jira] [Assigned] (SPARK-6806) SparkR examples in programming guide

2015-04-09 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-6806: - Assignee: Davies Liu SparkR examples in programming guide

[jira] [Resolved] (SPARK-6801) spark-submit is not able to identify Alive Master in case of multiple master

2015-04-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6801. -- Resolution: Duplicate spark-submit is not able to identify Alive Master in case of multiple master

[jira] [Commented] (SPARK-3947) [Spark SQL] UDAF Support

2015-04-09 Thread cynepia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487093#comment-14487093 ] cynepia commented on SPARK-3947: Can someone update on where do we stand on this issue?

[jira] [Updated] (SPARK-6811) Building binary R packages for SparkR

2015-04-09 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-6811: - Target Version/s: 1.4.0 Building binary R packages for SparkR

[jira] [Resolved] (SPARK-6804) System.exit(1) on error

2015-04-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6804. -- Resolution: Duplicate System.exit(1) on error --- Key:

[jira] [Created] (SPARK-6811) Building binary R packages for SparkR

2015-04-09 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-6811: Summary: Building binary R packages for SparkR Key: SPARK-6811 URL: https://issues.apache.org/jira/browse/SPARK-6811 Project: Spark Issue

[jira] [Created] (SPARK-6812) filter() on DataFrame does not work as expected

2015-04-09 Thread Davies Liu (JIRA)
Davies Liu created SPARK-6812: - Summary: filter() on DataFrame does not work as expected Key: SPARK-6812 URL: https://issues.apache.org/jira/browse/SPARK-6812 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

2015-04-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487934#comment-14487934 ] Marcelo Vanzin commented on SPARK-6229: --- Hi Jeff, Those kinds of differences are

[jira] [Commented] (SPARK-3947) [Spark SQL] UDAF Support

2015-04-09 Thread cynepia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487222#comment-14487222 ] cynepia commented on SPARK-3947: Hi Takeshi San, Thanks for the quick response. I would

[jira] [Commented] (SPARK-6691) Abstract and add a dynamic RateLimiter for Spark Streaming

2015-04-09 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487793#comment-14487793 ] Tathagata Das commented on SPARK-6691: -- Yes, I complete agree to the motivation.

[jira] [Created] (SPARK-6816) Add SparkConf API to configure SparkR

2015-04-09 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-6816: Summary: Add SparkConf API to configure SparkR Key: SPARK-6816 URL: https://issues.apache.org/jira/browse/SPARK-6816 Project: Spark Issue

[jira] [Commented] (SPARK-6691) Abstract and add a dynamic RateLimiter for Spark Streaming

2015-04-09 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486980#comment-14486980 ] Tathagata Das commented on SPARK-6691: -- [~jerryshao] This is a very good attempt!

[jira] [Created] (SPARK-6819) Support nested types in SparkR DataFrame

2015-04-09 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-6819: Summary: Support nested types in SparkR DataFrame Key: SPARK-6819 URL: https://issues.apache.org/jira/browse/SPARK-6819 Project: Spark Issue

[jira] [Created] (SPARK-6799) Add dataframe examples for SparkR

2015-04-09 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-6799: Summary: Add dataframe examples for SparkR Key: SPARK-6799 URL: https://issues.apache.org/jira/browse/SPARK-6799 Project: Spark Issue Type:

[jira] [Updated] (SPARK-2673) Improve Enable to attach Debugger to Executors easily

2015-04-09 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2673: -- Description: In current implementation, we are difficult to attach debugger to each Executor

[jira] [Created] (SPARK-6803) Support SparkR Streaming

2015-04-09 Thread Hao (JIRA)
Hao created SPARK-6803: -- Summary: Support SparkR Streaming Key: SPARK-6803 URL: https://issues.apache.org/jira/browse/SPARK-6803 Project: Spark Issue Type: New Feature Components: SparkR,

[jira] [Commented] (SPARK-6803) [SparkR] Support SparkR Streaming

2015-04-09 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487872#comment-14487872 ] Tathagata Das commented on SPARK-6803: -- I know of many use cases where python is

[jira] [Updated] (SPARK-2673) Improve Enable to attach Debugger to Executors easily

2015-04-09 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2673: -- Component/s: Web UI Improve Enable to attach Debugger to Executors easily

[jira] [Created] (SPARK-6823) Add a model.matrix like capability to DataFrames (modelDataFrame)

2015-04-09 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-6823: Summary: Add a model.matrix like capability to DataFrames (modelDataFrame) Key: SPARK-6823 URL: https://issues.apache.org/jira/browse/SPARK-6823

[jira] [Assigned] (SPARK-2673) Improve Enable to attach Debugger to Executors easily

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-2673: --- Assignee: Apache Spark Improve Enable to attach Debugger to Executors easily

[jira] [Commented] (SPARK-2673) Improve Enable to attach Debugger to Executors easily

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486897#comment-14486897 ] Apache Spark commented on SPARK-2673: - User 'sarutak' has created a pull request for

[jira] [Assigned] (SPARK-2673) Improve Enable to attach Debugger to Executors easily

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-2673: --- Assignee: (was: Apache Spark) Improve Enable to attach Debugger to Executors easily

[jira] [Created] (SPARK-6822) lapplyPartition passes empty list to function

2015-04-09 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-6822: Summary: lapplyPartition passes empty list to function Key: SPARK-6822 URL: https://issues.apache.org/jira/browse/SPARK-6822 Project: Spark

[jira] [Created] (SPARK-6805) ML Pipeline API in SparkR

2015-04-09 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-6805: Summary: ML Pipeline API in SparkR Key: SPARK-6805 URL: https://issues.apache.org/jira/browse/SPARK-6805 Project: Spark Issue Type: Umbrella

[jira] [Created] (SPARK-6817) DataFrame UDFs in R

2015-04-09 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-6817: Summary: DataFrame UDFs in R Key: SPARK-6817 URL: https://issues.apache.org/jira/browse/SPARK-6817 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-6767) Documentation error in Spark SQL Readme file

2015-04-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6767. -- Resolution: Fixed Fix Version/s: 1.4.0 1.3.2 Assignee: Tijo Thomas

[jira] [Assigned] (SPARK-3276) Provide a API to specify MIN_REMEMBER_DURATION for files to consider as input in streaming

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3276: --- Assignee: (was: Apache Spark) Provide a API to specify MIN_REMEMBER_DURATION for files

[jira] [Commented] (SPARK-3276) Provide a API to specify MIN_REMEMBER_DURATION for files to consider as input in streaming

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487073#comment-14487073 ] Apache Spark commented on SPARK-3276: - User 'emres' has created a pull request for

[jira] [Created] (SPARK-6804) System.exit(1) on error

2015-04-09 Thread Alberto (JIRA)
Alberto created SPARK-6804: -- Summary: System.exit(1) on error Key: SPARK-6804 URL: https://issues.apache.org/jira/browse/SPARK-6804 Project: Spark Issue Type: Improvement Reporter:

[jira] [Updated] (SPARK-6803) [SparkR] Support SparkR Streaming

2015-04-09 Thread Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao updated SPARK-6803: --- Summary: [SparkR] Support SparkR Streaming (was: Support SparkR Streaming) [SparkR] Support SparkR Streaming

[jira] [Commented] (SPARK-6735) Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.

2015-04-09 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487670#comment-14487670 ] Sandy Ryza commented on SPARK-6735: --- Hi [~twinkle], can you submit the PR against the

[jira] [Comment Edited] (SPARK-6735) Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.

2015-04-09 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487670#comment-14487670 ] Sandy Ryza edited comment on SPARK-6735 at 4/9/15 5:06 PM: --- Hi

[jira] [Created] (SPARK-6821) Refactor SerDe API in SparkR to be more developer friendly

2015-04-09 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-6821: Summary: Refactor SerDe API in SparkR to be more developer friendly Key: SPARK-6821 URL: https://issues.apache.org/jira/browse/SPARK-6821 Project:

[jira] [Created] (SPARK-6813) SparkR style guide

2015-04-09 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-6813: Summary: SparkR style guide Key: SPARK-6813 URL: https://issues.apache.org/jira/browse/SPARK-6813 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-6677) pyspark.sql nondeterministic issue with row fields

2015-04-09 Thread Stefano Parmesan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487010#comment-14487010 ] Stefano Parmesan commented on SPARK-6677: - Uhm, don't know what to say. Let's try

[jira] [Commented] (SPARK-6803) Support SparkR Streaming

2015-04-09 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487800#comment-14487800 ] Shivaram Venkataraman commented on SPARK-6803: -- Thanks [~hlin09]. The PySpark

[jira] [Updated] (SPARK-2673) Improve Enable to attach Debugger to Executors easily

2015-04-09 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2673: -- Summary: Improve Enable to attach Debugger to Executors easily (was: Improve Spark so that we

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

2015-04-09 Thread Jeffrey Turpin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487878#comment-14487878 ] Jeffrey Turpin commented on SPARK-6229: --- Hey Marcelo, I have been working on

[jira] [Created] (SPARK-6810) Performance benchmarks for SparkR

2015-04-09 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-6810: Summary: Performance benchmarks for SparkR Key: SPARK-6810 URL: https://issues.apache.org/jira/browse/SPARK-6810 Project: Spark Issue Type:

[jira] [Created] (SPARK-6800) Reading from JDBC with SQLContext, using lower/upper bounds and numPartitions gives incorrect results.

2015-04-09 Thread JIRA
Micael Capitão created SPARK-6800: - Summary: Reading from JDBC with SQLContext, using lower/upper bounds and numPartitions gives incorrect results. Key: SPARK-6800 URL:

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

2015-04-09 Thread Jeffrey Turpin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14488104#comment-14488104 ] Jeffrey Turpin commented on SPARK-6229: --- Hey Marcelo, So what I have done is to

[jira] [Commented] (SPARK-6770) DirectKafkaInputDStream has not been initialized when recovery from checkpoint

2015-04-09 Thread yangping wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486794#comment-14486794 ] yangping wu commented on SPARK-6770: Ok, Thank you very much for your reply. I will

[jira] [Resolved] (SPARK-6751) Spark History Server support multiple application attempts

2015-04-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-6751. -- Resolution: Duplicate Dup of SPARK-4705 Spark History Server support multiple application

[jira] [Assigned] (SPARK-5960) Allow AWS credentials to be passed to KinesisUtils.createStream()

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5960: --- Assignee: Apache Spark (was: Chris Fregly) Allow AWS credentials to be passed to

[jira] [Created] (SPARK-6820) Convert NAs to null type in SparkR DataFrames

2015-04-09 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-6820: Summary: Convert NAs to null type in SparkR DataFrames Key: SPARK-6820 URL: https://issues.apache.org/jira/browse/SPARK-6820 Project: Spark

[jira] [Created] (SPARK-6806) SparkR examples in programming guide

2015-04-09 Thread Davies Liu (JIRA)
Davies Liu created SPARK-6806: - Summary: SparkR examples in programming guide Key: SPARK-6806 URL: https://issues.apache.org/jira/browse/SPARK-6806 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-6807) Merge recent changes in SparkR-pkg into Spark

2015-04-09 Thread Davies Liu (JIRA)
Davies Liu created SPARK-6807: - Summary: Merge recent changes in SparkR-pkg into Spark Key: SPARK-6807 URL: https://issues.apache.org/jira/browse/SPARK-6807 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-6636) Use public DNS hostname everywhere in spark_ec2.py

2015-04-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14488245#comment-14488245 ] Nicholas Chammas commented on SPARK-6636: - [~aasted] - Can you elaborate on this?

[jira] [Commented] (SPARK-3947) [Spark SQL] UDAF Support

2015-04-09 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487278#comment-14487278 ] Takeshi Yamamuro commented on SPARK-3947: - Sorry, but Im not sure about the issue

[jira] [Commented] (SPARK-2451) Enable to load config file for Akka

2015-04-09 Thread Andrea Peruffo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487386#comment-14487386 ] Andrea Peruffo commented on SPARK-2451: --- As per:

[jira] [Updated] (SPARK-6804) System.exit(1) on error

2015-04-09 Thread Alberto (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alberto updated SPARK-6804: --- Description: We are developing a web application that is using Spark under the hood. Testing our app we have

[jira] [Assigned] (SPARK-2352) [MLLIB] Add Artificial Neural Network (ANN) to Spark

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-2352: --- Assignee: Bert Greevenbosch (was: Apache Spark) [MLLIB] Add Artificial Neural Network

[jira] [Commented] (SPARK-3937) Unsafe memory access inside of Snappy library

2015-04-09 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486810#comment-14486810 ] Guoqiang Li commented on SPARK-3937: The bug seems to be caused by

[jira] [Updated] (SPARK-2205) Unnecessary exchange operators in a join on multiple tables with the same join key.

2015-04-09 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-2205: Priority: Critical (was: Minor) Unnecessary exchange operators in a join on multiple tables with the same

[jira] [Commented] (SPARK-6816) Add SparkConf API to configure SparkR

2015-04-09 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14488237#comment-14488237 ] Shivaram Venkataraman commented on SPARK-6816: -- Comments from SparkR JIRA

[jira] [Commented] (SPARK-6770) DirectKafkaInputDStream has not been initialized when recovery from checkpoint

2015-04-09 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486782#comment-14486782 ] Saisai Shao commented on SPARK-6770: From my understanding, the basic scenario of your

[jira] [Commented] (SPARK-2949) SparkContext does not fate-share with ActorSystem

2015-04-09 Thread Andrea Peruffo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487467#comment-14487467 ] Andrea Peruffo commented on SPARK-2949: --- Which version of Spark? ActorSystem has a

[jira] [Resolved] (SPARK-6343) Make doc more explicit regarding network connectivity requirements

2015-04-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6343. -- Resolution: Fixed Fix Version/s: 1.4.0 1.3.2 Issue resolved by pull request

[jira] [Created] (SPARK-6808) Checkpointing after zipPartitions results in NODE_LOCAL execution

2015-04-09 Thread Xinghao Pan (JIRA)
Xinghao Pan created SPARK-6808: -- Summary: Checkpointing after zipPartitions results in NODE_LOCAL execution Key: SPARK-6808 URL: https://issues.apache.org/jira/browse/SPARK-6808 Project: Spark

[jira] [Assigned] (SPARK-3276) Provide a API to specify MIN_REMEMBER_DURATION for files to consider as input in streaming

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3276: --- Assignee: Apache Spark Provide a API to specify MIN_REMEMBER_DURATION for files to consider

[jira] [Commented] (SPARK-6803) [SparkR] Support SparkR Streaming

2015-04-09 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14488315#comment-14488315 ] Davies Liu commented on SPARK-6803: --- After a quick look over the prototype, the callback

[jira] [Updated] (SPARK-6343) Make doc more explicit regarding network connectivity requirements

2015-04-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6343: - Assignee: Peter Parente Make doc more explicit regarding network connectivity requirements

[jira] [Commented] (SPARK-6691) Abstract and add a dynamic RateLimiter for Spark Streaming

2015-04-09 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487323#comment-14487323 ] Saisai Shao commented on SPARK-6691: Hi [~tdas], thanks a lot for your comments. I

[jira] [Commented] (SPARK-6646) Spark 2.0: Rearchitecting Spark for Mobile Platforms

2015-04-09 Thread Marko Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14488225#comment-14488225 ] Marko Bonaci commented on SPARK-6646: - Wait a minute, don't postpone this one just

[jira] [Commented] (SPARK-5960) Allow AWS credentials to be passed to KinesisUtils.createStream()

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487177#comment-14487177 ] Apache Spark commented on SPARK-5960: - User 'mce' has created a pull request for this

[jira] [Assigned] (SPARK-5960) Allow AWS credentials to be passed to KinesisUtils.createStream()

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5960: --- Assignee: Chris Fregly (was: Apache Spark) Allow AWS credentials to be passed to

[jira] [Created] (SPARK-6814) Support sorting for any data type in SparkR

2015-04-09 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-6814: Summary: Support sorting for any data type in SparkR Key: SPARK-6814 URL: https://issues.apache.org/jira/browse/SPARK-6814 Project: Spark

[jira] [Updated] (SPARK-6800) Reading from JDBC with SQLContext, using lower/upper bounds and numPartitions gives incorrect results.

2015-04-09 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Micael Capitão updated SPARK-6800: -- Description: Having a Derby table with people info (id, name, age) defined like this: {code}

[jira] [Updated] (SPARK-6772) spark sql error when running code on large number of records

2015-04-09 Thread Aditya Parmar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Parmar updated SPARK-6772: - Description: Hi all , I am getting an Arrayoutboundsindex error when i try to run a simple

[jira] [Assigned] (SPARK-6796) Add the batch list to StreamingPage

2015-04-09 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-6796: Assignee: Shixiong Zhu Add the batch list to StreamingPage

[jira] [Commented] (SPARK-6770) DirectKafkaInputDStream has not been initialized when recovery from checkpoint

2015-04-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487103#comment-14487103 ] Sean Owen commented on SPARK-6770: -- That may be so, but it's not obvious that you simply

[jira] [Assigned] (SPARK-6807) Merge recent changes in SparkR-pkg into Spark

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6807: --- Assignee: Davies Liu (was: Apache Spark) Merge recent changes in SparkR-pkg into Spark

[jira] [Commented] (SPARK-6807) Merge recent changes in SparkR-pkg into Spark

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487687#comment-14487687 ] Apache Spark commented on SPARK-6807: - User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-6807) Merge recent changes in SparkR-pkg into Spark

2015-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6807: --- Assignee: Apache Spark (was: Davies Liu) Merge recent changes in SparkR-pkg into Spark

[jira] [Commented] (SPARK-6812) filter() on DataFrame does not work as expected

2015-04-09 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14488128#comment-14488128 ] Shivaram Venkataraman commented on SPARK-6812: -- Hmm - don't we have a unit

[jira] [Commented] (SPARK-3947) [Spark SQL] UDAF Support

2015-04-09 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487175#comment-14487175 ] Takeshi Yamamuro commented on SPARK-3947: - See SPARK-4233, we are refactoring the