[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-05-27 Thread koeninger
Github user koeninger commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-222164687 I think this PR is fairly out of date regarding things like needing to cache consumers on executors, getPreferredLocations, and a separate subproject for kafka-0.8

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-05-27 Thread Xaerxess
Github user Xaerxess commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-222110057 Kafka 0.10.0 is out, any plans of merging this PR or updating it with newest Kafka? --- If your project is set up for it, you can reply to this email and have your re

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-05-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-217477017 @markgrover Mind adding `Closes #10681` in the PR description so that merging script can close that together? --- If your project is set up for it, you can reply t

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191579126 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191579124 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191579006 **[Test build #52353 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52353/consoleFull)** for PR 10953 at commit [`229b773`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191575234 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191575227 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191574588 **[Test build #52352 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52352/consoleFull)** for PR 10953 at commit [`993c6fd`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191555926 **[Test build #52353 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52353/consoleFull)** for PR 10953 at commit [`229b773`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191545207 **[Test build #52352 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52352/consoleFull)** for PR 10953 at commit [`993c6fd`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191544638 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191544591 Hmm, this is weird, these tests were passing before. Anyways, I was able to circumvent them by setting SPARK_DRIVER_MEMORY to a higher value on my local box. I'll ki

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191357308 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191357302 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191357204 **[Test build #52322 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52322/consoleFull)** for PR 10953 at commit [`993c6fd`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191302375 **[Test build #52322 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52322/consoleFull)** for PR 10953 at commit [`993c6fd`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-01 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191021949 Will fix shortly. I was building against a custom built version of kafka because 0.9.1 wasn't out which had some blocker fixes, to make tests pass. On Mar 1, 2

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191016755 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191016757 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191016747 **[Test build #52280 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52280/consoleFull)** for PR 10953 at commit [`075226e`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-03-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-191016300 **[Test build #52280 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52280/consoleFull)** for PR 10953 at commit [`075226e`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-183444859 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-183444847 **[Test build #51193 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51193/consoleFull)** for PR 10953 at commit [`c422cd5`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-183444857 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-183440359 **[Test build #51193 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51193/consoleFull)** for PR 10953 at commit [`c422cd5`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-183032500 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-183032489 **[Test build #51125 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51125/consoleFull)** for PR 10953 at commit [`9983e7d`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-183032502 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-183021465 **[Test build #51125 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51125/consoleFull)** for PR 10953 at commit [`9983e7d`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-09 Thread markgrover
Github user markgrover commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r52361163 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/NewKafkaCluster.scala --- @@ -0,0 +1,156 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-09 Thread markgrover
Github user markgrover commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r52361057 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStreamBase.scala --- @@ -0,0 +1,158 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-09 Thread markgrover
Github user markgrover commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r52360764 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -610,6 +619,417 @@ object KafkaUtils { Set(to

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-09 Thread markgrover
Github user markgrover commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r52360194 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/NewDirectKafkaInputDStream.scala --- @@ -0,0 +1,134 @@ +/* --- End diff

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-181672442 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-181672445 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-181672272 **[Test build #50946 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50946/consoleFull)** for PR 10953 at commit [`846a85a`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-181633197 **[Test build #50946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50946/consoleFull)** for PR 10953 at commit [`846a85a`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread mariobriggs
Github user mariobriggs commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r52203773 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDDBase.scala --- @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread mariobriggs
Github user mariobriggs commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r52202954 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/NewKafkaCluster.scala --- @@ -0,0 +1,156 @@ +/* + * Licensed to the Apa

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread mariobriggs
Github user mariobriggs commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r52202548 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/NewDirectKafkaInputDStream.scala --- @@ -0,0 +1,134 @@ +/* --- End diff

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread mariobriggs
Github user mariobriggs commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r52201905 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -610,6 +619,417 @@ object KafkaUtils { Set(t

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread mariobriggs
Github user mariobriggs commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r52201159 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStreamBase.scala --- @@ -0,0 +1,158 @@ +/* + * License

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread vgromakowski
Github user vgromakowski commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-181407863 Authentication and access control are mandatory for enterprise deployments with security constraints, specially multi tenant ones. I have lots of interest for this

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-181328631 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-181328633 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-181328215 **[Test build #50916 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50916/consoleFull)** for PR 10953 at commit [`abd167c`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-181266094 **[Test build #50916 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50916/consoleFull)** for PR 10953 at commit [`abd167c`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-08 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-181264407 Thanks for your feedback, @harishreedharan I have done a significant refactoring of code based on your feedback, sharing code where possible. The new RDD, is now cal

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-179600365 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-179600368 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-179599885 **[Test build #50718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50718/consoleFull)** for PR 10953 at commit [`10a18b8`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-179564786 **[Test build #50718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50718/consoleFull)** for PR 10953 at commit [`10a18b8`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-03 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-179560652 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-179546246 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-179546248 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-03 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-179511363 I have a more fundamental question - given that this patch does not add a whole lot of new functionality but just ports the currently functionality to use the n

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-03 Thread harishreedharan
Github user harishreedharan commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r51799340 --- Diff: external/kafka-newapi/src/main/scala/org/apache/spark/streaming/kafka/newapi/DirectKafkaInputDStream.scala --- @@ -0,0 +1,208 @@ +/*

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-03 Thread harishreedharan
Github user harishreedharan commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r51798611 --- Diff: external/kafka-newapi/pom.xml --- @@ -0,0 +1,122 @@ + + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178954985 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178954982 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178954830 **[Test build #50615 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50615/consoleFull)** for PR 10953 at commit [`edfca9f`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178901409 **[Test build #50615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50615/consoleFull)** for PR 10953 at commit [`edfca9f`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178892437 **[Test build #50610 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50610/consoleFull)** for PR 10953 at commit [`7dcb848`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178892440 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178891809 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178891802 **[Test build #50613 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50613/consoleFull)** for PR 10953 at commit [`7dcb848`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178892002 **[Test build #50610 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50610/consoleFull)** for PR 10953 at commit [`7dcb848`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178891812 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178891202 **[Test build #50613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50613/consoleFull)** for PR 10953 at commit [`7dcb848`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178887726 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178886469 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178886470 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178886459 **[Test build #50609 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50609/consoleFull)** for PR 10953 at commit [`8fae471`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178884266 **[Test build #50609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50609/consoleFull)** for PR 10953 at commit [`8fae471`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178864810 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178864817 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178864366 **[Test build #50588 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50588/consoleFull)** for PR 10953 at commit [`1f77501`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178803662 **[Test build #50588 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50588/consoleFull)** for PR 10953 at commit [`1f77501`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178797235 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178797109 Ok, thanks, Marcelo. I didn't think I was because Jenkins didn't trigger on my explicit request but I will try again. --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178786575 You already are on it, given the tests for you PR started automatically when you submitted it. --- If your project is set up for it, you can reply to this email and hav

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178784728 Can someone add me to the whitelist please? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178775083 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178775094 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178763362 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178763349 The previous test failures seems like a timeout when fetching the PR from github: ``` > git fetch --tags --progress https://github.com/apache/spark.git +refs

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178756198 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178756201 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178746978 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178746983 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178746297 **[Test build #50568 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50568/consoleFull)** for PR 10953 at commit [`d932d75`](https://g

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178678612 **[Test build #50568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50568/consoleFull)** for PR 10953 at commit [`d932d75`](https://gi

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-02 Thread mariobriggs
Github user mariobriggs commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r51561776 --- Diff: external/kafka-newapi/src/main/scala/org/apache/spark/streaming/kafka/newapi/KafkaRDD.scala --- @@ -0,0 +1,237 @@ +/* + * Licensed to

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-01 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-178377734 @mariobriggs, thanks for your feedback. I have incorporated all but one of it (related to poll time). In that case, I have asked a follow up quesiton. If I missed so

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-02-01 Thread markgrover
Github user markgrover commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r51524399 --- Diff: external/kafka-newapi/src/main/scala/org/apache/spark/streaming/kafka/newapi/KafkaRDD.scala --- @@ -0,0 +1,237 @@ +/* + * Licensed to t

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-01-31 Thread mariobriggs
Github user mariobriggs commented on a diff in the pull request: https://github.com/apache/spark/pull/10953#discussion_r51381737 --- Diff: external/kafka-assembly/pom.xml --- @@ -43,6 +43,11 @@ org.apache.spark + spark-streaming-kafka-newapi_

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-177060819 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-177060820 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12177] [STREAMING] Update KafkaDStreams...

2016-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10953#issuecomment-177060738 **[Test build #50426 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50426/consoleFull)** for PR 10953 at commit [`82223f8`](https://g

  1   2   >