Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/247
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabl
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39817343
Looks good TD, merging this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39802027
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39802028
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13866/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39801451
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13865/
---
If your project is set up for it, you can r
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39801450
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39799987
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-3974
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39799424
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39799433
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11371039
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
---
@@ -21,15 +21,17 @@ import org.scalatest.{FunSuite, BeforeAndAfter}
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11371044
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
---
@@ -215,4 +271,29 @@ class StreamingContextSuite extends FunSuite w
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11371038
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/NetworkInputTracker.scala
---
@@ -17,20 +17,16 @@
package org.apache.spark.str
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11371036
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala
---
@@ -77,30 +82,79 @@ class JobGenerator(jobScheduler: JobScheduler)
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11371037
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobScheduler.scala
---
@@ -50,36 +50,54 @@ class JobScheduler(val ssc: StreamingContext)
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11367954
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
---
@@ -215,4 +271,29 @@ class StreamingContextSuite extends FunS
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11367919
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
---
@@ -21,15 +21,17 @@ import org.scalatest.{FunSuite, BeforeAnd
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11367807
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/NetworkInputTracker.scala
---
@@ -17,20 +17,16 @@
package org.apache.spa
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11367750
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobScheduler.scala
---
@@ -50,36 +50,54 @@ class JobScheduler(val ssc: StreamingCon
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11367644
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala
---
@@ -77,30 +82,79 @@ class JobGenerator(jobScheduler: JobSche
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11367369
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -112,7 +131,7 @@ abstract class NetworkReceiver[T: C
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39787095
Build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this f
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39787097
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13854/
---
If your project is set up for it, you can r
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39781404
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39781388
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11358984
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
---
@@ -133,18 +147,61 @@ class StreamingContextSuite extends FunSuite
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11319581
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
---
@@ -133,18 +147,61 @@ class StreamingContextSuite extends FunSu
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39545197
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13756/
---
If your project is set up for it, you can r
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39545196
Build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this f
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39541536
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39541530
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39534742
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13754/
---
If your project is set up for it, you can r
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39534741
Build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this f
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39534715
Hey @pwendell I did a pass as well. Some of the network receiver related
changes I have punted for now as those have actually changed with #300 and its
best to analyze them i
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11284020
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/util/RecurringTimer.scala
---
@@ -17,44 +17,80 @@
package org.apache.spark.streaming.uti
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39534643
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39534647
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11283933
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T: Clas
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11282497
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala
---
@@ -77,30 +82,79 @@ class JobGenerator(jobScheduler: JobScheduler)
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11282456
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala
---
@@ -77,30 +82,79 @@ class JobGenerator(jobScheduler: JobScheduler)
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11282425
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T: Clas
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11282410
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T: Clas
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11282396
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T:
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11282380
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T: Clas
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11281598
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T: Clas
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11281533
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T: Clas
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11281527
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T: Clas
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11281522
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -112,7 +131,7 @@ abstract class NetworkReceiver[T: ClassTa
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11281494
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala ---
@@ -430,12 +448,27 @@ class StreamingContext private[streaming] (
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11281342
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala ---
@@ -430,12 +448,27 @@ class StreamingContext private[streaming] (
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11281176
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala ---
@@ -158,6 +158,15 @@ class StreamingContext private[streaming] (
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39402799
Hey TD - Did a pass. Had some questions and surface level comment.
One thing that came up a few was the use of a busy-wait/Thread.sleep along
with interruption. I
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-39401108
@witgo I completely agree that a better lifecycle interface is necessary.
That has been addressed by a different PR #300. This PR only adds the changes
necessary to the other
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11235082
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala
---
@@ -77,30 +82,79 @@ class JobGenerator(jobScheduler: JobSchedu
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11235032
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala
---
@@ -77,30 +82,79 @@ class JobGenerator(jobScheduler: JobSchedu
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11234823
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T:
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11234745
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T:
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11234673
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/util/RecurringTimer.scala
---
@@ -17,44 +17,80 @@
package org.apache.spark.streaming
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11229778
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/util/RecurringTimer.scala
---
@@ -17,44 +17,80 @@
package org.apache.spark.streaming
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11229360
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/util/RecurringTimer.scala
---
@@ -17,44 +17,80 @@
package org.apache.spark.streaming
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11229496
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/util/RecurringTimer.scala
---
@@ -17,44 +17,80 @@
package org.apache.spark.streaming
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11228838
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T:
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11228760
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T:
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11228531
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T:
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11227952
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -112,7 +131,7 @@ abstract class NetworkReceiver[T: Cla
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11227829
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -124,84 +143,109 @@ abstract class NetworkReceiver[T:
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11227633
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala ---
@@ -430,12 +448,27 @@ class StreamingContext private[streaming] (
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11227580
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala ---
@@ -430,12 +448,27 @@ class StreamingContext private[streaming] (
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/247#discussion_r11227467
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala ---
@@ -158,6 +158,15 @@ class StreamingContext private[streaming] (
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-38854674
@pwendell
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled a
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-38854653
Jenkins, test this again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this f
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-38765648
One or more automated tests failed
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13491/
---
If your p
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-38765647
Build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-38765200
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fea
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-38765199
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/247#issuecomment-38764889
Add a lifecycle interface is a good idea?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project do
GitHub user tdas opened a pull request:
https://github.com/apache/spark/pull/247
[SPARK-1331] Added graceful shutdown to Spark Streaming
Current version of StreamingContext.stop() directly kills all the data
receivers (NetworkReceiver) without waiting for the data already received t
77 matches
Mail list logo
