[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68916422 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25106/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68916402 [Test build #25106 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25106/consoleFull) for PR 3912 at commit [`19827e5`](https://github.com/apache/spark/commit/19827e5ab0b24004b92e52344b6321342a50bc8a). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/3912#discussion_r22570298
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -138,7 +140,17 @@ class ReceiverTracker(ssc: StreamingContext,
skipReceiverLaunch: Boolean = false
receiverInfo(streamId) = ReceiverInfo(
streamId, s${typ}-${streamId}, receiverActor, true, host)
listenerBus.post(StreamingListenerReceiverStarted(receiverInfo(streamId)))
-logInfo(Registered receiver for stream + streamId + from +
sender.path.address)
+if(numAttempts == 0) {
+ logInfo(Receiver for stream + streamId + from +
sender.path.address +
--- End diff --
I think we can change to this style: `logInfo(s...$streamId from
${sender.path.address}...)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/3912#discussion_r22570265
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/receiver/ReceiverSupervisorImpl.scala
---
@@ -73,14 +73,16 @@ private[streaming] class ReceiverSupervisorImpl(
/** Timeout for Akka actor messages */
private val askTimeout = AkkaUtils.askTimeout(env.conf)
+ private var numRegisterAttemps = -1
/** Akka actor for receiving messages from the ReceiverTracker in the
driver */
private val actor = env.actorSystem.actorOf(
Props(new Actor {
override def preStart() {
-logInfo(Registered receiver + streamId)
-val msg = RegisterReceiver(
- streamId, receiver.getClass.getSimpleName,
Utils.localHostName(), self)
+numRegisterAttemps += 1
+logInfo(Register receiver + streamId + before it starts )
+val msg = RegisterReceiver(streamId,
receiver.getClass.getSimpleName,
--- End diff --
A simple question, maybe not relevant to this PR, whye we need to have two
`RegisterReceiver` messages, one in Akka preStart, another in
`onReceiverStart()`, I'm not sure is there any specific purpose?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68983524 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25144/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3912#discussion_r22572535
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/receiver/ReceiverSupervisorImpl.scala
---
@@ -73,14 +73,16 @@ private[streaming] class ReceiverSupervisorImpl(
/** Timeout for Akka actor messages */
private val askTimeout = AkkaUtils.askTimeout(env.conf)
+ private var numRegisterAttemps = -1
/** Akka actor for receiving messages from the ReceiverTracker in the
driver */
private val actor = env.actorSystem.actorOf(
Props(new Actor {
override def preStart() {
-logInfo(Registered receiver + streamId)
-val msg = RegisterReceiver(
- streamId, receiver.getClass.getSimpleName,
Utils.localHostName(), self)
+numRegisterAttemps += 1
+logInfo(Register receiver + streamId + before it starts )
+val msg = RegisterReceiver(streamId,
receiver.getClass.getSimpleName,
--- End diff --
@jerryshao yeah, I am confused too. IMHO, it is better to distinguish
register state and start state.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3912#discussion_r22572546
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -138,7 +140,17 @@ class ReceiverTracker(ssc: StreamingContext,
skipReceiverLaunch: Boolean = false
receiverInfo(streamId) = ReceiverInfo(
streamId, s${typ}-${streamId}, receiverActor, true, host)
listenerBus.post(StreamingListenerReceiverStarted(receiverInfo(streamId)))
-logInfo(Registered receiver for stream + streamId + from +
sender.path.address)
+if(numAttempts == 0) {
+ logInfo(Receiver for stream + streamId + from +
sender.path.address +
--- End diff --
ok
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68979288 [Test build #25139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25139/consoleFull) for PR 3912 at commit [`505257c`](https://github.com/apache/spark/commit/505257c239638a8828bef304104aec8a128d0c50). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68983150 [Test build #25144 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25144/consoleFull) for PR 3912 at commit [`9abb0db`](https://github.com/apache/spark/commit/9abb0db7dc904b21423331398fb7b779036d1cb1). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68983185 [Test build #25139 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25139/consoleFull) for PR 3912 at commit [`505257c`](https://github.com/apache/spark/commit/505257c239638a8828bef304104aec8a128d0c50). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68983189 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25139/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68983522 [Test build #25144 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25144/consoleFull) for PR 3912 at commit [`9abb0db`](https://github.com/apache/spark/commit/9abb0db7dc904b21423331398fb7b779036d1cb1). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68984072 [Test build #25147 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25147/consoleFull) for PR 3912 at commit [`4489d38`](https://github.com/apache/spark/commit/4489d38f59d0d8ebb29dc1f3d18789d926105fd0). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68903781 [Test build #25105 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25105/consoleFull) for PR 3912 at commit [`433e638`](https://github.com/apache/spark/commit/433e63895d8839cead053d77b8a386f4c7796d74). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/3912 [SPARK-5107][Streaming][Log]: A trick log info for the start of Receiver Receiver will register itself whenever it begins to start. But, it is trick to log the same information. Especially, at the preStart(), it will also register itself. Just like the receiver has started twice. Just like:  We can log the information more clearly. Like the attempt times to start. Of course, nothing matters performance or use. You can merge this pull request into a Git repository by running: $ git pull https://github.com/uncleGen/spark master-clean-150107 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3912.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3912 commit 433e63895d8839cead053d77b8a386f4c7796d74 Author: uncleGen husty...@gmail.com Date: 2015-01-06T17:49:31Z [SPARK-5107][Streaming][Log]: A trick log info for the start of Receiver --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68903917 [Test build #25105 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25105/consoleFull) for PR 3912 at commit [`433e638`](https://github.com/apache/spark/commit/433e63895d8839cead053d77b8a386f4c7796d74). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68903922 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25105/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5107][Streaming][Log]: A trick log info...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3912#issuecomment-68904627 [Test build #25106 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25106/consoleFull) for PR 3912 at commit [`19827e5`](https://github.com/apache/spark/commit/19827e5ab0b24004b92e52344b6321342a50bc8a). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
