[GitHub] spark pull request #19351: [SPARK-22127][CORE]The Master Register Applicatio...

[guoxiaolongzte]

Github user guoxiaolongzte closed the pull request at:

https://github.com/apache/spark/pull/19351


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19351: [SPARK-22127][CORE]The Master Register Applicatio...

[guoxiaolongzte]

Github user guoxiaolongzte commented on a diff in the pull request:

https://github.com/apache/spark/pull/19351#discussion_r141346612
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala 
---
@@ -265,6 +265,9 @@ private[deploy] class Master(
 val app = createApplication(description, driver)
 registerApplication(app)
 logInfo("Registered app " + description.name + " with ID " + 
app.id)
+if(app.state == ApplicationState.WAITING) {
--- End diff --

ok. I agree with your opinion. Thanks.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19351: [SPARK-22127][CORE]The Master Register Applicatio...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/19351#discussion_r141339131
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala 
---
@@ -265,6 +265,9 @@ private[deploy] class Master(
 val app = createApplication(description, driver)
 registerApplication(app)
 logInfo("Registered app " + description.name + " with ID " + 
app.id)
+if(app.state == ApplicationState.WAITING) {
--- End diff --

I don't think this is an abnormal case though; warn doesn't sound quite 
appropriate. It's already reported that it's waiting in the UI and so on.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19351: [SPARK-22127][CORE]The Master Register Applicatio...

[guoxiaolongzte]

Github user guoxiaolongzte commented on a diff in the pull request:

https://github.com/apache/spark/pull/19351#discussion_r141224687
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala 
---
@@ -265,6 +265,9 @@ private[deploy] class Master(
 val app = createApplication(description, driver)
 registerApplication(app)
 logInfo("Registered app " + description.name + " with ID " + 
app.id)
+if(app.state == ApplicationState.WAITING) {
--- End diff --

it is a warning.
Because there is no resource allocation itself may lead to spark business 
has been waiting, there will be problems.

Now I fix as shown below，Do you think this is correct?

 if(app.state == ApplicationState.WAITING) {
  logWarning("App need to wait worker Launch executor," +
"maybe worker does not have extra resources to allocate")
}


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19351: [SPARK-22127][CORE]The Master Register Applicatio...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/19351#discussion_r141032457
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala 
---
@@ -265,6 +265,9 @@ private[deploy] class Master(
 val app = createApplication(description, driver)
 registerApplication(app)
 logInfo("Registered app " + description.name + " with ID " + 
app.id)
+if(app.state == ApplicationState.WAITING) {
--- End diff --

Nit: space. The text of the warning needs to be fixed up; it's not clear 
what it means. Why is it abnormal to have to wait for resources? doesn't seem 
like a warning


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19351: [SPARK-22127][CORE]The Master Register Applicatio...

[guoxiaolongzte]

GitHub user guoxiaolongzte opened a pull request:

https://github.com/apache/spark/pull/19351

[SPARK-22127][CORE]The Master Register Application Function requires an 
warn log to increase the waiting status

## What changes were proposed in this pull request?

The Master register application function requires an alarm log to increase 
the waiting status.

When I create a spark application, I apply for the resources to reach the 
ceiling, the current Worker does not have enough resources to allocate 
Executor, this time spark application state is waiting.
But I can not know from the spark master log this situation, which led to 
my positioning problems difficult, I mistakenly thought that the master or 
worker process dead.

All that I added to the warting state of the warn log, which better helps 
us locate this kind of problem.

1.WAITING  app:

![1](https://user-images.githubusercontent.com/26266482/30858530-a3f3c5e2-a2f2-11e7-9899-041cf3eac118.png)

2.master log:

![2](https://user-images.githubusercontent.com/26266482/30858550-bd07ed38-a2f2-11e7-91db-952101976502.png)


## How was this patch tested?
manual tests

Please review http://spark.apache.org/contributing.html before opening a 
pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/guoxiaolongzte/spark SPARK-22127

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/19351.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #19351


commit 3a756caf6c9015f32068b8c28c65ade151411392
Author: guoxiaolong 
Date:   2017-09-26T11:37:40Z

[SPARK-22127]The Master Register Application Function requires an warn log 
to increase the waiting status




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org