[jira] [Commented] (LIVY-664) Spark application still running when Livy session creating was rejected
[ https://issues.apache.org/jira/browse/LIVY-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931300#comment-16931300 ] Yiheng Wang commented on LIVY-664: -- session stop code: [https://github.com/apache/incubator-livy/blob/master/server/src/main/scala/org/apache/livy/sessions/SessionManager.scala#L103] > Spark application still running when Livy session creating was rejected > > > Key: LIVY-664 > URL: https://issues.apache.org/jira/browse/LIVY-664 > Project: Livy > Issue Type: Bug >Reporter: Oleksandr Shevchenko >Priority: Major > Attachments: image-2019-09-08-20-38-50-195.png, > image-2019-09-08-20-39-18-569.png > > Time Spent: 10m > Remaining Estimate: 0h > > Steps for reproduce: > 1. Create a session with some name > 2. Create a second session with the same name > 2.1 Second session creating will be rejected since duplicated session name > is not allowed. > 2.2 Spark application will be submitted but Livy session won't be created > > Result: Spark application was submitted but Livy session is not created > Expected result: Livy session creating rejected AND Spark application should > be finished with failed or killed state or even should not be submitted. > !image-2019-09-08-20-38-50-195.png! > !image-2019-09-08-20-39-18-569.png! -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (LIVY-664) Spark application still running when Livy session creating was rejected
[ https://issues.apache.org/jira/browse/LIVY-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931261#comment-16931261 ] Yiheng Wang commented on LIVY-664: -- Oh, I check the wrong branch... Sorry for the confusion. > Spark application still running when Livy session creating was rejected > > > Key: LIVY-664 > URL: https://issues.apache.org/jira/browse/LIVY-664 > Project: Livy > Issue Type: Bug >Reporter: Oleksandr Shevchenko >Priority: Major > Attachments: image-2019-09-08-20-38-50-195.png, > image-2019-09-08-20-39-18-569.png > > Time Spent: 10m > Remaining Estimate: 0h > > Steps for reproduce: > 1. Create a session with some name > 2. Create a second session with the same name > 2.1 Second session creating will be rejected since duplicated session name > is not allowed. > 2.2 Spark application will be submitted but Livy session won't be created > > Result: Spark application was submitted but Livy session is not created > Expected result: Livy session creating rejected AND Spark application should > be finished with failed or killed state or even should not be submitted. > !image-2019-09-08-20-38-50-195.png! > !image-2019-09-08-20-39-18-569.png! -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (LIVY-664) Spark application still running when Livy session creating was rejected
[
https://issues.apache.org/jira/browse/LIVY-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928495#comment-16928495
]
Oleksandr Shevchenko commented on LIVY-664:
---
Thanks for your comment [~yihengw]!
Don't see that duplicated session stops now, we just throw an exception
([https://github.com/apache/incubator-livy/blob/def6318c84f32a09f219065691857f11ca74e7cb/server/src/main/scala/org/apache/livy/sessions/SessionManager.scala#L99]).
Could you show where we stop this session?
I have tried to add `session.stop()` before throwing IllegalArgumentException
and it works fine in the case when YARN/Spark standalone application isn't
started yet but if application is already in ACCEPTED or RUNNING state stop
session doesn't work.
By design, in this case, the application should be killed here
([https://github.com/apache/incubator-livy/blob/80daadef02ae57b2a5487c6f92e0f7df558d4864/server/src/main/scala/org/apache/livy/server/interactive/InteractiveSession.scala#L474]),
I guess. But we have bad error handling in RSCClient.java and error will not
be propagated and also `app` in the following code is None since
`session.start()` didn't execute:
{code}
app.foreach {
warn(s"Failed to stop RSCDriver. Killing it...")
_.kill()
}
{code}
Better to check the session name before start application, but I don't see
where we actually do it. So, I put off this bug fix since little busy now. I
can continue my work on this issue if you help me to find where an application
is submitted (hope this is not a Python code, but I didn't find related
Scala/Java code for the first time).
Thanks!
> Spark application still running when Livy session creating was rejected
>
>
> Key: LIVY-664
> URL: https://issues.apache.org/jira/browse/LIVY-664
> Project: Livy
> Issue Type: Bug
>Reporter: Oleksandr Shevchenko
>Priority: Major
> Attachments: image-2019-09-08-20-38-50-195.png,
> image-2019-09-08-20-39-18-569.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Steps for reproduce:
> 1. Create a session with some name
> 2. Create a second session with the same name
> 2.1 Second session creating will be rejected since duplicated session name
> is not allowed.
> 2.2 Spark application will be submitted but Livy session won't be created
>
> Result: Spark application was submitted but Livy session is not created
> Expected result: Livy session creating rejected AND Spark application should
> be finished with failed or killed state or even should not be submitted.
> !image-2019-09-08-20-38-50-195.png!
> !image-2019-09-08-20-39-18-569.png!
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (LIVY-664) Spark application still running when Livy session creating was rejected
[ https://issues.apache.org/jira/browse/LIVY-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928486#comment-16928486 ] Yiheng Wang commented on LIVY-664: -- In the code, it will stop the duplicated session when finding there're duplicated names. In my test, I find the stop session may fail. I guess it may due to some race condition. I'm looking into it. Another suggestion I think it good. We should check the name duplication before submitting the application. I'll raise a patch to fix that. > Spark application still running when Livy session creating was rejected > > > Key: LIVY-664 > URL: https://issues.apache.org/jira/browse/LIVY-664 > Project: Livy > Issue Type: Bug >Reporter: Oleksandr Shevchenko >Priority: Major > Attachments: image-2019-09-08-20-38-50-195.png, > image-2019-09-08-20-39-18-569.png > > > Steps for reproduce: > 1. Create a session with some name > 2. Create a second session with the same name > 2.1 Second session creating will be rejected since duplicated session name > is not allowed. > 2.2 Spark application will be submitted but Livy session won't be created > > Result: Spark application was submitted but Livy session is not created > Expected result: Livy session creating rejected AND Spark application should > be finished with failed or killed state or even should not be submitted. > !image-2019-09-08-20-38-50-195.png! > !image-2019-09-08-20-39-18-569.png! -- This message was sent by Atlassian Jira (v8.3.2#803003)
