[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[
https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923311#comment-15923311
]
Hive QA commented on HIVE-16104:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12858549/HIVE-16104.05.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 10343 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4109/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4109/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4109/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12858549 - PreCommit-HIVE-Build
> LLAP: preemption may be too aggressive if the pre-empted task doesn't die
> immediately
> -
>
> Key: HIVE-16104
> URL: https://issues.apache.org/jira/browse/HIVE-16104
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-16104.01.patch, HIVE-16104.02.patch,
> HIVE-16104.03.patch, HIVE-16104.04.patch, HIVE-16104.05.patch,
> HIVE-16104.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[ https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15922973#comment-15922973 ] Siddharth Seth commented on HIVE-16104: --- +1 on the latest patch. Thanks! > LLAP: preemption may be too aggressive if the pre-empted task doesn't die > immediately > - > > Key: HIVE-16104 > URL: https://issues.apache.org/jira/browse/HIVE-16104 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-16104.01.patch, HIVE-16104.02.patch, > HIVE-16104.03.patch, HIVE-16104.04.patch, HIVE-16104.05.patch, > HIVE-16104.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[
https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15922804#comment-15922804
]
Siddharth Seth commented on HIVE-16104:
---
Mostly looks good.
{code}
// TODO: not clear if we should wait for our last victim, or this. CR feedback.
{code}
Didn't get this comment. Should we wait for the specific preemption? - no, if
some other fragment completes.
If another fragment completes, we'll forget about what was preempted, and when
it was preempted. Don't think we should try fixing this corner case here.
FixedClock - May want to use "ControlledClock" which is already available.
The TODO on the thread should go.
On the test, think we should increase the sleep time + timeouts, or move to a
mechanism where the sleep is not required (that's where the clock was supposed
to help). 500ms typically causes a problems on slow/busy boxes, and will result
in test flakiness. A direct notifyAll to break out of the condition is ideal -
that gets into the area of whether the main code should be modified for tests.
Will probably require a wrapper wait interface which the test can control...
IAC, increased timeouts is probably the simplest at the moment.
> LLAP: preemption may be too aggressive if the pre-empted task doesn't die
> immediately
> -
>
> Key: HIVE-16104
> URL: https://issues.apache.org/jira/browse/HIVE-16104
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-16104.01.patch, HIVE-16104.02.patch,
> HIVE-16104.03.patch, HIVE-16104.04.patch, HIVE-16104.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[
https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906042#comment-15906042
]
Hive QA commented on HIVE-16104:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12857429/HIVE-16104.04.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 10341 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4082/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4082/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4082/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12857429 - PreCommit-HIVE-Build
> LLAP: preemption may be too aggressive if the pre-empted task doesn't die
> immediately
> -
>
> Key: HIVE-16104
> URL: https://issues.apache.org/jira/browse/HIVE-16104
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-16104.01.patch, HIVE-16104.02.patch,
> HIVE-16104.03.patch, HIVE-16104.04.patch, HIVE-16104.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[
https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905932#comment-15905932
]
Hive QA commented on HIVE-16104:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12857429/HIVE-16104.04.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 10341 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4079/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4079/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4079/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12857429 - PreCommit-HIVE-Build
> LLAP: preemption may be too aggressive if the pre-empted task doesn't die
> immediately
> -
>
> Key: HIVE-16104
> URL: https://issues.apache.org/jira/browse/HIVE-16104
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-16104.01.patch, HIVE-16104.02.patch,
> HIVE-16104.03.patch, HIVE-16104.04.patch, HIVE-16104.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[ https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905856#comment-15905856 ] Sergey Shelukhin commented on HIVE-16104: - Updated the loop, the lock and tests > LLAP: preemption may be too aggressive if the pre-empted task doesn't die > immediately > - > > Key: HIVE-16104 > URL: https://issues.apache.org/jira/browse/HIVE-16104 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-16104.01.patch, HIVE-16104.02.patch, > HIVE-16104.03.patch, HIVE-16104.04.patch, HIVE-16104.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[ https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904375#comment-15904375 ] Siddharth Seth commented on HIVE-16104: --- bq. Sorry didn't get that. You mean for the waiting for completion? Wouldn't that block other ops like adding tasks? No. It's a wait on the giant lock. So there's no lock held. When a new operation happens, it can signal this. (The same lock used in the main scheduling loop) - which gets a signal from all other opeartions. Also, I think the loop can be simplified. trySChedule does not need to be done twice. > LLAP: preemption may be too aggressive if the pre-empted task doesn't die > immediately > - > > Key: HIVE-16104 > URL: https://issues.apache.org/jira/browse/HIVE-16104 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-16104.01.patch, HIVE-16104.02.patch, > HIVE-16104.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[
https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904326#comment-15904326
]
Sergey Shelukhin commented on HIVE-16104:
-
See
https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime%28%29,
it can be negative. Anyway there's no reason to have a magic value when it can
be null.
{quote}
That's a style choice. There's nothing wrong with using an executor with a
single thread. If anything, this should have been
Executors.newSingleThreadExecutor
{quote}
That's complexity for no reason.. so not really a style choice.
{quote}
Think we're better off using the mega lock in the scheduler for the wait - that
way it gets interrupted if some other task completes, another task asks to be
scheduled, etc. (Doesn't need to wait for the specific fragment to end)
{quote}
Sorry didn't get that. You mean for the waiting for completion? Wouldn't that
block other ops like adding tasks?
{quote}
Using isComplete as the wait mechanism has a race when the setIsCompleted()
invocation happens in TaskRunnerCallable.
{quote}
What is the race? The wait is on the lock and so both wait and notify have to
execute when the lock is taken. isComplete is also only examined under the same
lock. So either the checker sees isComplete=true (after someone has set it and
ran notify) and doesn't wait, or it sees false, so whoever notifies-All cannot
notify before the checker releases the lock, cause he needs the lock to notify.
Even if he could, at worst checker would wait needlessly and return value of
isComplete. Also isComplete is never unset, so that makes it even less
problematic cause noone will set it to false.
> LLAP: preemption may be too aggressive if the pre-empted task doesn't die
> immediately
> -
>
> Key: HIVE-16104
> URL: https://issues.apache.org/jira/browse/HIVE-16104
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-16104.01.patch, HIVE-16104.02.patch,
> HIVE-16104.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[
https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904273#comment-15904273
]
Siddharth Seth commented on HIVE-16104:
---
bq. Wrt special values, monotonic/nanotime can take any value so special values
should not be used.
How about "-1" as a value, don't think we'll see that as a Time.monotonicNow
bq. #3 is completely unrelated, strange that you would ask for it here I'll
make the change.
Think it is related, since the change increases the chances of hitting this
(seen while walking through the patch)
bq. The class has so many TODO comments that adding another relevant one should
not be a problem
That's a style choice. There's nothing wrong with using an executor with a
single thread. If anything, this should have been
Executors.newSingleThreadExecutor
Think we're better off using the mega lock in the scheduler for the wait - that
way it gets interrupted if some other task completes, another task asks to be
scheduled, etc. (Doesn't need to wait for the specific fragment to end)
Using isComplete as the wait mechanism has a race when the setIsCompleted()
invocation happens in TaskRunnerCallable.
{code}
lastVictim = handleScheduleAttemptedRejection(task);
// We killed something.
lastKillTimeMs = clock.getTime();
{code}
Should this check for lastVictim being null?
> LLAP: preemption may be too aggressive if the pre-empted task doesn't die
> immediately
> -
>
> Key: HIVE-16104
> URL: https://issues.apache.org/jira/browse/HIVE-16104
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-16104.01.patch, HIVE-16104.02.patch,
> HIVE-16104.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[ https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904079#comment-15904079 ] Sergey Shelukhin commented on HIVE-16104: - Wrt special values, monotonic/nanotime can take any value so special values should not be used. #3 is completely unrelated, strange that you would ask for it here ;) I'll make the change. The class has so many TODO comments that adding another relevant one should not be a problem > LLAP: preemption may be too aggressive if the pre-empted task doesn't die > immediately > - > > Key: HIVE-16104 > URL: https://issues.apache.org/jira/browse/HIVE-16104 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-16104.01.patch, HIVE-16104.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[
https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902605#comment-15902605
]
Hive QA commented on HIVE-16104:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12856901/HIVE-16104.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 10335 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4040/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4040/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4040/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12856901 - PreCommit-HIVE-Build
> LLAP: preemption may be too aggressive if the pre-empted task doesn't die
> immediately
> -
>
> Key: HIVE-16104
> URL: https://issues.apache.org/jira/browse/HIVE-16104
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-16104.01.patch, HIVE-16104.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[
https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902341#comment-15902341
]
Siddharth Seth commented on HIVE-16104:
---
Thank you. Still some parts which should not have changed... Anyway. Actual
review comments this time
Thread.sleep(PREEMPTION_KILL_GRACE_SLEEP_MS); - Think it is possible to replace
this with a timed wait. Similar to the way the main scheduling loop waits for a
fragment to complete or get scheduled before it tries to schedule the next one.
That way we won't actually be sleeping for the entire 100ms.
Another nice to have bit would be to allow the element to go back into the
queue, and fetch the latest from the queue - which could be at a higher
priority. Don't think this is critical though.
In TaskRunnerCallable - killTask needs a minor change. shouldRunTask = false
needs to be set independent of the (taskRunner) condition that it is under.
Having a fragment wait outside the queue increases the possibility of an issue
like this being hit. (The nice to have about putting the fragment back in the
queue and waiting for the next schedule attempt would limit the possibility of
other such issues, if they exist, as well)
Any chance of adding tests? The test class is setup with some controlled
scheduling constructs to help with this. Suspect it'll need more work with the
timed wait though.
System.nanoTime - replace with the Clock instance already used in the class.
(Helps with unit tests to simulate time changes, may not be plugged in yet).
Nit: lastKillTimeNs - Long(null) vs long (-1/special value for unset)?
Not required.
{code}
// TODO: this can all be replaced by a Thread with a Runnable and a catch block
{code}
> LLAP: preemption may be too aggressive if the pre-empted task doesn't die
> immediately
> -
>
> Key: HIVE-16104
> URL: https://issues.apache.org/jira/browse/HIVE-16104
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-16104.01.patch, HIVE-16104.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[ https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902006#comment-15902006 ] Siddharth Seth commented on HIVE-16104: --- We generally don't refactor sections which interfere with the review. And in most cases, unrelated changes are avoided. Don't see any reason to make the additional changes here. > LLAP: preemption may be too aggressive if the pre-empted task doesn't die > immediately > - > > Key: HIVE-16104 > URL: https://issues.apache.org/jira/browse/HIVE-16104 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-16104.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[ https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901897#comment-15901897 ] Sergey Shelukhin commented on HIVE-16104: - We don't normally have separate jiras for refactoring so I don't see why not perform small refactors in other changes... > LLAP: preemption may be too aggressive if the pre-empted task doesn't die > immediately > - > > Key: HIVE-16104 > URL: https://issues.apache.org/jira/browse/HIVE-16104 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-16104.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[ https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901833#comment-15901833 ] Siddharth Seth commented on HIVE-16104: --- bq. The lock in trySchedule is unnecessary so I removed it and renamed the method; Not required, or relevant to this patch. bq. preemption was surrounded by a loop because previously, if the first task in queue was finishable it would bail without preempting anything even if there are more tasks. I believe this is mostly harmless - the same task will be picked up in the next loop. Unrelated to the jira, but probably a relevant change.There's some change around switching the condition to return early (negative condition - return, instead of the existing conditions met-> execute) - which is unnecessary. Similar condition changes elsewhere as well. bq. I can merge updateQueueMetric back into being copy-pasted in 3 places... also one if was refactored because it has lots of repetitive code. bq. Another method was added because something that was previously called in one place is now called in 2 places and I didn't want to copy-paste it. These would be relevant to the jira? - since additional invocations are happening because of the change. > LLAP: preemption may be too aggressive if the pre-empted task doesn't die > immediately > - > > Key: HIVE-16104 > URL: https://issues.apache.org/jira/browse/HIVE-16104 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-16104.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[ https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900574#comment-15900574 ] Sergey Shelukhin commented on HIVE-16104: - RB https://reviews.apache.org/r/57405/ for review where whitespace changes make the patch too verbose > LLAP: preemption may be too aggressive if the pre-empted task doesn't die > immediately > - > > Key: HIVE-16104 > URL: https://issues.apache.org/jira/browse/HIVE-16104 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-16104.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[ https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900571#comment-15900571 ] Sergey Shelukhin commented on HIVE-16104: - Some things that look like refactoring are not actually refactoring. The lock in trySchedule is unnecessary so I removed it and renamed the method; preemption was surrounded by a loop because previously, if the first task in queue was finishable it would bail without preempting anything even if there are more tasks. I can merge updateQueueMetric back into being copy-pasted in 3 places... also one if was refactored because it has lots of repetitive code. > LLAP: preemption may be too aggressive if the pre-empted task doesn't die > immediately > - > > Key: HIVE-16104 > URL: https://issues.apache.org/jira/browse/HIVE-16104 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-16104.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[ https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900560#comment-15900560 ] Siddharth Seth commented on HIVE-16104: --- Looking. Can you please remove the unnecessary parts of the patch - formatting changes, refactored if/else statements. That makes it difficult to review, and is not required. > LLAP: preemption may be too aggressive if the pre-empted task doesn't die > immediately > - > > Key: HIVE-16104 > URL: https://issues.apache.org/jira/browse/HIVE-16104 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-16104.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[ https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898335#comment-15898335 ] Sergey Shelukhin commented on HIVE-16104: - [~sseth] ping? > LLAP: preemption may be too aggressive if the pre-empted task doesn't die > immediately > - > > Key: HIVE-16104 > URL: https://issues.apache.org/jira/browse/HIVE-16104 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-16104.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16104) LLAP: preemption may be too aggressive if the pre-empted task doesn't die immediately
[
https://issues.apache.org/jira/browse/HIVE-16104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895542#comment-15895542
]
Hive QA commented on HIVE-16104:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12855954/HIVE-16104.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 26 failed/errored test(s), 10285 tests
executed
*Failed tests:*
{noformat}
TestCommandProcessorFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=272)
TestDbTxnManager - did not produce a TEST-*.xml file (likely timed out)
(batchId=272)
TestDummyTxnManager - did not produce a TEST-*.xml file (likely timed out)
(batchId=272)
TestHiveInputSplitComparator - did not produce a TEST-*.xml file (likely timed
out) (batchId=272)
TestIndexType - did not produce a TEST-*.xml file (likely timed out)
(batchId=272)
TestSplitFilter - did not produce a TEST-*.xml file (likely timed out)
(batchId=272)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[escape_comments]
(batchId=229)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table]
(batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_between_in]
(batchId=119)
org.apache.hive.beeline.TestSchemaTool.testNestedScriptsForDerby (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testNestedScriptsForMySQL (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testNestedScriptsForOracle (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testPostgresFilter (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testSchemaInit (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testSchemaInitDryRun (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testSchemaUpgradeDryRun (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testScriptMultiRowComment (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testScriptWithDelimiter (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testScripts (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testValidateLocations (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testValidateNullValues (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testValidateSchemaTables (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testValidateSchemaVersions (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testValidateSequences (batchId=212)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3934/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3934/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3934/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 26 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12855954 - PreCommit-HIVE-Build
> LLAP: preemption may be too aggressive if the pre-empted task doesn't die
> immediately
> -
>
> Key: HIVE-16104
> URL: https://issues.apache.org/jira/browse/HIVE-16104
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-16104.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
