[
https://issues.apache.org/jira/browse/HIVE-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13685857#comment-13685857
]
Alan Gates commented on HIVE-4679:
----------------------------------
After applying this patch many of the webhcat e2e tests fail. Some of them are
returning: ./test_harness.pl::TestDriverCurl::checkResStatusCode INFO Check
failed: status_code 200 expected, test returned <400> and some
check_job_exit_value failed. got exitValue 5, expected 0INFO:
TestDriverCurl::run() at 1141:Test TestPig_2 FAILED at 1371494116
> WebHCat can deadlock Hadoop if the number of concurrently running tasks if
> higher or equal than the number of mappers
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-4679
> URL: https://issues.apache.org/jira/browse/HIVE-4679
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
> Affects Versions: 0.10.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Fix For: 0.12.0
>
> Attachments: HIVE-4679.patch
>
>
> o In the current Templeton design, each time a Job is submitted thru the REST
> API (it can be Pig/Hive or MR job), it will consume one Hadoop map slot.
> Given that the number of map slots is finite in the cluster (16 node cluster
> will have 32 map slots), in some circumstances, a user can deadlock the
> cluster if Templeton job submission pipeline takes over all map slots
> (Templeton map tasks will wait for the actual underlying jobs to complete,
> what will never happen, given that Hadoop has no free map slots to schedule
> new tasks).
> o HCat queries use a different mechanism and do not contribute to the
> deadlock.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
