[
https://issues.apache.org/jira/browse/PIG-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044844#comment-14044844
]
Ruchir commented on PIG-3866:
-----------------------------
This is with respect to our Open Source Project https://github.com/eBay/oink :
PIG Self-Serve. Generally it is found that in shared cluster environment, when
there are large number of users who want to submit PIG jobs, on-boarding
process is very cumbersome and have lots of overhead. Defining REST APIs and
allowing user to submit from any machine make it very simple. Actual PIG jobs
will be submitted as MR job, so memory limit is not an issue.
> Create ThreadLocal classloader per PigContext
> ---------------------------------------------
>
> Key: PIG-3866
> URL: https://issues.apache.org/jira/browse/PIG-3866
> Project: Pig
> Issue Type: Wish
> Components: impl
> Affects Versions: 0.12.0
> Reporter: Ruchir
> Fix For: 0.14.0
>
> Attachments: diff
>
>
> For shared Hadoop environment, where there are large number of customers who
> submit PIG jobs for execution, it becomes very cumbersome to handle all
> support issue (ex providing access to CLI machine to every user, limiting
> user for submission etc).
> There is need to provide REST based interface for submitting PIG job/view PIG
> job status/view PIG request output, so that there is single entry point to
> cluster. With this, users are abstracted of Hadoop configuration and need to
> get access on machine. Also given it is hosted service, it is possible to
> provide rate limiting, QoS etc to user.
> We have designed such PIG self-service which provide REST APIs for all
> operations and this webapp runs under web container. In order to run multiple
> requests in same JVM, we need to create classloader per request thread.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
