[
https://issues.apache.org/jira/browse/TEZ-692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052065#comment-14052065
]
Bikas Saha edited comment on TEZ-692 at 7/4/14 1:09 AM:
--------------------------------------------------------
This is in an incompatible change. Steps to make code compile again.
1) TezSession is gone. TezClient is the only interface. Session/Non-session
mode is determined via TezConfiguration tez.am.mode.session or hard-coded in
the constructor. Recommendation is to not hard-code. If properly written, the
code should be the same both ways and session/non-session can be decided via
configuration when executing the job.
2) Remove AMConfiguration - If app master specific credentials/resources were
being passed via AMConfiguration, then they need to be passed in via the new
constructor.
3) start() needs to be called before submitting a DAG.
4) call waitTillReady() before submitting the DAG.
5) stop() needs to be called at the end to release connection to cluster,
ideally in a finally block.
6) TezCient does not generate a YARN appId for the app master before submitting
the DAG. The appId is available after DAG is submitted. This is best practice
to handle forward looking HA scenarios when YARN will return the appId after
app submission. Users relying on appId as a unique string should use some other
method.
The new code should be a lot simpler and concise than earlier.
Non-session mode is the default. If you want to use sessions, then please set
it in the conf or use the constructor to explicitly enable it.
was (Author: bikassaha):
This is in an incompatible change. Steps to make code compile again.
1) TezSession is gone. TezClient is the only interface. Session/Non-session
mode is determined via TezConfiguration tez.am.mode.session or hard-coded in
the constructor. Recommendation is to not hard-code. If properly written, the
code should be the same both ways and session/non-session can be decided via
configuration when executing the job.
2) Remove AMConfiguration - If app master specific credentials/resources were
being passed via AMConfiguration, then they need to be passed in via the new
constructor.
3) start() needs to be called before submitting a DAG.
4) call waitTillReady() before submitting the DAG.
5) stop() needs to be called at the end to release connection to cluster,
ideally in a finally block.
6) TezCient does not generate a YARN appId for the app master before submitting
the DAG. The appId is available after DAG is submitted. This is best practice
to handle forward looking HA scenarios when YARN will return the appId after
app submission. Users relying on appId as a unique string should use some other
method.
The new code should be a lot simpler and concise than earlier.
> Unify job submission in either TezClient or TezSession
> ------------------------------------------------------
>
> Key: TEZ-692
> URL: https://issues.apache.org/jira/browse/TEZ-692
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-692.1.patch, TEZ-692.2.patch, TEZ-692.3.patch,
> TEZ-692.4.patch, TEZ-692.5.patch
>
>
> Its confusing to have 2 ways to create and submit a tez job. The developer
> has to spend time thinking about and deciding which method to use.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
