[
https://issues.apache.org/jira/browse/TEZ-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962197#comment-13962197
]
Bikas Saha commented on TEZ-700:
--------------------------------
For the latest patch
We typically throw TezUncheckedException instead of RuntimeException. Btw,
should it be an error (exception) if the session has shutdown? Are we assuming
that the session is exclusive to this connection and hence should not shutdown
unless we ask it to? What is the user expected to do if the session has
shutdown? Is that action facilitated by throwing/catching an unchecked
exception?
{code}+ if (status.equals(TezSessionStatus.SHUTDOWN)) {
+ throw new RuntimeException("TezSession has already shutdown");
+ }{code}
The sleep value should probably be larger (say 500 ms) to prevent DOSing the
AM. Secondly, if we return on interrupted exception then the user will assume
that the session is ready. That is probably not correct.
{code}+ try {
+ Thread.sleep(SLEEP_FOR_READY);
+ } catch (InterruptedException e) {
+ return;
+ }{code}
Its not clear to me to print vertex progress once for INIT and then print
further while RUNNING. E.g. if the user has asked us not to print progress then
why print progress once for INIT? We could just print "Waiting for DAG to start
running" once or something like that. There is no need to check state for that,
right? Simply print that and then do the print vertex progress, if asked to do
so, while DAG is running. Then finally print "DAG completed with state FOO"
when the state is no longer running. Additionally we can print the final vertex
progress if the user has asked for vertex progress.
Can we allocate this object once (private static)? {code}+ private void
log(String message) {
+ final SimpleDateFormat dateFormat = new SimpleDateFormat("yy/MM/dd
HH:mm:ss");{code}
Array or Collection? What is easier for the user? What would the user already
have given our other API's, that can be directly passed to this method without
needing any conversion?
{code}+ public DAGStatus waitForCompletionWithStatusUpdates(String[]
vertexNames) throws IOException, TezException;{code}
How will the user enable/disable counters printing? That can be excessive
printing with many counters? I would suggest accepting a parameter
Set<StatusGetOpts> statusOptions (already being accepted as an argument to the
underlying DAGClient.get*Status() code. This way we can print whatever the user
wants by observing what is set by the user (and also simply passing those same
options to DAGClient to get the values). This makes the print API transparent
to future additions to StatusGetOpts.
> Helper API's to monitor a DAG to completion
> -------------------------------------------
>
> Key: TEZ-700
> URL: https://issues.apache.org/jira/browse/TEZ-700
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Bikas Saha
> Assignee: Mohammad Kamrul Islam
> Attachments: TEZ-700.1.patch, TEZ-700.2.patch, TEZ-700.3.patch,
> TEZ-700.4.patch
>
>
> There is a lot of boiler plate code to monitor a dag progress to completion.
> A simple API to monitor the job to completion will be reduce this burden.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
