[
https://issues.apache.org/jira/browse/PIG-4976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15477192#comment-15477192
]
Koji Noguchi commented on PIG-4976:
-----------------------------------
{code:title=ExecutableManager.java}
324 try {
325 t = (Tuple) inp.result;
326 inputHandler.putNext(t);
327 } catch (IOException e) {
328 // if input type is synchronous then it could
329 // be related to the process terminating
330 if(inputHandler.getInputType() ==
InputType.SYNCHRONOUS) {
331 LOG.warn("Exception while trying to write
to stream binary's input", e);
332 // could be because the process
333 // died OR closed the input stream
334 // we will only call close() here and not
335 // worry about deducing whether the process
died
336 // normally or abnormally - if there was
any real
337 // issue the ProcessOutputThread should see
338 // a non zero exit code from the process
and send
339 // a POStatus.STATUS_ERR back - what if we
got
340 // an IOException because there was only an
issue with
341 // writing to input of the binary -
hmm..hope that means
342 // the process died abnormally!!
343 close();
344 return;
345 } else {
346 // asynchronous case - then this is a real
exception
347 LOG.error("Exception while trying to write
to stream binary's input", e);
348 // send POStatus.STATUS_ERR to POStream to
signal the error
349 // Generally the ProcessOutputThread would
do this but now
350 // we should do it here since neither the
process nor the
351 // ProcessOutputThread will ever be spawned
352 Result res = new Result(POStatus.STATUS_ERR,
353 "Exception while trying to write to
stream binary's input" + e.getMessage());
354 sendOutput(poStream.getBinaryOutputQueue(),
res);
355 throw e;
356 }
357 }
...
{code}
Can we wrap "close()" with try-block at line 343 and send
{{sendOutput(poStream.getBinaryOutputQueue(), res);}} when catching any
Throwable?
btw,
bq. The bug exist in both Tez and MR, and it is not a regression.
This bug does exist on both Tez and MR, but I believe it only hangs longer than
default 10 mins timeout on Tez.
[~rohini], [~jeagles] taught me that this is because
tez.task.progress.stuck.interval-ms is not being set in Pig and we need
TEZ-3317 fixed first for that.
> streaming job with store clause stuck if the script fail
> --------------------------------------------------------
>
> Key: PIG-4976
> URL: https://issues.apache.org/jira/browse/PIG-4976
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.17.0
>
> Attachments: PIG-4976-1.patch
>
>
> When investigating PIG-4972, I also notice Pig job stuck when the perl script
> have syntax error. This happens if we have output clause in stream
> specification (means use a file as staging). The bug exist in both Tez and
> MR, and it is not a regression.
> Here is an example:
> {code}
> define CMD `perl kk.pl` output('foo') ship('kk.pl');
> A = load 'studenttab10k' as (name, age, gpa);
> B = foreach A generate name;
> C = stream B through CMD;
> store C into 'ooo';
> {code}
> kk.pl is any perl script contain a syntax error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
