[jira] Commented: (PIG-1424) Error logs of streaming should not be placed in output location
[ https://issues.apache.org/jira/browse/PIG-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12884365#action_12884365 ] Ashutosh Chauhan commented on PIG-1424: --- This turns out to be much more involved then I initially thought. Assumption that output/input location is a file based path exists at more then one place in Pig. In particular, Streaming kind of make this explicit assumption and has it in the semantics. We need to be careful about streaming semantics before we fix this. More at: http://wiki.apache.org/pig/PigStreamingFunctionalSpec Error logs of streaming should not be placed in output location --- Key: PIG-1424 URL: https://issues.apache.org/jira/browse/PIG-1424 Project: Pig Issue Type: Bug Components: impl Affects Versions: 0.7.0 Reporter: Ashutosh Chauhan Fix For: 0.8.0 This becomes a problem when output location is anything other then a filesystem. Output will be written to DB but where the logs generated by streaming should go? Clearly, they cant be written into DB. This blocks PIG-1229 which introduces writing to DB from Pig. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1424) Error logs of streaming should not be placed in output location
[ https://issues.apache.org/jira/browse/PIG-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12871902#action_12871902 ] Ashutosh Chauhan commented on PIG-1424: --- Till we figure out a proper solution for this, one possibility is to wrap the code in my previous comment into try-catch block. That will unblock PIG-1229 for commit. We can leave this ticket open if we feel there is a need for a better solution. Error logs of streaming should not be placed in output location --- Key: PIG-1424 URL: https://issues.apache.org/jira/browse/PIG-1424 Project: Pig Issue Type: Bug Components: impl Affects Versions: 0.7.0 Reporter: Ashutosh Chauhan Fix For: 0.8.0 This becomes a problem when output location is anything other then a filesystem. Output will be written to DB but where the logs generated by streaming should go? Clearly, they cant be written into DB. This blocks PIG-1229 which introduces writing to DB from Pig. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1424) Error logs of streaming should not be placed in output location
[
https://issues.apache.org/jira/browse/PIG-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12869688#action_12869688
]
Ashutosh Chauhan commented on PIG-1424:
---
Since all the logs generated by Pig in backend end up in log directory of task
tracker, logs generated by streaming binary should also go there and not into
the output location.
The place where this setting of location happens is in JobControlCompiler.java,
line 460:
{code}
conf.set(pig.streaming.log.dir,
new Path(outputPath, LOG_DIR).toString());
{code}
Error logs of streaming should not be placed in output location
---
Key: PIG-1424
URL: https://issues.apache.org/jira/browse/PIG-1424
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.7.0
Reporter: Ashutosh Chauhan
Fix For: 0.8.0
This becomes a problem when output location is anything other then a
filesystem. Output will be written to DB but where the logs generated by
streaming should go? Clearly, they cant be written into DB. This blocks
PIG-1229 which introduces writing to DB from Pig.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
