[
https://issues.apache.org/jira/browse/PIG-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202052#comment-13202052
]
Thejas M Nair commented on PIG-2497:
------------------------------------
+1 . Created PIG-2516 to track other issues remaining in sh implementation that
Daniel found.
> Order of execution of fs, store and sh commands in Pig is not maintained
> ------------------------------------------------------------------------
>
> Key: PIG-2497
> URL: https://issues.apache.org/jira/browse/PIG-2497
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.1
> Reporter: Viraj Bhat
> Assignee: Daniel Dai
> Fix For: 0.10, 0.9.3, 0.11
>
> Attachments: PIG-2497-1.patch
>
>
> I have a pig script like this :
> --Load data, process it and store to two outputs
> {code}
> a = load 'dummy.txt' as (cookie: chararray,timestamp: long,url: chararray);
> b = group a by (cookie);
> c = foreach b generate group, COUNT_STAR(a);
> store c into '$COUNT_OUTPUT' using PigStorage();
> store b into '$GRID_OUTPUT' using PigStorage();
> --Remove local file, copy to local and remove processed file from grid
> sh rm -rf '$LOCAL_OUTPUT';
> fs -getmerge '$GRID_OUTPUT' '$LOCAL_OUTPUT';
> fs -rmr '$GRID_OUTPUT';
> {code}
> Pig does not guarantee the order of command execution in the above script
> i.e. the "store" "sh rm...", "fs -getmerge ..." and "fs -rmr ..." will not be
> executed in the written order.
> Pig guarantees that "fs" commands and pig "store" commands will be executed
> in sequence. But "sh" commands will get executed before anything else (in
> normal multi-query mode) because "sh" commands are executed when the parser
> sees them. They go through a different code path within Pig. This behavior
> needs to be changed.
> Thanks
> Viraj
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
