[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14043760#comment-14043760
]
Alan Gates commented on PIG-2122:
-
+1 for the patch.
[~olgan], I don't see the backwards compatibility issue. By definition this is
for interactive sessions, so users can't have existing scripts that change
behavior. I suppose someone somewhere might regularly use $x in his
interactive session and expect it to come out as $x rather than complain that
it can't make the substitution, but that seems 1) unlikely, and 2) easy to fix.
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
> Components: grunt
>Affects Versions: 0.8.0, 0.8.1, 0.12.0
>Reporter: Grant Ingersoll
>Assignee: Daniel Dai
>Priority: Minor
> Fix For: 0.14.0
>
> Attachments: PIG-2122-1.patch
>
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026138#comment-14026138
]
Gaurav Phapale commented on PIG-2122:
-
It would be great if this issue is fixed.
I'm not able to understand how adding parameter substitution will affect back
comp.
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
> Components: grunt
>Affects Versions: 0.8.0, 0.8.1, 0.12.0
>Reporter: Grant Ingersoll
>Assignee: Daniel Dai
>Priority: Minor
> Fix For: 0.14.0
>
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470566#comment-13470566
]
SnowCrash commented on PIG-2122:
[~aniket486]
Your workaround is not really much help if you're trying to debug a script
(i.e. line by line, DUMPing or ILLUSTRATing as you go. We want to be able to
use the interactive shell with the existing script (including params) but
execute it line by line.
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466619#comment-13466619
]
Aniket Mokashi commented on PIG-2122:
-
There is a workaround to this that you can use-
* Store your script line in a file (with $params included).
* Start grunt interactively
* type- run -param a=b -param c=d myscript.pig
This should do the trick.
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464326#comment-13464326
]
Joshua Hartman commented on PIG-2122:
-
This irritates me on a weekly basis. I'm often running ad-hoc analysis in grunt
with snippets taken from some heavier scripts I've written and I get bitten ALL
the time.
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259783#comment-13259783
]
Gianmarco De Francisci Morales commented on PIG-2122:
-
On the backward compatibility issue, I must say that I think modifying Grunt
and interactive mode is not as bad as modifying the batch mode.
I don't feel changing Grunt should be a big issue as most of the production job
that I know are batch scripts which would be unaffected.
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259781#comment-13259781
]
Gianmarco De Francisci Morales commented on PIG-2122:
-
Hi,
I can agree that the feature is useful.
However to debug a Pig script you can use the -debug and -dryrun options:
{code}
–dryrun
Flag. With this option, the script is not run and a fully
substituted Pig script produced in the current working
directory named original_script_name.substituted
{code}
This should help you avoid manual search/replace.
Cheers,
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259606#comment-13259606
]
Janne Jalkanen commented on PIG-2122:
-
Upvoting; this makes debugging Pig scripts *really* hard using Grunt. I mostly
run a part of the script simply by cut-n-pasting from the editor into a Grunt
shell, then stop to examine the intermediate values. Having to do manual
search-replace for variables makes them useless.
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101302#comment-13101302
]
Jeff Wartes commented on PIG-2122:
--
I found this JIRA issue because:
1) I was copying a script into the shell for testing and was confused why it
was wasn't working.
2) Reading the documentation and attempting alternative methods to set
parameters didn't work either.
I mention this because I hit *both* of cases previously presented as reasons to
fix this issue. The second, because of the first.
If this issue cannot be fixed, I'd suggest the grunt shell throw a more helpful
error when it encounters preprocessor directives, and warn when it sees likely
attempts at dollar-sign interpolation.
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049926#comment-13049926
]
Olga Natkovich commented on PIG-2122:
-
Backward compatibility is a requirement because we have many customers and many
of them uset this feature especially in production environment.
In my opinion design/implementation that does not provide backward
comptaibility for batch processing is a non-starter.
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049922#comment-13049922
]
Grant Ingersoll commented on PIG-2122:
--
I know it would be nice, but is back compat really a requirement for a pre 1.0
release?
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049920#comment-13049920
]
Olga Natkovich commented on PIG-2122:
-
I think the trick would be to maintain backward compatibility.
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049910#comment-13049910
]
Dmitriy V. Ryaboy commented on PIG-2122:
This really does diminish usefulness of Grunt significantly.
There isn't an easy way to apply the existing preprocessor here, which is a
shame, but it's an issue that comes up often. It would be great if someone took
a crack at fixing this.. theoretically having a in-memory variable map and
doing line-by-line substitution before handing off to the parser should work,
right?
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049847#comment-13049847
]
Grant Ingersoll commented on PIG-2122:
--
Seems to me one of the major use cases of the Grunt shell is to test your
scripts line by line. If you don't have parameter substitution, then you have
to futz around with hand editing out the substitutions to run it in Grunt, and
then add it back in before you submit it to run in file mode.
The other use case is, the documentation doesn't work! I would bet that many
people simply try out what is on the website as examples, so when things like
declare, etc. as copy and pasted from the website don't work, they lose
confidence in the tool.
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2122) Parameter Substitution doesn't work in the Grunt shell
[
https://issues.apache.org/jira/browse/PIG-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049829#comment-13049829
]
Olga Natkovich commented on PIG-2122:
-
This is by design. We did not see any need for parameter substitution in the
interactive mode. The current implementation is based on a preprocessor applied
to the entire script.
We would need a compelling case to reconsider.
> Parameter Substitution doesn't work in the Grunt shell
> --
>
> Key: PIG-2122
> URL: https://issues.apache.org/jira/browse/PIG-2122
> Project: Pig
> Issue Type: Bug
>Affects Versions: 0.8.0, 0.8.1
>Reporter: Grant Ingersoll
>Priority: Minor
>
> Simple param substitution and things like %declare (as copied out of the
> docs) don't work in the grunt shell.
> #Start Pig with: Start Pig with: bin/pig -x local -p time=FOO
> {quote}
> foo = LOAD '/user/grant/foo.txt' AS (a:chararray, b:chararray, c:chararray);
> Y = foreach foo generate *, '$time';
> dump Y;
> {quote}
> Output:
> {quote}
> 2011-06-13 20:22:24,197 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,$time)
> (4 5 6,,,$time)
> {quote}
> Same script, stored in junk.pig, run as: bin/pig -x local -p time=FOO junk.pig
> {quote}
> 2011-06-13 20:23:38,864 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> (1 2 3,,,FOO)
> (4 5 6,,,FOO)
> {quote}
> Also, things like don't work (nor does %declare):
> {quote}
> grunt> %default DATE '20090101';
> 2011-06-13 20:18:19,943 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1000: Error during parsing. Encountered " "%default "" at line 1,
> column 1.
> Was expecting one of:
>
> "cat" ...
> "fs" ...
> "sh" ...
> "cd" ...
> "cp" ...
> "copyFromLocal" ...
> "copyToLocal" ...
> "dump" ...
> "describe" ...
> "aliases" ...
> "explain" ...
> "help" ...
> "kill" ...
> "ls" ...
> "mv" ...
> "mkdir" ...
> "pwd" ...
> "quit" ...
> "register" ...
> "rm" ...
> "rmf" ...
> "set" ...
> "illustrate" ...
> "run" ...
> "exec" ...
> "scriptDone" ...
> "" ...
> ...
> ";" ...
>
> Details at logfile:
> /Users/grant.ingersoll/projects/apache/pig/release-0.8.1/pig_1308002917912.log
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
