[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971772#comment-14971772
]
opensauce commented on PIG-3114:
I had a problem in pigunit 0.15.0. It looks like this is fixed only if you
unoverride your STORE method. Be sure to build the PigTest object (and pig unit
objects in general) the same was as the TestPigTest class. This class has
testMacro which I used as a template.
https://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/pigunit/TestPigTest.java
{code}
@Test
// Script should only be registered once, otherwise Pig will complain
// Macro defined twice. See PIG-3114
public void testMacro() throws ParseException, IOException {
String[] args = {
"n=3",
"reducers=1",
"input=top_queries_input_data.txt",
"output=top_3_queries",
};
test = new PigTest(PIG_SCRIPT_MACRO, args);
// By default PigUnit removes all the STORE and DUMP
test.unoverride("STORE");
test.runScript();
assertTrue(cluster.delete(new Path("top_3_queries")));
}
{code}
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Daniel Dai
> Fix For: 0.12.0
>
> Attachments: PIG-3114-2.patch, PIG-3114-3.patch, PIG-3114.patch,
> PIG-3114.patch, PatchedPigTest.java, bug_pig.tbz2
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.pa
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14969847#comment-14969847
]
anoop bhatti commented on PIG-3114:
---
Has there been any update for this ticket? I am getting this error for the
versions above as well as 0.15.0.
Yet it looks like this ticket has been closed.
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Daniel Dai
> Fix For: 0.12.0
>
> Attachments: PIG-3114-2.patch, PIG-3114-3.patch, PIG-3114.patch,
> PIG-3114.patch, PatchedPigTest.java, bug_pig.tbz2
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337976#comment-14337976
]
Daniel Dai commented on PIG-3114:
-
The original fix does not apply to the new function introduced in PIG-2692. I
cannot find a easy fix for that. We do need to register the query multiple
times to get alias mock working, but that causes "Duplicated macro" error.
One simple work around is to change the test to register macro separately:
{code}
String[] script = {
"data = LOAD 'input' AS (query:CHARARRAY);",
"queries_group = GROUP data BY query;",
"queries_count = FOREACH queries_group GENERATE group AS query,
COUNT(data) AS total;",
"queries_ordered = my_macro_1(queries_count, query);",
"queries_limit = LIMIT queries_ordered 2;",
"STORE queries_limit INTO 'output';"
};
PigTest test = new PigTest(script, new String[]{});
test.getPigServer().registerQuery("DEFINE my_macro_1 (QUERY, A) RETURNS
C {" +
"$C = ORDER $QUERY BY total DESC, $A;" +
"} ;");
test.assertOutput("data", new String[] {"1"},
"queries_limit", new String[] {"(1)"});
{code}
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Daniel Dai
> Fix For: 0.12.0
>
> Attachments: PIG-3114-2.patch, PIG-3114-3.patch, PIG-3114.patch,
> PIG-3114.patch, PatchedPigTest.java, bug_pig.tbz2
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(Que
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14335819#comment-14335819
]
Jerome Serrano commented on PIG-3114:
-
It looks like the issue is still present in version 0.12.0, 0.12.1, 0.13.0 and
0.14.0. The same script (test.pig) is failing with the exact same error. I'm
attaching the code I used to reproduce the error (bug_pig.tbz2).
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Daniel Dai
> Fix For: 0.12.0
>
> Attachments: PIG-3114-2.patch, PIG-3114-3.patch, PIG-3114.patch,
> PIG-3114.patch, PatchedPigTest.java
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13780216#comment-13780216
]
Rohini Palaniswamy commented on PIG-3114:
-
+1
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12.0
>
> Attachments: PatchedPigTest.java, PIG-3114-2.patch, PIG-3114-3.patch,
> PIG-3114.patch, PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779470#comment-13779470
]
Rohini Palaniswamy commented on PIG-3114:
-
patch does not have top_queries_macro.pig. Looks good otherwise
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12.0
>
> Attachments: PatchedPigTest.java, PIG-3114-2.patch, PIG-3114.patch,
> PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742612#comment-13742612
]
Sajid Raza commented on PIG-3114:
-
Sorry, I haven't looked into the Cloudera's version of Pig. I'd suggest poking
through their source code snapshot.
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12
>
> Attachments: PatchedPigTest.java, PIG-3114.patch, PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13723539#comment-13723539
]
Ruslan Al-Fakikh commented on PIG-3114:
---
Sajid,
The patch didn't work for me:( I am getting the same "Reason: Duplicated macro
name..."
Also I see the message "No matching static PigServer field found to patch."
appearing in the console two times. My Pig version is
[cloudera@localhost pig-scripts]$ pig -version
Apache Pig version 0.11.0-cdh4.3.0 (rexported)
compiled May 27 2013, 20:48:21
Any thoughts? Any help would be much appreciated.
I also noticed another problem with Macros:
Here org.apache.pig.pigunit.PigTest.PigTest(String[], String[], String)
the STORE and DUMP statements are erased as far as I understand, but if there
is a STORE statement in my Macro, it doesn't get erased:(
Should we open another jira for it?
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12
>
> Attachments: PatchedPigTest.java, PIG-3114.patch, PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_g
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13719934#comment-13719934
]
Ruslan Al-Fakikh commented on PIG-3114:
---
Sajid Raza:
Thanks a lot!
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12
>
> Attachments: PatchedPigTest.java, PIG-3114.patch, PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13719060#comment-13719060
]
Sajid Raza commented on PIG-3114:
-
Attached the workaround to this JIRA.
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12
>
> Attachments: PatchedPigTest.java, PIG-3114.patch, PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718179#comment-13718179
]
Ruslan Al-Fakikh commented on PIG-3114:
---
Sajid Raza:
Can you please paste the code for your workaround?
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12
>
> Attachments: PIG-3114.patch, PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690641#comment-13690641
]
Sajid Raza commented on PIG-3114:
-
BTW, to get around the issues I created my own sub-class of PigTest with an
overridden registerScript method that resets the PigServer instance on every
invocation.
Doing so is inefficient since I'm reinitializing PigServer and registerScript
is called a number of times.
Why does PigTest register the text of the script multiple times with the
PigServer?
I would suggest lazily parsing the script when the user calls assertOutput.
Registering the script once when the input alias has been specified would
probably make life easier.
Either way, it would be nice to have a component that a) does not rely on
global (static) state, and b) that allows the end-user some transparency into
how it manages resources.
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12
>
> Attachments: PIG-3114.patch, PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690637#comment-13690637
]
Sajid Raza commented on PIG-3114:
-
I see the same behavior, and I would urge the maintainers to come to an
agreement on the fix.
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12
>
> Attachments: PIG-3114.patch, PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583936#comment-13583936
]
Rohini Palaniswamy commented on PIG-3114:
-
Then we should probably look at adding a reInitialize method to PigTest which
will reset the PigServer instance and can be used in tests like this.
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12
>
> Attachments: PIG-3114.patch, PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582532#comment-13582532
]
Chetan Nadgire commented on PIG-3114:
-
Taking a look. PigTest checks for existing instance of PigServer and reuses it
instead of creating new instance, will debug more.
Also did not see any unit test failures after applying this patch.
Right now pigunit build is failing on branch 0.11, opened
https://issues.apache.org/jira/browse/PIG-3201 to track this issue.
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12
>
> Attachments: PIG-3114.patch, PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message is a
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13581782#comment-13581782
]
Rohini Palaniswamy commented on PIG-3114:
-
[~chetan.nadgire],
Is it not possible for you to use a different PigServer instance for each
test case? Clearing the scriptCache in the registerScript() might break test
cases of other users who register script multiple times with different scripts.
Also clearing just the current DAG might not work when there are nested
scripts (Look for the graphs LinkedList in PigServer which stores multiple
graphs)
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12
>
> Attachments: PIG-3114.patch, PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fin
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13560148#comment-13560148
]
Daniel Dai commented on PIG-3114:
-
Looks good. Would you mind add a unit test case?
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Affects Versions: 0.11
>Reporter: Chetan Nadgire
>Assignee: Chetan Nadgire
> Fix For: 0.12
>
> Attachments: PIG-3114.patch, PIG-3114.patch
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13549272#comment-13549272
]
Chetan Nadgire commented on PIG-3114:
-
Here's my investigation:
I am using assertOutput(aliasInput, input, alias, expected) to verify output.
1. 'PigTest::assertOutput(String aliasInput, String[] input, String alias,
String[] expected)' calls 'PigTest::registerScript()' to parse input query and
to substitute aliases with input/output relations and load/store commands and
generate query to be submitted to pigserver.
It copies input (from pigtest) to destination file and calls
'PigTest::assertOutput(String alias, String[] expected)'
2. PigTest::assertOutput(String alias, String[] expected) again calls
PigTest::registerScript() and submits generated query to pigserver again. Note
that PigTest uses same instance on PigServer
It then appends newline to expected output and calls
'assertEquals(StringUtils.join(expected, "\n"),
StringUtils.join(getAlias(alias), "\n"));'
3. PigTest::getAlias() also calls registerScript()
So PigServer::registerScript() it called 3 times with modification to same
query. PigServer::Graph::scriptCache is used to store query and since same
PigServer is used for three calls, same query gets appended in scriptCache.
When this query is parse it errors out due to multiple definitions of macro.
Here's flow:
PigTest::assertOutput(String aliasInput, String[] input, String alias, String[]
expected)
-> PigTest::registerScript()
-> PigServer::registerScript()
query submitted to pigserver (scriptCache):
DEFINE my_macro_1 (QUERY, A) RETURNS C {
$C = ORDER $QUERY BY total DESC, $A;
}
data = LOAD 'input' AS (query:CHARARRAY);
queries_group = GROUP data BY query;
queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
total;
queries_ordered = my_macro_1(queries_count, query);
queries_limit = LIMIT queries_ordered 2;
STORE queries_limit INTO 'output';
PigTest::assertOutput(String alias, String[] expected)
-> PigTest::registerScript()
-> PigServer::registerScript()
query submitted to pigserver (scriptCache):
DEFINE my_macro_1 (QUERY, A) RETURNS C {
$C = ORDER $QUERY BY total DESC, $A;
}
data = LOAD 'input' AS (query:CHARARRAY);
queries_group = GROUP data BY query;
queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
total;
queries_ordered = my_macro_1(queries_count, query);
queries_limit = LIMIT queries_ordered 2;
STORE queries_limit INTO 'output';
DEFINE my_macro_1 (QUERY, A) RETURNS C {
$C = ORDER $QUERY BY total DESC, $A;
}
data = LOAD 'input' AS (query:CHARARRAY);
queries_group = GROUP data BY query;
queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
total;
queries_ordered = my_macro_1(queries_count, query);
queries_limit = LIMIT queries_ordered 2;
STORE queries_limit INTO 'output';
PigTest::assertEquals(String expected, String current)
-> PigTest::registerScript()
-> PigServer::registerScript()
query submitted to pigserver (scriptCache):
DEFINE my_macro_1 (QUERY, A) RETURNS C {
$C = ORDER $QUERY BY total DESC, $A;
}
data = LOAD 'input' AS (query:CHARARRAY);
queries_group = GROUP data BY query;
queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
total;
queries_ordered = my_macro_1(queries_count, query);
queries_limit = LIMIT queries_ordered 2;
STORE queries_limit INTO 'output';
DEFINE my_macro_1 (QUERY, A) RETURNS C {
$C = ORDER $QUERY BY total DESC, $A;
}
data = LOAD 'input' AS (query:CHARARRAY);
queries_group = GROUP data BY query;
queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
total;
queries_ordered = my_macro_1(queries_count, query);
queries_limit = LIMIT queries_ordered 2;
STORE queries_limit INTO 'output';
DEFINE my_macro_1 (QUERY, A) RETURNS C {
$C = ORDER $QUERY BY total DESC, $A;
}
data = LOAD 'input' AS (query:CHARARRAY);
queries_group = GROUP data BY query;
queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
total;
queries_ordered = my_macro_1(queries_count, query);
queries_limit = LIMIT queries_ordered 2;
STORE queries_limit INTO 'output';
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Reporter: Chetan Nadgire
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script wit
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13549127#comment-13549127
]
Daniel Dai commented on PIG-3114:
-
You are right. I can see it with pigunit. It runs fine with command line.
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Reporter: Chetan Nadgire
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[ https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13547657#comment-13547657 ] Chetan Nadgire commented on PIG-3114: - Hi Daniel, I am building pig.jar and pigunit.jar from branch-0.11. When I tried branch-0.10 getting different error: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. null at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1606) at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1549) at org.apache.pig.PigServer.registerQuery(PigServer.java:549) at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:968) at org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190) at org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:53) at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160) at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:251) at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:79) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197) Caused by: java.lang.NullPointerException at java.io.File.(File.java:222) at org.apache.pig.parser.QueryParserUtils.getFileFromImportSearchPath(QueryParserUtils.java:205) at org.apache.pig.parser.QueryParserDriver.getMacroFile(QueryParserDriver.java:352) at org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:411) at org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:270) at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:171) at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1598) ... 29 more Thanks, Chetan > Duplicated macro name error when using pigunit > -- > > Key: PIG-3114 > URL: https://issues.apache.org/jira/browse/PIG-3114 > Project: Pig > Issue Type: Bug > Components: parser >Reporter: Chetan Nadgire > > I'm using PigUnit to test a pig script within which a macro is defined. > Pig runs fine on cluster but getting parsing error with pigunit. > So I tried very basic pig script with macro and getting similar error. > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during > parsing. null. Reason: Duplicated macro name 'my_macro_1' > at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607) > at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546) > at org.apache.pig.PigServer.registerQuery(PigServer.java:516) > at > org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988) > at > org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194) > at > org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56) > at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160) > at org.apache.pig.pigunit.
[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
[
https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13547534#comment-13547534
]
Daniel Dai commented on PIG-3114:
-
Runs good for me with 0.10.1. Which version are you using?
> Duplicated macro name error when using pigunit
> --
>
> Key: PIG-3114
> URL: https://issues.apache.org/jira/browse/PIG-3114
> Project: Pig
> Issue Type: Bug
> Components: parser
>Reporter: Chetan Nadgire
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. null. Reason: Duplicated macro name 'my_macro_1'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> at
> org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: null. Reason: Duplicated macro name
> 'my_macro_1'
> at
> org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> at
> org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> ... 30 more
>
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
> $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data = LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS
> total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using
> it results in parsing error.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
