[jira] [Commented] (PIG-1990) support casting of complex types with empty inner schema to complex type with non-empty inner schema

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027983#comment-13027983
 ] 

Daniel Dai commented on PIG-1990:
-

+1

> support casting of complex types with empty inner schema to complex type with 
> non-empty inner schema
> 
>
> Key: PIG-1990
> URL: https://issues.apache.org/jira/browse/PIG-1990
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-1990.1.patch
>
>
> Use case like the following should be supported - 
> {code}
> a = load '1.txt' as (t:tuple());
> b = foreach a generate (tuple(int))t;
> {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (PIG-2016) -dot option does not work with explain and new logical plan

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai resolved PIG-2016.
-

  Resolution: Fixed
Hadoop Flags: [Reviewed]

Test-patch result:
 [exec] -1 overall.  
 [exec] 
 [exec] +1 @author.  The patch does not contain any @author tags.
 [exec] 
 [exec] +1 tests included.  The patch appears to include 7 new or 
modified tests.
 [exec] 
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec] 
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec] 
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
warnings.
 [exec] 
 [exec] -1 release audit.  The applied patch generated 563 release 
audit warnings (more than the trunk's current 562 warnings).

Ignore release audit warnings since all new source file contains proper header

Unit test pass.

Patch committed to both trunk and 0.9 branch.

> -dot option does not work with explain and new logical plan
> ---
>
> Key: PIG-2016
> URL: https://issues.apache.org/jira/browse/PIG-2016
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.0, 0.8.1, 0.9.0
>Reporter: Alan Gates
>Assignee: Daniel Dai
>Priority: Minor
> Fix For: 0.9.0
>
> Attachments: PIG-2016-1.patch, PIG-2016-2.patch
>
>
> If you specify -dot in explain, it is supposed to produce a file with the 
> graphs in .dot format.  While the physical plan and map reduce plan are 
> correctly output in .dot format, the new logical plan is still output in text 
> format.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2016) -dot option does not work with explain and new logical plan

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-2016:


Attachment: PIG-2016-2.patch

PIG-2016-2.patch fix findbug and javac warnings.

> -dot option does not work with explain and new logical plan
> ---
>
> Key: PIG-2016
> URL: https://issues.apache.org/jira/browse/PIG-2016
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.0, 0.8.1, 0.9.0
>Reporter: Alan Gates
>Assignee: Daniel Dai
>Priority: Minor
> Fix For: 0.9.0
>
> Attachments: PIG-2016-1.patch, PIG-2016-2.patch
>
>
> If you specify -dot in explain, it is supposed to produce a file with the 
> graphs in .dot format.  While the physical plan and map reduce plan are 
> correctly output in .dot format, the new logical plan is still output in text 
> format.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2030) Merged join/cogroup does not automatically ship loader

[Daniel Dai (JIRA)]

Merged join/cogroup does not automatically ship loader
--

 Key: PIG-2030
 URL: https://issues.apache.org/jira/browse/PIG-2030
 Project: Pig
  Issue Type: Bug
  Components: impl
Affects Versions: 0.9.0
Reporter: Daniel Dai
Assignee: Daniel Dai
 Fix For: 0.9.0
 Attachments: PIG-2030-1.patch

The following script fail due to TableLoader class not found (If the jar is in 
classpath):
{code}
a = load '/user/pig/tests/data/zebra/singlefile/studentsortedtab10k' using 
org.apache.hadoop.zebra.pig.TableLoader('', 'sorted');
b = load '/user/pig/tests/data/zebra/singlefile/votersortedtab10k' using 
org.apache.hadoop.zebra.pig.TableLoader('', 'sorted');
g = cogroup a by $0, b by $0 using 'merge';
store g into '/user/pig/out/jianyong.1304374720/ZebraMapCogrp_1.out';
{code}

If we use register, the error goes away. However, Pig always ship jars 
containing LoadFunc automatically. It should be the same for merged 
cogroup/join.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2030) Merged join/cogroup does not automatically ship loader

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-2030:


Attachment: PIG-2030-1.patch

> Merged join/cogroup does not automatically ship loader
> --
>
> Key: PIG-2030
> URL: https://issues.apache.org/jira/browse/PIG-2030
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.9.0
>
> Attachments: PIG-2030-1.patch
>
>
> The following script fail due to TableLoader class not found (If the jar is 
> in classpath):
> {code}
> a = load '/user/pig/tests/data/zebra/singlefile/studentsortedtab10k' using 
> org.apache.hadoop.zebra.pig.TableLoader('', 'sorted');
> b = load '/user/pig/tests/data/zebra/singlefile/votersortedtab10k' using 
> org.apache.hadoop.zebra.pig.TableLoader('', 'sorted');
> g = cogroup a by $0, b by $0 using 'merge';
> store g into '/user/pig/out/jianyong.1304374720/ZebraMapCogrp_1.out';
> {code}
> If we use register, the error goes away. However, Pig always ship jars 
> containing LoadFunc automatically. It should be the same for merged 
> cogroup/join.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2029) Inconsistency in Pig Stats reports

[Viraj Bhat (JIRA)]

Inconsistency in Pig Stats reports 
---

 Key: PIG-2029
 URL: https://issues.apache.org/jira/browse/PIG-2029
 Project: Pig
  Issue Type: Bug
  Components: impl
Affects Versions: 0.8.1, 0.9.0
Reporter: Viraj Bhat
 Fix For: 0.8.1, 0.9.0


I have a Pig script which reports varying Stats for the same M/R job (same 
inputs). Sometimes the PigStats reports all the stats (such as 
Maps,Reduces,MaxMapTime,MinMapTime,AvgMapTime,MaxReduceTime, MinReduceTime and 
AvgReduceTime) for the M/R job as 0. Sometimes it reports it correctly.

Enclosed are the stderr logs for 2 runs, you can notice that for Run 1 
job_201103091134_556600 from Run 1; has 0 against all the columns whereas in 
Run 2, Hadoop job job_201104272229_75693 has some valid values. 

The actual Job Tracker link shows that they are non empty. This points to a bug 
in the interaction of the PigStats module with the Jobtracker.

Run 1:
{quote}
Job Stats (time in seconds):
JobId   MapsReduces MaxMapTime  MinMapTIme  AvgMapTime  
MaxReduceTime   MinReduceTime   AvgReduceTime   Alias   Feature Outputs
job_201103091134_556458 160 100 552 191 368 1257371 
392 
IN,SP10P,SP11P,SP12P,SP13P,SP16P,SP17P,SP18P,SP20P,SP21P,SP22P,SP23P,SP24P,SP26P,SP27P,SP28P,SP29P,SP30P,SP31P,SP32P,SP33P,SP34P,SP4P,SP6P,SP7P,SP8P,SP9P
   DISTINCT,MULTI_QUERY
job_201103091134_556600 0   0   0   0   0   0   0   
0   UNION5  MULTI_QUERY,MAP_ONLY/user/viraj/dir,,
job_201103091134_556601 7   100 17  8   14  200 15  
27  CNJOIN25,GNJOIN25,sampleNJOIN25 GROUP_BY,COMBINER   
job_201103091134_556602 0   0   0   0   0   0   0   
0   CNJOIN3,GNJOIN3,sampleNJOIN3GROUP_BY,COMBINER   
job_201103091134_556603 0   0   0   0   0   0   0   
0   CNJOIN15,GNJOIN15,sampleNJOIN15 GROUP_BY,COMBINER   
job_201103091134_556604 2   100 13  7   10  34  13  
31  CNJOIN19,GNJOIN19,sampleNJOIN19 GROUP_BY,COMBINER   
job_201103091134_556644 0   0   0   0   0   0   0   
0   ONJOIN15SAMPLER 
job_201103091134_556645 0   0   0   0   0   0   0   
0   ONJOIN25SAMPLER 
job_201103091134_556646 0   0   0   0   0   0   0   
0   ONJOIN3 SAMPLER 
job_201103091134_556654 0   0   0   0   0   0   0   
0   ONJOIN19SAMPLER 
job_201103091134_556662 0   0   0   0   0   0   0   
0   ONJOIN19ORDER_BY,COMBINER
..
{quote}


Run 2:
{quote}

Job Stats (time in seconds):
JobId   MapsReduces MaxMapTime  MinMapTIme  AvgMapTime  
MaxReduceTime   MinReduceTime   AvgReduceTime   Alias   Feature Outputs
job_201104272229_75503  159 100 484 192 353 396 308 
321 
IN,SP10P,SP11P,SP12P,SP13P,SP16P,SP17P,SP18P,SP20P,SP21P,SP22P,SP23P,SP24P,SP26P,SP27P,SP28P,SP29P,SP30P,SP31P,SP32P,SP33P,SP34P,SP4P,SP6P,SP7P,SP8P,SP9P
   DISTINCT,MULTI_QUERY
job_201104272229_75693  18  0   31  14  24  0   0   
UNION5 MULTI_QUERY,MAP_ONLY /user/viraj/dir,
job_201104272229_75694  7   100 34  13  22  46  20  
25  CNJOIN25,GNJOIN25,sampleNJOIN25 GROUP_BY,COMBINER   
job_201104272229_75695  125 100 19  11  15  32  18  
26  CNJOIN3,GNJOIN3,sampleNJOIN3GROUP_BY,COMBINER   
job_201104272229_75698  1   100 12  12  12  13  9   
11  CNJOIN15,GNJOIN15,sampleNJOIN15 GROUP_BY,COMBINER   
job_201104272229_75702  2   100 21  5   13  35  22  
26  CNJOIN19,GNJOIN19,sampleNJOIN19 GROUP_BY,COMBINER   
job_201104272229_75724  1   1   4   4   4   11  11  
11  ONJOIN15SAMPLER 
job_201104272229_75725  0   0   0   0   0   0   0   
ONJOIN25SAMPLER 
job_201104272229_75726  6   1   8   6   8   24  24  
24  ONJOIN3 SAMPLER 
job_201104272229_75729  0   0   0   0   0   0   0   
ONJOIN19SAMPLER 
job_201104272229_75752  1   100 5   5   5   12  9   
11  ONJOIN19ORDER_BY,COMBINER
..
{quote}

Viraj

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-1998) Allow macro to return void

[Richard Ding (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding updated PIG-1998:
--

  Resolution: Fixed
Hadoop Flags: [Reviewed]
  Status: Resolved  (was: Patch Available)

Patch committed to trunk and 0.9 branch.

> Allow macro to return void
> --
>
> Key: PIG-1998
> URL: https://issues.apache.org/jira/browse/PIG-1998
> Project: Pig
>  Issue Type: Improvement
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Richard Ding
>Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1998_1.patch, PIG-1998_2.patch, PIG-1998_3.patch
>
>
> Pig macro is allowed to not have output alias. But this property isn't clear 
> from macro definition and macro invocation (macro inline). Here we propose to 
> make it clear:
> 1. If a macro doesn't output any alias, it must specify void as return value. 
> For example:
> {code}  
> define mymacro(...) returns void {
>... ...
> };
> {code}
> 2. If a macro doesn't output any alias, it must be invoked without return 
> value. For example, to invoke above macro, just specify:
> {code}
> mymacro(...);
> {code}
> 3. Any non-void return alias in the macro definition must exist in the macro 
> body and be prefixed with $. For example:
> {code}  
> define mymacro(...) returns B {
>... ...
>$B = filter ...;
> };
> {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2028) Speed up multiquery unit tests

[Dmitriy V. Ryaboy (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027944#comment-13027944
 ] 

Dmitriy V. Ryaboy commented on PIG-2028:


Nice speed improvement!
A nitpick: if the test is interrupted, it will leave test files in the working 
directory; that's messy and causes failures on reruns (since the files already 
exist). It's better to write to and load from a temp directory created inside 
java.io.tmpdir, and to set deleteOnExit() on them.

> Speed up multiquery unit tests 
> ---
>
> Key: PIG-2028
> URL: https://issues.apache.org/jira/browse/PIG-2028
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Richard Ding
>Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-2028.patch
>
>
> Switch TestMultiQueryBasic and TestMultiQuery to use LOCAL mode. The results 
> on my laptop:
> Using Mini Cluster:
> TestMultiQueryBasic: 17 min 17 sec
> TestMultiQuery:  23 min 2 sec
> Using LOCAL mode:
> TestMultiQueryBasic: 4 min 17 sec
> TestMultiQuery:  5 min 51 sec

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1775) Removal of old logical plan

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027943#comment-13027943
 ] 

Thejas M Nair commented on PIG-1775:


Comments on - PIG-1775.patch -
TestLogToPhyCompiler.testComplexForeach - this comment can be removed - // 
Commented out due to PIG-2020

TestMapSideCogroup.testFailure1 -  fix for PIG-2018 has been checked in , can 
this be enabled?

TestLogicalPlanBuilder.testQuery18 -  fix for PIG-1996 has been checked in , 
can this be enabled?

TestLogicalPlanBuilder.testQuery96 - verification part of the test is missing

TestLogicalPlanBuilder.testQuery97 onwards are commented out, should they also 
be migrated ?

src/org/apache/pig/impl/util/Utils.java - getSchemaFromString has been changed 
to always return null, should it be re-enabled ?

> Removal of old logical plan
> ---
>
> Key: PIG-1775
> URL: https://issues.apache.org/jira/browse/PIG-1775
> Project: Pig
>  Issue Type: Improvement
>Affects Versions: 0.9.0
>Reporter: Yan Zhou
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-1775.patch
>
>
> The new logical plan will only be used and the old logical plan will be 
> removed once the new one is stable enough. It is scheduled for the 0.9 
> release.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1999) Macro alias masker should consider schema context

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027938#comment-13027938
 ] 

Xuefu Zhang commented on PIG-1999:
--

+1 Patch PIG-1999_2.patch looks good.

> Macro alias masker should consider schema context 
> --
>
> Key: PIG-1999
> URL: https://issues.apache.org/jira/browse/PIG-1999
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Richard Ding
>Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1999_1.patch, PIG-1999_2.patch
>
>
> Macro alias masker doesn't consider the current schema context. This results 
> errors when deciding with alias to mask. Here is an example:
> {code}
> define toBytearray(in, intermediate) returns e { 
>a = load '$in' as (name:chararray, age:long, gpa: float);
>b = group a by  name;
>c = foreach b generate a, (1,2,3);
>store c into '$intermediate' using BinStorage();
>d = load '$intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, 
> t2:tuple(a,b,c));
>$e = foreach d generate COUNT(b), t2.a, t2.b, t2.c;
> };
>  
> f = toBytearray ('data', 'output1');
> {code} 
> Now the alias masker mistakes b in COUNT(b) as an alias instead of b in the 
> current schema.
> The workaround is to not use alias as as names in the schema definition. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1998) Allow macro to return void

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027926#comment-13027926
 ] 

Xuefu Zhang commented on PIG-1998:
--

+1 Patch PIG-1998_3.patch looks good.

> Allow macro to return void
> --
>
> Key: PIG-1998
> URL: https://issues.apache.org/jira/browse/PIG-1998
> Project: Pig
>  Issue Type: Improvement
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Richard Ding
>Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1998_1.patch, PIG-1998_2.patch, PIG-1998_3.patch
>
>
> Pig macro is allowed to not have output alias. But this property isn't clear 
> from macro definition and macro invocation (macro inline). Here we propose to 
> make it clear:
> 1. If a macro doesn't output any alias, it must specify void as return value. 
> For example:
> {code}  
> define mymacro(...) returns void {
>... ...
> };
> {code}
> 2. If a macro doesn't output any alias, it must be invoked without return 
> value. For example, to invoke above macro, just specify:
> {code}
> mymacro(...);
> {code}
> 3. Any non-void return alias in the macro definition must exist in the macro 
> body and be prefixed with $. For example:
> {code}  
> define mymacro(...) returns B {
>... ...
>$B = filter ...;
> };
> {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2028) Speed up multiquery unit tests

[Richard Ding (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding updated PIG-2028:
--

Attachment: PIG-2028.patch

> Speed up multiquery unit tests 
> ---
>
> Key: PIG-2028
> URL: https://issues.apache.org/jira/browse/PIG-2028
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Richard Ding
>Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-2028.patch
>
>
> Switch TestMultiQueryBasic and TestMultiQuery to use LOCAL mode. The results 
> on my laptop:
> Using Mini Cluster:
> TestMultiQueryBasic: 17 min 17 sec
> TestMultiQuery:  23 min 2 sec
> Using LOCAL mode:
> TestMultiQueryBasic: 4 min 17 sec
> TestMultiQuery:  5 min 51 sec

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2028) Speed up multiquery unit tests

[Richard Ding (JIRA)]

Speed up multiquery unit tests 
---

 Key: PIG-2028
 URL: https://issues.apache.org/jira/browse/PIG-2028
 Project: Pig
  Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Richard Ding
Assignee: Richard Ding
 Fix For: 0.9.0


Switch TestMultiQueryBasic and TestMultiQuery to use LOCAL mode. The results on 
my laptop:

Using Mini Cluster:

TestMultiQueryBasic: 17 min 17 sec
TestMultiQuery:  23 min 2 sec

Using LOCAL mode:

TestMultiQueryBasic: 4 min 17 sec
TestMultiQuery:  5 min 51 sec




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2025) org.apache.pig.test.udf.evalfunc.TOMAP is missing package declaration

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated PIG-2025:


   Resolution: Fixed
Fix Version/s: 0.10
   Status: Resolved  (was: Patch Available)

Patch checked in, thanks Gianmarco.

> org.apache.pig.test.udf.evalfunc.TOMAP is missing package declaration
> -
>
> Key: PIG-2025
> URL: https://issues.apache.org/jira/browse/PIG-2025
> Project: Pig
>  Issue Type: Bug
>Reporter: Gianmarco De Francisci Morales
>Assignee: Gianmarco De Francisci Morales
>Priority: Trivial
> Fix For: 0.10
>
> Attachments: PIG-2025.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1883) Pig's progress estimation should account for parallel job executions

[jirapos...@reviews.apache.org (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027863#comment-13027863
 ] 

jirapos...@reviews.apache.org commented on PIG-1883:



---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/683/#review631
---


This doesn't lend itself well to automated testing.  Any thoughts on how to 
test how the new progress indicator does versus the existing one?  Have you run 
any initial tests to measure this?


trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/Launcher.java


I don't understand the logic here.  Why is it 0% done if ANY job is 
waiting, etc.?  Some of the jobs may be done and some partially done and some 
not even started.



trunk/src/org/apache/pig/impl/plan/OperatorPlan.java


This code shouldn't be in OperatorPlan.  We want to keep that as clean as 
possible.  Instead you should build a new Walker type that can do this 
calculation.



trunk/src/org/apache/pig/impl/plan/OperatorPlan.java


You have tabs here and some other spots.  Please make sure you use 4 spaces 
rather than tabs.



trunk/src/org/apache/pig/tools/pigstats/PigProgressNotificationListener.java


Why is a separate method needed here?  When users turn on the new progress 
indicator I assume they don't get the old one too.  Given that the interfaces 
are the same it seems one method should suffice here.



trunk/src/org/apache/pig/tools/pigstats/ScriptState.java


It looks like this comment got attached to the run method.  Also, the 
method has only one parameter, but two are listed in the comment.


- Alan


On 2011-05-02 20:41:04, Alan Gates wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/683/
bq.  ---
bq.  
bq.  (Updated 2011-05-02 20:41:04)
bq.  
bq.  
bq.  Review request for pig.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This is Laukik's patch for PIG-1883
bq.  
bq.  
bq.  This addresses bug PIG-1883.
bq.  https://issues.apache.org/jira/browse/PIG-1883
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.trunk/src/org/apache/pig/Main.java 1097661 
bq.
trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/Launcher.java
 1097661 
bq.
trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java
 1097661 
bq.trunk/src/org/apache/pig/impl/plan/OperatorPlan.java 1097661 
bq.trunk/src/org/apache/pig/scripting/SyncProgressNotificationAdaptor.java 
1097661 
bq.
trunk/src/org/apache/pig/tools/pigstats/PigProgressNotificationListener.java 
1097661 
bq.trunk/src/org/apache/pig/tools/pigstats/ScriptState.java 1097661 
bq.trunk/test/org/apache/pig/test/TestOperatorPlan.java 1097661 
bq.  
bq.  Diff: https://reviews.apache.org/r/683/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Alan
bq.  
bq.



> Pig's progress estimation should account for parallel job executions
> 
>
> Key: PIG-1883
> URL: https://issues.apache.org/jira/browse/PIG-1883
> Project: Pig
>  Issue Type: Improvement
>Reporter: Laukik Chitnis
>Assignee: Laukik Chitnis
> Attachments: PIG-1883-2.patch
>
>
> Currently, Pig's progress estimation is based on the percentage of jobs 
> completed out of the total number of MR jobs. However, since the MR operators 
> are arranged in a DAG (and hence more than 1 job might be submitted for 
> execution in parallel), the progress estimation can be improved by 
> considering the number of jobs in the critical path, instead of just the 
> total number of jobs.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1890) Fix piggybank unit test TestAvroStorage

[Ken Goodhope (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027862#comment-13027862
 ] 

Ken Goodhope commented on PIG-1890:
---

I have been working on some fixes to AvroStorage already.  I should be able to 
make sure this issue gets addressed in those fixes as will.  Will have it done 
sometime this week.

> Fix piggybank unit test TestAvroStorage
> ---
>
> Key: PIG-1890
> URL: https://issues.apache.org/jira/browse/PIG-1890
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Jakob Homan
> Fix For: 0.9.0
>
> Attachments: PIG-1890-1.patch
>
>
> TestAvroStorage fail on trunk. There are two reasons:
> 1. After PIG-1680, we call LoadFunc.setLocation one more time.
> 2. The schema for AvroStorage seems to be wrong. For example, in first test 
> case testArrayDefault, the schema for "in" is set to "PIG_WRAPPER: (FIELD: 
> {PIG_WRAPPER: (ARRAY_ELEM: float)})". It seems PIG_WRAPPER is redundant. This 
> issue is hidden until PIG-1188 checked in.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: Review request for PIG-1883-2.patch

[Alan Gates]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/683/#review631
---


This doesn't lend itself well to automated testing.  Any thoughts on how to 
test how the new progress indicator does versus the existing one?  Have you run 
any initial tests to measure this?


trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/Launcher.java


I don't understand the logic here.  Why is it 0% done if ANY job is 
waiting, etc.?  Some of the jobs may be done and some partially done and some 
not even started.



trunk/src/org/apache/pig/impl/plan/OperatorPlan.java


This code shouldn't be in OperatorPlan.  We want to keep that as clean as 
possible.  Instead you should build a new Walker type that can do this 
calculation.



trunk/src/org/apache/pig/impl/plan/OperatorPlan.java


You have tabs here and some other spots.  Please make sure you use 4 spaces 
rather than tabs.



trunk/src/org/apache/pig/tools/pigstats/PigProgressNotificationListener.java


Why is a separate method needed here?  When users turn on the new progress 
indicator I assume they don't get the old one too.  Given that the interfaces 
are the same it seems one method should suffice here.



trunk/src/org/apache/pig/tools/pigstats/ScriptState.java


It looks like this comment got attached to the run method.  Also, the 
method has only one parameter, but two are listed in the comment.


- Alan


On 2011-05-02 20:41:04, Alan Gates wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/683/
> ---
> 
> (Updated 2011-05-02 20:41:04)
> 
> 
> Review request for pig.
> 
> 
> Summary
> ---
> 
> This is Laukik's patch for PIG-1883
> 
> 
> This addresses bug PIG-1883.
> https://issues.apache.org/jira/browse/PIG-1883
> 
> 
> Diffs
> -
> 
>   trunk/src/org/apache/pig/Main.java 1097661 
>   
> trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/Launcher.java
>  1097661 
>   
> trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java
>  1097661 
>   trunk/src/org/apache/pig/impl/plan/OperatorPlan.java 1097661 
>   trunk/src/org/apache/pig/scripting/SyncProgressNotificationAdaptor.java 
> 1097661 
>   
> trunk/src/org/apache/pig/tools/pigstats/PigProgressNotificationListener.java 
> 1097661 
>   trunk/src/org/apache/pig/tools/pigstats/ScriptState.java 1097661 
>   trunk/test/org/apache/pig/test/TestOperatorPlan.java 1097661 
> 
> Diff: https://reviews.apache.org/r/683/diff
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Alan
> 
>

[jira] [Commented] (PIG-1890) Fix piggybank unit test TestAvroStorage

[Olga Natkovich (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027855#comment-13027855
 ] 

Olga Natkovich commented on PIG-1890:
-

Hi Jacob, 

Are you planning to address the additional issue for 0.9 or should we delay 
this?

> Fix piggybank unit test TestAvroStorage
> ---
>
> Key: PIG-1890
> URL: https://issues.apache.org/jira/browse/PIG-1890
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Daniel Dai
>Assignee: Jakob Homan
> Fix For: 0.9.0
>
> Attachments: PIG-1890-1.patch
>
>
> TestAvroStorage fail on trunk. There are two reasons:
> 1. After PIG-1680, we call LoadFunc.setLocation one more time.
> 2. The schema for AvroStorage seems to be wrong. For example, in first test 
> case testArrayDefault, the schema for "in" is set to "PIG_WRAPPER: (FIELD: 
> {PIG_WRAPPER: (ARRAY_ELEM: float)})". It seems PIG_WRAPPER is redundant. This 
> issue is hidden until PIG-1188 checked in.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (PIG-1622) DEFINE streaming options are ill defined and not properly documented

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang resolved PIG-1622.
--

  Resolution: Fixed
Hadoop Flags: [Incompatible change]

> DEFINE streaming options are ill defined and not properly documented
> 
>
> Key: PIG-1622
> URL: https://issues.apache.org/jira/browse/PIG-1622
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Alan Gates
>Assignee: Xuefu Zhang
>Priority: Minor
> Fix For: 0.9.0
>
> Attachments: PIG-1622-1.patch, PIG-1622-2.patch, PIG-1622.patch
>
>
> According to the documentation 
> (http://hadoop.apache.org/pig/docs/r0.7.0/piglatin_ref2.html#DEFINE) the 
> syntax for DEFINE when used to define a streaming command is:
> DEFINE cmd INPUT(stdin|path) OUTPUT(stdout|stderr|path) SHIP(path [, path, 
> ...]) CACHE (path [, path, ...])
> However, the actual parser accepts something pretty different.  Consider the 
> following script:
> {code}
> define strm `wc -l` INPUT(stdin) 
> CACHE('/Users/gates/.vimrc#myvim') 
> OUTPUT(stdin)
> INPUT('/tmp/fred') 
> OUTPUT('/tmp/bob')
> SHIP('/Users/gates/.bashrc') 
> SHIP('/Users/gates/.vimrc') 
> CACHE('/Users/gates/.bashrc#mybash')
> stderr('/tmp/errors' limit 10);
> A = load '/Users/gates/test/data/studenttab10';
> B = stream A through strm;
> dump B;
> {code}
> The above actually parsers.  I see several issues here:
> # What do multiple INPUT and OUTPUT statements mean in the context of 
> streaming?  These should not be allowed.
> # The documentation implies an order (INPUT, OUTPUT, SHIP, CACHE) that is not 
> enforced by the parser.  We should either enforce the order in the parser or 
> update the documentation.  Most likely the latter to avoid breaking existing 
> scripts.
> # Why are multiple SHIP and CACHE clauses allowed when each can take multiple 
> paths?  It seems we should only allow one of each.
> # The error clause is completely different that what is given in the 
> documentation.  I suspect this is a documentation error and the grammar 
> supported by the parser here is what we want.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-1622) DEFINE streaming options are ill defined and not properly documented

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated PIG-1622:
-

Assignee: Xuefu Zhang  (was: Corinne Chandel)

> DEFINE streaming options are ill defined and not properly documented
> 
>
> Key: PIG-1622
> URL: https://issues.apache.org/jira/browse/PIG-1622
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Alan Gates
>Assignee: Xuefu Zhang
>Priority: Minor
> Fix For: 0.9.0
>
> Attachments: PIG-1622-1.patch, PIG-1622-2.patch, PIG-1622.patch
>
>
> According to the documentation 
> (http://hadoop.apache.org/pig/docs/r0.7.0/piglatin_ref2.html#DEFINE) the 
> syntax for DEFINE when used to define a streaming command is:
> DEFINE cmd INPUT(stdin|path) OUTPUT(stdout|stderr|path) SHIP(path [, path, 
> ...]) CACHE (path [, path, ...])
> However, the actual parser accepts something pretty different.  Consider the 
> following script:
> {code}
> define strm `wc -l` INPUT(stdin) 
> CACHE('/Users/gates/.vimrc#myvim') 
> OUTPUT(stdin)
> INPUT('/tmp/fred') 
> OUTPUT('/tmp/bob')
> SHIP('/Users/gates/.bashrc') 
> SHIP('/Users/gates/.vimrc') 
> CACHE('/Users/gates/.bashrc#mybash')
> stderr('/tmp/errors' limit 10);
> A = load '/Users/gates/test/data/studenttab10';
> B = stream A through strm;
> dump B;
> {code}
> The above actually parsers.  I see several issues here:
> # What do multiple INPUT and OUTPUT statements mean in the context of 
> streaming?  These should not be allowed.
> # The documentation implies an order (INPUT, OUTPUT, SHIP, CACHE) that is not 
> enforced by the parser.  We should either enforce the order in the parser or 
> update the documentation.  Most likely the latter to avoid breaking existing 
> scripts.
> # Why are multiple SHIP and CACHE clauses allowed when each can take multiple 
> paths?  It seems we should only allow one of each.
> # The error clause is completely different that what is given in the 
> documentation.  I suspect this is a documentation error and the grammar 
> supported by the parser here is what we want.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1883) Pig's progress estimation should account for parallel job executions

[jirapos...@reviews.apache.org (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027830#comment-13027830
 ] 

jirapos...@reviews.apache.org commented on PIG-1883:



---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/683/
---

Review request for pig.


Summary
---

This is Laukik's patch for PIG-1883


This addresses bug PIG-1883.
https://issues.apache.org/jira/browse/PIG-1883


Diffs
-

  trunk/src/org/apache/pig/Main.java 1097661 
  
trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/Launcher.java
 1097661 
  
trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java
 1097661 
  trunk/src/org/apache/pig/impl/plan/OperatorPlan.java 1097661 
  trunk/src/org/apache/pig/scripting/SyncProgressNotificationAdaptor.java 
1097661 
  trunk/src/org/apache/pig/tools/pigstats/PigProgressNotificationListener.java 
1097661 
  trunk/src/org/apache/pig/tools/pigstats/ScriptState.java 1097661 
  trunk/test/org/apache/pig/test/TestOperatorPlan.java 1097661 

Diff: https://reviews.apache.org/r/683/diff


Testing
---


Thanks,

Alan



> Pig's progress estimation should account for parallel job executions
> 
>
> Key: PIG-1883
> URL: https://issues.apache.org/jira/browse/PIG-1883
> Project: Pig
>  Issue Type: Improvement
>Reporter: Laukik Chitnis
>Assignee: Laukik Chitnis
> Attachments: PIG-1883-2.patch
>
>
> Currently, Pig's progress estimation is based on the percentage of jobs 
> completed out of the total number of MR jobs. However, since the MR operators 
> are arranged in a DAG (and hence more than 1 job might be submitted for 
> execution in parallel), the progress estimation can be improved by 
> considering the number of jobs in the critical path, instead of just the 
> total number of jobs.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Review Request: Review request for PIG-1883-2.patch

[Alan Gates]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/683/
---

Review request for pig.


Summary
---

This is Laukik's patch for PIG-1883


This addresses bug PIG-1883.
https://issues.apache.org/jira/browse/PIG-1883


Diffs
-

  trunk/src/org/apache/pig/Main.java 1097661 
  
trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/Launcher.java
 1097661 
  
trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java
 1097661 
  trunk/src/org/apache/pig/impl/plan/OperatorPlan.java 1097661 
  trunk/src/org/apache/pig/scripting/SyncProgressNotificationAdaptor.java 
1097661 
  trunk/src/org/apache/pig/tools/pigstats/PigProgressNotificationListener.java 
1097661 
  trunk/src/org/apache/pig/tools/pigstats/ScriptState.java 1097661 
  trunk/test/org/apache/pig/test/TestOperatorPlan.java 1097661 

Diff: https://reviews.apache.org/r/683/diff


Testing
---


Thanks,

Alan

[jira] [Commented] (PIG-2016) -dot option does not work with explain and new logical plan

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027826#comment-13027826
 ] 

Alan Gates commented on PIG-2016:
-

+1, looks good

> -dot option does not work with explain and new logical plan
> ---
>
> Key: PIG-2016
> URL: https://issues.apache.org/jira/browse/PIG-2016
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.0, 0.8.1, 0.9.0
>Reporter: Alan Gates
>Assignee: Daniel Dai
>Priority: Minor
> Fix For: 0.9.0
>
> Attachments: PIG-2016-1.patch
>
>
> If you specify -dot in explain, it is supposed to produce a file with the 
> graphs in .dot format.  While the physical plan and map reduce plan are 
> correctly output in .dot format, the new logical plan is still output in text 
> format.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2027) NPE if Pig don't have permission for log file

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-2027:


Attachment: PIG-2027-1.patch

> NPE if Pig don't have permission for log file
> -
>
> Key: PIG-2027
> URL: https://issues.apache.org/jira/browse/PIG-2027
> Project: Pig
>  Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Trivial
> Fix For: 0.10
>
> Attachments: PIG-2027-1.patch
>
>
> If specify a log file to Pig, but Pig don't have write permission, if any 
> failure in Pig script, we will get a NPE in addition to Pig script failure:
> 2011-05-02 13:18:36,493 [main] ERROR org.apache.pig.tools.grunt.Grunt - 
> java.lang.NullPointerException
> at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:172)
> at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:79)
> at 
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:131)
> at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:180)
> at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:152)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> at org.apache.pig.Main.run(Main.java:554)
> at org.apache.pig.Main.main(Main.java:109)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2019) smoketest-jar target has to depend on pigunit-jar to guarantee inclusion of test classes

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027820#comment-13027820
 ] 

Konstantin Boudnik commented on PIG-2019:
-

Right, I understand. I wanted to make sure that I am not missing any tests 
which can be run against a real cluster (e.g. smoke tests). Thanks.

> smoketest-jar target has to depend on pigunit-jar to guarantee inclusion of 
> test classes
> 
>
> Key: PIG-2019
> URL: https://issues.apache.org/jira/browse/PIG-2019
> Project: Pig
>  Issue Type: Bug
>  Components: build
>Affects Versions: 0.9.0
>Reporter: Konstantin Boudnik
>Assignee: Konstantin Boudnik
> Fix For: 0.10
>
> Attachments: PIG-2019.patch
>
>
> pigsmoke artifact uses classes from pigunit for real cluster testing. 
> However, in the deployment phase the artifact comes out without the classes 
> added. It happens because of the execution order of targets during deployment.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2019) smoketest-jar target has to depend on pigunit-jar to guarantee inclusion of test classes

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated PIG-2019:


   Resolution: Fixed
Fix Version/s: (was: 0.9.0)
   0.10
   Status: Resolved  (was: Patch Available)

Patch checked in.  Thanks Cos.

> smoketest-jar target has to depend on pigunit-jar to guarantee inclusion of 
> test classes
> 
>
> Key: PIG-2019
> URL: https://issues.apache.org/jira/browse/PIG-2019
> Project: Pig
>  Issue Type: Bug
>  Components: build
>Affects Versions: 0.9.0
>Reporter: Konstantin Boudnik
>Assignee: Konstantin Boudnik
> Fix For: 0.10
>
> Attachments: PIG-2019.patch
>
>
> pigsmoke artifact uses classes from pigunit for real cluster testing. 
> However, in the deployment phase the artifact comes out without the classes 
> added. It happens because of the execution order of targets during deployment.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2027) NPE if Pig don't have permission for log file

[Daniel Dai (JIRA)]

NPE if Pig don't have permission for log file
-

 Key: PIG-2027
 URL: https://issues.apache.org/jira/browse/PIG-2027
 Project: Pig
  Issue Type: Bug
Reporter: Daniel Dai
Assignee: Daniel Dai
Priority: Trivial
 Fix For: 0.10


If specify a log file to Pig, but Pig don't have write permission, if any 
failure in Pig script, we will get a NPE in addition to Pig script failure:

2011-05-02 13:18:36,493 [main] ERROR org.apache.pig.tools.grunt.Grunt - 
java.lang.NullPointerException
at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:172)
at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:79)
at 
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:131)
at 
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:180)
at 
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:152)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
at org.apache.pig.Main.run(Main.java:554)
at org.apache.pig.Main.main(Main.java:109)



--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2019) smoketest-jar target has to depend on pigunit-jar to guarantee inclusion of test classes

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027814#comment-13027814
 ] 

Alan Gates commented on PIG-2019:
-

I'm not saying it's the wrong framework.  I just wanted to make sure this 
wasn't supposed to include some actual tests as well.

> smoketest-jar target has to depend on pigunit-jar to guarantee inclusion of 
> test classes
> 
>
> Key: PIG-2019
> URL: https://issues.apache.org/jira/browse/PIG-2019
> Project: Pig
>  Issue Type: Bug
>  Components: build
>Affects Versions: 0.9.0
>Reporter: Konstantin Boudnik
>Assignee: Konstantin Boudnik
> Fix For: 0.9.0
>
> Attachments: PIG-2019.patch
>
>
> pigsmoke artifact uses classes from pigunit for real cluster testing. 
> However, in the deployment phase the artifact comes out without the classes 
> added. It happens because of the execution order of targets during deployment.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2019) smoketest-jar target has to depend on pigunit-jar to guarantee inclusion of test classes

[Konstantin Boudnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027782#comment-13027782
 ] 

Konstantin Boudnik commented on PIG-2019:
-

It seems to be right unless there any other tests which could be run against a 
real cluster that I missing. I thought that PigUnit was a framework right for 
this purpose, wasn't it?

Please correct me if I am wrong and I'd fix the patch accordingly.

> smoketest-jar target has to depend on pigunit-jar to guarantee inclusion of 
> test classes
> 
>
> Key: PIG-2019
> URL: https://issues.apache.org/jira/browse/PIG-2019
> Project: Pig
>  Issue Type: Bug
>  Components: build
>Affects Versions: 0.9.0
>Reporter: Konstantin Boudnik
>Assignee: Konstantin Boudnik
> Fix For: 0.9.0
>
> Attachments: PIG-2019.patch
>
>
> pigsmoke artifact uses classes from pigunit for real cluster testing. 
> However, in the deployment phase the artifact comes out without the classes 
> added. It happens because of the execution order of targets during deployment.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2019) smoketest-jar target has to depend on pigunit-jar to guarantee inclusion of test classes

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027769#comment-13027769
 ] 

Alan Gates commented on PIG-2019:
-

When I run ant smoketests-jar now, the resulting smoketests.jar contains:

{code}
META-INF/
META-INF/MANIFEST.MF
org/
org/apache/
org/apache/pig/
org/apache/pig/test/
org/apache/pig/test/pigunit/
org/apache/pig/test/pigunit/pig/
org/apache/pig/test/pigunit/TestPigTest.class
org/apache/pig/test/pigunit/pig/TestGruntParser$1.class
org/apache/pig/test/pigunit/pig/TestGruntParser.class
test/
test/data/
test/data/pigunit/
test/data/pigunit/top_queries.pig
test/data/pigunit/top_queries_expected_top_3.txt
test/data/pigunit/top_queries_input_data.txt
test/data/pigunit/top_queries_params.txt
{code}

Does that look correct? That seems a little thin.  Smoketest just runs the 
PigUnit tests?

> smoketest-jar target has to depend on pigunit-jar to guarantee inclusion of 
> test classes
> 
>
> Key: PIG-2019
> URL: https://issues.apache.org/jira/browse/PIG-2019
> Project: Pig
>  Issue Type: Bug
>  Components: build
>Affects Versions: 0.9.0
>Reporter: Konstantin Boudnik
>Assignee: Konstantin Boudnik
> Fix For: 0.9.0
>
> Attachments: PIG-2019.patch
>
>
> pigsmoke artifact uses classes from pigunit for real cluster testing. 
> However, in the deployment phase the artifact comes out without the classes 
> added. It happens because of the execution order of targets during deployment.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1999) Macro alias masker should consider schema context

[Richard Ding (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027735#comment-13027735
 ] 

Richard Ding commented on PIG-1999:
---

test-patch result:

{code}
 [exec] +1 overall.  
 [exec] 
 [exec] +1 @author.  The patch does not contain any @author tags.
 [exec] 
 [exec] +1 tests included.  The patch appears to include 3 new or 
modified tests.
 [exec] 
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec] 
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec] 
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
warnings.
 [exec] 
 [exec] +1 release audit.  The applied patch does not increase the 
total number of release audit warnings.
{code}

> Macro alias masker should consider schema context 
> --
>
> Key: PIG-1999
> URL: https://issues.apache.org/jira/browse/PIG-1999
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Richard Ding
>Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1999_1.patch, PIG-1999_2.patch
>
>
> Macro alias masker doesn't consider the current schema context. This results 
> errors when deciding with alias to mask. Here is an example:
> {code}
> define toBytearray(in, intermediate) returns e { 
>a = load '$in' as (name:chararray, age:long, gpa: float);
>b = group a by  name;
>c = foreach b generate a, (1,2,3);
>store c into '$intermediate' using BinStorage();
>d = load '$intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, 
> t2:tuple(a,b,c));
>$e = foreach d generate COUNT(b), t2.a, t2.b, t2.c;
> };
>  
> f = toBytearray ('data', 'output1');
> {code} 
> Now the alias masker mistakes b in COUNT(b) as an alias instead of b in the 
> current schema.
> The workaround is to not use alias as as names in the schema definition. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1775) Removal of old logical plan

[Jeremy Hanna (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027716#comment-13027716
 ] 

Jeremy Hanna commented on PIG-1775:
---

Oh okay - I had misunderstood.  Thank you for the clarification.  I look 
forward to using 0.9 in our environment as it comes closer to being released.

> Removal of old logical plan
> ---
>
> Key: PIG-1775
> URL: https://issues.apache.org/jira/browse/PIG-1775
> Project: Pig
>  Issue Type: Improvement
>Affects Versions: 0.9.0
>Reporter: Yan Zhou
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-1775.patch
>
>
> The new logical plan will only be used and the old logical plan will be 
> removed once the new one is stable enough. It is scheduled for the 0.9 
> release.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1775) Removal of old logical plan

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027715#comment-13027715
 ] 

Xuefu Zhang commented on PIG-1775:
--

In 0.9.0, there is no way to disable new logical plan, with or without this 
ticket. If you have problem using 0.9, please log tickets accordingly.

> Removal of old logical plan
> ---
>
> Key: PIG-1775
> URL: https://issues.apache.org/jira/browse/PIG-1775
> Project: Pig
>  Issue Type: Improvement
>Affects Versions: 0.9.0
>Reporter: Yan Zhou
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-1775.patch
>
>
> The new logical plan will only be used and the old logical plan will be 
> removed once the new one is stable enough. It is scheduled for the 0.9 
> release.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1775) Removal of old logical plan

[Jeremy Hanna (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027712#comment-13027712
 ] 

Jeremy Hanna commented on PIG-1775:
---

Right - the ticket is about removing the ability to disable the new logical 
plan, though.  That's what had me a bit concerned.  I will do some testing with 
0.8.1 as soon as I am able and want to contribute in various ways, such as 
submitting bugs.  However, given the problems NLP has had in the 0.8.x line 
thus far, it just seemed like a quick removal.

> Removal of old logical plan
> ---
>
> Key: PIG-1775
> URL: https://issues.apache.org/jira/browse/PIG-1775
> Project: Pig
>  Issue Type: Improvement
>Affects Versions: 0.9.0
>Reporter: Yan Zhou
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-1775.patch
>
>
> The new logical plan will only be used and the old logical plan will be 
> removed once the new one is stable enough. It is scheduled for the 0.9 
> release.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1775) Removal of old logical plan

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027710#comment-13027710
 ] 

Xuefu Zhang commented on PIG-1775:
--

This ticket is not about logical plan replacement, which has already happened, 
but it's about migrating those test cases that are based on the old logical 
plan. Of course, dead code will be removed accordingly.

> Removal of old logical plan
> ---
>
> Key: PIG-1775
> URL: https://issues.apache.org/jira/browse/PIG-1775
> Project: Pig
>  Issue Type: Improvement
>Affects Versions: 0.9.0
>Reporter: Yan Zhou
>Assignee: Xuefu Zhang
> Fix For: 0.9.0
>
> Attachments: PIG-1775.patch
>
>
> The new logical plan will only be used and the old logical plan will be 
> removed once the new one is stable enough. It is scheduled for the 0.9 
> release.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-1821) UDFContext.getUDFProperties does not handle collisions in hashcode of udf classname (+ arg hashcodes)

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027676#comment-13027676
 ] 

Thejas M Nair commented on PIG-1821:


PIG-1821.1.patch - unit tests passed (except TestStoreInstances, which is 
failing in trunk). test-patch failed because of no new unit tests. There are no 
new unit tests because it is not easy to create a test case to produce the 
problem this could have caused.


> UDFContext.getUDFProperties does not handle collisions in hashcode of udf 
> classname (+ arg hashcodes)
> -
>
> Key: PIG-1821
> URL: https://issues.apache.org/jira/browse/PIG-1821
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.8.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-1821.1.patch
>
>
> In code below, if generateKey() returns same value for two udfs, the udfs 
> would end up sharing the properties object. 
> {code}
> private HashMap udfConfs = new HashMap Properties>();
> public Properties getUDFProperties(Class c) {
> Integer k = generateKey(c);
> Properties p = udfConfs.get(k);
> if (p == null) {
> p = new Properties();
> udfConfs.put(k, p);
> }
> return p;
> }
> private int generateKey(Class c) {
> return c.getName().hashCode();
> }
> public Properties getUDFProperties(Class c, String[] args) {
> Integer k = generateKey(c, args);
> Properties p = udfConfs.get(k);
> if (p == null) {
> p = new Properties();
> udfConfs.put(k, p);
> }
> return p;
> }
> private int generateKey(Class c, String[] args) {
> int hc = c.getName().hashCode();
> for (int i = 0; i < args.length; i++) {
> hc <<= 1;
> hc ^= args[i].hashCode();
> }
> return hc;
> }
> {code}
> To prevent this, a new class (say X) that can hold the classname and args 
> should be created, and instead of HashMap,  HashMap Properties> should be used. Then HahsMap will deal with the collisions. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-1821) UDFContext.getUDFProperties does not handle collisions in hashcode of udf classname (+ arg hashcodes)

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-1821:
---

Attachment: PIG-1821.1.patch

> UDFContext.getUDFProperties does not handle collisions in hashcode of udf 
> classname (+ arg hashcodes)
> -
>
> Key: PIG-1821
> URL: https://issues.apache.org/jira/browse/PIG-1821
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.8.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-1821.1.patch
>
>
> In code below, if generateKey() returns same value for two udfs, the udfs 
> would end up sharing the properties object. 
> {code}
> private HashMap udfConfs = new HashMap Properties>();
> public Properties getUDFProperties(Class c) {
> Integer k = generateKey(c);
> Properties p = udfConfs.get(k);
> if (p == null) {
> p = new Properties();
> udfConfs.put(k, p);
> }
> return p;
> }
> private int generateKey(Class c) {
> return c.getName().hashCode();
> }
> public Properties getUDFProperties(Class c, String[] args) {
> Integer k = generateKey(c, args);
> Properties p = udfConfs.get(k);
> if (p == null) {
> p = new Properties();
> udfConfs.put(k, p);
> }
> return p;
> }
> private int generateKey(Class c, String[] args) {
> int hc = c.getName().hashCode();
> for (int i = 0; i < args.length; i++) {
> hc <<= 1;
> hc ^= args[i].hashCode();
> }
> return hc;
> }
> {code}
> To prevent this, a new class (say X) that can hold the classname and args 
> should be created, and instead of HashMap,  HashMap Properties> should be used. Then HahsMap will deal with the collisions. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-1821) UDFContext.getUDFProperties does not handle collisions in hashcode of udf classname (+ arg hashcodes)

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-1821:
---

Status: Patch Available  (was: Open)

> UDFContext.getUDFProperties does not handle collisions in hashcode of udf 
> classname (+ arg hashcodes)
> -
>
> Key: PIG-1821
> URL: https://issues.apache.org/jira/browse/PIG-1821
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.8.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-1821.1.patch
>
>
> In code below, if generateKey() returns same value for two udfs, the udfs 
> would end up sharing the properties object. 
> {code}
> private HashMap udfConfs = new HashMap Properties>();
> public Properties getUDFProperties(Class c) {
> Integer k = generateKey(c);
> Properties p = udfConfs.get(k);
> if (p == null) {
> p = new Properties();
> udfConfs.put(k, p);
> }
> return p;
> }
> private int generateKey(Class c) {
> return c.getName().hashCode();
> }
> public Properties getUDFProperties(Class c, String[] args) {
> Integer k = generateKey(c, args);
> Properties p = udfConfs.get(k);
> if (p == null) {
> p = new Properties();
> udfConfs.put(k, p);
> }
> return p;
> }
> private int generateKey(Class c, String[] args) {
> int hc = c.getName().hashCode();
> for (int i = 0; i < args.length; i++) {
> hc <<= 1;
> hc ^= args[i].hashCode();
> }
> return hc;
> }
> {code}
> To prevent this, a new class (say X) that can hold the classname and args 
> should be created, and instead of HashMap,  HashMap Properties> should be used. Then HahsMap will deal with the collisions. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2026) e2e tests in eclipse classpath

[Gianmarco De Francisci Morales (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gianmarco De Francisci Morales updated PIG-2026:


Attachment: PIG-2026.patch

> e2e tests in eclipse classpath
> --
>
> Key: PIG-2026
> URL: https://issues.apache.org/jira/browse/PIG-2026
> Project: Pig
>  Issue Type: Bug
>Reporter: Gianmarco De Francisci Morales
>Priority: Trivial
> Attachments: PIG-2026.patch
>
>
> e2e tests under test/e2e/pig/udfs/java should have their own entry as a 
> source dir in eclipse .classpath

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2026) e2e tests in eclipse classpath

[Gianmarco De Francisci Morales (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gianmarco De Francisci Morales updated PIG-2026:


Assignee: Gianmarco De Francisci Morales
  Status: Patch Available  (was: Open)

> e2e tests in eclipse classpath
> --
>
> Key: PIG-2026
> URL: https://issues.apache.org/jira/browse/PIG-2026
> Project: Pig
>  Issue Type: Bug
>Reporter: Gianmarco De Francisci Morales
>Assignee: Gianmarco De Francisci Morales
>Priority: Trivial
> Attachments: PIG-2026.patch
>
>
> e2e tests under test/e2e/pig/udfs/java should have their own entry as a 
> source dir in eclipse .classpath

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2026) e2e tests in eclipse classpath

[Gianmarco De Francisci Morales (JIRA)]

e2e tests in eclipse classpath
--

 Key: PIG-2026
 URL: https://issues.apache.org/jira/browse/PIG-2026
 Project: Pig
  Issue Type: Bug
Reporter: Gianmarco De Francisci Morales
Priority: Trivial
 Attachments: PIG-2026.patch

e2e tests under test/e2e/pig/udfs/java should have their own entry as a source 
dir in eclipse .classpath

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2025) org.apache.pig.test.udf.evalfunc.TOMAP is missing package declaration

[Gianmarco De Francisci Morales (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gianmarco De Francisci Morales updated PIG-2025:


Attachment: PIG-2025.patch

> org.apache.pig.test.udf.evalfunc.TOMAP is missing package declaration
> -
>
> Key: PIG-2025
> URL: https://issues.apache.org/jira/browse/PIG-2025
> Project: Pig
>  Issue Type: Bug
>Reporter: Gianmarco De Francisci Morales
>Assignee: Gianmarco De Francisci Morales
>Priority: Trivial
> Attachments: PIG-2025.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2025) org.apache.pig.test.udf.evalfunc.TOMAP is missing package declaration

[Gianmarco De Francisci Morales (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gianmarco De Francisci Morales updated PIG-2025:


Status: Patch Available  (was: Open)

> org.apache.pig.test.udf.evalfunc.TOMAP is missing package declaration
> -
>
> Key: PIG-2025
> URL: https://issues.apache.org/jira/browse/PIG-2025
> Project: Pig
>  Issue Type: Bug
>Reporter: Gianmarco De Francisci Morales
>Assignee: Gianmarco De Francisci Morales
>Priority: Trivial
> Attachments: PIG-2025.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2025) org.apache.pig.test.udf.evalfunc.TOMAP is missing package declaration

[Gianmarco De Francisci Morales (JIRA)]

org.apache.pig.test.udf.evalfunc.TOMAP is missing package declaration
-

 Key: PIG-2025
 URL: https://issues.apache.org/jira/browse/PIG-2025
 Project: Pig
  Issue Type: Bug
Reporter: Gianmarco De Francisci Morales
Assignee: Gianmarco De Francisci Morales
Priority: Trivial




--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2024) Incorrect jar paths in .classpath template for eclipse

[Gianmarco De Francisci Morales (JIRA)]

Incorrect jar paths in .classpath template for eclipse
--

 Key: PIG-2024
 URL: https://issues.apache.org/jira/browse/PIG-2024
 Project: Pig
  Issue Type: Bug
Reporter: Gianmarco De Francisci Morales
Assignee: Gianmarco De Francisci Morales
Priority: Minor
 Attachments: PIG-2024.patch

The jars listed in .eclipse.templates/.classpath are outdated.
Importing the project in eclipse after using ant eclipse-files generates build 
path errors.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2024) Incorrect jar paths in .classpath template for eclipse

[Gianmarco De Francisci Morales (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gianmarco De Francisci Morales updated PIG-2024:


Status: Patch Available  (was: Open)

> Incorrect jar paths in .classpath template for eclipse
> --
>
> Key: PIG-2024
> URL: https://issues.apache.org/jira/browse/PIG-2024
> Project: Pig
>  Issue Type: Bug
>Reporter: Gianmarco De Francisci Morales
>Assignee: Gianmarco De Francisci Morales
>Priority: Minor
> Attachments: PIG-2024.patch
>
>
> The jars listed in .eclipse.templates/.classpath are outdated.
> Importing the project in eclipse after using ant eclipse-files generates 
> build path errors.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2024) Incorrect jar paths in .classpath template for eclipse

[Gianmarco De Francisci Morales (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gianmarco De Francisci Morales updated PIG-2024:


Attachment: PIG-2024.patch

> Incorrect jar paths in .classpath template for eclipse
> --
>
> Key: PIG-2024
> URL: https://issues.apache.org/jira/browse/PIG-2024
> Project: Pig
>  Issue Type: Bug
>Reporter: Gianmarco De Francisci Morales
>Assignee: Gianmarco De Francisci Morales
>Priority: Minor
> Attachments: PIG-2024.patch
>
>
> The jars listed in .eclipse.templates/.classpath are outdated.
> Importing the project in eclipse after using ant eclipse-files generates 
> build path errors.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-1883) Pig's progress estimation should account for parallel job executions

[Laukik Chitnis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laukik Chitnis updated PIG-1883:


Attachment: PIG-1883-2.patch

This patch adds methods to compute the critical path in an operator plan based 
on the number of nodes. It also uses the minimum progress along these many 
number of jobs to calculate the total progress. A new cmd line option is also 
added to enable this for progress reporting instead of the old estimation 
technique based purely on the total number of jobs.

> Pig's progress estimation should account for parallel job executions
> 
>
> Key: PIG-1883
> URL: https://issues.apache.org/jira/browse/PIG-1883
> Project: Pig
>  Issue Type: Improvement
>Reporter: Laukik Chitnis
>Assignee: Laukik Chitnis
> Attachments: PIG-1883-2.patch
>
>
> Currently, Pig's progress estimation is based on the percentage of jobs 
> completed out of the total number of MR jobs. However, since the MR operators 
> are arranged in a DAG (and hence more than 1 job might be submitted for 
> execution in parallel), the progress estimation can be improved by 
> considering the number of jobs in the critical path, instead of just the 
> total number of jobs.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira