explain does not show the inner plans of the MapReduce plan
-----------------------------------------------------------

                 Key: PIG-1416
                 URL: https://issues.apache.org/jira/browse/PIG-1416
             Project: Pig
          Issue Type: Bug
          Components: impl
            Reporter: Thejas M Nair
             Fix For: 0.8.0


The  inner plan in MR plan is very useful in understanding query plans where 
the inner plans are not present in the Physical or Logical plans. For example, 
in case of "order by", the sampling MR job is not part of the physical plan, 
and the reduce has a POSort which is not shown in explain . 

In following example, notice that POSort is not shown the MR plan .

{code}

grunt> l = load 'file.txt' as (a, b, c);
grunt> g = group l by a;
grunt> f = foreach g { s = order l by $0; generate s; }
grunt> explain f;

grunt> explain f;
#-----------------------------------------------
# Logical Plan:
#-----------------------------------------------
Store 1-137 Schema: {s: {a: bytearray,b: bytearray,c: bytearray}} Type: Unknown
|
|---ForEach 1-136 Schema: {s: {a: bytearray,b: bytearray,c: bytearray}} Type: 
bag
    |   |
    |   Project 1-132 Projections:  [*]  Overloaded: false FieldSchema: s: 
bag({a: bytearray,b: bytearray,c: bytearray}) Type: bag
    |   Input: SORT 1-133|
    |   |---SORT 1-133 Schema: {a: bytearray,b: bytearray,c: bytearray} Type: 
bag
    |       |   |
    |       |   Project 1-134 Projections: [0] Overloaded: false FieldSchema: 
a: bytearray Type: bytearray
    |       |   Input: Project 1-135 Projections: [1] Overloaded: true
    |       |
    |       |---Project 1-135 Projections: [1] Overloaded: true FieldSchema: l: 
tuple({a: bytearray,b: bytearray,c: bytearray}) Type: tuple
    |           Input: CoGroup 1-126
    |
    |---CoGroup 1-126 Schema: {group: bytearray,l: {a: bytearray,b: 
bytearray,c: bytearray}} Type: bag
        |   |
        |   Project 1-125 Projections: [0] Overloaded: false FieldSchema: a: 
bytearray Type: bytearray
        |   Input: Load 1-124
        |
        |---Load 1-124 Schema: {a: bytearray,b: bytearray,c: bytearray} Type: 
bag

#-----------------------------------------------
# Physical Plan:
#-----------------------------------------------
Store(fakefile:org.apache.pig.builtin.PigStorage) - 1-148
|
|---New For Each(false)[bag] - 1-147
    |   |
    |   RelationToExpressionProject[bag][*] - 1-146
    |   |
    |   |---POSort[bag]() - 1-145
    |       |   |
    |       |   Project[bytearray][0] - 1-144
    |       |
    |       |---Project[tuple][1] - 1-143
    |
    |---Package[tuple]{bytearray} - 1-140
        |
        |---Global Rearrange[tuple] - 1-139
            |
            |---Local Rearrange[tuple]{bytearray}(false) - 1-141
                |   |
                |   Project[bytearray][0] - 1-142
                |
                
|---Load(file:///Users/tejas/trunk_oby/file.txt:org.apache.pig.builtin.PigStorage)
 - 1-138

2010-05-13 15:47:32,102 [main] INFO  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
 - MR plan size before optimization: 1
2010-05-13 15:47:32,102 [main] INFO  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
 - MR plan size after optimization: 1
#--------------------------------------------------
# Map Reduce Plan
#--------------------------------------------------
MapReduce node 1-149
Map Plan
Local Rearrange[tuple]{bytearray}(false) - 1-141
|   |
|   Project[bytearray][0] - 1-142
|
|---Load(file:///Users/tejas/trunk_oby/file.txt:org.apache.pig.builtin.PigStorage)
 - 1-138--------
Reduce Plan
Store(fakefile:org.apache.pig.builtin.PigStorage) - 1-148
|
|---New For Each(false)[bag] - 1-147
    |   |
    |   RelationToExpressionProject[bag][*] - 1-146
    |   |
    |   |---Project[tuple][1] - 1-143
    |
    |---Package[tuple]{bytearray} - 1-140--------
Global sort: false
----------------

{code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to