[
https://issues.apache.org/jira/browse/PIG-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair resolved PIG-1416.
--------------------------------
Resolution: Invalid
The secondary key optimizer is removing the POSort, that is why it does not
appear in MR plan. Closing as invalid.
> explain does not show the inner plans of the MapReduce plan
> -----------------------------------------------------------
>
> Key: PIG-1416
> URL: https://issues.apache.org/jira/browse/PIG-1416
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Thejas M Nair
> Fix For: 0.8.0
>
>
> The inner plan in MR plan is very useful in understanding query plans where
> the inner plans are not present in the Physical or Logical plans. For
> example, in case of "order by", the sampling MR job is not part of the
> physical plan, and the reduce has a POSort which is not shown in explain .
> In following example, notice that POSort is not shown the MR plan .
> {code}
> grunt> l = load 'file.txt' as (a, b, c);
> grunt> g = group l by a;
> grunt> f = foreach g { s = order l by $0; generate s; }
> grunt> explain f;
> grunt> explain f;
> #-----------------------------------------------
> # Logical Plan:
> #-----------------------------------------------
> Store 1-137 Schema: {s: {a: bytearray,b: bytearray,c: bytearray}} Type:
> Unknown
> |
> |---ForEach 1-136 Schema: {s: {a: bytearray,b: bytearray,c: bytearray}} Type:
> bag
> | |
> | Project 1-132 Projections: [*] Overloaded: false FieldSchema: s:
> bag({a: bytearray,b: bytearray,c: bytearray}) Type: bag
> | Input: SORT 1-133|
> | |---SORT 1-133 Schema: {a: bytearray,b: bytearray,c: bytearray} Type:
> bag
> | | |
> | | Project 1-134 Projections: [0] Overloaded: false FieldSchema:
> a: bytearray Type: bytearray
> | | Input: Project 1-135 Projections: [1] Overloaded: true
> | |
> | |---Project 1-135 Projections: [1] Overloaded: true FieldSchema:
> l: tuple({a: bytearray,b: bytearray,c: bytearray}) Type: tuple
> | Input: CoGroup 1-126
> |
> |---CoGroup 1-126 Schema: {group: bytearray,l: {a: bytearray,b:
> bytearray,c: bytearray}} Type: bag
> | |
> | Project 1-125 Projections: [0] Overloaded: false FieldSchema: a:
> bytearray Type: bytearray
> | Input: Load 1-124
> |
> |---Load 1-124 Schema: {a: bytearray,b: bytearray,c: bytearray} Type:
> bag
> #-----------------------------------------------
> # Physical Plan:
> #-----------------------------------------------
> Store(fakefile:org.apache.pig.builtin.PigStorage) - 1-148
> |
> |---New For Each(false)[bag] - 1-147
> | |
> | RelationToExpressionProject[bag][*] - 1-146
> | |
> | |---POSort[bag]() - 1-145
> | | |
> | | Project[bytearray][0] - 1-144
> | |
> | |---Project[tuple][1] - 1-143
> |
> |---Package[tuple]{bytearray} - 1-140
> |
> |---Global Rearrange[tuple] - 1-139
> |
> |---Local Rearrange[tuple]{bytearray}(false) - 1-141
> | |
> | Project[bytearray][0] - 1-142
> |
>
> |---Load(file:///Users/tejas/trunk_oby/file.txt:org.apache.pig.builtin.PigStorage)
> - 1-138
> 2010-05-13 15:47:32,102 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size before optimization: 1
> 2010-05-13 15:47:32,102 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size after optimization: 1
> #--------------------------------------------------
> # Map Reduce Plan
> #--------------------------------------------------
> MapReduce node 1-149
> Map Plan
> Local Rearrange[tuple]{bytearray}(false) - 1-141
> | |
> | Project[bytearray][0] - 1-142
> |
> |---Load(file:///Users/tejas/trunk_oby/file.txt:org.apache.pig.builtin.PigStorage)
> - 1-138--------
> Reduce Plan
> Store(fakefile:org.apache.pig.builtin.PigStorage) - 1-148
> |
> |---New For Each(false)[bag] - 1-147
> | |
> | RelationToExpressionProject[bag][*] - 1-146
> | |
> | |---Project[tuple][1] - 1-143
> |
> |---Package[tuple]{bytearray} - 1-140--------
> Global sort: false
> ----------------
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.