[jira] Commented: (PIG-1644) New logical plan: Plan.connect with position is misused in some places
[
https://issues.apache.org/jira/browse/PIG-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12914637#action_12914637
]
Thejas M Nair commented on PIG-1644:
Looks good. +1
Please commit after test-patch and unit tests pass.
New logical plan: Plan.connect with position is misused in some places
--
Key: PIG-1644
URL: https://issues.apache.org/jira/browse/PIG-1644
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.8.0
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.8.0
Attachments: PIG-1644-1.patch, PIG-1644-2.patch
When we replace/remove/insert a node, we will use disconnect/connect methods
of OperatorPlan. When we disconnect an edge, we shall save the position of
the edge in origination and destination, and use this position when connect
to the new predecessor/successor. Some of the pattens are:
Insert a new node:
{code}
PairInteger, Integer pos = plan.disconnect(pred, succ);
plan.connect(pred, pos.first, newnode, 0);
plan.connect(newnode, 0, succ, pos.second);
{code}
Remove a node:
{code}
PairInteger, Integer pos1 = plan.disconnect(pred, nodeToRemove);
PairInteger, Integer pos2 = plan.disconnect(nodeToRemove, succ);
plan.connect(pred, pos1.first, succ, pos2.second);
{code}
Replace a node:
{code}
PairInteger, Integer pos1 = plan.disconnect(pred, nodeToReplace);
PairInteger, Integer pos2 = plan.disconnect(nodeToReplace, succ);
plan.connect(pred, pos1.first, newNode, pos1.second);
plan.connect(newNode, pos2.first, succ, pos2.second);
{code}
There are couple of places of we does not follow this pattern, that results
some error. For example, the following script fail:
{code}
a = load '1.txt' as (a0, a1, a2, a3);
b = foreach a generate a0, a1, a2;
store b into 'aaa';
c = order b by a2;
d = foreach c generate a2;
store d into 'bbb';
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1644) New logical plan: Plan.connect with position is misused in some places
[
https://issues.apache.org/jira/browse/PIG-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12914130#action_12914130
]
Thejas M Nair commented on PIG-1644:
These operations will be fairly common in the optimizer. I think it would be
good to have functions in the OperatorPlan that support these operations, that
will reduce the chances of bugs and also make the code more readable.
New logical plan: Plan.connect with position is misused in some places
--
Key: PIG-1644
URL: https://issues.apache.org/jira/browse/PIG-1644
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.8.0
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.8.0
Attachments: PIG-1644-1.patch
When we replace/remove/insert a node, we will use disconnect/connect methods
of OperatorPlan. When we disconnect an edge, we shall save the position of
the edge in origination and destination, and use this position when connect
to the new predecessor/successor. Some of the pattens are:
Insert a new node:
{code}
PairInteger, Integer pos = plan.disconnect(pred, succ);
plan.connect(pred, pos.first, newnode, 0);
plan.connect(newnode, 0, succ, pos.second);
{code}
Remove a node:
{code}
PairInteger, Integer pos1 = plan.disconnect(pred, nodeToRemove);
PairInteger, Integer pos2 = plan.disconnect(nodeToRemove, succ);
plan.connect(pred, pos1.first, succ, pos2.second);
{code}
Replace a node:
{code}
PairInteger, Integer pos1 = plan.disconnect(pred, nodeToReplace);
PairInteger, Integer pos2 = plan.disconnect(nodeToReplace, succ);
plan.connect(pred, pos1.first, newNode, pos1.second);
plan.connect(newNode, pos2.first, succ, pos2.second);
{code}
There are couple of places of we does not follow this pattern, that results
some error. For example, the following script fail:
{code}
a = load '1.txt' as (a0, a1, a2, a3);
b = foreach a generate a0, a1, a2;
store b into 'aaa';
c = order b by a2;
d = foreach c generate a2;
store d into 'bbb';
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1644) New logical plan: Plan.connect with position is misused in some places
[
https://issues.apache.org/jira/browse/PIG-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12914147#action_12914147
]
Daniel Dai commented on PIG-1644:
-
Yes, I think we can do replace/remove/insert. They should be simple and clear
enough to use. Here is the new methods adding to OperatorPlan:
{code}
replace(Operator oldOperator, Operator newOperator)
remove(Operator operatorToRemove) // Connect all its successors to
predecessor/connect all it's predecessors to successor
insertBefore(Operator operatorToInsert, Operator pos) // Insert
operatorToInsert before pos, connect all pos's predecessors to operatorToInsert
insertAfter(Operator operatorToInsert, Operator pos) // Insert operatorToInsert
after pos, connect operatorToInsert to all pos's successor
{code}
How does it sounds?
New logical plan: Plan.connect with position is misused in some places
--
Key: PIG-1644
URL: https://issues.apache.org/jira/browse/PIG-1644
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.8.0
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.8.0
Attachments: PIG-1644-1.patch
When we replace/remove/insert a node, we will use disconnect/connect methods
of OperatorPlan. When we disconnect an edge, we shall save the position of
the edge in origination and destination, and use this position when connect
to the new predecessor/successor. Some of the pattens are:
Insert a new node:
{code}
PairInteger, Integer pos = plan.disconnect(pred, succ);
plan.connect(pred, pos.first, newnode, 0);
plan.connect(newnode, 0, succ, pos.second);
{code}
Remove a node:
{code}
PairInteger, Integer pos1 = plan.disconnect(pred, nodeToRemove);
PairInteger, Integer pos2 = plan.disconnect(nodeToRemove, succ);
plan.connect(pred, pos1.first, succ, pos2.second);
{code}
Replace a node:
{code}
PairInteger, Integer pos1 = plan.disconnect(pred, nodeToReplace);
PairInteger, Integer pos2 = plan.disconnect(nodeToReplace, succ);
plan.connect(pred, pos1.first, newNode, pos1.second);
plan.connect(newNode, pos2.first, succ, pos2.second);
{code}
There are couple of places of we does not follow this pattern, that results
some error. For example, the following script fail:
{code}
a = load '1.txt' as (a0, a1, a2, a3);
b = foreach a generate a0, a1, a2;
store b into 'aaa';
c = order b by a2;
d = foreach c generate a2;
store d into 'bbb';
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1644) New logical plan: Plan.connect with position is misused in some places
[
https://issues.apache.org/jira/browse/PIG-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12914167#action_12914167
]
Thejas M Nair commented on PIG-1644:
I think insertAsPredecessor and insertAsSuccessor (instead of insertBefore and
insertAfter) will convey the idea of what it does a little better.
New logical plan: Plan.connect with position is misused in some places
--
Key: PIG-1644
URL: https://issues.apache.org/jira/browse/PIG-1644
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.8.0
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.8.0
Attachments: PIG-1644-1.patch
When we replace/remove/insert a node, we will use disconnect/connect methods
of OperatorPlan. When we disconnect an edge, we shall save the position of
the edge in origination and destination, and use this position when connect
to the new predecessor/successor. Some of the pattens are:
Insert a new node:
{code}
PairInteger, Integer pos = plan.disconnect(pred, succ);
plan.connect(pred, pos.first, newnode, 0);
plan.connect(newnode, 0, succ, pos.second);
{code}
Remove a node:
{code}
PairInteger, Integer pos1 = plan.disconnect(pred, nodeToRemove);
PairInteger, Integer pos2 = plan.disconnect(nodeToRemove, succ);
plan.connect(pred, pos1.first, succ, pos2.second);
{code}
Replace a node:
{code}
PairInteger, Integer pos1 = plan.disconnect(pred, nodeToReplace);
PairInteger, Integer pos2 = plan.disconnect(nodeToReplace, succ);
plan.connect(pred, pos1.first, newNode, pos1.second);
plan.connect(newNode, pos2.first, succ, pos2.second);
{code}
There are couple of places of we does not follow this pattern, that results
some error. For example, the following script fail:
{code}
a = load '1.txt' as (a0, a1, a2, a3);
b = foreach a generate a0, a1, a2;
store b into 'aaa';
c = order b by a2;
d = foreach c generate a2;
store d into 'bbb';
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1644) New logical plan: Plan.connect with position is misused in some places
[
https://issues.apache.org/jira/browse/PIG-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12914317#action_12914317
]
Daniel Dai commented on PIG-1644:
-
After looking into the existing code, seems insertBetween is a more useful
method. So I want to drop insertBefore/insertAfter, and add insertBetween
{code}
insertBetween(Operator pred, Operator operatorToInsert, Operator succ)
{code}
New logical plan: Plan.connect with position is misused in some places
--
Key: PIG-1644
URL: https://issues.apache.org/jira/browse/PIG-1644
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.8.0
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.8.0
Attachments: PIG-1644-1.patch
When we replace/remove/insert a node, we will use disconnect/connect methods
of OperatorPlan. When we disconnect an edge, we shall save the position of
the edge in origination and destination, and use this position when connect
to the new predecessor/successor. Some of the pattens are:
Insert a new node:
{code}
PairInteger, Integer pos = plan.disconnect(pred, succ);
plan.connect(pred, pos.first, newnode, 0);
plan.connect(newnode, 0, succ, pos.second);
{code}
Remove a node:
{code}
PairInteger, Integer pos1 = plan.disconnect(pred, nodeToRemove);
PairInteger, Integer pos2 = plan.disconnect(nodeToRemove, succ);
plan.connect(pred, pos1.first, succ, pos2.second);
{code}
Replace a node:
{code}
PairInteger, Integer pos1 = plan.disconnect(pred, nodeToReplace);
PairInteger, Integer pos2 = plan.disconnect(nodeToReplace, succ);
plan.connect(pred, pos1.first, newNode, pos1.second);
plan.connect(newNode, pos2.first, succ, pos2.second);
{code}
There are couple of places of we does not follow this pattern, that results
some error. For example, the following script fail:
{code}
a = load '1.txt' as (a0, a1, a2, a3);
b = foreach a generate a0, a1, a2;
store b into 'aaa';
c = order b by a2;
d = foreach c generate a2;
store d into 'bbb';
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
