johnyangk commented on a change in pull request #115: [NEMO-96] Modularize
DataSkewPolicy to use MetricVertex and BarrierVertex
URL: https://github.com/apache/incubator-nemo/pull/115#discussion_r214386521
##########
File path:
runtime/executor/src/main/java/edu/snu/nemo/runtime/executor/task/TaskExecutor.java
##########
@@ -231,27 +235,33 @@ public TaskExecutor(final Task task,
private void processElementRecursively(final VertexHarness vertexHarness,
final Object dataElement) {
final IRVertex irVertex = vertexHarness.getIRVertex();
final OutputCollectorImpl outputCollector =
vertexHarness.getOutputCollector();
+
if (irVertex instanceof SourceVertex) {
outputCollector.emit(dataElement);
} else if (irVertex instanceof OperatorVertex) {
final Transform transform = ((OperatorVertex) irVertex).getTransform();
transform.onData(dataElement);
- } else if (irVertex instanceof MetricCollectionBarrierVertex) {
- outputCollector.emit(dataElement);
- setIRVertexPutOnHold((MetricCollectionBarrierVertex) irVertex);
+ } else if (irVertex instanceof MetricCollectionVertex) {
+ final Transform transform = ((MetricCollectionVertex)
irVertex).getTransform();
+ transform.onData(dataElement);
+ } else if (irVertex instanceof AggregationBarrierVertex) {
+ final Transform transform = ((AggregationBarrierVertex)
irVertex).getTransform();
+ transform.onData(dataElement);
Review comment:
I think this shows that AggregationBarrierVertex may also become another
OperatorVertex, instead of its own IRVertex subclass.
The `setIRVertexPutOnHold` below can be performed by checking an execution
property of the vertex.
I'd like to have as few IRVertex subclasses as possible. (e.g., just
SourceVertex and OperatorVertex)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
