[jira] [Commented] (GIRAPH-1146) Keep track of number of supersteps when possible
[
https://issues.apache.org/jira/browse/GIRAPH-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998634#comment-15998634
]
ASF GitHub Bot commented on GIRAPH-1146:
Github user ikabiljo commented on a diff in the pull request:
https://github.com/apache/giraph/pull/36#discussion_r115049370
--- Diff:
giraph-block-app/src/main/java/org/apache/giraph/block_app/framework/block/PieceCount.java
---
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.giraph.block_app.framework.block;
+
+import com.google.common.base.Objects;
+
+/**
+ * Number of pieces
+ */
+public class PieceCount {
+ private boolean known;
+ private int count;
+
+ public PieceCount(int count) {
+known = true;
+this.count = count;
+ }
+
+ private PieceCount() {
+known = false;
+ }
+
+ public static PieceCount createUnknownCount() {
+return new PieceCount();
+ }
+
+
+ public PieceCount add(PieceCount other) {
+if (!this.known || !other.known) {
+ known = false;
+} else {
+ count += other.count;
+}
+return this;
+ }
+
+ public PieceCount multiply(int value) {
+count *= value;
+return this;
+ }
+
+ public int getCount() {
+return known ? count : Integer.MAX_VALUE;
--- End diff --
this might easily lead to overflow if anything is done with this number.
You should either fatal (better), or return 1M or something here.
> Keep track of number of supersteps when possible
>
>
> Key: GIRAPH-1146
> URL: https://issues.apache.org/jira/browse/GIRAPH-1146
> Project: Giraph
> Issue Type: New Feature
>Reporter: Maja Kabiljo
>Assignee: Maja Kabiljo
>Priority: Minor
>
> In many cases we know how many supersteps are there going to be. We can keep
> track of it and log it with progress.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (GIRAPH-1146) Keep track of number of supersteps when possible
[
https://issues.apache.org/jira/browse/GIRAPH-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998637#comment-15998637
]
ASF GitHub Bot commented on GIRAPH-1146:
Github user ikabiljo commented on a diff in the pull request:
https://github.com/apache/giraph/pull/36#discussion_r115049666
--- Diff:
giraph-block-app/src/main/java/org/apache/giraph/block_app/framework/piece/delegate/DelegatePiece.java
---
@@ -249,6 +250,15 @@ public void
forAllPossiblePieces(Consumer consumer) {
}
}
+ @Override
+ public PieceCount getPieceCount() {
+PieceCount ret = new PieceCount(0);
--- End diff --
this should be 1, this executes all pieces simultaneously.
> Keep track of number of supersteps when possible
>
>
> Key: GIRAPH-1146
> URL: https://issues.apache.org/jira/browse/GIRAPH-1146
> Project: Giraph
> Issue Type: New Feature
>Reporter: Maja Kabiljo
>Assignee: Maja Kabiljo
>Priority: Minor
>
> In many cases we know how many supersteps are there going to be. We can keep
> track of it and log it with progress.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (GIRAPH-1146) Keep track of number of supersteps when possible
[
https://issues.apache.org/jira/browse/GIRAPH-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998635#comment-15998635
]
ASF GitHub Bot commented on GIRAPH-1146:
Github user ikabiljo commented on a diff in the pull request:
https://github.com/apache/giraph/pull/36#discussion_r115049981
--- Diff:
giraph-block-app/src/main/java/org/apache/giraph/block_app/framework/BlockUtils.java
---
@@ -147,6 +148,11 @@ public static void
initAndCheckConfig(GiraphConfiguration conf) {
checkBlockTypes(
executionBlock, blockFactory.createExecutionStage(immConf),
immConf);
+PieceCount pieceCount = executionBlock.getPieceCount();
+if (pieceCount.isKnown()) {
+ GiraphConstants.SUPERSTEP_COUNT.set(conf, pieceCount.getCount());
--- End diff --
shouldn't it be pieceCount.getCount() + 1 ?
> Keep track of number of supersteps when possible
>
>
> Key: GIRAPH-1146
> URL: https://issues.apache.org/jira/browse/GIRAPH-1146
> Project: Giraph
> Issue Type: New Feature
>Reporter: Maja Kabiljo
>Assignee: Maja Kabiljo
>Priority: Minor
>
> In many cases we know how many supersteps are there going to be. We can keep
> track of it and log it with progress.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (GIRAPH-1146) Keep track of number of supersteps when possible
[
https://issues.apache.org/jira/browse/GIRAPH-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998636#comment-15998636
]
ASF GitHub Bot commented on GIRAPH-1146:
Github user ikabiljo commented on a diff in the pull request:
https://github.com/apache/giraph/pull/36#discussion_r115049146
--- Diff:
giraph-block-app/src/main/java/org/apache/giraph/block_app/framework/block/EmptyBlock.java
---
@@ -36,4 +36,9 @@
@Override
public void forAllPossiblePieces(Consumer consumer) {
}
+
+ @Override
+ public PieceCount getPieceCount() {
+return new PieceCount(1);
--- End diff --
should be 0
> Keep track of number of supersteps when possible
>
>
> Key: GIRAPH-1146
> URL: https://issues.apache.org/jira/browse/GIRAPH-1146
> Project: Giraph
> Issue Type: New Feature
>Reporter: Maja Kabiljo
>Assignee: Maja Kabiljo
>Priority: Minor
>
> In many cases we know how many supersteps are there going to be. We can keep
> track of it and log it with progress.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (GIRAPH-1146) Keep track of number of supersteps when possible
[
https://issues.apache.org/jira/browse/GIRAPH-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998670#comment-15998670
]
ASF GitHub Bot commented on GIRAPH-1146:
Github user majakabiljo commented on a diff in the pull request:
https://github.com/apache/giraph/pull/36#discussion_r115053845
--- Diff:
giraph-block-app/src/main/java/org/apache/giraph/block_app/framework/block/PieceCount.java
---
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.giraph.block_app.framework.block;
+
+import com.google.common.base.Objects;
+
+/**
+ * Number of pieces
+ */
+public class PieceCount {
+ private boolean known;
+ private int count;
+
+ public PieceCount(int count) {
+known = true;
+this.count = count;
+ }
+
+ private PieceCount() {
+known = false;
+ }
+
+ public static PieceCount createUnknownCount() {
+return new PieceCount();
+ }
+
+
+ public PieceCount add(PieceCount other) {
+if (!this.known || !other.known) {
+ known = false;
+} else {
+ count += other.count;
+}
+return this;
+ }
+
+ public PieceCount multiply(int value) {
+count *= value;
+return this;
+ }
+
+ public int getCount() {
+return known ? count : Integer.MAX_VALUE;
--- End diff --
Good point, I'll throw instead
> Keep track of number of supersteps when possible
>
>
> Key: GIRAPH-1146
> URL: https://issues.apache.org/jira/browse/GIRAPH-1146
> Project: Giraph
> Issue Type: New Feature
>Reporter: Maja Kabiljo
>Assignee: Maja Kabiljo
>Priority: Minor
>
> In many cases we know how many supersteps are there going to be. We can keep
> track of it and log it with progress.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (GIRAPH-1146) Keep track of number of supersteps when possible
[
https://issues.apache.org/jira/browse/GIRAPH-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998668#comment-15998668
]
ASF GitHub Bot commented on GIRAPH-1146:
Github user majakabiljo commented on a diff in the pull request:
https://github.com/apache/giraph/pull/36#discussion_r115053732
--- Diff:
giraph-block-app/src/main/java/org/apache/giraph/block_app/framework/BlockUtils.java
---
@@ -147,6 +148,11 @@ public static void
initAndCheckConfig(GiraphConfiguration conf) {
checkBlockTypes(
executionBlock, blockFactory.createExecutionStage(immConf),
immConf);
+PieceCount pieceCount = executionBlock.getPieceCount();
+if (pieceCount.isKnown()) {
+ GiraphConstants.SUPERSTEP_COUNT.set(conf, pieceCount.getCount());
--- End diff --
There will be X+1 supersteps, but they are going to be supersteps 0..X.
Actually I can make +1 here and then -1 in the logging part to keep it clear.
> Keep track of number of supersteps when possible
>
>
> Key: GIRAPH-1146
> URL: https://issues.apache.org/jira/browse/GIRAPH-1146
> Project: Giraph
> Issue Type: New Feature
>Reporter: Maja Kabiljo
>Assignee: Maja Kabiljo
>Priority: Minor
>
> In many cases we know how many supersteps are there going to be. We can keep
> track of it and log it with progress.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
