[jira] [Commented] (GIRAPH-1146) Keep track of number of supersteps when possible
[ https://issues.apache.org/jira/browse/GIRAPH-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998634#comment-15998634 ] ASF GitHub Bot commented on GIRAPH-1146: Github user ikabiljo commented on a diff in the pull request: https://github.com/apache/giraph/pull/36#discussion_r115049370 --- Diff: giraph-block-app/src/main/java/org/apache/giraph/block_app/framework/block/PieceCount.java --- @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.giraph.block_app.framework.block; + +import com.google.common.base.Objects; + +/** + * Number of pieces + */ +public class PieceCount { + private boolean known; + private int count; + + public PieceCount(int count) { +known = true; +this.count = count; + } + + private PieceCount() { +known = false; + } + + public static PieceCount createUnknownCount() { +return new PieceCount(); + } + + + public PieceCount add(PieceCount other) { +if (!this.known || !other.known) { + known = false; +} else { + count += other.count; +} +return this; + } + + public PieceCount multiply(int value) { +count *= value; +return this; + } + + public int getCount() { +return known ? count : Integer.MAX_VALUE; --- End diff -- this might easily lead to overflow if anything is done with this number. You should either fatal (better), or return 1M or something here. > Keep track of number of supersteps when possible > > > Key: GIRAPH-1146 > URL: https://issues.apache.org/jira/browse/GIRAPH-1146 > Project: Giraph > Issue Type: New Feature >Reporter: Maja Kabiljo >Assignee: Maja Kabiljo >Priority: Minor > > In many cases we know how many supersteps are there going to be. We can keep > track of it and log it with progress. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (GIRAPH-1146) Keep track of number of supersteps when possible
[ https://issues.apache.org/jira/browse/GIRAPH-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998637#comment-15998637 ] ASF GitHub Bot commented on GIRAPH-1146: Github user ikabiljo commented on a diff in the pull request: https://github.com/apache/giraph/pull/36#discussion_r115049666 --- Diff: giraph-block-app/src/main/java/org/apache/giraph/block_app/framework/piece/delegate/DelegatePiece.java --- @@ -249,6 +250,15 @@ public void forAllPossiblePieces(Consumer consumer) { } } + @Override + public PieceCount getPieceCount() { +PieceCount ret = new PieceCount(0); --- End diff -- this should be 1, this executes all pieces simultaneously. > Keep track of number of supersteps when possible > > > Key: GIRAPH-1146 > URL: https://issues.apache.org/jira/browse/GIRAPH-1146 > Project: Giraph > Issue Type: New Feature >Reporter: Maja Kabiljo >Assignee: Maja Kabiljo >Priority: Minor > > In many cases we know how many supersteps are there going to be. We can keep > track of it and log it with progress. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (GIRAPH-1146) Keep track of number of supersteps when possible
[ https://issues.apache.org/jira/browse/GIRAPH-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998635#comment-15998635 ] ASF GitHub Bot commented on GIRAPH-1146: Github user ikabiljo commented on a diff in the pull request: https://github.com/apache/giraph/pull/36#discussion_r115049981 --- Diff: giraph-block-app/src/main/java/org/apache/giraph/block_app/framework/BlockUtils.java --- @@ -147,6 +148,11 @@ public static void initAndCheckConfig(GiraphConfiguration conf) { checkBlockTypes( executionBlock, blockFactory.createExecutionStage(immConf), immConf); +PieceCount pieceCount = executionBlock.getPieceCount(); +if (pieceCount.isKnown()) { + GiraphConstants.SUPERSTEP_COUNT.set(conf, pieceCount.getCount()); --- End diff -- shouldn't it be pieceCount.getCount() + 1 ? > Keep track of number of supersteps when possible > > > Key: GIRAPH-1146 > URL: https://issues.apache.org/jira/browse/GIRAPH-1146 > Project: Giraph > Issue Type: New Feature >Reporter: Maja Kabiljo >Assignee: Maja Kabiljo >Priority: Minor > > In many cases we know how many supersteps are there going to be. We can keep > track of it and log it with progress. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (GIRAPH-1146) Keep track of number of supersteps when possible
[ https://issues.apache.org/jira/browse/GIRAPH-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998636#comment-15998636 ] ASF GitHub Bot commented on GIRAPH-1146: Github user ikabiljo commented on a diff in the pull request: https://github.com/apache/giraph/pull/36#discussion_r115049146 --- Diff: giraph-block-app/src/main/java/org/apache/giraph/block_app/framework/block/EmptyBlock.java --- @@ -36,4 +36,9 @@ @Override public void forAllPossiblePieces(Consumer consumer) { } + + @Override + public PieceCount getPieceCount() { +return new PieceCount(1); --- End diff -- should be 0 > Keep track of number of supersteps when possible > > > Key: GIRAPH-1146 > URL: https://issues.apache.org/jira/browse/GIRAPH-1146 > Project: Giraph > Issue Type: New Feature >Reporter: Maja Kabiljo >Assignee: Maja Kabiljo >Priority: Minor > > In many cases we know how many supersteps are there going to be. We can keep > track of it and log it with progress. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (GIRAPH-1146) Keep track of number of supersteps when possible
[ https://issues.apache.org/jira/browse/GIRAPH-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998670#comment-15998670 ] ASF GitHub Bot commented on GIRAPH-1146: Github user majakabiljo commented on a diff in the pull request: https://github.com/apache/giraph/pull/36#discussion_r115053845 --- Diff: giraph-block-app/src/main/java/org/apache/giraph/block_app/framework/block/PieceCount.java --- @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.giraph.block_app.framework.block; + +import com.google.common.base.Objects; + +/** + * Number of pieces + */ +public class PieceCount { + private boolean known; + private int count; + + public PieceCount(int count) { +known = true; +this.count = count; + } + + private PieceCount() { +known = false; + } + + public static PieceCount createUnknownCount() { +return new PieceCount(); + } + + + public PieceCount add(PieceCount other) { +if (!this.known || !other.known) { + known = false; +} else { + count += other.count; +} +return this; + } + + public PieceCount multiply(int value) { +count *= value; +return this; + } + + public int getCount() { +return known ? count : Integer.MAX_VALUE; --- End diff -- Good point, I'll throw instead > Keep track of number of supersteps when possible > > > Key: GIRAPH-1146 > URL: https://issues.apache.org/jira/browse/GIRAPH-1146 > Project: Giraph > Issue Type: New Feature >Reporter: Maja Kabiljo >Assignee: Maja Kabiljo >Priority: Minor > > In many cases we know how many supersteps are there going to be. We can keep > track of it and log it with progress. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (GIRAPH-1146) Keep track of number of supersteps when possible
[ https://issues.apache.org/jira/browse/GIRAPH-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998668#comment-15998668 ] ASF GitHub Bot commented on GIRAPH-1146: Github user majakabiljo commented on a diff in the pull request: https://github.com/apache/giraph/pull/36#discussion_r115053732 --- Diff: giraph-block-app/src/main/java/org/apache/giraph/block_app/framework/BlockUtils.java --- @@ -147,6 +148,11 @@ public static void initAndCheckConfig(GiraphConfiguration conf) { checkBlockTypes( executionBlock, blockFactory.createExecutionStage(immConf), immConf); +PieceCount pieceCount = executionBlock.getPieceCount(); +if (pieceCount.isKnown()) { + GiraphConstants.SUPERSTEP_COUNT.set(conf, pieceCount.getCount()); --- End diff -- There will be X+1 supersteps, but they are going to be supersteps 0..X. Actually I can make +1 here and then -1 in the logging part to keep it clear. > Keep track of number of supersteps when possible > > > Key: GIRAPH-1146 > URL: https://issues.apache.org/jira/browse/GIRAPH-1146 > Project: Giraph > Issue Type: New Feature >Reporter: Maja Kabiljo >Assignee: Maja Kabiljo >Priority: Minor > > In many cases we know how many supersteps are there going to be. We can keep > track of it and log it with progress. -- This message was sent by Atlassian JIRA (v6.3.15#6346)