[jira] [Resolved] (GIRAPH-43) GiraphJob: Counter Limit is not set
[ https://issues.apache.org/jira/browse/GIRAPH-43?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-43. --- Resolution: Not A Problem Resolving as not a problem, since this is the intended behavior in 203. If it needs to be addressed from Giraph's side, another JIRA can handle it. > GiraphJob: Counter Limit is not set > --- > > Key: GIRAPH-43 > URL: https://issues.apache.org/jira/browse/GIRAPH-43 > Project: Giraph > Issue Type: Bug > Components: graph >Affects Versions: 0.70.0 >Reporter: Severin Corsten >Priority: Minor > > I have got a problem regarding Counters: > For testing and benchmarking I added a lot of timers bud now the task fails > with this Exception: > org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded > limits on number of counters - Counters=120 Limit=120 > I don't understand why this happens. Within GiraphJob the counter limit is > set to 512. > In the cluster configuration there is no value specified. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-44) Add documentation about counter limits in Hadoop 0.203+
[ https://issues.apache.org/jira/browse/GIRAPH-44?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-44. --- Resolution: Fixed I've committed this. Resolving as fixed. Thanks for the contribution, Mitul! > Add documentation about counter limits in Hadoop 0.203+ > --- > > Key: GIRAPH-44 > URL: https://issues.apache.org/jira/browse/GIRAPH-44 > Project: Giraph > Issue Type: Improvement > Components: documentation >Reporter: Jakob Homan >Assignee: Mitul Tiwari >Priority: Minor > Labels: newbie > Attachments: giraph-44.patch > > > GIRAPH-43 noted that the counter limits introduced in Hadoop 203 may limit > Giraph's ability to track superstep information, and collide with users' > counters as well. We should note in the documentation that with these > versions of Hadoop the limit (mapreduce.job.counters.max in .23 and > mapreduce.job.counters.limit in .203) should be increased. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-71) SequenceFileVertexInputFormat missing license header; rat fails
[ https://issues.apache.org/jira/browse/GIRAPH-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-71. --- Resolution: Fixed I've committed this. Resolving as fixed. > SequenceFileVertexInputFormat missing license header; rat fails > --- > > Key: GIRAPH-71 > URL: https://issues.apache.org/jira/browse/GIRAPH-71 > Project: Giraph > Issue Type: Bug > Components: lib >Affects Versions: 0.70.0 >Reporter: Jakob Homan >Assignee: Jakob Homan > Fix For: 0.70.0 > > Attachments: GIRAPH-71.patch > > > SequenceFileVertexInputFormat needs the Apache license header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-79) Change the menu layout of the site
[ https://issues.apache.org/jira/browse/GIRAPH-79?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-79. --- Resolution: Fixed Assignee: Hyunsik Choi I've committed this and updated the site. Thanks, Hyunsik! > Change the menu layout of the site > -- > > Key: GIRAPH-79 > URL: https://issues.apache.org/jira/browse/GIRAPH-79 > Project: Giraph > Issue Type: Task > Components: site >Reporter: Hyunsik Choi >Assignee: Hyunsik Choi > Labels: site > Attachments: GIRAPH-79_1.patch, GIRAPH-79_2.patch > > > The current site has the basic menu layout generated by maven site plugin. > This layout is restricted to embrace new contents. > I would like to suggest the following menu layout. > http://people.apache.org/~hyunsik/giraph/site/index.html > Although the layout includes most existing contents, it has two addition > categories, Giraph and Documentation. I think that this layout is simpler and > is good to add new contents. > Anyone has any other suggestions? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-47) Export Worker's Context/State to vertices through pre/post/Application/Superstep
[ https://issues.apache.org/jira/browse/GIRAPH-47?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-47. --- Resolution: Fixed > Export Worker's Context/State to vertices through > pre/post/Application/Superstep > > > Key: GIRAPH-47 > URL: https://issues.apache.org/jira/browse/GIRAPH-47 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.70.0 >Reporter: Claudio Martella >Assignee: Claudio Martella > Attachments: GIRAPH-47.diff, GIRAPH-47.diff, GIRAPH-47.diff, > GIRAPH-47.final > > > It would be quite useful for vertices to reach some worker-related > information stored i.e. in the GraphState class. > This information could be exported as a parameter to > pre/post/Application/Superstep like this: > public void preApplication(Configurable workerObject); > public void postApplication(Configurable workerObject); > public void preSuperstep(Configurable workerObject); > public void postSuperstep(Configurable workerObject); > public Configurable getWorkerObject(); > Another possibility is to add a Context inner class to BasicVertex to store > this information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-99) Make AdjacencyListVertexReader and its constructor public
[ https://issues.apache.org/jira/browse/GIRAPH-99?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-99. --- Resolution: Fixed Fix Version/s: 0.70.0 Assignee: Kohei Ozaki I've committed this. Resolving as fixed. Thanks for the contribution, Kohei! > Make AdjacencyListVertexReader and its constructor public > - > > Key: GIRAPH-99 > URL: https://issues.apache.org/jira/browse/GIRAPH-99 > Project: Giraph > Issue Type: Wish > Components: lib >Reporter: Kohei Ozaki >Assignee: Kohei Ozaki >Priority: Minor > Labels: patch > Fix For: 0.70.0 > > Attachments: GIRAPH-99.diff > > > Hi, > I'd like to write a class inherited from AdjacencyListVertexReader > to make a library using Giraph (like git.io/ALVR), > but AdjacencyListVertexReader is a private class and its constructor are > private. > I guess making it public is useful to handle a more complex input format > specified by the data structure of algorithms. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-51) Provide unit testing tool for Giraph algorithms
[ https://issues.apache.org/jira/browse/GIRAPH-51?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-51. --- Resolution: Fixed Fix Version/s: 0.70.0 I've committed this. Thanks for the contribution, Sebastian! > Provide unit testing tool for Giraph algorithms > --- > > Key: GIRAPH-51 > URL: https://issues.apache.org/jira/browse/GIRAPH-51 > Project: Giraph > Issue Type: Improvement >Reporter: Jakob Homan >Assignee: Sebastian Schelter > Fix For: 0.70.0 > > Attachments: GIRAPH-51-2.patch, GIRAPH-51-3.patch, GIRAPH-51.patch > > > It would be nice to have a little tool, similar to MRUnit, that would allow > Giraph application writers to quickly unit test their algorithms. The tool > could take a Vertex implementation, a set of input and expected output and > verify that after the specified number of supersteps, we've gotten what we > expect. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-123) the wiki is not publicly accessible
[ https://issues.apache.org/jira/browse/GIRAPH-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-123. Resolution: Fixed Assignee: Jakob Homan INFRA has done this for us. Let us know if you run into any issues with it. Resolving. > the wiki is not publicly accessible > --- > > Key: GIRAPH-123 > URL: https://issues.apache.org/jira/browse/GIRAPH-123 > Project: Giraph > Issue Type: Bug > Components: documentation >Reporter: André Kelpe >Assignee: Jakob Homan >Priority: Minor > > When I try to read the documentation on the wiki I end up on a login screen. > Can you please make the wiki open for the public. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-129) enable creation of javadoc and sources jars
[ https://issues.apache.org/jira/browse/GIRAPH-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-129. Resolution: Fixed Fix Version/s: 0.1.0 I've committed this. Resolving as fixed. Thanks for the contribution, André! > enable creation of javadoc and sources jars > --- > > Key: GIRAPH-129 > URL: https://issues.apache.org/jira/browse/GIRAPH-129 > Project: Giraph > Issue Type: Improvement > Components: build >Affects Versions: 0.1.0 >Reporter: André Kelpe >Assignee: André Kelpe >Priority: Minor > Fix For: 0.1.0 > > Attachments: GIRAPH-129.patch > > > It is pretty useful to enable the creation if javadoc and sources jars during > the build, so that people using IDEs like eclipse can easily jump into the > code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-131) enable creation of test-jars to simplify testing in downstream projects
[ https://issues.apache.org/jira/browse/GIRAPH-131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-131. Resolution: Fixed Fix Version/s: 0.1.0 Assignee: André Kelpe I've committed this. Resolving as fixed. Thanks, André! > enable creation of test-jars to simplify testing in downstream projects > --- > > Key: GIRAPH-131 > URL: https://issues.apache.org/jira/browse/GIRAPH-131 > Project: Giraph > Issue Type: Improvement >Reporter: André Kelpe >Assignee: André Kelpe >Priority: Minor > Fix For: 0.1.0 > > Attachments: GIRAPH-131.patch > > > Attached patch enables the creation of test-jars, which are the tests > packaged in a separate jar file. This makes it possible to use the > super-useful test infrastructure in MockUtils in downstream projects. If you > add the patch, you will get a ${giraph.version}-tests.jar, which can be used > for downstream testing like this: > > org.apache.giraph > giraph > ${giraph.version} > test-jar > test > > P.S.: The patch also resets the version to 0.1-SNAPSHOT as discussed in > GIRAPH-129 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-133) Typo in JavaDoc in BspCase::remove
[ https://issues.apache.org/jira/browse/GIRAPH-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-133. Resolution: Fixed Fix Version/s: 0.2.0 Assignee: Harsh J Committed. Resolving. > Typo in JavaDoc in BspCase::remove > -- > > Key: GIRAPH-133 > URL: https://issues.apache.org/jira/browse/GIRAPH-133 > Project: Giraph > Issue Type: Improvement >Reporter: Jakob Homan >Assignee: Harsh J >Priority: Trivial > Labels: newbie > Fix For: 0.2.0 > > Attachments: GIRAPH-133.patch > > > Configuration is spelled wrong in the javadoc: > {noformat}/** > * Helper method to remove a path if it exists. > * > * @param conf Configutation > * @param path Path to remove > * @throws IOException > */ > public static void remove(Configuration conf, Path path) > throws IOException { > FileSystem hdfs = FileSystem.get(conf); > hdfs.delete(path, true); > }{noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-137) De-duplicate pagerank implementation in PageRankBenchmark
[ https://issues.apache.org/jira/browse/GIRAPH-137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-137. Resolution: Fixed Fix Version/s: 0.2.0 Assignee: Harsh J Committed. Resolving. > De-duplicate pagerank implementation in PageRankBenchmark > - > > Key: GIRAPH-137 > URL: https://issues.apache.org/jira/browse/GIRAPH-137 > Project: Giraph > Issue Type: Improvement >Reporter: Jakob Homan >Assignee: Harsh J >Priority: Minor > Labels: newbie > Fix For: 0.2.0 > > Attachments: GIRAPH-137.patch > > > Currently in PageRankBenchmark we have the code for pagerank duplicated in > each of the implementations of Vertex: > {noformat}public static class PageRankHashMapVertex extends HashMapVertex< > LongWritable, DoubleWritable, DoubleWritable, DoubleWritable> { > @Override > public void compute(Iterator msgIterator) { > if (getSuperstep() >= 1) { > double sum = 0; > while (msgIterator.hasNext()) { > sum += msgIterator.next().get(); > } > DoubleWritable vertexValue = > new DoubleWritable((0.15f / getNumVertices()) + 0.85f * >sum); > setVertexValue(vertexValue); > } > if (getSuperstep() < getConf().getInt(SUPERSTEP_COUNT, -1)) { > long edges = getNumOutEdges(); > sendMsgToAllEdges( > new DoubleWritable(getVertexValue().get() / edges)); > } else { > voteToHalt(); > } > } > } > public static class PageRankEdgeListVertex extends EdgeListVertex< > LongWritable, DoubleWritable, DoubleWritable, DoubleWritable> { > @Override > public void compute(Iterator msgIterator) { > if (getSuperstep() >= 1) { > double sum = 0; > while (msgIterator.hasNext()) { > sum += msgIterator.next().get(); > } > DoubleWritable vertexValue = > new DoubleWritable((0.15f / getNumVertices()) + 0.85f * >sum); > setVertexValue(vertexValue); > } > if (getSuperstep() < getConf().getInt(SUPERSTEP_COUNT, -1)) { > long edges = getNumOutEdges(); > sendMsgToAllEdges( > new DoubleWritable(getVertexValue().get() / edges)); > } else { > voteToHalt(); > } > } > }{noformat} > This code can be consolidated into private class and the two implementations > just extend that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-130) Fix Javadoc warnings
[ https://issues.apache.org/jira/browse/GIRAPH-130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-130. Resolution: Fixed Fix Version/s: 0.2.0 Assignee: Harsh J > Fix Javadoc warnings > > > Key: GIRAPH-130 > URL: https://issues.apache.org/jira/browse/GIRAPH-130 > Project: Giraph > Issue Type: Bug >Reporter: Jakob Homan >Assignee: Harsh J >Priority: Minor > Labels: newbie > Fix For: 0.2.0 > > Attachments: GIRAPH-130.patch > > > We've accumulated a fair number of javadoc warnings recently: > {noformat}[WARNING] Javadoc Warnings > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:146: > warning - Tag @link: reference not found: GraphPartitioner > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:129: > warning - @param argument "superstep" is not a parameter name. > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:146: > warning - Tag @link: reference not found: GraphPartitioner > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/CommunicationsInterface.java:84: > warning - @param argument "vertexIndex" is not a parameter name. > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/CommunicationsInterface.java:84: > warning - @param argument "msgList" is not a parameter name. > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java:32: > warning - Tag @link: reference not found: VertexIdMessage > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/VertexCombiner.java:46: > warning - Tag @link: reference not found: messages > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/VertexCombiner.java:46: > warning - Tag @link: reference not found: messages > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/AggregatorWriter.java:60: > warning - @param argument "map" is not a parameter name. > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:146: > warning - Tag @link: reference not found: GraphPartitioner > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:146: > warning - Tag @link: reference not found: GraphPartitioner > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/GiraphJob.java:432: > warning - @param argument "graphPartitionerClass" is not a parameter name. > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/VertexCombiner.java:46: > warning - Tag @link: reference not found: messages > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/partition/MasterGraphPartitioner.java:62: > warning - Tag @link: reference not found: GraphPartitioner > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/partition/MasterGraphPartitioner.java:62: > warning - Tag @link: reference not found: GraphPartitioner > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/partition/MasterGraphPartitioner.java:62: > warning - @param argument "availableWorkerInfos" is not a parameter name. > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/partition/PartitionBalancer.java:176: > warning - @param argument "allPartitionStatsList" is not a parameter name. > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java:32: > warning - Tag @link: reference not found: VertexIdMessage > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java:32: > warning - Tag @link: reference not found: VertexIdMessage > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:146: > warning - Tag @link: reference not found: GraphPartitioner > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java:32: > warning - Tag @link: reference not found: VertexIdMessage > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:146: > warning - Tag @link: reference not found: GraphPartitioner > [WARNING] > /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java:32: > warning - Tag @link: reference not found: VertexIdMessage > {noformat} > It would be good to fix these. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issue
[jira] [Resolved] (GIRAPH-85) Simplify return expression in RPCCommunications::getRPCProxy
[ https://issues.apache.org/jira/browse/GIRAPH-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-85. --- Resolution: Fixed > Simplify return expression in RPCCommunications::getRPCProxy > > > Key: GIRAPH-85 > URL: https://issues.apache.org/jira/browse/GIRAPH-85 > Project: Giraph > Issue Type: Improvement >Affects Versions: 0.2.0 >Reporter: Jakob Homan >Assignee: Eli Reisman > Labels: newbie > Fix For: 0.2.0 > > Attachments: GIRAPH-85-3.patch, GIRAPH-85-3.patch, GIRAPH-85.patch, > GIRAPH-85.patch > > > Twice in RPCCommunications::getRPCProxy a local variable, proxy, is created > and immediately returned. We can simplify this to just return the value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira