[jira] [Resolved] (GIRAPH-43) GiraphJob: Counter Limit is not set
[ https://issues.apache.org/jira/browse/GIRAPH-43?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-43. --- Resolution: Not A Problem Resolving as not a problem, since this is the intended behavior in 203. If it needs to be addressed from Giraph's side, another JIRA can handle it. > GiraphJob: Counter Limit is not set > --- > > Key: GIRAPH-43 > URL: https://issues.apache.org/jira/browse/GIRAPH-43 > Project: Giraph > Issue Type: Bug > Components: graph >Affects Versions: 0.70.0 >Reporter: Severin Corsten >Priority: Minor > > I have got a problem regarding Counters: > For testing and benchmarking I added a lot of timers bud now the task fails > with this Exception: > org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded > limits on number of counters - Counters=120 Limit=120 > I don't understand why this happens. Within GiraphJob the counter limit is > set to 512. > In the cluster configuration there is no value specified. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-44) Add documentation about counter limits in Hadoop 0.203+
[ https://issues.apache.org/jira/browse/GIRAPH-44?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-44. --- Resolution: Fixed I've committed this. Resolving as fixed. Thanks for the contribution, Mitul! > Add documentation about counter limits in Hadoop 0.203+ > --- > > Key: GIRAPH-44 > URL: https://issues.apache.org/jira/browse/GIRAPH-44 > Project: Giraph > Issue Type: Improvement > Components: documentation >Reporter: Jakob Homan >Assignee: Mitul Tiwari >Priority: Minor > Labels: newbie > Attachments: giraph-44.patch > > > GIRAPH-43 noted that the counter limits introduced in Hadoop 203 may limit > Giraph's ability to track superstep information, and collide with users' > counters as well. We should note in the documentation that with these > versions of Hadoop the limit (mapreduce.job.counters.max in .23 and > mapreduce.job.counters.limit in .203) should be increased. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-71) SequenceFileVertexInputFormat missing license header; rat fails
[ https://issues.apache.org/jira/browse/GIRAPH-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-71. --- Resolution: Fixed I've committed this. Resolving as fixed. > SequenceFileVertexInputFormat missing license header; rat fails > --- > > Key: GIRAPH-71 > URL: https://issues.apache.org/jira/browse/GIRAPH-71 > Project: Giraph > Issue Type: Bug > Components: lib >Affects Versions: 0.70.0 >Reporter: Jakob Homan >Assignee: Jakob Homan > Fix For: 0.70.0 > > Attachments: GIRAPH-71.patch > > > SequenceFileVertexInputFormat needs the Apache license header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-79) Change the menu layout of the site
[ https://issues.apache.org/jira/browse/GIRAPH-79?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-79. --- Resolution: Fixed Assignee: Hyunsik Choi I've committed this and updated the site. Thanks, Hyunsik! > Change the menu layout of the site > -- > > Key: GIRAPH-79 > URL: https://issues.apache.org/jira/browse/GIRAPH-79 > Project: Giraph > Issue Type: Task > Components: site >Reporter: Hyunsik Choi >Assignee: Hyunsik Choi > Labels: site > Attachments: GIRAPH-79_1.patch, GIRAPH-79_2.patch > > > The current site has the basic menu layout generated by maven site plugin. > This layout is restricted to embrace new contents. > I would like to suggest the following menu layout. > http://people.apache.org/~hyunsik/giraph/site/index.html > Although the layout includes most existing contents, it has two addition > categories, Giraph and Documentation. I think that this layout is simpler and > is good to add new contents. > Anyone has any other suggestions? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-47) Export Worker's Context/State to vertices through pre/post/Application/Superstep
[ https://issues.apache.org/jira/browse/GIRAPH-47?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-47. --- Resolution: Fixed > Export Worker's Context/State to vertices through > pre/post/Application/Superstep > > > Key: GIRAPH-47 > URL: https://issues.apache.org/jira/browse/GIRAPH-47 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.70.0 >Reporter: Claudio Martella >Assignee: Claudio Martella > Attachments: GIRAPH-47.diff, GIRAPH-47.diff, GIRAPH-47.diff, > GIRAPH-47.final > > > It would be quite useful for vertices to reach some worker-related > information stored i.e. in the GraphState class. > This information could be exported as a parameter to > pre/post/Application/Superstep like this: > public void preApplication(Configurable workerObject); > public void postApplication(Configurable workerObject); > public void preSuperstep(Configurable workerObject); > public void postSuperstep(Configurable workerObject); > public Configurable getWorkerObject(); > Another possibility is to add a Context inner class to BasicVertex to store > this information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-99) Make AdjacencyListVertexReader and its constructor public
[ https://issues.apache.org/jira/browse/GIRAPH-99?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-99. --- Resolution: Fixed Fix Version/s: 0.70.0 Assignee: Kohei Ozaki I've committed this. Resolving as fixed. Thanks for the contribution, Kohei! > Make AdjacencyListVertexReader and its constructor public > - > > Key: GIRAPH-99 > URL: https://issues.apache.org/jira/browse/GIRAPH-99 > Project: Giraph > Issue Type: Wish > Components: lib >Reporter: Kohei Ozaki >Assignee: Kohei Ozaki >Priority: Minor > Labels: patch > Fix For: 0.70.0 > > Attachments: GIRAPH-99.diff > > > Hi, > I'd like to write a class inherited from AdjacencyListVertexReader > to make a library using Giraph (like git.io/ALVR), > but AdjacencyListVertexReader is a private class and its constructor are > private. > I guess making it public is useful to handle a more complex input format > specified by the data structure of algorithms. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-51) Provide unit testing tool for Giraph algorithms
[ https://issues.apache.org/jira/browse/GIRAPH-51?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-51. --- Resolution: Fixed Fix Version/s: 0.70.0 I've committed this. Thanks for the contribution, Sebastian! > Provide unit testing tool for Giraph algorithms > --- > > Key: GIRAPH-51 > URL: https://issues.apache.org/jira/browse/GIRAPH-51 > Project: Giraph > Issue Type: Improvement >Reporter: Jakob Homan >Assignee: Sebastian Schelter > Fix For: 0.70.0 > > Attachments: GIRAPH-51-2.patch, GIRAPH-51-3.patch, GIRAPH-51.patch > > > It would be nice to have a little tool, similar to MRUnit, that would allow > Giraph application writers to quickly unit test their algorithms. The tool > could take a Vertex implementation, a set of input and expected output and > verify that after the specified number of supersteps, we've gotten what we > expect. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-123) the wiki is not publicly accessible
[ https://issues.apache.org/jira/browse/GIRAPH-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-123. Resolution: Fixed Assignee: Jakob Homan INFRA has done this for us. Let us know if you run into any issues with it. Resolving. > the wiki is not publicly accessible > --- > > Key: GIRAPH-123 > URL: https://issues.apache.org/jira/browse/GIRAPH-123 > Project: Giraph > Issue Type: Bug > Components: documentation >Reporter: André Kelpe >Assignee: Jakob Homan >Priority: Minor > > When I try to read the documentation on the wiki I end up on a login screen. > Can you please make the wiki open for the public. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-129) enable creation of javadoc and sources jars
[ https://issues.apache.org/jira/browse/GIRAPH-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-129. Resolution: Fixed Fix Version/s: 0.1.0 I've committed this. Resolving as fixed. Thanks for the contribution, André! > enable creation of javadoc and sources jars > --- > > Key: GIRAPH-129 > URL: https://issues.apache.org/jira/browse/GIRAPH-129 > Project: Giraph > Issue Type: Improvement > Components: build >Affects Versions: 0.1.0 >Reporter: André Kelpe >Assignee: André Kelpe >Priority: Minor > Fix For: 0.1.0 > > Attachments: GIRAPH-129.patch > > > It is pretty useful to enable the creation if javadoc and sources jars during > the build, so that people using IDEs like eclipse can easily jump into the > code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-131) enable creation of test-jars to simplify testing in downstream projects
[
https://issues.apache.org/jira/browse/GIRAPH-131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan resolved GIRAPH-131.
Resolution: Fixed
Fix Version/s: 0.1.0
Assignee: André Kelpe
I've committed this. Resolving as fixed. Thanks, André!
> enable creation of test-jars to simplify testing in downstream projects
> ---
>
> Key: GIRAPH-131
> URL: https://issues.apache.org/jira/browse/GIRAPH-131
> Project: Giraph
> Issue Type: Improvement
>Reporter: André Kelpe
>Assignee: André Kelpe
>Priority: Minor
> Fix For: 0.1.0
>
> Attachments: GIRAPH-131.patch
>
>
> Attached patch enables the creation of test-jars, which are the tests
> packaged in a separate jar file. This makes it possible to use the
> super-useful test infrastructure in MockUtils in downstream projects. If you
> add the patch, you will get a ${giraph.version}-tests.jar, which can be used
> for downstream testing like this:
>
> org.apache.giraph
> giraph
> ${giraph.version}
> test-jar
> test
>
> P.S.: The patch also resets the version to 0.1-SNAPSHOT as discussed in
> GIRAPH-129
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-133) Typo in JavaDoc in BspCase::remove
[
https://issues.apache.org/jira/browse/GIRAPH-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan resolved GIRAPH-133.
Resolution: Fixed
Fix Version/s: 0.2.0
Assignee: Harsh J
Committed. Resolving.
> Typo in JavaDoc in BspCase::remove
> --
>
> Key: GIRAPH-133
> URL: https://issues.apache.org/jira/browse/GIRAPH-133
> Project: Giraph
> Issue Type: Improvement
>Reporter: Jakob Homan
>Assignee: Harsh J
>Priority: Trivial
> Labels: newbie
> Fix For: 0.2.0
>
> Attachments: GIRAPH-133.patch
>
>
> Configuration is spelled wrong in the javadoc:
> {noformat}/**
> * Helper method to remove a path if it exists.
> *
> * @param conf Configutation
> * @param path Path to remove
> * @throws IOException
> */
> public static void remove(Configuration conf, Path path)
> throws IOException {
> FileSystem hdfs = FileSystem.get(conf);
> hdfs.delete(path, true);
> }{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-137) De-duplicate pagerank implementation in PageRankBenchmark
[
https://issues.apache.org/jira/browse/GIRAPH-137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan resolved GIRAPH-137.
Resolution: Fixed
Fix Version/s: 0.2.0
Assignee: Harsh J
Committed. Resolving.
> De-duplicate pagerank implementation in PageRankBenchmark
> -
>
> Key: GIRAPH-137
> URL: https://issues.apache.org/jira/browse/GIRAPH-137
> Project: Giraph
> Issue Type: Improvement
>Reporter: Jakob Homan
>Assignee: Harsh J
>Priority: Minor
> Labels: newbie
> Fix For: 0.2.0
>
> Attachments: GIRAPH-137.patch
>
>
> Currently in PageRankBenchmark we have the code for pagerank duplicated in
> each of the implementations of Vertex:
> {noformat}public static class PageRankHashMapVertex extends HashMapVertex<
> LongWritable, DoubleWritable, DoubleWritable, DoubleWritable> {
> @Override
> public void compute(Iterator msgIterator) {
> if (getSuperstep() >= 1) {
> double sum = 0;
> while (msgIterator.hasNext()) {
> sum += msgIterator.next().get();
> }
> DoubleWritable vertexValue =
> new DoubleWritable((0.15f / getNumVertices()) + 0.85f *
>sum);
> setVertexValue(vertexValue);
> }
> if (getSuperstep() < getConf().getInt(SUPERSTEP_COUNT, -1)) {
> long edges = getNumOutEdges();
> sendMsgToAllEdges(
> new DoubleWritable(getVertexValue().get() / edges));
> } else {
> voteToHalt();
> }
> }
> }
> public static class PageRankEdgeListVertex extends EdgeListVertex<
> LongWritable, DoubleWritable, DoubleWritable, DoubleWritable> {
> @Override
> public void compute(Iterator msgIterator) {
> if (getSuperstep() >= 1) {
> double sum = 0;
> while (msgIterator.hasNext()) {
> sum += msgIterator.next().get();
> }
> DoubleWritable vertexValue =
> new DoubleWritable((0.15f / getNumVertices()) + 0.85f *
>sum);
> setVertexValue(vertexValue);
> }
> if (getSuperstep() < getConf().getInt(SUPERSTEP_COUNT, -1)) {
> long edges = getNumOutEdges();
> sendMsgToAllEdges(
> new DoubleWritable(getVertexValue().get() / edges));
> } else {
> voteToHalt();
> }
> }
> }{noformat}
> This code can be consolidated into private class and the two implementations
> just extend that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-130) Fix Javadoc warnings
[
https://issues.apache.org/jira/browse/GIRAPH-130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan resolved GIRAPH-130.
Resolution: Fixed
Fix Version/s: 0.2.0
Assignee: Harsh J
> Fix Javadoc warnings
>
>
> Key: GIRAPH-130
> URL: https://issues.apache.org/jira/browse/GIRAPH-130
> Project: Giraph
> Issue Type: Bug
>Reporter: Jakob Homan
>Assignee: Harsh J
>Priority: Minor
> Labels: newbie
> Fix For: 0.2.0
>
> Attachments: GIRAPH-130.patch
>
>
> We've accumulated a fair number of javadoc warnings recently:
> {noformat}[WARNING] Javadoc Warnings
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:146:
> warning - Tag @link: reference not found: GraphPartitioner
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:129:
> warning - @param argument "superstep" is not a parameter name.
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:146:
> warning - Tag @link: reference not found: GraphPartitioner
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/CommunicationsInterface.java:84:
> warning - @param argument "vertexIndex" is not a parameter name.
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/CommunicationsInterface.java:84:
> warning - @param argument "msgList" is not a parameter name.
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java:32:
> warning - Tag @link: reference not found: VertexIdMessage
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/VertexCombiner.java:46:
> warning - Tag @link: reference not found: messages
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/VertexCombiner.java:46:
> warning - Tag @link: reference not found: messages
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/AggregatorWriter.java:60:
> warning - @param argument "map" is not a parameter name.
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:146:
> warning - Tag @link: reference not found: GraphPartitioner
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:146:
> warning - Tag @link: reference not found: GraphPartitioner
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/GiraphJob.java:432:
> warning - @param argument "graphPartitionerClass" is not a parameter name.
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/VertexCombiner.java:46:
> warning - Tag @link: reference not found: messages
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/partition/MasterGraphPartitioner.java:62:
> warning - Tag @link: reference not found: GraphPartitioner
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/partition/MasterGraphPartitioner.java:62:
> warning - Tag @link: reference not found: GraphPartitioner
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/partition/MasterGraphPartitioner.java:62:
> warning - @param argument "availableWorkerInfos" is not a parameter name.
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/graph/partition/PartitionBalancer.java:176:
> warning - @param argument "allPartitionStatsList" is not a parameter name.
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java:32:
> warning - Tag @link: reference not found: VertexIdMessage
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java:32:
> warning - Tag @link: reference not found: VertexIdMessage
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:146:
> warning - Tag @link: reference not found: GraphPartitioner
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java:32:
> warning - Tag @link: reference not found: VertexIdMessage
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java:146:
> warning - Tag @link: reference not found: GraphPartitioner
> [WARNING]
> /Users/jhoman/repos/giraph/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java:32:
> warning - Tag @link: reference not found: VertexIdMessage
> {noformat}
> It would be good to fix these.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issue
[jira] [Resolved] (GIRAPH-85) Simplify return expression in RPCCommunications::getRPCProxy
[ https://issues.apache.org/jira/browse/GIRAPH-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved GIRAPH-85. --- Resolution: Fixed > Simplify return expression in RPCCommunications::getRPCProxy > > > Key: GIRAPH-85 > URL: https://issues.apache.org/jira/browse/GIRAPH-85 > Project: Giraph > Issue Type: Improvement >Affects Versions: 0.2.0 >Reporter: Jakob Homan >Assignee: Eli Reisman > Labels: newbie > Fix For: 0.2.0 > > Attachments: GIRAPH-85-3.patch, GIRAPH-85-3.patch, GIRAPH-85.patch, > GIRAPH-85.patch > > > Twice in RPCCommunications::getRPCProxy a local variable, proxy, is created > and immediately returned. We can simplify this to just return the value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
