GitHub user dlogothetis opened a pull request:
https://github.com/apache/giraph/pull/91
Remove the use of atomic counter in LongByteMappingStore
I don't see any particular value in this keeping this counter, which incurs
high overhead is it is accessed on every entry. There are better ways to update
it as values are added, but i don't think it's worth it.
Tests
- mvn -Phadoop_facebook clean install
- mvn -Phadop_2 clean install
- Ran jobs that read a mapping, this reduces the time to load mapping by up
to 50%.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dlogothetis/giraph mapping_perf
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/giraph/pull/91.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #91
----
commit db149d968790fb42f5a87e45245843c3005b79ae
Author: Dionysios Logothetis <dlogothetis@...>
Date: 2018-10-24T21:05:55Z
Remove atomic counter overhead in LongByteMappingStore
commit 497278cdc8aa24a18ccdc3bb701939cbe8de6fb7
Author: Dionysios Logothetis <dlogothetis@...>
Date: 2018-10-24T21:55:41Z
Get size from the right map
----
---
