[
https://issues.apache.org/jira/browse/GIRAPH-996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dionysios Logothetis updated GIRAPH-996:
----------------------------------------
Attachment: GIRAPH-996.patch
If message request size exceeds the default maximum size by a factor (e.g. by
2.0), then we print a warning to the log and also to the command line. The
factor is configurable and its default value is 2.0.
The check is inserted at SendMessageCache.sendMessageRequest() and
SendOneMessageToManyCache.sendMessageToAllRequest(). Some of these requests may
end up being local, so this may just generate some warning for requests that
don't really go through netty. But after doRequest() is called there is no way
to distinguish between message requests, edge requests, which have a different
max request size value. The code will become unnecessarily messy.
To test:
1) Run a job with giraph.msgRequestSize set to 1. This will make all requests
big and print the warning.
e.g. Add conf.set("giraph.msgRequestSize","1") in PageRankComputationTest and
run this locally:
mvn -DfailIfNoTests=false -Dtest=PageRankComputationTest test
2) Do all unit tests.
> Large requests degrade performance. Print out warnings.
> -------------------------------------------------------
>
> Key: GIRAPH-996
> URL: https://issues.apache.org/jira/browse/GIRAPH-996
> Project: Giraph
> Issue Type: Task
> Reporter: Dionysios Logothetis
> Assignee: Dionysios Logothetis
> Priority: Minor
> Attachments: GIRAPH-996.patch
>
>
> Sometimes performance degrades significantly or a job starts using too much
> off heap memory when requests larger than expected are being sent through
> netty. We should print warnings when large requests are sent, and also send
> warnings to command line to make sure users notice them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
