GitHub user roshannaik opened a pull request:
https://github.com/apache/storm/pull/2019
STORM-2423 - Join Bolt should use explicit instead of default window
anchoring of emitted tuples
Default anchoring will anchor each emitted tuple to every tuple in current
window. This requires a very large numbers of ACKs from any downstream bolt. If
topology.debug is enabled, it also worsens the load on the system significantly.
Letting the topo run in this mode (in particular with max.spout.pending
disabled), could lead to the worker running out of memory and crashing.
*Fix* Join Bolt should avoid using default window anchoring, and explicitly
anchor each emitted tuple with the exact matching tuples form each inputs
streams. This reduces the complexity of the tuple trees and consequently the
reduces burden on the ACKing & messaging subsystems.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/roshannaik/storm STORM-2423
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/2019.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2019
----
commit a3cd13474846756a7edf3061fd60a947bec078f8
Author: Roshan Naik <[email protected]>
Date: 2017-03-18T02:46:28Z
STORM-2423 - Join Bolt : Use explicit instead of default window anchoring
of emitted tuples
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
