[GitHub] storm pull request: Include topic information with Kafka metrics
GitHub user chawco opened a pull request: https://github.com/apache/storm/pull/470 Include topic information with Kafka metrics Currently, Kafka metrics are emitted with partition information, but no information about which topic that partition belongs to. This patch includes the topic information in the Storm metrics emitted by the Kafka spout so that multiple input topics may be tracked from a single topology. You can merge this pull request into a Git repository by running: $ git pull https://github.com/chawco/storm storm-kafka-metrics-with-topic Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/470.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #470 commit 66a20c689dfbd79d12a7e9557837bf5a2e920e06 Author: Craig Hawco Date: 2015-03-19T15:35:59Z Include topic information with Kafka metrics. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: Include topic information with Kafka metrics
Github user chawco commented on the pull request: https://github.com/apache/storm/pull/470#issuecomment-83682537 Sure thing, I've filed [STORM-713 ](https://issues.apache.org/jira/browse/STORM-713) for this issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: Include topic information with Kafka metrics
Github user chawco commented on the pull request: https://github.com/apache/storm/pull/470#issuecomment-83695774 I've amended the offending commit. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1767: STORM-2194: Report error and die, not report error...
GitHub user chawco opened a pull request: https://github.com/apache/storm/pull/1767 STORM-2194: Report error and die, not report error or die We should still kill our executor after encountering an unhandled exception. This change logs what happens as before, but also ensures we always call suicide-fn to ensure our worker exits. You can merge this pull request into a Git repository by running: $ git pull https://github.com/chawco/storm STORM-2194 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1767.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1767 commit 952fe22f0ec2dc1193f0d951d16f8ee0eb58e817 Author: Craig Hawco Date: 2016-11-09T20:02:04Z STORM-2194: Report error and die, not report error or die --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request #1768: STORM-2194: Report error and die, not report error...
GitHub user chawco opened a pull request: https://github.com/apache/storm/pull/1768 STORM-2194: Report error and die, not report error or die This appears to have just ported an existing bug from executor.clj -- this is what I believe the expected behaviour is/was. You can merge this pull request into a Git repository by running: $ git pull https://github.com/chawco/storm master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1768.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1768 commit f3a64f08cf056f638e703f420d025c1a81af6fcc Author: Craig Hawco Date: 2016-11-09T20:07:03Z STORM-2194: Report error and die, not report error or die --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1767: STORM-2194: Report error and die, not report error or die
Github user chawco commented on the issue: https://github.com/apache/storm/pull/1767 So, there's some ambiguity in place here that would need to get resolved. Basically, there's two sources for InterruptedException/InterruptedIOException -- one from Storm itself, and one from the user's code (See: [STORM-2194](https://issues.apache.org/jira/browse/STORM-2194) If we want to have different behaviour from these two sources we do need some solution to disambiguate the uses. Basically, the problem is this: If anything in the user's thread raises these exceptions, the executor thread will terminate *but the worker will not*. This leaves the entire topology in a zombified state. I find it difficult to see that this behaviour is "as intended". @revans2 can you suggest a way we can disambiguate between Storm initiated and user initiated exceptions here? I'm having a real tough time thinking how we could accomplish that. Alternatively, can you propose an alternate implementation for signalling this shutdown? I'm happy to take on that work to make things shutdown in a more appropriate manner, as long as we get to fix this zombie topology behaviour. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm issue #1768: STORM-2194: Report error and die, not report error or die
Github user chawco commented on the issue: https://github.com/apache/storm/pull/1768 I've left some follow up comments in [1767](https://github.com/apache/storm/pull/1767). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
