Re: Companies Using Apache Storm Listing - XenonStack
Hi XenonStack, Thanks. We'll update the site. Den tor. 13. sep. 2018 kl. 13.21 skrev XenonStack A Stack Innovator < busin...@xenonstack.com>: > At XenonStack we use Storm for building real-time data integration systems > and Enabling Predictive analysis >
[GitHub] storm-site pull request #7: Add XenonStack to powered-by
GitHub user srdo opened a pull request: https://github.com/apache/storm-site/pull/7 Add XenonStack to powered-by You can merge this pull request into a Git repository by running: $ git pull https://github.com/srdo/storm-site add-xenonstack Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm-site/pull/7.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7 commit 7e3ca9649f5954e8e0a2e8346e3a59d56c88e90f Author: Stig Rohde Døssing Date: 2018-09-16T11:26:51Z Add XenonStack to powered-by ---
[GitHub] storm pull request #2837: Remove powered-by.md, it lives in the storm-site r...
GitHub user srdo opened a pull request: https://github.com/apache/storm/pull/2837 Remove powered-by.md, it lives in the storm-site repository now Copied off the mailing list post: We have a powered-by.md file in the Storm repository's /docs directory. There's also a powered-by file in the storm-site repo root, as well as in each release directory. The storm-site root powered-by is the one linked to by the Storm site, and I don't believe it's being updated based on the Storm repo powered-by. The Storm repo powered-by ends up in the release-specific documentation instead, e.g. https://storm.apache.org/releases/2.0.0-SNAPSHOT/Powered-By.html. I think we might as well treat Powered-By.md the same as the getting-help.md file, which seems to only be present in the storm-site repo. I'd like to delete powered-by from the Storm repo, so we only have the one in the storm-site root. You can merge this pull request into a Git repository by running: $ git pull https://github.com/srdo/storm remove-powered-by Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/2837.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2837 commit 0e765f89e674f7e0a05746f5cce8ace193180837 Author: Stig Rohde Døssing Date: 2018-09-16T11:23:30Z Remove powered-by.md, it lives in the storm-site repository now ---
[GitHub] storm issue #2829: STORM-3222: Fix KafkaSpout internals to use LinkedList in...
Github user srdo commented on the issue: https://github.com/apache/storm/pull/2829 Even if we change the spout to emit all the fetched records in one `nextTuple` call, the number and size of records returned in a fetch is limited by the `max.poll.records` KafkaConsumer setting (500 by default), as well as the `fetch.max.bytes` setting (50MB by default). ---
Re: Is the powered-by file in the Storm repository necessary?
Having it in the site is sufficient IMO as well. On Saturday, September 15, 2018, 4:16:05 AM PDT, Stig Rohde Døssing wrote: Hi, We have a powered-by.md file in the Storm repository's /docs directory. There's also a powered-by file in the storm-site repo root, as well as in each release directory. The storm-site root powered-by is the one linked to by the Storm site, and I don't believe it's being updated based on the Storm repo powered-by. The Storm repo powered-by ends up in the release-specific documentation instead, e.g. https://storm.apache.org/releases/2.0.0-SNAPSHOT/Powered-By.html. I think we might as well treat Powered-By.md the same as the getting-help.md file, which seems to only be present in the storm-site repo. I'd like to delete powered-by from the Storm repo, so we only have the one in the storm-site root. Any opinions?
[GitHub] storm issue #2829: STORM-3222: Fix KafkaSpout internals to use LinkedList in...
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2829 @arunmahadevan .. need to be careful when that we are not doing too many emits in a single nextTuple()... it can cause a OOM situation by flooding the pendingQ. Worth checking if there is any measurable gain in combining many emits into a single nextTuple() for KafkaSpout. ---
[GitHub] storm issue #2829: STORM-3222: Fix KafkaSpout internals to use LinkedList in...
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2829 @HeartSaVioR 1) pendingEmitsQ prevents nextTuple() from blocking when downstream queue is full. It holds the overflow emits (one or more) that occurred within a **single** nextTuple() invocation. If Spout executor notices this Q is not empty, it will try to process any ACKs before entering wait strategy. Purpose of this Q is to prevent deadlock under BP. 2) **topology.max.spout.pending** Looked like a candidate for elimination given that STORM-2306's BP. Although it is not absolutely necessary to use it in 2.0.. during benchmarking I noticed that it could sometimes have sizable impact on performance. Why ? it remains a mystery. It is now a perf related tunable (in ACK mode).. and not really a BP mechanism. ---
[GitHub] storm issue #2789: STORM-3173: flush metrics to ScheduledReporter on shutdow...
Github user srdo commented on the issue: https://github.com/apache/storm/pull/2789 @zd-project I'm happy to take a look at implementing https://github.com/apache/storm/pull/2764 once the non-static PR is merged. I don't think it makes sense to mix more stuff into the current PR, and if it doesn't get approved, it'll be wasted work. ---
