[GitHub] storm issue #2454: STORM-2847: Ensure spout can handle being activated and d...
Github user hmcl commented on the issue: https://github.com/apache/storm/pull/2454 +1. @srdo can you please squash. ---
[GitHub] storm pull request #2428: STORM-2826: Set key/value deserializer fields when...
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2428 ---
[GitHub] storm issue #2463: STORM-2859: Fix a number of issues with NormalizedResourc...
Github user srdo commented on the issue: https://github.com/apache/storm/pull/2463 Here's the run from a branch that contains this branch, STORM-2858 and some minor travis.yml changes. https://travis-ci.org/srdo/storm/builds/317153261 ---
[GitHub] storm issue #2462: STORM-2858: Fix worker-launcher build by erroring out if ...
Github user srdo commented on the issue: https://github.com/apache/storm/pull/2462 Here's the run from a branch that contains this branch, STORM-2859 and some minor travis.yml changes. https://travis-ci.org/srdo/storm/builds/317153261 ---
[GitHub] storm issue #2463: STORM-2859: Fix a number of issues with NormalizedResourc...
Github user srdo commented on the issue: https://github.com/apache/storm/pull/2463 There's also a TODO I'd like some input on https://github.com/apache/storm/pull/2463/files#diff-0f94945ef866045b8717f781620fd59dR190 ---
[GitHub] storm pull request #2463: STORM-2859: Fix a number of issues with Normalized...
GitHub user srdo opened a pull request: https://github.com/apache/storm/pull/2463 STORM-2859: Fix a number of issues with NormalizedResources, and prev⦠â¦ent it from leaking static state in tests. This is part of making the build run on the new Travis Ubuntu image. Travis is probably running the tests in a slightly different order than before. NormalizedResources had a static field used to track the generic resource types registered for supervisors. It was leaking from the GenericResourceAwareSchedulerTest to DefaultResourceAwareSchedulerTest, where it was causing a test failure. When the unused resource type showed up in the test, NormalizedResources would return 0.0 from calculateMinPercentageUsedBy for all racks because there was a resource type where the cluster had 0 of that resource in total. This breaks rack sorting and causes the test to fail. I'm not entirely sure why NormalizedResources used the static field + double arrays for tracking resources, but I've replaced it with regular resourceName -> resourceValue maps. Let me know if this is a problem. Other issues fixed: * calculateMinPercentageUsedBy threw division by 0 error if total cpu or memory was 0. * Removed static map between resource names and array indices. It was causing tests to leak information to each other. Rewrote parts of NormalizedResources to make this map unnecessary. * Added check that all resources in a rack/node are also present in the total. * calculateAvg was being inconsistent about the divisor in the average between cpu/memory and other resources. Fixed so skipped resources are never counted in the divisor. * calculateMin didn't handle resource totals being 0 very well. The method defined (anyNumber)/0 to result in 0, so if any resource in the total had a 0 value, all racks/nodes would be prioritized equally. Change calculateMin and calculateAvg to simply skip any resources where the total is 0 in the calculation. I think it makes sense to disregard such resources and fall back on prioritizing by any remaining non-0 resources. It was already done for CPU and memory in calculateAvg. In case there are no non-0 resources, we can just return 100% for both average and minimum to disable prioritization, since any rack/node is as good as any other. * Fixed up NormalizedResources constructor so it doesn't call an overridable method. You can merge this pull request into a Git repository by running: $ git pull https://github.com/srdo/storm STORM-2859 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/2463.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2463 commit b654a4f065a496a273601e05888e79b35a991653 Author: Stig Rohde Døssing Date: 2017-12-13T15:47:31Z STORM-2859: Fix a number of issues with NormalizedResources, and prevent it from leaking static state in tests. ---
[GitHub] storm pull request #2462: STORM-2858: Fix worker-launcher build by erroring ...
GitHub user srdo opened a pull request: https://github.com/apache/storm/pull/2462 STORM-2858: Fix worker-launcher build by erroring out if asprintf fai⦠â¦ls to allocate memory. Replace make-maven-plugin with a build shell script. This is part of making the build run on the new Travis Ubuntu image. I got an error when building storm-core with -Pnative because GCC has marked asprintf as a function where you shouldn't ignore the return value. I've put in return value handling that exits the worker-launcher process if asprintf fails to allocate memory. As part of this I replaced make-maven-plugin, because it's fairly old (last release in 2009, there doesn't seem to be a page for the plugin anymore) and opaque (had to decompile the jar to figure out what it was doing). I think a build script + exec-maven-plugin is less magical. The make process output now also goes in the terminal when building, which it didn't with make-maven-plugin. You can merge this pull request into a Git repository by running: $ git pull https://github.com/srdo/storm STORM-2858 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/2462.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2462 commit 25a2f337a629ed55fde772052ea8ce7015b69462 Author: Stig Rohde Døssing Date: 2017-12-13T15:47:31Z STORM-2858: Fix worker-launcher build by erroring out if asprintf fails to allocate memory. Replace make-maven-plugin with a build shell script. ---
[GitHub] storm issue #2460: STORM-2851 1.x
Github user hmcl commented on the issue: https://github.com/apache/storm/pull/2460 +1 ---
[GitHub] storm issue #2443: STORM-2406 [Storm SQL] Change underlying API to Streams A...
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/2443 Rebased to apply dropping standalone mode. ---
[GitHub] storm pull request #2447: STORM-2845 Drop standalone mode of Storm SQL
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2447 ---
[GitHub] storm issue #2456: YSTORM-4457: Fix for wouldFit
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/2456 Some guidelines for contribution: 1. Please file an issue to Apache JIRA, rather than referring internal issue. 2. Please make your title of PR (and summary of JIRA) self-describing. Not only code change is important. Leaving context against the code change is more important for future. ---
[GitHub] storm issue #2459: STORM-2855: Revert to 2017Q4 Ubuntu image in Travis to fi...
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/2459 I just merged cause it is not about code but environment hence no need to wait 24hrs. ---
[GitHub] storm pull request #2459: STORM-2855: Revert to 2017Q4 Ubuntu image in Travi...
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/2459 ---
