[GitHub] zookeeper issue #590: [ZOOKEEPER-1177] Add the memory optimized watch manage...

[lvfangmin]

Github user lvfangmin commented on the issue:

    https://github.com/apache/zookeeper/pull/590
  
    Added JMH micro benchmark for the watch manager:
    
    * It shows **big win** for the watch heavy cases, with the current 
implementation, it uses more than 50MB memory to store 1M watches, with 
WatchManagerOptimized it only uses around 0.2MB. 
    * It also makes add and trigger watches more efficient, since 
WatchManagerOptimized doesn't maintain the reverse map. 
    * In sparse watches use case, the WatchManagerOptimized is expected to use 
a bit more memory because it needs extra effort to maintain those bit set. In 
the test it shows around 10% more memory usage.
    
    Here are more result about the throughput/latency related with 
WatchManager: 
    
    ```
    Benchmark                               (pathCount)    (watchManagerClass)  
(watcherCount)  Mode  Cnt   Score    Error  Units
    WatchBench.testAddConcentrateWatch            10000           WatchManager  
           N/A  avgt    9   5.382 ±  0.968  ms/op
    WatchBench.testAddConcentrateWatch            10000  WatchManagerOptimized  
           N/A  avgt    9   0.696 ±  0.133  ms/op
    WatchBench.testAddSparseWatch                 10000           WatchManager  
         10000  avgt    9   4.889 ±  1.585  ms/op
    WatchBench.testAddSparseWatch                 10000  WatchManagerOptimized  
         10000  avgt    9   4.794 ±  1.068  ms/op
    WatchBench.testTriggerConcentrateWatch            1           WatchManager  
             1  avgt    9  ≈ 10⁻⁴           ms/op
    WatchBench.testTriggerConcentrateWatch            1           WatchManager  
          1000  avgt    9   0.037 ±  0.002  ms/op
    WatchBench.testTriggerConcentrateWatch            1  WatchManagerOptimized  
             1  avgt    9  ≈ 10⁻⁴           ms/op
    WatchBench.testTriggerConcentrateWatch            1  WatchManagerOptimized  
          1000  avgt    9   0.025 ±  0.001  ms/op
    WatchBench.testTriggerConcentrateWatch         1000           WatchManager  
             1  avgt    9   0.048 ±  0.003  ms/op
    WatchBench.testTriggerConcentrateWatch         1000           WatchManager  
          1000  avgt    9  71.838 ±  4.043  ms/op
    WatchBench.testTriggerConcentrateWatch         1000  WatchManagerOptimized  
             1  avgt    9   0.079 ±  0.002  ms/op
    WatchBench.testTriggerConcentrateWatch         1000  WatchManagerOptimized  
          1000  avgt    9  26.135 ±  0.223  ms/op
    WatchBench.testTriggerSparseWatch             10000           WatchManager  
         10000  avgt    9   1.207 ±  0.035  ms/op
    WatchBench.testTriggerSparseWatch             10000  WatchManagerOptimized  
         10000  avgt    9   1.321 ±  0.019  ms/op
    ```
    
    You can try the following command to run the micro benchmark:
    ```
    $ ant clean package
    $ ant clean package -buildfile 
zookeeper-contrib/zookeeper-contrib-fatjar/build.xml
    $ java -jar build/contrib/fatjar/zookeeper-dev-fatjar.jar jmh
    ```
    
    @maoling @anmolnar hope this gives you a more vivid comparison between the 
old and new watch manager implementation.


---