----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47283/ -----------------------------------------------------------
Review request for samza, Boris Shkolnik and Yi Pan (Data Infrastructure). Repository: samza Description ------- Continuation of the Refactoring from RB:44920: 1. SamzaAppState.finishedContainers was declared as a Set. The UI thread invokes the size() on the set, while others could modify the set causing a race. Since, the only place finishedContainers is read is from the UI, we can simply use an AtomicInteger to keep track of the counts. (instead of the Set). 2. Changed all the UI and scaml files to work with the refactored code as in https://reviews.apache.org/r/44920 3. Modify the AMRest servlets, and AMWeb servlets to work with the refactoring as in https://reviews.apache.org/r/44920 . We now have the AppMaster UI with the refactored patch :-) 3. Deleted several classes in the old code-path. Maintaining 2 code-paths increases tech debt. 4. Refactored out some MockClasses so that they can be re-used else-where too. 5. Refactored and Rewrote unit tests to work with the RB:44920 Diffs ----- samza-core/src/main/java/org/apache/samza/clustermanager/AbstractContainerAllocator.java PRE-CREATION samza-core/src/main/java/org/apache/samza/clustermanager/ContainerProcessManager.java PRE-CREATION samza-core/src/main/java/org/apache/samza/clustermanager/SamzaApplicationState.java PRE-CREATION samza-core/src/main/scala/org/apache/samza/metrics/ContainerProcessManagerMetrics.scala PRE-CREATION samza-core/src/test/java/org/apache/samza/clustermanager/TestContainerAllocator.java PRE-CREATION samza-core/src/test/java/org/apache/samza/clustermanager/TestContainerProcessManager.java PRE-CREATION samza-core/src/test/java/org/apache/samza/clustermanager/TestHostAwareContainerAllocator.java PRE-CREATION samza-shell/src/main/bash/run-am.sh 9545a96953baaff17ad14962e02bc12aadbb1101 samza-yarn/src/main/java/org/apache/samza/job/yarn/AbstractContainerAllocator.java b4789e62beb1120f11a8101664b10c34ae930e58 samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerAllocator.java 24ac41067e34cf5a445bb036db9ea324eaafa7df samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerFailure.java 1d15651096e52a2e323a8e2f658fad3ea8e9c709 samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerRequestState.java 57ce35099973b7fc3414c450e3246cb9f204289b samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerUtil.java e8976bce934a324c48475bcd64d392119cc44b40 samza-yarn/src/main/java/org/apache/samza/job/yarn/HostAwareContainerAllocator.java 1d101fa80e6367d54f06455c242222195e4c0091 samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaAppState.java 77280bab8aeb242b34b5b780c84e6deab1a45f51 samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaContainerRequest.java 4a04eb6b3b054ee85988e1b26ececc800bbc7861 samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaTaskManager.java caee6e6c182d3cf86bd4fe193f8b1797605b2480 samza-yarn/src/main/java/org/apache/samza/job/yarn/YarnAppState.java PRE-CREATION samza-yarn/src/main/resources/scalate/WEB-INF/views/index.scaml 93660c73e507b1a6076e743fcec56a41d22a5f39 samza-yarn/src/main/scala/org/apache/samza/job/yarn/SamzaAppMaster.scala 80deb3b18c094d83af67535f9d0156f18ae3f5e4 samza-yarn/src/main/scala/org/apache/samza/job/yarn/SamzaAppMasterLifecycle.scala 2a5c0d8092aa6f0a7f9b1f5fa56f9f4d4919d579 samza-yarn/src/main/scala/org/apache/samza/job/yarn/SamzaAppMasterMetrics.scala 054d8b68033535ff0cab7cca84b71455e201a715 samza-yarn/src/main/scala/org/apache/samza/job/yarn/SamzaAppMasterService.scala 3adf86f83123f6d66fc18ef9feed95b551c8398f samza-yarn/src/main/scala/org/apache/samza/job/yarn/SamzaYarnAppMasterService.scala PRE-CREATION samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala 62ddb261f0b8f0f24468875d6d84da05908a3c62 samza-yarn/src/main/scala/org/apache/samza/webapp/ApplicationMasterRestServlet.scala a40ab72a71c4fa82bd87aa03caec8936e609bf68 samza-yarn/src/main/scala/org/apache/samza/webapp/ApplicationMasterWebServlet.scala 605332a02f365248fc9ec3525ed9d3721db71e8d samza-yarn/src/test/java/org/apache/samza/job/yarn/TestContainerAllocator.java e21aded3c5e302593ed1cb5675da81aa6e3e743f samza-yarn/src/test/java/org/apache/samza/job/yarn/TestContainerAllocatorCommon.java 5badd29af601ce0a046ef7dce4739e31514c4e63 samza-yarn/src/test/java/org/apache/samza/job/yarn/TestContainerRequestState.java 402fe784120ac40ed542d9fa60d6a6d7df9c8cda samza-yarn/src/test/java/org/apache/samza/job/yarn/TestHostAwareContainerAllocator.java ead7200487f27ed31e30576721ef16a064a28bc7 samza-yarn/src/test/java/org/apache/samza/job/yarn/TestSamzaContainerRequest.java ad0f4d3d9db9111a7c0087b6da9c724dcc736726 samza-yarn/src/test/java/org/apache/samza/job/yarn/TestSamzaTaskManager.java faa697db49ec1c11d76c88d919a356a5ae409a15 samza-yarn/src/test/java/org/apache/samza/job/yarn/util/TestUtil.java d4c9c96144c58e1acc53726265a452f2467cd554 samza-yarn/src/test/scala/org/apache/samza/job/yarn/MockSystemAdmin.scala PRE-CREATION samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMaster.scala 3f056c4d39fc8e3a2ed42fbcc73aa6eca1a2287c samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMasterLifecycle.scala 30cf34fe1fd3f74537d16e8a51b467cd50835357 samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMasterService.scala 7f5d9f4af088589d4287c26737bae25567c157d7 Diff: https://reviews.apache.org/r/47283/diff/ Testing ------- Added Unit tests. Manually tested with a sample hello-world job. Tested that the UI was showing up correctly and template files were being populated. Thanks, Jagadish Venkatraman
