Re: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v12]
On Wed, 7 Jun 2023 14:06:31 GMT, Kelvin Nilsen wrote: > Hi Thomas, > > Thank you for your followup comments. I am in total agreement that it is a > shame the challenges we have faced and the progress we have made is not > better documented in the history of JBS tickets. I have been the worst > offender. I apologize. Please, no need to apologize. I understand that during early development one needs to move quickly. I just thought that your team's experience with tuning Shenandoah is valuable, and it is regrettable when it is lost. > You are correct that the change is to N, the number of times in a row that we > perform degenerated GC before we automatically upgrade to Full GC. It is > still possible that we will upgrade to Full GC before N is reached, because > there are other situations, such as lack of progress by degenerated GC, that > will cause us to upgrade to Full even before N is reached. > > The comment is still valid as written. During degenerated GC, the mutator > threads are all blocked, so the ONLY kind of allocation failure that can > occur during degenerated GC is a GC-worker-thread allocation for the purpose > of evacuating memory. If we experience an "evacuation failure" during > degenerated GC. we will upgrade to Full GC. Thank you for the thorough explanation. - PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1581232734
Re: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v12]
On Wed, 7 Jun 2023 13:37:44 GMT, Kelvin Nilsen wrote: >> OpenJDK Colleagues: >> >> Please review this proposed integration of Generational mode for Shenandoah >> GC under https://bugs.openjdk.org/browse/JDK-8307314. >> >> Generational mode of Shenandoah is enabled by adding >> `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a >> command line that already specifies ` -XX:+UseShenandoahGC`. The >> implementation automatically adjusts the sizes of old generation and young >> generation to efficiently utilize the entire heap capacity. Generational >> mode of Shenandoah resembles G1 in the following regards: >> >> 1. Old-generation marking runs concurrently during the time that multiple >> young generation collections run to completion. >> 2. After old-generation marking completes, we perform a sequence of mixed >> collections. Each mixed collection combines collection of young generation >> with evacuation of a portion of the old-generation regions identified for >> collection based on old-generation marking information. >> 3. Unlike G1, young-generation collections and evacuations are entirely >> concurrent, as with single-generation Shenandoah. >> 4. As with single-generation Shenandoah, there is no explicit notion of eden >> and survivor space within the young generation. In practice, regions that >> were most recently allocated tend to have large amounts of garbage and these >> regions tend to be collected with very little effort. Young-generation >> objects that survive garbage collection tend to accumulate in regions that >> hold survivor objects. These regions tend to have smaller amounts of >> garbage, and are less likely to be collected. If they survive a sufficient >> number of young-generation collections, the “survivor” regions are promoted >> into the old generation. >> >> We expect to refine heuristics as we gain experience with more production >> workloads. In the future, we plan to remove the “experimental” qualifier >> from generational mode, at which time we expect that generational mode will >> become the default mode for Shenandoah. >> >> **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, >> gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, >> HyperAlloc, and multiple AWS production workload simulators. We test on >> Linux x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and >> Windows x64. > > Kelvin Nilsen has updated the pull request incrementally with one additional > commit since the last revision: > > Remove more extraneous copyright notices Hi Thomas, Thank you for your followup comments. I am in total agreement that it is a shame the challenges we have faced and the progress we have made is not better documented in the history of JBS tickets. I have been the worst offender. I apologize. One aspect of this problem is that our work has included a large degree of uncertainty and "research", and it is not always clear to us what needs to be addressed until after we finish and test certain fixes as integrated with a variety of other fixes. We will commit to being more engaged with JBS from this point forward, both for any further work done on the Shenandoah branch, and definitely for work done on tip. You are correct that the change is to N, the number of times in a row that we perform degenerated GC before we automatically upgrade to Full GC. It is still possible that we will upgrade to Full GC before N is reached, because there are other situations, such as lack of progress by degenerated GC, that will cause us to upgrade to Full even before N is reached. The comment is still valid as written. During degenerated GC, the mutator threads are all blocked, so the ONLY kind of allocation failure that can occur during degenerated GC is a GC-worker-thread allocation for the purpose of evacuating memory. If we experience an "evacuation failure" during degenerated GC. we will upgrade to Full GC. - PR Comment: https://git.openjdk.org/jdk/pull/14185#issuecomment-1580896496
Re: RFR: JDK-8307314: Implementation: Generational Shenandoah (Experimental) [v12]
> OpenJDK Colleagues: > > Please review this proposed integration of Generational mode for Shenandoah > GC under https://bugs.openjdk.org/browse/JDK-8307314. > > Generational mode of Shenandoah is enabled by adding > `-XX:+UnlockExperimentalVMOptions -XX:ShenandoahGCMode=generational` to a > command line that already specifies ` -XX:+UseShenandoahGC`. The > implementation automatically adjusts the sizes of old generation and young > generation to efficiently utilize the entire heap capacity. Generational > mode of Shenandoah resembles G1 in the following regards: > > 1. Old-generation marking runs concurrently during the time that multiple > young generation collections run to completion. > 2. After old-generation marking completes, we perform a sequence of mixed > collections. Each mixed collection combines collection of young generation > with evacuation of a portion of the old-generation regions identified for > collection based on old-generation marking information. > 3. Unlike G1, young-generation collections and evacuations are entirely > concurrent, as with single-generation Shenandoah. > 4. As with single-generation Shenandoah, there is no explicit notion of eden > and survivor space within the young generation. In practice, regions that > were most recently allocated tend to have large amounts of garbage and these > regions tend to be collected with very little effort. Young-generation > objects that survive garbage collection tend to accumulate in regions that > hold survivor objects. These regions tend to have smaller amounts of > garbage, and are less likely to be collected. If they survive a sufficient > number of young-generation collections, the “survivor” regions are promoted > into the old generation. > > We expect to refine heuristics as we gain experience with more production > workloads. In the future, we plan to remove the “experimental” qualifier > from generational mode, at which time we expect that generational mode will > become the default mode for Shenandoah. > > **Testing**: We continuously run jtreg tiers 1-4 + hotspot_gc_shenandoah, > gcstress, jck compiler, jck runtime, Dacapo, SpecJBB, SpecVM, Extremem, > HyperAlloc, and multiple AWS production workload simulators. We test on Linux > x64 and aarch64, Alpine x64 and aarch64, macOS x64 and aarch64, and Windows > x64. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Remove more extraneous copyright notices - Changes: - all: https://git.openjdk.org/jdk/pull/14185/files - new: https://git.openjdk.org/jdk/pull/14185/files/88958669..8e5c3b73 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=11 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14185&range=10-11 Stats: 10 lines in 10 files changed: 0 ins; 10 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/14185.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14185/head:pull/14185 PR: https://git.openjdk.org/jdk/pull/14185
