>From Ian Maxon <ima...@uci.edu>: Ian Maxon has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18271 )
Change subject: [ASTERIXDB-3396][CONF] Tweak memory, '-add-opens' flags ...................................................................... [ASTERIXDB-3396][CONF] Tweak memory, '-add-opens' flags - user model changes: no - storage format changes: no - interface changes: no Details: In JDK17, we are encountering module access errors that appear in the log. This is due to not having the right access flags added to the JVM Additionally, the NCService integration tests are failing due to memory overuse. This patch tweaks the default allocation to be 50% instead of 60% of the host memory. Change-Id: I13ec2c1063754b93d1fad551362134c0d1c593d5 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18271 Reviewed-by: Ian Maxon <ima...@uci.edu> Reviewed-by: Michael Blow <mb...@apache.org> Integration-Tests: Jenkins <jenk...@fulliautomatix.ics.uci.edu> Tested-by: Jenkins <jenk...@fulliautomatix.ics.uci.edu> --- M hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/main/java/org/apache/hyracks/control/nc/service/NCService.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/test/java/org/apache/hyracks/control/nc/service/NCServiceTest.java 2 files changed, 45 insertions(+), 3 deletions(-) Approvals: Ian Maxon: Looks good to me, but someone else must approve Michael Blow: Looks good to me, approved Jenkins: Verified; Verified Objections: Anon. E. Moose #1000171: Violations found diff --git a/hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/main/java/org/apache/hyracks/control/nc/service/NCService.java b/hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/main/java/org/apache/hyracks/control/nc/service/NCService.java index 97dc344..96d3ecb 100644 --- a/hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/main/java/org/apache/hyracks/control/nc/service/NCService.java +++ b/hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/main/java/org/apache/hyracks/control/nc/service/NCService.java @@ -31,6 +31,7 @@ import java.net.ServerSocket; import java.net.Socket; import java.util.ArrayList; +import java.util.Arrays; import java.util.Date; import java.util.List; import java.util.Map; @@ -85,6 +86,12 @@ */ private static OperatingSystemMXBean osMXBean; + public static final String[] DEFAULT_ADD_OPENS = + { "--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED", + "--add-opens=java.management/sun.management=ALL-UNNAMED", + "--add-opens=java.base/java.lang=ALL-UNNAMED", "--add-opens=java.base/java.nio=ALL-UNNAMED", + "--add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED" }; + private static List<String> buildCommand() throws IOException { List<String> cList = new ArrayList<>(); @@ -134,12 +141,20 @@ // Sets up memory parameter if it is not specified. if (!jvmargs.contains("-Xmx")) { long ramSize = ((com.sun.management.OperatingSystemMXBean) osMXBean).getTotalPhysicalMemorySize(); - int proportionalRamSize = (int) Math.ceil(0.6 * ramSize / (1024 * 1024)); + int proportionalRamSize = (int) Math.ceil(0.5 * ramSize / (1024 * 1024)); //if under 32bit JVM, use less than 1GB heap by default. otherwise use proportional ramsize. int heapSize = "32".equals(System.getProperty("sun.arch.data.model")) ? (proportionalRamSize <= 1024 ? proportionalRamSize : 1024) : proportionalRamSize; jvmargs = jvmargs + " -Xmx" + heapSize + "m"; } + + // Squelch some module access warnings and errors from JDK9+ + if (!jvmargs.contains("-add-opens")) { + StringBuilder jvmArgsBuilder = new StringBuilder(jvmargs); + Arrays.stream(DEFAULT_ADD_OPENS).map(s -> jvmArgsBuilder.append(s)); + jvmargs = jvmArgsBuilder.toString(); + } + env.put("JAVA_OPTS", jvmargs.trim()); LOGGER.info("Setting JAVA_OPTS to " + jvmargs); } diff --git a/hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/test/java/org/apache/hyracks/control/nc/service/NCServiceTest.java b/hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/test/java/org/apache/hyracks/control/nc/service/NCServiceTest.java index 00d7ca4..bc04320 100644 --- a/hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/test/java/org/apache/hyracks/control/nc/service/NCServiceTest.java +++ b/hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/test/java/org/apache/hyracks/control/nc/service/NCServiceTest.java @@ -42,7 +42,7 @@ String sizeStr = prefix.substring(0, prefix.length() - 1); int size = Integer.parseInt(sizeStr); long ramSize = ((com.sun.management.OperatingSystemMXBean) osMXBean).getTotalPhysicalMemorySize(); - int base = 1024 * 1024 * 5; - Assert.assertTrue(size == ramSize * 3 / base + ((ramSize * 3) % base == 0 ? 0 : 1)); + int base = 1024 * 1024; + Assert.assertTrue(size == (ramSize / 2) / base + (((ramSize / 2) % base) == 0 ? 0 : 1)); } } -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18271 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: trinity Gerrit-Change-Id: I13ec2c1063754b93d1fad551362134c0d1c593d5 Gerrit-Change-Number: 18271 Gerrit-PatchSet: 4 Gerrit-Owner: Ian Maxon <ima...@uci.edu> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ian Maxon <ima...@uci.edu> Gerrit-Reviewer: Jenkins <jenk...@fulliautomatix.ics.uci.edu> Gerrit-Reviewer: Michael Blow <mb...@apache.org> Gerrit-Reviewer: Wail Alkowaileet <wael....@gmail.com> Gerrit-MessageType: merged
