Re: RFR: 8235637: jhsdb jmap from OpenJDK 11.0.5 doesn't work if prelink is enabled
Hi Fairoz, Looks good! I want you to backport this change to both jdk11u and 8u. Thanks, Yasumasa On 2019/12/12 12:10, Fairoz Matte wrote: Hi, Please review this small change, Updating error handling, to make sure "lib_base_diff = 0" is still a valid scenario even after calc_prelinked_load_address() call. JBS - https://bugs.openjdk.java.net/browse/JDK-8235637 Webrev - http://cr.openjdk.java.net/~fmatte/8235637/webrev.00/ This patch is provided by Yasumasa Suenaga Thanks, Fairoz
Re: RFR: 8226575: OperatingSystemMXBean should be made container aware
Hi Bob, David, Mandy, and Serguei, Thank you for reviewing this change! Best regards, Daniil On 12/11/19, 9:21 AM, "Bob Vandette" wrote: Yes, I defer to Mandy on the best way to express the various Java exceptions. I’m ok with the changes. Thanks for getting this done for JDK14! Bob. > On Dec 11, 2019, at 12:12 PM, Daniil Titov wrote: > > Hi Serguei, > > Thank you for your comments. I will correct this nits before pushing the changes. > > Hi Bob and David, > >> [Mandy Chung] >>> I reviewed Metrics and Subsystem in this version. >>> I don't need to see a new webrev. > > As I understood Mandy finished reviewing this fix. Just wanted to confirm with you if you are okey with that version of the fix (webrev.06) ? > > Mach5 testing: tier1-tier6 and open/test/hotspot/jtreg/containers/docker tests passed. > > Thank you, > Daniil > > > > On 12/9/19, 6:02 PM, "serguei.spit...@oracle.com" wrote: > >Hi Daniil, > >It is not a full review, just some minor comments. >In fact, I do not see real problems yet. > > http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java.frames.html > > 55 public long getTotalSwapSpaceSize() { > 56 if (containerMetrics != null) { > 57 long limit = containerMetrics.getMemoryAndSwapLimit(); > 58 // The memory limit metrics is not available if JVM >runs on Linux host ( not in a docker container) > 59 // or if a docker container was started without >specifying a memory limit ( without '--memory=' > 60 // Docker option). In latter case there is no limit on >how much memory the container can use and > 61 // it can use as much memory as the host's OS allows. > 62 long memLimit = containerMetrics.getMemoryLimit(); > 63 if (limit >= 0 && memLimit >= 0) { > 64 return limit - memLimit; > 65 } > 66 } > 67 return getTotalSwapSpaceSize0(); > 68 } > > Unneeded space after brackets '('. > Do we need to check if the (limit - memLimit) value is negative? > The same question is for getFreeSwapSpaceSize(): > memSwapLimit - memLimit - (memSwapUsage - memUsage) > > and getFreeMemorySize(): > 101 return limit - usage; > > 81 // If this happens just retry the loop for >a few iterations > > Dot is missed at the end of comment. > > > http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/test/hotspot/jtreg/containers/docker/CheckOperatingSystemMXBean.java.html > > 34 System.out.println(String.format("Runtime.availableProcessors: >%d", Runtime.getRuntime().availableProcessors())); > 35 > System.out.println(String.format("OperatingSystemMXBean.getAvailableProcessors: >%d", osBean.getAvailableProcessors())); > 36 > System.out.println(String.format("OperatingSystemMXBean.getTotalMemorySize: >%d", osBean.getTotalMemorySize())); > 37 > System.out.println(String.format("OperatingSystemMXBean.getTotalPhysicalMemorySize: >%d", osBean.getTotalPhysicalMemorySize())); > 38 > System.out.println(String.format("OperatingSystemMXBean.getFreeMemorySize: >%d", osBean.getFreeMemorySize())); > 39 > System.out.println(String.format("OperatingSystemMXBean.getFreePhysicalMemorySize: >%d", osBean.getFreePhysicalMemorySize())); > 40 > System.out.println(String.format("OperatingSystemMXBean.getTotalSwapSpaceSize: >%d", osBean.getTotalSwapSpaceSize())); > 41 > System.out.println(String.format("OperatingSystemMXBean.getFreeSwapSpaceSize: >%d", osBean.getFreeSwapSpaceSize())); > 42 >System.out.println(String.format("OperatingSystemMXBean.getCpuLoad: %f", >osBean.getCpuLoad())); > 43 > System.out.println(String.format("OperatingSystemMXBean.getSystemCpuLoad: >%f", osBean.getSystemCpuLoad())); > > > To make the above lines a little bit shorter I'd suggest to define a >log() method like this: > private static void log(String msg) ( System.out.println(msg(; } > > 34 log(String.format("Runtime.availableProcessors: %d", >Runtime.getRuntime().availableProcessors())); > 35 log(String.format("OperatingSystemMXBean.getAvailableProcessors: >%d", osBean.getAvai
RFR: 8235637: jhsdb jmap from OpenJDK 11.0.5 doesn't work if prelink is enabled
Hi, Please review this small change, Updating error handling, to make sure "lib_base_diff = 0" is still a valid scenario even after calc_prelinked_load_address() call. JBS - https://bugs.openjdk.java.net/browse/JDK-8235637 Webrev - http://cr.openjdk.java.net/~fmatte/8235637/webrev.00/ This patch is provided by Yasumasa Suenaga Thanks, Fairoz
Re: Removal of SA javascript support
I discussed with Kris about this in OpenJDK Committers' Workshop last year. In case of .NET Core, SOS is provided to integrate runtime debugging feature to native debugger. If same feature will be provided, I'm very happy! For example, GDB and WinDbg provides remote debug server. If (CL)HSDB can connect to native debugger, we can gather data in Java layer via them. I think we can delegate most of memory access to native debugger. Also we can run custom scripts on GDB. Then I think we need minimum support for Java call frames, OOP, and SymbolTable. Yasumasa On 2019/12/11 21:47, sundararajan.athijegannat...@oracle.com wrote: Effectively you're asking for SA as API. I don't think that is a good idea. That implies supporting hotspot data structures as Java *API*. That will be maintainability nightmare - we've to keep tracking hotspot data structures in SA code. That itself is problematic. API would be next level nightmare. -Sundar On 11/12/19 11:57 am, Yasumasa Suenaga wrote: Hi, IMHO we need to export all packages in SA if we do not provide new API for SA. sa.js in jdk.hotspot.agent could access all SA classes until JDK 8 (before Jigsaw), so we could make various functions if we need. OTOH we cannot know what classes are needed by the SA users. All packages in jdk.hotspot.agent module provides features, and they require other packages. For example, sun.jvm.hotspot.oops.Oop requires sun.jvm.hotspot.types, and it requires sun.jvm.hotspot.debugger . It is difficult to track and to export minimally. (I worked for it in JDK-8157947, but I gave up...) Thus I guess it is a big challenge to export SA classes without refactoring. If we provide new API for SA plugin, I guess we need to work some refactoring. Yasumasa On 2019/12/11 15:00, Chris Plummer wrote: On 12/10/19 9:56 PM, Yasumasa Suenaga wrote: On 2019/12/11 14:39, Krystal Mok wrote: Hi Yasumasa, That's a very nice idea. Basically what you're asking for is exposing the Command interface [1] so that plugins can implement it and get dynamically loaded / registered into CLHSDB / HSDB, right? Yes, but we also need proxy API to access internal SA objects e.g. CodeCache, JavaThread, TypeDataBase, etc... Yes, or export them. I should have read this email before posting my previous one. Chris Yasumasa [1]: http://hg.openjdk.java.net/jdk/jdk/file/c71ec1f09f21/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/CommandProcessor.java#l246 - Kris On Tue, Dec 10, 2019 at 9:33 PM Yasumasa Suenaga mailto:suen...@oss.nttdata.com>> wrote: Hi Chris, It's a sad proposal, but I agree with you. To maintain SA in JS is difficult since Jigsaw. However I want SA to implement pluggable feature. I use custom script to list compiled codes in CodeCache. I guess other troubleshooters also want similar feature (via jsload) in future if they encounter JVM crash. Thanks, Yasumasa On 2019/12/11 11:52, Chris Plummer wrote: > Hi, > > I like to propose the removal of SA javascript support. Few people even realize this support exists, and hopefully even fewer are using it since I'd like to remove it. Since I'm new to this myself, let me first explain what I know about it's existence, and then explain why I want to remove it. > > If you run "jhsdb clhsdb", there are jsload and jseval commands. Don't look for them in anything post JDK 8. I'll explain why later. jsload is used to load a javascript file. In that file you can register new clhsdb commands that are written in javascript. You can also evaluate javascript using the jseval command. Some of this is explained in [1], which is the only place I can find any reference to this support. It does not appear to be officially supported, nor is there any oracle provided documentation. > > There also appear to be a few clhsdb commands that are written in javascript. Doing a grep for "registerCommand" in sa.js shows the following: > > registerCommand("class", "class name", "jclass"); > registerCommand("classes", "classes", "jclasses"); > registerCommand("dumpclass", "dumpclass { address | name } [ directory ]", "dclass"); > registerCommand("dumpheap", "dumpheap [ file ]", "dumpHeap"); > registerCommand("mem", "mem address [ length ]", "printMem"); > registerCommand("sysprops", "sysprops", "sysProps"); > registerCommand("whatis", "whatis address", "printWhatis"); > > Once again, don't go looking for these in anything newer than JDK8. You won't find them. Again the only documentation I can fine is [1]. > > The other use of Javascript is the SOQL command (Simple Object Query Language), a tool used to query the heap, and also the JSDB command. The only SOQL documentation I could find is the blog reference [2]. I could not find HSDB documentation, but I believe is is a javascript support for looking at hotspot. So once again, neither of these seem t
Re: PING: RFR: 8234624: jstack mixed mode should refer DWARF
Hi Dmitry, Thanks for your comment! On 2019/12/12 0:34, Dmitry Samersoff wrote: Hello Yasumasa, Please, 1. Consider to use mmap for reading elf sections. Did you pointed `read_section_data()`? lib->eh_frame.data = read_section_data(lib->fd, &ehdr, sh); I do not change implementation of `read_section_data()`. If you want to change to use mmap, I think it should be fixed as another issue. 2. Please move all platfrom-specific parts of native code to a separate file/directory. Current patch will brake AARCH64 build. Unfortunately JDK libraries (shared libraries excepts HotSpot) seem not to care CPU type in makefiles. http://hg.openjdk.java.net/jdk/jdk/file/f22d91b2d072/make/common/JdkNativeCompilation.gmk#l38 I believe my patch do not call platform-specific function(s). Can you share your concern? 3. I didn't find any tests here. How did your test the changes? It can be tested in TestJhsdbJstackMixed and ClhsdbPstack whether mixed jstack can work without error. We can add the test whether native frames exist in the result, but I found same issue on Windows. So I do not want to add it now. libproc_impl.c 131: If is not necessary, free handles NULLPTR gracefully. Thanks, I will fix it. Yasumasa -Dmitry On 04.12.19 03:54, Yasumasa Suenaga wrote: PING: Could you review it? JBS: https://bugs.openjdk.java.net/browse/JDK-8234624 webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.01/ This bug is targeted to JDK 14. Thanks, Yasumasa On 2019/11/28 21:39, Yasumasa Suenaga wrote: Hi, I refactored LinuxAMD64CFrame.java . It works fine in serviceability/sa tests and all tests on submit repo (mach5-one-ysuenaga-JDK-8234624-2-20191128-0928-7059923). Could you review new webrev? http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.01/ The diff from previous webrev is here: http://hg.openjdk.java.net/jdk/submit/rev/4bc47efbc90b Thanks, Yasumasa On 2019/11/25 14:08, Yasumasa Suenaga wrote: Hi all, Please review this change: JBS: https://bugs.openjdk.java.net/browse/JDK-8234624 webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.00/ According to 2.7 Stack Unwind Algorithm in System V Application Binary Interface AMD64 Architecture Processor Supplement [1], we need to use DWARF in .eh_frame or .debug_frame for stack unwinding. As JDK-8022183 said, omit-frame-pointer is enabled by default since GCC 4.6, so system library (e.g. libc) might be compiled with this feature. However `jhsdb jstack --mixed` does not do so, it uses base pointer register (RBP). So it might be lack of stack frames. I guess JDK-8219201 is caused by same issue. Thanks, Yasumasa [1] https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf
Re: RFR: 8226575: OperatingSystemMXBean should be made container aware
Hi Daniil, Got it, thanks! Serguei On 12/11/19 15:33, Daniil Titov wrote: Hi Serguei, Thank you for reviewing this change. Just wanted to add that the only "volatile" metrics are "usage" ones ( memoryUsage and memoryAndSwapLimit). The "limit" metrics (memoryLimit and memoryAndSwapLimit) are set when the container starts and are not subjects to change. The only method that reads more than one "volatile" metric is getFreeSwapSpaceSize() and it has a code that retries if the calculated swapUsage is negative as a result of non-atomic reads. Thank you, Daniil On 12/11/19, 3:13 PM, "serguei.spit...@oracle.com" wrote: Hi Daniil, One my concerns was a non-atomic read of multiple metrics before comparison. It creates a potential to get a mismatch in result. However, the probability to get a negative value is pretty low, I think. The other concern (if incorrect metrics are returned) is covered by JDK-8235522. Revising all concerns in JDK-8235522 sounds good to me. Thanks, Serguei On 12/10/19 10:29, Daniil Titov wrote: > Hi Serguei, > >>Do we need to check if the (limit - memLimit) value is negative? >>The same question is for getFreeSwapSpaceSize(): >> memSwapLimit - memLimit - (memSwapUsage - memUsage) >> >>and getFreeMemorySize(): >> 101 return limit - usage; > I don't think we need such check here. If it happens in fact it means the serious system malfunction and a negative value this method > returns would indicate this (currently the native methods already returns -1 if something went wrong). But we could revise it in the follow > up issue I created for that [1]. > > [1] https://bugs.openjdk.java.net/browse/JDK-8235522 > > Thank you, > Daniil > > On 12/9/19, 6:02 PM, "serguei.spit...@oracle.com" wrote: > > Hi Daniil, > > It is not a full review, just some minor comments. > In fact, I do not see real problems yet. > > http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java.frames.html > > 55 public long getTotalSwapSpaceSize() { > 56 if (containerMetrics != null) { > 57 long limit = containerMetrics.getMemoryAndSwapLimit(); > 58 // The memory limit metrics is not available if JVM > runs on Linux host ( not in a docker container) > 59 // or if a docker container was started without > specifying a memory limit ( without '--memory=' > 60 // Docker option). In latter case there is no limit on > how much memory the container can use and > 61 // it can use as much memory as the host's OS allows. > 62 long memLimit = containerMetrics.getMemoryLimit(); > 63 if (limit >= 0 && memLimit >= 0) { > 64 return limit - memLimit; > 65 } > 66 } > 67 return getTotalSwapSpaceSize0(); > 68 } > > Unneeded space after brackets '('. > Do we need to check if the (limit - memLimit) value is negative? > The same question is for getFreeSwapSpaceSize(): > memSwapLimit - memLimit - (memSwapUsage - memUsage) > > and getFreeMemorySize(): > 101 return limit - usage; > > 81 // If this happens just retry the loop for > a few iterations > > Dot is missed at the end of comment. > > > http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/test/hotspot/jtreg/containers/docker/CheckOperatingSystemMXBean.java.html > > 34 System.out.println(String.format("Runtime.availableProcessors: > %d", Runtime.getRuntime().availableProcessors())); > 35 > System.out.println(String.format("OperatingSystemMXBean.getAvailableProcessors: > %d", osBean.getAvailableProcessors())); > 36 > System.out.println(String.format("OperatingSystemMXBean.getTotalMemorySize: > %d", osBean.getTotalMemorySize())); > 37 > System.out.println(String.format("OperatingSystemMXBean.getTotalPhysicalMemorySize: > %d", osBean.getTotalPhysicalMemorySize())); > 38 > System.out.println(String.format("OperatingSystemMXBean.getFreeMemorySize: > %d", osBean.getFreeMemorySize())); > 39 > System.out.println(String.format("OperatingSystemMXBean.getFreePhysicalMemorySize: > %d", osBean.getFreePhysicalMemorySiz
Re: RFR: 8226575: OperatingSystemMXBean should be made container aware
Hi Serguei, Thank you for reviewing this change. Just wanted to add that the only "volatile" metrics are "usage" ones ( memoryUsage and memoryAndSwapLimit). The "limit" metrics (memoryLimit and memoryAndSwapLimit) are set when the container starts and are not subjects to change. The only method that reads more than one "volatile" metric is getFreeSwapSpaceSize() and it has a code that retries if the calculated swapUsage is negative as a result of non-atomic reads. Thank you, Daniil On 12/11/19, 3:13 PM, "serguei.spit...@oracle.com" wrote: Hi Daniil, One my concerns was a non-atomic read of multiple metrics before comparison. It creates a potential to get a mismatch in result. However, the probability to get a negative value is pretty low, I think. The other concern (if incorrect metrics are returned) is covered by JDK-8235522. Revising all concerns in JDK-8235522 sounds good to me. Thanks, Serguei On 12/10/19 10:29, Daniil Titov wrote: > Hi Serguei, > >>Do we need to check if the (limit - memLimit) value is negative? >>The same question is for getFreeSwapSpaceSize(): >> memSwapLimit - memLimit - (memSwapUsage - memUsage) >> >>and getFreeMemorySize(): >> 101 return limit - usage; > I don't think we need such check here. If it happens in fact it means the serious system malfunction and a negative value this method > returns would indicate this (currently the native methods already returns -1 if something went wrong). But we could revise it in the follow > up issue I created for that [1]. > > [1] https://bugs.openjdk.java.net/browse/JDK-8235522 > > Thank you, > Daniil > > On 12/9/19, 6:02 PM, "serguei.spit...@oracle.com" wrote: > > Hi Daniil, > > It is not a full review, just some minor comments. > In fact, I do not see real problems yet. > > http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java.frames.html > > 55 public long getTotalSwapSpaceSize() { > 56 if (containerMetrics != null) { > 57 long limit = containerMetrics.getMemoryAndSwapLimit(); > 58 // The memory limit metrics is not available if JVM > runs on Linux host ( not in a docker container) > 59 // or if a docker container was started without > specifying a memory limit ( without '--memory=' > 60 // Docker option). In latter case there is no limit on > how much memory the container can use and > 61 // it can use as much memory as the host's OS allows. > 62 long memLimit = containerMetrics.getMemoryLimit(); > 63 if (limit >= 0 && memLimit >= 0) { > 64 return limit - memLimit; > 65 } > 66 } > 67 return getTotalSwapSpaceSize0(); > 68 } > > Unneeded space after brackets '('. > Do we need to check if the (limit - memLimit) value is negative? > The same question is for getFreeSwapSpaceSize(): > memSwapLimit - memLimit - (memSwapUsage - memUsage) > > and getFreeMemorySize(): > 101 return limit - usage; > > 81 // If this happens just retry the loop for > a few iterations > > Dot is missed at the end of comment. > > > http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/test/hotspot/jtreg/containers/docker/CheckOperatingSystemMXBean.java.html > > 34 System.out.println(String.format("Runtime.availableProcessors: > %d", Runtime.getRuntime().availableProcessors())); > 35 > System.out.println(String.format("OperatingSystemMXBean.getAvailableProcessors: > %d", osBean.getAvailableProcessors())); > 36 > System.out.println(String.format("OperatingSystemMXBean.getTotalMemorySize: > %d", osBean.getTotalMemorySize())); > 37 > System.out.println(String.format("OperatingSystemMXBean.getTotalPhysicalMemorySize: > %d", osBean.getTotalPhysicalMemorySize())); > 38 > System.out.println(String.format("OperatingSystemMXBean.getFreeMemorySize: > %d", osBean.getFreeMemorySize())); > 39 > System.out.println(String.format("OperatingSystemMXBean.getFreePhysicalMemorySize: > %d", osBean.getFreePhysicalMemorySize())); > 40 > System.out.println(String.format("OperatingSystemMXBean.getTotalSwapSp
Re: RFR: 8226575: OperatingSystemMXBean should be made container aware
Typo fixed... .. that the only "volatile" metrics are "usage" ones ( memoryUsage and *memoryAndSwapUsage*). Best regards, Daniil On 12/11/19, 3:33 PM, "Daniil Titov" wrote: Hi Serguei, Thank you for reviewing this change. Just wanted to add that the only "volatile" metrics are "usage" ones ( memoryUsage and memoryAndSwapLimit). The "limit" metrics (memoryLimit and memoryAndSwapLimit) are set when the container starts and are not subjects to change. The only method that reads more than one "volatile" metric is getFreeSwapSpaceSize() and it has a code that retries if the calculated swapUsage is negative as a result of non-atomic reads. Thank you, Daniil On 12/11/19, 3:13 PM, "serguei.spit...@oracle.com" wrote: Hi Daniil, One my concerns was a non-atomic read of multiple metrics before comparison. It creates a potential to get a mismatch in result. However, the probability to get a negative value is pretty low, I think. The other concern (if incorrect metrics are returned) is covered by JDK-8235522. Revising all concerns in JDK-8235522 sounds good to me. Thanks, Serguei On 12/10/19 10:29, Daniil Titov wrote: > Hi Serguei, > >>Do we need to check if the (limit - memLimit) value is negative? >>The same question is for getFreeSwapSpaceSize(): >> memSwapLimit - memLimit - (memSwapUsage - memUsage) >> >>and getFreeMemorySize(): >> 101 return limit - usage; > I don't think we need such check here. If it happens in fact it means the serious system malfunction and a negative value this method > returns would indicate this (currently the native methods already returns -1 if something went wrong). But we could revise it in the follow > up issue I created for that [1]. > > [1] https://bugs.openjdk.java.net/browse/JDK-8235522 > > Thank you, > Daniil > > On 12/9/19, 6:02 PM, "serguei.spit...@oracle.com" wrote: > > Hi Daniil, > > It is not a full review, just some minor comments. > In fact, I do not see real problems yet. > > http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java.frames.html > > 55 public long getTotalSwapSpaceSize() { > 56 if (containerMetrics != null) { > 57 long limit = containerMetrics.getMemoryAndSwapLimit(); > 58 // The memory limit metrics is not available if JVM > runs on Linux host ( not in a docker container) > 59 // or if a docker container was started without > specifying a memory limit ( without '--memory=' > 60 // Docker option). In latter case there is no limit on > how much memory the container can use and > 61 // it can use as much memory as the host's OS allows. > 62 long memLimit = containerMetrics.getMemoryLimit(); > 63 if (limit >= 0 && memLimit >= 0) { > 64 return limit - memLimit; > 65 } > 66 } > 67 return getTotalSwapSpaceSize0(); > 68 } > > Unneeded space after brackets '('. > Do we need to check if the (limit - memLimit) value is negative? > The same question is for getFreeSwapSpaceSize(): > memSwapLimit - memLimit - (memSwapUsage - memUsage) > > and getFreeMemorySize(): > 101 return limit - usage; > > 81 // If this happens just retry the loop for > a few iterations > > Dot is missed at the end of comment. > > > http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/test/hotspot/jtreg/containers/docker/CheckOperatingSystemMXBean.java.html > > 34 System.out.println(String.format("Runtime.availableProcessors: > %d", Runtime.getRuntime().availableProcessors())); > 35 > System.out.println(String.format("OperatingSystemMXBean.getAvailableProcessors: > %d", osBean.getAvailableProcessors())); > 36 > System.out.println(String.format("OperatingSystemMXBean.getTotalMemorySize: > %d", osBean.getTotalMemorySize()));
Re: RFR: 8226575: OperatingSystemMXBean should be made container aware
Hi Daniil, One my concerns was a non-atomic read of multiple metrics before comparison. It creates a potential to get a mismatch in result. However, the probability to get a negative value is pretty low, I think. The other concern (if incorrect metrics are returned) is covered by JDK-8235522. Revising all concerns in JDK-8235522 sounds good to me. Thanks, Serguei On 12/10/19 10:29, Daniil Titov wrote: Hi Serguei, Do we need to check if the (limit - memLimit) value is negative? The same question is for getFreeSwapSpaceSize(): memSwapLimit - memLimit - (memSwapUsage - memUsage) and getFreeMemorySize(): 101 return limit - usage; I don't think we need such check here. If it happens in fact it means the serious system malfunction and a negative value this method returns would indicate this (currently the native methods already returns -1 if something went wrong). But we could revise it in the follow up issue I created for that [1]. [1] https://bugs.openjdk.java.net/browse/JDK-8235522 Thank you, Daniil On 12/9/19, 6:02 PM, "serguei.spit...@oracle.com" wrote: Hi Daniil, It is not a full review, just some minor comments. In fact, I do not see real problems yet. http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java.frames.html 55 public long getTotalSwapSpaceSize() { 56 if (containerMetrics != null) { 57 long limit = containerMetrics.getMemoryAndSwapLimit(); 58 // The memory limit metrics is not available if JVM runs on Linux host ( not in a docker container) 59 // or if a docker container was started without specifying a memory limit ( without '--memory=' 60 // Docker option). In latter case there is no limit on how much memory the container can use and 61 // it can use as much memory as the host's OS allows. 62 long memLimit = containerMetrics.getMemoryLimit(); 63 if (limit >= 0 && memLimit >= 0) { 64 return limit - memLimit; 65 } 66 } 67 return getTotalSwapSpaceSize0(); 68 } Unneeded space after brackets '('. Do we need to check if the (limit - memLimit) value is negative? The same question is for getFreeSwapSpaceSize(): memSwapLimit - memLimit - (memSwapUsage - memUsage) and getFreeMemorySize(): 101 return limit - usage; 81 // If this happens just retry the loop for a few iterations Dot is missed at the end of comment. http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/test/hotspot/jtreg/containers/docker/CheckOperatingSystemMXBean.java.html 34 System.out.println(String.format("Runtime.availableProcessors: %d", Runtime.getRuntime().availableProcessors())); 35 System.out.println(String.format("OperatingSystemMXBean.getAvailableProcessors: %d", osBean.getAvailableProcessors())); 36 System.out.println(String.format("OperatingSystemMXBean.getTotalMemorySize: %d", osBean.getTotalMemorySize())); 37 System.out.println(String.format("OperatingSystemMXBean.getTotalPhysicalMemorySize: %d", osBean.getTotalPhysicalMemorySize())); 38 System.out.println(String.format("OperatingSystemMXBean.getFreeMemorySize: %d", osBean.getFreeMemorySize())); 39 System.out.println(String.format("OperatingSystemMXBean.getFreePhysicalMemorySize: %d", osBean.getFreePhysicalMemorySize())); 40 System.out.println(String.format("OperatingSystemMXBean.getTotalSwapSpaceSize: %d", osBean.getTotalSwapSpaceSize())); 41 System.out.println(String.format("OperatingSystemMXBean.getFreeSwapSpaceSize: %d", osBean.getFreeSwapSpaceSize())); 42 System.out.println(String.format("OperatingSystemMXBean.getCpuLoad: %f", osBean.getCpuLoad())); 43 System.out.println(String.format("OperatingSystemMXBean.getSystemCpuLoad: %f", osBean.getSystemCpuLoad())); To make the above lines a little bit shorter I'd suggest to define a log() method like this: private static void log(String msg) ( System.out.println(msg(; } 34 log(String.format("Runtime.availableProcessors: %d", Runtime.getRuntime().availableProcessors())); 35 log(String.format("OperatingSystemMXBean.getAvailableProcessors: %d", osBean.getAvailableProcessors())); 36 log(String.format("OperatingSystemMXBean.getTotalMemorySize: %d", osBean.getTotalMemorySize())); 37 log(String.format("OperatingSystemMXBean.getTotalPhysicalMemorySize: %d"
Re: Removal of SA javascript support
Regarding maintaining hotspot data structures in SA, I think we often break it without knowing, especially when we are adding data structures that are not currently exposed by SA. Does anyone have a sense of the state of SA in newer versions of the JDK. Is SA still doing what you expect, or do you see a declining level of usefulness because SA is getting more out-of-sync? Thanks On 12/11/19 7:03 AM, Dmitry Samersoff wrote: Sundar, Supporting hotspot data structure in SA is already a maintenance nightmare ;) So we can consider to provide high level API, like find_class_by_name to script writer. It allows anybody who are interesting with quick prototyping write his own program on top of SA with any language they want. -Dmitry On 11.12.19 15:47, sundararajan.athijegannat...@oracle.com wrote: Effectively you're asking for SA as API. I don't think that is a good idea. That implies supporting hotspot data structures as Java *API*. That will be maintainability nightmare - we've to keep tracking hotspot data structures in SA code. That itself is problematic. API would be next level nightmare. -Sundar On 11/12/19 11:57 am, Yasumasa Suenaga wrote: Hi, IMHO we need to export all packages in SA if we do not provide new API for SA. sa.js in jdk.hotspot.agent could access all SA classes until JDK 8 (before Jigsaw), so we could make various functions if we need. OTOH we cannot know what classes are needed by the SA users. All packages in jdk.hotspot.agent module provides features, and they require other packages. For example, sun.jvm.hotspot.oops.Oop requires sun.jvm.hotspot.types, and it requires sun.jvm.hotspot.debugger . It is difficult to track and to export minimally. (I worked for it in JDK-8157947, but I gave up...) Thus I guess it is a big challenge to export SA classes without refactoring. If we provide new API for SA plugin, I guess we need to work some refactoring. Yasumasa On 2019/12/11 15:00, Chris Plummer wrote: On 12/10/19 9:56 PM, Yasumasa Suenaga wrote: On 2019/12/11 14:39, Krystal Mok wrote: Hi Yasumasa, That's a very nice idea. Basically what you're asking for is exposing the Command interface [1] so that plugins can implement it and get dynamically loaded / registered into CLHSDB / HSDB, right? Yes, but we also need proxy API to access internal SA objects e.g. CodeCache, JavaThread, TypeDataBase, etc... Yes, or export them. I should have read this email before posting my previous one. Chris Yasumasa [1]: http://hg.openjdk.java.net/jdk/jdk/file/c71ec1f09f21/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/CommandProcessor.java#l246 - Kris On Tue, Dec 10, 2019 at 9:33 PM Yasumasa Suenaga mailto:suen...@oss.nttdata.com>> wrote: Hi Chris, It's a sad proposal, but I agree with you. To maintain SA in JS is difficult since Jigsaw. However I want SA to implement pluggable feature. I use custom script to list compiled codes in CodeCache. I guess other troubleshooters also want similar feature (via jsload) in future if they encounter JVM crash. Thanks, Yasumasa On 2019/12/11 11:52, Chris Plummer wrote: > Hi, > > I like to propose the removal of SA javascript support. Few people even realize this support exists, and hopefully even fewer are using it since I'd like to remove it. Since I'm new to this myself, let me first explain what I know about it's existence, and then explain why I want to remove it. > > If you run "jhsdb clhsdb", there are jsload and jseval commands. Don't look for them in anything post JDK 8. I'll explain why later. jsload is used to load a javascript file. In that file you can register new clhsdb commands that are written in javascript. You can also evaluate javascript using the jseval command. Some of this is explained in [1], which is the only place I can find any reference to this support. It does not appear to be officially supported, nor is there any oracle provided documentation. > > There also appear to be a few clhsdb commands that are written in javascript. Doing a grep for "registerCommand" in sa.js shows the following: > > registerCommand("class", "class name", "jclass"); > registerCommand("classes", "classes", "jclasses"); > registerCommand("dumpclass", "dumpclass { address | name } [ directory ]", "dclass"); > registerCommand("dumpheap", "dumpheap [ file ]", "dumpHeap"); > registerCommand("mem", "mem address [ length ]", "printMem"); > registerCommand("sysprops", "sysprops", "sysProps"); > registerCommand("whatis", "whatis address", "printWhatis"); > > Once again, don't go looking for these in anything newer than JDK8. You won't find them. Again the only documentation I can fine is [1]. > > The other use of Javascript is the SOQL command (Simple Object Query Language), a tool used to query the heap, and also the JSDB command. The only SOQL documentation I co
Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents
On 12/12/2019 1:07 am, Reingruber, Richard wrote: Hi David, > Most of the details here are in areas I can comment on in detail, but I > did take an initial general look at things. Thanks for taking the time! Apologies the above should read: "Most of the details here are in areas I *can't* comment on in detail ..." David > The only thing that jumped out at me is that I think the > DeoptimizeObjectsALotThread should be a hidden thread. > > + bool is_hidden_from_external_view() const { return true; } Yes, it should. Will add the method like above. > Also I don't see any testing of the DeoptimizeObjectsALotThread. Without > active testing this will just bit-rot. DeoptimizeObjectsALot is meant for stress testing with a larger workload. I will add a minimal test to keep it fresh. > Also on the tests I don't understand your @requires clause: > > @requires ((vm.compMode != "Xcomp") & vm.compiler2.enabled & > (vm.opt.TieredCompilation != true)) > > This seems to require that TieredCompilation is disabled, but tiered is > our normal mode of operation. ?? > I removed the clause. I guess I wanted to target the tests towards the code they are supposed to test, and it's easier to analyze failures w/o tiered compilation and with just one compiler thread. Additionally I will make use of compiler.whitebox.CompilerWhiteBoxTest.THRESHOLD in the tests. Thanks, Richard. -Original Message- From: David Holmes Sent: Mittwoch, 11. Dezember 2019 08:03 To: Reingruber, Richard ; serviceability-dev@openjdk.java.net; hotspot-compiler-...@openjdk.java.net; hotspot-runtime-...@openjdk.java.net Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, On 11/12/2019 7:45 am, Reingruber, Richard wrote: Hi, I would like to get reviews please for http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.3/ Corresponding RFE: https://bugs.openjdk.java.net/browse/JDK-8227745 Fixes also https://bugs.openjdk.java.net/browse/JDK-8233915 And potentially https://bugs.openjdk.java.net/browse/JDK-8214584 [1] Vladimir Kozlov kindly put webrev.3 through tier1-8 testing without issues (thanks!). In addition the change is being tested at SAP since I posted the first RFR some months ago. The intention of this enhancement is to benefit performance wise from escape analysis even if JVMTI agents request capabilities that allow them to access local variable values. E.g. if you start-up with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n, then escape analysis is disabled right from the beginning, well before a debugger attaches -- if ever one should do so. With the enhancement, escape analysis will remain enabled until and after a debugger attaches. EA based optimizations are reverted just before an agent acquires the reference to an object. In the JBS item you'll find more details. Most of the details here are in areas I can comment on in detail, but I did take an initial general look at things. The only thing that jumped out at me is that I think the DeoptimizeObjectsALotThread should be a hidden thread. + bool is_hidden_from_external_view() const { return true; } Also I don't see any testing of the DeoptimizeObjectsALotThread. Without active testing this will just bit-rot. Also on the tests I don't understand your @requires clause: @requires ((vm.compMode != "Xcomp") & vm.compiler2.enabled & (vm.opt.TieredCompilation != true)) This seems to require that TieredCompilation is disabled, but tiered is our normal mode of operation. ?? Thanks, David Thanks, Richard. [1] Experimental fix for JDK-8214584 based on JDK-8227745 http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.patch
Re: RFR(S): 8234277:ClhsdbLauncher should enable verbose exceptions and do a better job of detecting SA failures
Hi Chris, The change looks good to me. Best regards, Daniil On 12/4/19, 5:39 PM, "serviceability-dev on behalf of Chris Plummer" wrote: Can I get one more review please? thanks, Chris On 12/3/19 1:10 PM, serguei.spit...@oracle.com wrote: > Hi Chris, > > It looks good. > > Thanks, > Serguei > > On 12/3/19 12:45 PM, Chris Plummer wrote: >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8234277 >> http://cr.openjdk.java.net/~cjplummer/8234277/webrev.00/ >> >> No longer redirect stderr for the jhsdb/clhsdb process. It results in >> not seeing attach failures in the output, so OutputAnalyer can't >> check for them. >> >> Execute "verbose true" as the first clhsdb command after launching. >> This will result in verboseExceptions being true in >> CommandProcessor.java, so full exception traces will appear in the >> output. This will make debugging future SA test failures a lot easier. >> >> Add an extra check for any DebuggerException. This is mainly for >> detecting that the attached failed. This previously was going >> un-noticed, and instead the test would later fail because it noticed >> some other issue, like missing output, which isn't very informative. >> >> Add checks for other unexpected SA exceptions that are caught and >> printed by CommandProcessor. These will always have an "Error: " >> prefix, making them easy to detect. >> >> Problem list ClhsdbScanOops.java. With the new error checking, it >> will now always fail on windows due to JDK-8230731 and on macos and >> linux due to JDK-8235220. These failures are not "new" per se, but >> are just now being properly detected. >> >> thanks, >> >> Chris >
Re: 8226575: OperatingSystemMXBean should be made container aware
Yes, I defer to Mandy on the best way to express the various Java exceptions. I’m ok with the changes. Thanks for getting this done for JDK14! Bob. > On Dec 11, 2019, at 12:12 PM, Daniil Titov wrote: > > Hi Serguei, > > Thank you for your comments. I will correct this nits before pushing the > changes. > > Hi Bob and David, > >> [Mandy Chung] >>> I reviewed Metrics and Subsystem in this version. >>> I don't need to see a new webrev. > > As I understood Mandy finished reviewing this fix. Just wanted to confirm > with you if you are okey with that version of the fix (webrev.06) ? > > Mach5 testing: tier1-tier6 and open/test/hotspot/jtreg/containers/docker > tests passed. > > Thank you, > Daniil > > > > On 12/9/19, 6:02 PM, "serguei.spit...@oracle.com" > wrote: > >Hi Daniil, > >It is not a full review, just some minor comments. >In fact, I do not see real problems yet. > > > http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java.frames.html > > 55 public long getTotalSwapSpaceSize() { > 56 if (containerMetrics != null) { > 57 long limit = containerMetrics.getMemoryAndSwapLimit(); > 58 // The memory limit metrics is not available if JVM >runs on Linux host ( not in a docker container) > 59 // or if a docker container was started without >specifying a memory limit ( without '--memory=' > 60 // Docker option). In latter case there is no limit on >how much memory the container can use and > 61 // it can use as much memory as the host's OS allows. > 62 long memLimit = containerMetrics.getMemoryLimit(); > 63 if (limit >= 0 && memLimit >= 0) { > 64 return limit - memLimit; > 65 } > 66 } > 67 return getTotalSwapSpaceSize0(); > 68 } > > Unneeded space after brackets '('. > Do we need to check if the (limit - memLimit) value is negative? > The same question is for getFreeSwapSpaceSize(): > memSwapLimit - memLimit - (memSwapUsage - memUsage) > > and getFreeMemorySize(): > 101 return limit - usage; > > 81 // If this happens just retry the loop for >a few iterations > > Dot is missed at the end of comment. > > > > http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/test/hotspot/jtreg/containers/docker/CheckOperatingSystemMXBean.java.html > > 34 System.out.println(String.format("Runtime.availableProcessors: >%d", Runtime.getRuntime().availableProcessors())); > 35 > > System.out.println(String.format("OperatingSystemMXBean.getAvailableProcessors: > >%d", osBean.getAvailableProcessors())); > 36 > > System.out.println(String.format("OperatingSystemMXBean.getTotalMemorySize: >%d", osBean.getTotalMemorySize())); > 37 > > System.out.println(String.format("OperatingSystemMXBean.getTotalPhysicalMemorySize: > >%d", osBean.getTotalPhysicalMemorySize())); > 38 >System.out.println(String.format("OperatingSystemMXBean.getFreeMemorySize: >%d", osBean.getFreeMemorySize())); > 39 > > System.out.println(String.format("OperatingSystemMXBean.getFreePhysicalMemorySize: > >%d", osBean.getFreePhysicalMemorySize())); > 40 > > System.out.println(String.format("OperatingSystemMXBean.getTotalSwapSpaceSize: > >%d", osBean.getTotalSwapSpaceSize())); > 41 > > System.out.println(String.format("OperatingSystemMXBean.getFreeSwapSpaceSize: >%d", osBean.getFreeSwapSpaceSize())); > 42 >System.out.println(String.format("OperatingSystemMXBean.getCpuLoad: %f", >osBean.getCpuLoad())); > 43 >System.out.println(String.format("OperatingSystemMXBean.getSystemCpuLoad: >%f", osBean.getSystemCpuLoad())); > > > To make the above lines a little bit shorter I'd suggest to define a >log() method like this: > private static void log(String msg) ( System.out.println(msg(; } > > 34 log(String.format("Runtime.availableProcessors: %d", >Runtime.getRuntime().availableProcessors())); > 35 log(String.format("OperatingSystemMXBean.getAvailableProcessors: >%d", osBean.getAvailableProcessors())); > 36 log(String.format("OperatingSystemMXBean.getTotalMemorySize: %d", >osBean.getTotalMemorySize())); > 37 >log(String.format("OperatingSystemMXBean.getTotalPhysicalMemorySize: >%d", osBean.getTotalPhysicalMemorySize())); > 38 log(String.format("OperatingSystemMXBean.getFreeMemorySize: %d", >osBean.getFreeMemorySize())); > 39 >log(String.format("OperatingSystemMXBean.getFreePhysicalMemorySize: %d", >osBean.getFreePhysicalMemorySize())); > 40 log(String.format("OperatingSy
Re: 8226575: OperatingSystemMXBean should be made container aware
Hi Serguei, Thank you for your comments. I will correct this nits before pushing the changes. Hi Bob and David, > [Mandy Chung] >> I reviewed Metrics and Subsystem in this version. >> I don't need to see a new webrev. As I understood Mandy finished reviewing this fix. Just wanted to confirm with you if you are okey with that version of the fix (webrev.06) ? Mach5 testing: tier1-tier6 and open/test/hotspot/jtreg/containers/docker tests passed. Thank you, Daniil On 12/9/19, 6:02 PM, "serguei.spit...@oracle.com" wrote: Hi Daniil, It is not a full review, just some minor comments. In fact, I do not see real problems yet. http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java.frames.html 55 public long getTotalSwapSpaceSize() { 56 if (containerMetrics != null) { 57 long limit = containerMetrics.getMemoryAndSwapLimit(); 58 // The memory limit metrics is not available if JVM runs on Linux host ( not in a docker container) 59 // or if a docker container was started without specifying a memory limit ( without '--memory=' 60 // Docker option). In latter case there is no limit on how much memory the container can use and 61 // it can use as much memory as the host's OS allows. 62 long memLimit = containerMetrics.getMemoryLimit(); 63 if (limit >= 0 && memLimit >= 0) { 64 return limit - memLimit; 65 } 66 } 67 return getTotalSwapSpaceSize0(); 68 } Unneeded space after brackets '('. Do we need to check if the (limit - memLimit) value is negative? The same question is for getFreeSwapSpaceSize(): memSwapLimit - memLimit - (memSwapUsage - memUsage) and getFreeMemorySize(): 101 return limit - usage; 81 // If this happens just retry the loop for a few iterations Dot is missed at the end of comment. http://cr.openjdk.java.net/~dtitov/8226575/webrev.05/test/hotspot/jtreg/containers/docker/CheckOperatingSystemMXBean.java.html 34 System.out.println(String.format("Runtime.availableProcessors: %d", Runtime.getRuntime().availableProcessors())); 35 System.out.println(String.format("OperatingSystemMXBean.getAvailableProcessors: %d", osBean.getAvailableProcessors())); 36 System.out.println(String.format("OperatingSystemMXBean.getTotalMemorySize: %d", osBean.getTotalMemorySize())); 37 System.out.println(String.format("OperatingSystemMXBean.getTotalPhysicalMemorySize: %d", osBean.getTotalPhysicalMemorySize())); 38 System.out.println(String.format("OperatingSystemMXBean.getFreeMemorySize: %d", osBean.getFreeMemorySize())); 39 System.out.println(String.format("OperatingSystemMXBean.getFreePhysicalMemorySize: %d", osBean.getFreePhysicalMemorySize())); 40 System.out.println(String.format("OperatingSystemMXBean.getTotalSwapSpaceSize: %d", osBean.getTotalSwapSpaceSize())); 41 System.out.println(String.format("OperatingSystemMXBean.getFreeSwapSpaceSize: %d", osBean.getFreeSwapSpaceSize())); 42 System.out.println(String.format("OperatingSystemMXBean.getCpuLoad: %f", osBean.getCpuLoad())); 43 System.out.println(String.format("OperatingSystemMXBean.getSystemCpuLoad: %f", osBean.getSystemCpuLoad())); To make the above lines a little bit shorter I'd suggest to define a log() method like this: private static void log(String msg) ( System.out.println(msg(; } 34 log(String.format("Runtime.availableProcessors: %d", Runtime.getRuntime().availableProcessors())); 35 log(String.format("OperatingSystemMXBean.getAvailableProcessors: %d", osBean.getAvailableProcessors())); 36 log(String.format("OperatingSystemMXBean.getTotalMemorySize: %d", osBean.getTotalMemorySize())); 37 log(String.format("OperatingSystemMXBean.getTotalPhysicalMemorySize: %d", osBean.getTotalPhysicalMemorySize())); 38 log(String.format("OperatingSystemMXBean.getFreeMemorySize: %d", osBean.getFreeMemorySize())); 39 log(String.format("OperatingSystemMXBean.getFreePhysicalMemorySize: %d", osBean.getFreePhysicalMemorySize())); 40 log(String.format("OperatingSystemMXBean.getTotalSwapSpaceSize: %d", osBean.getTotalSwapSpaceSize())); 41 log(String.format("OperatingSystemMXBean.getFreeSwapSpaceSize: %d", osBean.getFreeSwapSpaceSize())); 42 log(String.format("OperatingSystemMXBean.getCpuLoad: %f", osBean.getCpuLoad())); 43 log(String.format("Operat
Re: PING: RFR: 8234624: jstack mixed mode should refer DWARF
Hello Yasumasa, Please, 1. Consider to use mmap for reading elf sections. 2. Please move all platfrom-specific parts of native code to a separate file/directory. Current patch will brake AARCH64 build. 3. I didn't find any tests here. How did your test the changes? libproc_impl.c 131: If is not necessary, free handles NULLPTR gracefully. -Dmitry On 04.12.19 03:54, Yasumasa Suenaga wrote: > PING: Could you review it? > > JBS: https://bugs.openjdk.java.net/browse/JDK-8234624 > webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.01/ > > This bug is targeted to JDK 14. > > > Thanks, > > Yasumasa > > > On 2019/11/28 21:39, Yasumasa Suenaga wrote: >> Hi, >> >> I refactored LinuxAMD64CFrame.java . It works fine in >> serviceability/sa tests and >> all tests on submit repo >> (mach5-one-ysuenaga-JDK-8234624-2-20191128-0928-7059923). >> Could you review new webrev? >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.01/ >> >> The diff from previous webrev is here: >> http://hg.openjdk.java.net/jdk/submit/rev/4bc47efbc90b >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2019/11/25 14:08, Yasumasa Suenaga wrote: >>> Hi all, >>> >>> Please review this change: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8234624 >>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8234624/webrev.00/ >>> >>> >>> According to 2.7 Stack Unwind Algorithm in System V Application >>> Binary Interface AMD64 >>> Architecture Processor Supplement [1], we need to use DWARF in >>> .eh_frame or .debug_frame >>> for stack unwinding. >>> >>> As JDK-8022183 said, omit-frame-pointer is enabled by default since >>> GCC 4.6, so system >>> library (e.g. libc) might be compiled with this feature. >>> >>> However `jhsdb jstack --mixed` does not do so, it uses base pointer >>> register (RBP). >>> So it might be lack of stack frames. >>> >>> I guess JDK-8219201 is caused by same issue. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> [1] >>> https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf >>>
RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents
Hi David, > Most of the details here are in areas I can comment on in detail, but I > did take an initial general look at things. Thanks for taking the time! > The only thing that jumped out at me is that I think the > DeoptimizeObjectsALotThread should be a hidden thread. > > + bool is_hidden_from_external_view() const { return true; } Yes, it should. Will add the method like above. > Also I don't see any testing of the DeoptimizeObjectsALotThread. Without > active testing this will just bit-rot. DeoptimizeObjectsALot is meant for stress testing with a larger workload. I will add a minimal test to keep it fresh. > Also on the tests I don't understand your @requires clause: > > @requires ((vm.compMode != "Xcomp") & vm.compiler2.enabled & > (vm.opt.TieredCompilation != true)) > > This seems to require that TieredCompilation is disabled, but tiered is > our normal mode of operation. ?? > I removed the clause. I guess I wanted to target the tests towards the code they are supposed to test, and it's easier to analyze failures w/o tiered compilation and with just one compiler thread. Additionally I will make use of compiler.whitebox.CompilerWhiteBoxTest.THRESHOLD in the tests. Thanks, Richard. -Original Message- From: David Holmes Sent: Mittwoch, 11. Dezember 2019 08:03 To: Reingruber, Richard ; serviceability-dev@openjdk.java.net; hotspot-compiler-...@openjdk.java.net; hotspot-runtime-...@openjdk.java.net Subject: Re: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, On 11/12/2019 7:45 am, Reingruber, Richard wrote: > Hi, > > I would like to get reviews please for > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.3/ > > Corresponding RFE: > https://bugs.openjdk.java.net/browse/JDK-8227745 > > Fixes also https://bugs.openjdk.java.net/browse/JDK-8233915 > And potentially https://bugs.openjdk.java.net/browse/JDK-8214584 [1] > > Vladimir Kozlov kindly put webrev.3 through tier1-8 testing without issues > (thanks!). In addition the > change is being tested at SAP since I posted the first RFR some months ago. > > The intention of this enhancement is to benefit performance wise from escape > analysis even if JVMTI > agents request capabilities that allow them to access local variable values. > E.g. if you start-up > with -agentlib:jdwp=transport=dt_socket,server=y,suspend=n, then escape > analysis is disabled right > from the beginning, well before a debugger attaches -- if ever one should do > so. With the > enhancement, escape analysis will remain enabled until and after a debugger > attaches. EA based > optimizations are reverted just before an agent acquires the reference to an > object. In the JBS item > you'll find more details. Most of the details here are in areas I can comment on in detail, but I did take an initial general look at things. The only thing that jumped out at me is that I think the DeoptimizeObjectsALotThread should be a hidden thread. + bool is_hidden_from_external_view() const { return true; } Also I don't see any testing of the DeoptimizeObjectsALotThread. Without active testing this will just bit-rot. Also on the tests I don't understand your @requires clause: @requires ((vm.compMode != "Xcomp") & vm.compiler2.enabled & (vm.opt.TieredCompilation != true)) This seems to require that TieredCompilation is disabled, but tiered is our normal mode of operation. ?? Thanks, David > Thanks, > Richard. > > [1] Experimental fix for JDK-8214584 based on JDK-8227745 > > http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.patch >
Re: Removal of SA javascript support
Sundar, Supporting hotspot data structure in SA is already a maintenance nightmare ;) So we can consider to provide high level API, like find_class_by_name to script writer. It allows anybody who are interesting with quick prototyping write his own program on top of SA with any language they want. -Dmitry On 11.12.19 15:47, sundararajan.athijegannat...@oracle.com wrote: > Effectively you're asking for SA as API. I don't think that is a good > idea. That implies supporting hotspot data structures as Java *API*. > That will be maintainability nightmare - we've to keep tracking hotspot > data structures in SA code. That itself is problematic. API would be > next level nightmare. > > -Sundar > > On 11/12/19 11:57 am, Yasumasa Suenaga wrote: >> Hi, >> >> IMHO we need to export all packages in SA if we do not provide new API >> for SA. >> sa.js in jdk.hotspot.agent could access all SA classes until JDK 8 >> (before Jigsaw), so we could make various functions if we need. >> >> OTOH we cannot know what classes are needed by the SA users. All >> packages in jdk.hotspot.agent module provides features, and they >> require other packages. For example, sun.jvm.hotspot.oops.Oop requires >> sun.jvm.hotspot.types, and it requires sun.jvm.hotspot.debugger . >> It is difficult to track and to export minimally. >> (I worked for it in JDK-8157947, but I gave up...) >> >> Thus I guess it is a big challenge to export SA classes without >> refactoring. >> If we provide new API for SA plugin, I guess we need to work some >> refactoring. >> >> >> Yasumasa >> >> >> On 2019/12/11 15:00, Chris Plummer wrote: >>> On 12/10/19 9:56 PM, Yasumasa Suenaga wrote: On 2019/12/11 14:39, Krystal Mok wrote: > Hi Yasumasa, > > That's a very nice idea. Basically what you're asking for is > exposing the Command interface [1] so that plugins can implement it > and get dynamically loaded / registered into CLHSDB / HSDB, right? Yes, but we also need proxy API to access internal SA objects e.g. CodeCache, JavaThread, TypeDataBase, etc... >>> Yes, or export them. I should have read this email before posting my >>> previous one. >>> >>> Chris Yasumasa > [1]: > http://hg.openjdk.java.net/jdk/jdk/file/c71ec1f09f21/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/CommandProcessor.java#l246 > > > - Kris > > On Tue, Dec 10, 2019 at 9:33 PM Yasumasa Suenaga > mailto:suen...@oss.nttdata.com>> wrote: > > Hi Chris, > > It's a sad proposal, but I agree with you. To maintain SA in JS > is difficult since Jigsaw. > However I want SA to implement pluggable feature. > I use custom script to list compiled codes in CodeCache. > > I guess other troubleshooters also want similar feature (via > jsload) in future if they encounter JVM crash. > > > Thanks, > > Yasumasa > > > On 2019/12/11 11:52, Chris Plummer wrote: > > Hi, > > > > I like to propose the removal of SA javascript support. Few > people even realize this support exists, and hopefully even fewer > are using it since I'd like to remove it. Since I'm new to this > myself, let me first explain what I know about it's existence, and > then explain why I want to remove it. > > > > If you run "jhsdb clhsdb", there are jsload and jseval > commands. Don't look for them in anything post JDK 8. I'll explain > why later. jsload is used to load a javascript file. In that file > you can register new clhsdb commands that are written in > javascript. You can also evaluate javascript using the jseval > command. Some of this is explained in [1], which is the only place > I can find any reference to this support. It does not appear to be > officially supported, nor is there any oracle provided documentation. > > > > There also appear to be a few clhsdb commands that are > written in javascript. Doing a grep for "registerCommand" in sa.js > shows the following: > > > > registerCommand("class", "class name", "jclass"); > > registerCommand("classes", "classes", "jclasses"); > > registerCommand("dumpclass", "dumpclass { address | name } > [ directory ]", "dclass"); > > registerCommand("dumpheap", "dumpheap [ file ]", "dumpHeap"); > > registerCommand("mem", "mem address [ length ]", "printMem"); > > registerCommand("sysprops", "sysprops", "sysProps"); > > registerCommand("whatis", "whatis address", "printWhatis"); > > > > Once again, don't go looking for these in anything newer > than JDK8. You won't find them. Again the only documentation I can > fine is [1]. > > > > The other use of Javascript is the SOQL command (Simple > Object Query Language), a tool used to query the heap,
Re: Removal of SA javascript support
Replacing one scripting language with another (jython) does not solve anything. You'd still face the same issues - accessing module private stuff from SA module from scripts. Besides you'll have a new problem in addition. How to bundle jython? We've been using bundled scripting engine (nashorn) so far. -Sundar On 11/12/19 1:14 pm, Dmitry Samersoff wrote: Hello Chris, I'm supporting you with this decision. PS: For people who want SA scripting - One thing I experimented with a long time ago - has been exporting of some SA capabilities to jython. This might be the way to go. -Dmitry On 11.12.19 05:52, Chris Plummer wrote: Hi, I like to propose the removal of SA javascript support. Few people even realize this support exists, and hopefully even fewer are using it since I'd like to remove it. Since I'm new to this myself, let me first explain what I know about it's existence, and then explain why I want to remove it. If you run "jhsdb clhsdb", there are jsload and jseval commands. Don't look for them in anything post JDK 8. I'll explain why later. jsload is used to load a javascript file. In that file you can register new clhsdb commands that are written in javascript. You can also evaluate javascript using the jseval command. Some of this is explained in [1], which is the only place I can find any reference to this support. It does not appear to be officially supported, nor is there any oracle provided documentation. There also appear to be a few clhsdb commands that are written in javascript. Doing a grep for "registerCommand" in sa.js shows the following: registerCommand("class", "class name", "jclass"); registerCommand("classes", "classes", "jclasses"); registerCommand("dumpclass", "dumpclass { address | name } [ directory ]", "dclass"); registerCommand("dumpheap", "dumpheap [ file ]", "dumpHeap"); registerCommand("mem", "mem address [ length ]", "printMem"); registerCommand("sysprops", "sysprops", "sysProps"); registerCommand("whatis", "whatis address", "printWhatis"); Once again, don't go looking for these in anything newer than JDK8. You won't find them. Again the only documentation I can fine is [1]. The other use of Javascript is the SOQL command (Simple Object Query Language), a tool used to query the heap, and also the JSDB command. The only SOQL documentation I could find is the blog reference [2]. I could not find HSDB documentation, but I believe is is a javascript support for looking at hotspot. So once again, neither of these seem to be officially supported or documented. The real purpose of the email is to propose removal of this support. Here are the reasons: (1) It's broken, and has been since 9. See [3]. This is why you don't see the javascript related commands in clhsdb. Javascript fails to initialize, so none of the javascript related commands are registered. (2) Nashorn is deprecated and will be removed eventually. (3) We have very little understanding of the javascript support. (4) No resources to work on it (unless there is a community volunteer). (5) Very questionable value (lack of users). The fact this support has been broken since JDK 9 and no bug was filed until I did so this week is a good indication of that. Another is that there are no other SA Javascript related bugs filed. Lastly, the lack of any official documentation and only minimal mention of it on the web is another good indication of it's (lack of) value. Also, regarding the 7 commands listed above that would be lost (but currently don't work now anyway), if they are really wanted, they could be implemented in java instead of javascript. I'd like to remove javascript support in two steps. The first is simply disable the clhsdb code that tries to initialize the javascript support. I'd like to do this in 14 (actually as soon as possible). I'd like to actually do this now even if we decide to keep javascript support and eventually fix it because it will get rid of the warning you see whenever you attach from clhsdb: Warning! JS Engine can't start, some commands will not be available. This warning will become more of an issue for the clhsdb tests after I push [4] because then you will also see the full stacktrace for the underlying exception that caused the Javascript to fail to start. Besides being unnecessary noise in passing test cases, it can also be misleading in any test that fails because the exception will be unrelated to the failure. This is actually what got me going down this path of what the javascript support is all about. The next step would be to strip out all Javascript related code, including the SOQL and JSDB tools. This would be done in 15. Please let me know what you think. thanks, Chris [1] https://cr.openjdk.java.net/~minqi/6830717/raw_files/new/agent/doc/clhsdb.html [2] http://javatroubleshooting.blogspot.com/2015/12/serviceability-agent-part-3.html [3] https://bugs.openjdk.java.net/browse/JDK-8235594 [4] https://bugs.openjdk.java.net/browse/JDK-82
Re: Removal of SA javascript support
Effectively you're asking for SA as API. I don't think that is a good idea. That implies supporting hotspot data structures as Java *API*. That will be maintainability nightmare - we've to keep tracking hotspot data structures in SA code. That itself is problematic. API would be next level nightmare. -Sundar On 11/12/19 11:57 am, Yasumasa Suenaga wrote: Hi, IMHO we need to export all packages in SA if we do not provide new API for SA. sa.js in jdk.hotspot.agent could access all SA classes until JDK 8 (before Jigsaw), so we could make various functions if we need. OTOH we cannot know what classes are needed by the SA users. All packages in jdk.hotspot.agent module provides features, and they require other packages. For example, sun.jvm.hotspot.oops.Oop requires sun.jvm.hotspot.types, and it requires sun.jvm.hotspot.debugger . It is difficult to track and to export minimally. (I worked for it in JDK-8157947, but I gave up...) Thus I guess it is a big challenge to export SA classes without refactoring. If we provide new API for SA plugin, I guess we need to work some refactoring. Yasumasa On 2019/12/11 15:00, Chris Plummer wrote: On 12/10/19 9:56 PM, Yasumasa Suenaga wrote: On 2019/12/11 14:39, Krystal Mok wrote: Hi Yasumasa, That's a very nice idea. Basically what you're asking for is exposing the Command interface [1] so that plugins can implement it and get dynamically loaded / registered into CLHSDB / HSDB, right? Yes, but we also need proxy API to access internal SA objects e.g. CodeCache, JavaThread, TypeDataBase, etc... Yes, or export them. I should have read this email before posting my previous one. Chris Yasumasa [1]: http://hg.openjdk.java.net/jdk/jdk/file/c71ec1f09f21/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/CommandProcessor.java#l246 - Kris On Tue, Dec 10, 2019 at 9:33 PM Yasumasa Suenaga mailto:suen...@oss.nttdata.com>> wrote: Hi Chris, It's a sad proposal, but I agree with you. To maintain SA in JS is difficult since Jigsaw. However I want SA to implement pluggable feature. I use custom script to list compiled codes in CodeCache. I guess other troubleshooters also want similar feature (via jsload) in future if they encounter JVM crash. Thanks, Yasumasa On 2019/12/11 11:52, Chris Plummer wrote: > Hi, > > I like to propose the removal of SA javascript support. Few people even realize this support exists, and hopefully even fewer are using it since I'd like to remove it. Since I'm new to this myself, let me first explain what I know about it's existence, and then explain why I want to remove it. > > If you run "jhsdb clhsdb", there are jsload and jseval commands. Don't look for them in anything post JDK 8. I'll explain why later. jsload is used to load a javascript file. In that file you can register new clhsdb commands that are written in javascript. You can also evaluate javascript using the jseval command. Some of this is explained in [1], which is the only place I can find any reference to this support. It does not appear to be officially supported, nor is there any oracle provided documentation. > > There also appear to be a few clhsdb commands that are written in javascript. Doing a grep for "registerCommand" in sa.js shows the following: > > registerCommand("class", "class name", "jclass"); > registerCommand("classes", "classes", "jclasses"); > registerCommand("dumpclass", "dumpclass { address | name } [ directory ]", "dclass"); > registerCommand("dumpheap", "dumpheap [ file ]", "dumpHeap"); > registerCommand("mem", "mem address [ length ]", "printMem"); > registerCommand("sysprops", "sysprops", "sysProps"); > registerCommand("whatis", "whatis address", "printWhatis"); > > Once again, don't go looking for these in anything newer than JDK8. You won't find them. Again the only documentation I can fine is [1]. > > The other use of Javascript is the SOQL command (Simple Object Query Language), a tool used to query the heap, and also the JSDB command. The only SOQL documentation I could find is the blog reference [2]. I could not find HSDB documentation, but I believe is is a javascript support for looking at hotspot. So once again, neither of these seem to be officially supported or documented. > > The real purpose of the email is to propose removal of this support. Here are the reasons: > > (1) It's broken, and has been since 9. See [3]. This is why you don't see the javascript related commands in clhsdb. Javascript fails to initialize, so none of the javascript related commands are registered. > (2) Nashorn is deprecated and will be removed eventually. > (3) We have very little understanding of the javascript support. > (4) No resources to work on it (unless there is a community volunteer). > (5) Very question