On Fri, Nov 14, 2014 at 6:00 PM, Maynard Johnson <mayna...@us.ibm.com> wrote:
> On 10/07/2014 08:35 AM, Maynard Johnson wrote:
>> On 10/07/2014 03:58 AM, Volker Simonis wrote:
>>> Hi Maynard,
>>>
>>> I'm now back from JavaOne and can look at this issue. Could you please
>>> share your current implementation so I can reproduce your problem more
>>> easily.
>> See attachment. The two patches in the attached tar file apply to a
>> jdk9-dev snapshot from July. I haven't even tried forward-porting to
>> current upstream code, so I don't know how well they would apply.
>>>
>>> By the way, you can find the ppc frame layout description of all the
>>> different frame types (native, interpreted, compiled) in
>>> /hotspot/src/cpu/ppc/vm/frame_ppc.hpp. The different frame::sender()
>>> implementations (in frame_x86.cpp, frame_ppc.cpp, ..) contain all the
>> Yes, I've been studying those files, but I freely admit I don't have a good
>> grasp of the code yet.
>>> gory details about how to walk a frame. That's what you have to
>>> implement in Java in order to get a full stack trace from the
>>> serviceability tools. On x86 the frame pointer (i.e. the ebp register
>>> points to the last frame pointer while (frame pointer - 1) points to
>>> the return pc.
>> Ummm . . . Stack addresses grow downward, and I was under the impression
>> 'return pc' was the word on the stack directly before the ebp, which would
>> mean return pc is at 'frame pointer + 1'. Or am I off base here?
>> Nevertheless, my question below concerns the "pc", not the "return pc".
>> Perhaps I'm misunderstanding "pc" in this context; but even so, the x86 code
>> still seems wrong:
>>
>> this.pc = raw_sp.getAddressAt(-1 * VM.getVM().getAddressSize());
>>
>> If, in fact, 'this.pc' is supposed to represent 'return pc' in this context,
>> then I would think the code should be:
>>
>> this.pc = raw_fp.getAddressAt(VM.getVM().getAddressSize());
>>
>> I hope you can help set me on the right track. As you can see, I'm lost in
>> the weeds right now. :-)
> Hi, Volker,
> I presume you've been too busy to help out with this, so I had another go at
> it, and was able to make
> quite a bit of progress. The main issue was that I was unwittingly building
> my openJDK source
> with the "--with-jvm-interpreter=cpp" configure option! Once I discovered
> that and removed that option,
> the stack started making sense, since my new ppc64 agent's reflection of the
> stack assumes the use
> of the template interpreter (as does the x86 agent code which the ppc64 stuff
> is based on).
>
> So now I have jmap, jsadebugd, and jstack working for ppc64 on both big
> endian and little endian -- almost.
> There are a couple issues yet with jstack that I'm stuck on. I will post the
> patches as an RFC
> and document those issues. If you could find time to review the patches and
> perhaps help out with
> the open issues, I would appreciate it. Thanks!
>
Hi Maynard,
sorry, I was quite busy indeed.
I'm really happy that you've made some progress in the meantime and I
promise to look at your changes next week.
Thank you and best regards,
Volker
> -Maynard
>>
>> -Maynard
>>>
>>> Regards,
>>> Volker
>>>
>>>
>>> On Tue, Sep 30, 2014 at 12:42 AM, Maynard Johnson <mayna...@us.ibm.com>
>>> wrote:
>>>> On 07/09/2014 12:38 PM, Volker Simonis wrote:
>>>>> On Wed, Jul 9, 2014 at 3:45 PM, Maynard Johnson <mayna...@us.ibm.com>
>>>>> wrote:
>>>>>> On 07/04/2014 10:59 AM, Volker Simonis wrote:
>>>>>>> Hi Maynard,
>>>>>>>
>>>>>>> we (i.e. SAP) do not currently support the SA agent on Linux/PPC64 and
>>>>>>> AIX (we have other proprietary servicibility tools). Because of that
>>>>>>> (and because SA isn't specified by the SE specification) porting the
>>>>>>> SA agent was no top priority until now. But there are no technical
>>>>>>> reasons why it should not work (it's just a lack of resources). Of
>>>>>>> course contributions are always highly welcome:)
>>>>>>>
>>>>>>> That said, the SA agent library and jar file actually gets build. If
>>>>>>> you do a complete build you'll find them under:
>>>>>>>
>>>>>>> hotspot/linux_ppc64_compiler2/generated/sa-jdi.jar
>>>>>>> hotspot/linux_ppc64_compiler2/{product,fastdebug,debug}/libsaproc.so
>>>>>>>
>>>>>>> in the build directory. They are just not copied into the jdk
>>>>>>> workspace and the created images because they don't work out of the
>>>>>>> box.
>>>>>>>
>>>>>>> The following two patches for the jdk9 top-level and hotspot
>>>>>>> repositories respectively should fix the build such that the agent
>>>>>>> files will be correctly copied into the images.
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/sa_toplevel
>>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/sa_hotspot/
>>>>>>>
>>>>>>> They will get you to the point where for example 'jstack' will run up
>>>>>>> to the following point:
>>>>>> Ok, great. This should be enough to get me started. I should have time
>>>>>> to begin on this later this week or early next week.
>>>>>
>>>>> Hi Maynard,
>>>>>
>>>>> great to welcome you in the ppc64 porting team:)
>>>>>
>>>>>> I may come knocking at your "door" for some occasional help, but I'll
>>>>>> try to keep that to a minimum.
>>>> Hi, Volker. Knock, knock. :-)
>>>> I was preoccupied for a while this summer rolling out the latest release
>>>> of oprofile (for which I'm the maintainer), but am now coming back to this
>>>> task. I've implemented what I believe are all of the necessary
>>>> ppc64-specific Java files to enable the jstack and jmap tools to work on
>>>> core files. I've also updated
>>>> hotspot/agent/src/os/linux/LinuxDebuggerLocal.c to implement the
>>>> accumulation of register data on ppc64 vi ptrace. But now I've run into a
>>>> problem I need help with.
>>>>
>>>> When I run jstack on my POWER7 system, it gets stuck in a loop in
>>>> sun.jvm.hotspot.tools.StackTrace::run. There's an inner for-loop there
>>>> where cur.getLastJavaVFrameDbg() is called ('cur' is a JavaThread). For
>>>> the first JavaThread, we do return from getLastJavaVFrameDbg(), just as we
>>>> do when running jstack on my Intel laptop. But for the second JavaThread,
>>>> we never return from getLastJavaVFrameDbg() on ppc64. I believe the root
>>>> of the problem is in my new sun.jvm.hotspot.runtime.ppc64.PPC64Frame
>>>> class. The getLastJavaVFrameDbg method calls getCurrentFrameGuess, which
>>>> is implemented in the new
>>>> sun.jvm.hotspot.runtime.linux_ppc64.LinuxPPC64JavaThreadPDAccess class. In
>>>> both ppc64 and x86, this first level xxxCurrentFrameGuess object is
>>>> instantiated with a 'pc' value of null, so getCurrentFrameGuess then new's
>>>> up a xxxFrame object, passing in the SP and FP, but no PC. The
>>>> implementation of the PPC64Frame(Address,Address) constructor is currently
>>>> identical to the X86Frame !
> c!
>> ons!
>>>> tructor, b
>>>> ut is almost certainly incorrect. In this constructor, the 'pc' is set as
>>>> follows:
>>>> this.pc = raw_sp.getAddressAt(-1 * VM.getVM().getAddressSize());
>>>>
>>>> This works fine on X86, but not on ppc64. But I'm not understanding how
>>>> this even works on X86. From what I understand, the data below the stack
>>>> pointer on X86 is the "red zone". How is that being used as a pc? But more
>>>> importantly, do you know how I can ascertain what the 'pc' value should be
>>>> for ppc64?
>>>>
>>>> Thanks in advance for any assistance you can give.
>>>>
>>>> -Maynard
>>>>
>>>>>
>>>>> Please feel free to ask any questions. The OpenJDK project and
>>>>> especially the HotSpot part are known to take some getting used to.
>>>>>
>>>>>> I was wondering if a bug report should be opened in JBS, just to record
>>>>>> that the issue is being worked. Thoughts?
>>>>>
>>>>> I have opened "8049715: PPC64: First steps to enable SA on
>>>>> Linux/PPC64" (https://bugs.openjdk.java.net/browse/JDK-8049715) for
>>>>> the patch which I sent you with the last mail. I've already sent out
>>>>> webrevs for that change and hopefully it will be fixed within the next
>>>>> few days.
>>>>>
>>>>> For the actual port of the ppc64-specific stuff I opened bug "8049716
>>>>> PPC64: Implement SA on Linux/PPC64"
>>>>> (https://bugs.openjdk.java.net/browse/JDK-8049716). I can also help
>>>>> with hosting the webrevs, once you have a running version.
>>>>>
>>>>> Regards,
>>>>> Volker
>>>>>
>>>>>>
>>>>>> -Maynard
>>>>>>>
>>>>>>>> images/j2sdk-image/bin/jstack ./jdk/bin/java core.13547
>>>>>>> Attaching to core core.13547 from executable ./jdk/bin/java, please
>>>>>>> wait...
>>>>>>> WARNING: Hotspot VM version
>>>>>>> 1.9.0-internal-debug-d046063_2014_07_04_11_46-b00 does not match with
>>>>>>> SA version 1.9.0-internal-debug-d046063_2014_07_04_11_46-b00. You may
>>>>>>> see unexpected results.
>>>>>>> Debugger attached successfully.
>>>>>>> Server compiler detected.
>>>>>>> JVM version is 1.9.0-internal-debug-d046063_2014_07_04_11_46-b00
>>>>>>> Deadlock Detection:
>>>>>>>
>>>>>>> Exception in thread "main" java.lang.reflect.InvocationTargetException
>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>> at
>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>>> at
>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>> at java.lang.reflect.Method.invoke(Method.java:484)
>>>>>>> at sun.tools.jstack.JStack.runJStackTool(JStack.java:140)
>>>>>>> at sun.tools.jstack.JStack.main(JStack.java:106)
>>>>>>> Caused by: java.lang.ExceptionInInitializerError
>>>>>>> at sun.jvm.hotspot.runtime.VM.getThreads(VM.java:610)
>>>>>>> at
>>>>>>> sun.jvm.hotspot.runtime.DeadlockDetector.print(DeadlockDetector.java:54)
>>>>>>> at
>>>>>>> sun.jvm.hotspot.runtime.DeadlockDetector.print(DeadlockDetector.java:39)
>>>>>>> at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:62)
>>>>>>> at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>>>>>>> at sun.jvm.hotspot.tools.JStack.run(JStack.java:66)
>>>>>>> at sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
>>>>>>> at sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
>>>>>>> at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
>>>>>>> at sun.jvm.hotspot.tools.JStack.main(JStack.java:92)
>>>>>>> ... 6 more
>>>>>>> Caused by: java.lang.RuntimeException: OS/CPU combination linux/ppc64
>>>>>>> not yet supported
>>>>>>> at sun.jvm.hotspot.runtime.Threads.initialize(Threads.java:97)
>>>>>>> at sun.jvm.hotspot.runtime.Threads.access$000(Threads.java:42)
>>>>>>> at sun.jvm.hotspot.runtime.Threads$1.update(Threads.java:52)
>>>>>>> at
>>>>>>> sun.jvm.hotspot.runtime.VM.registerVMInitializedObserver(VM.java:394)
>>>>>>> at sun.jvm.hotspot.runtime.Threads.<clinit>(Threads.java:50)
>>>>>>> ... 16 more
>>>>>>>
>>>>>>> And that's the point where I was saying that "contributions are always
>>>>>>> highly welcome:)"
>>>>>>>
>>>>>>> Now all the Linux/PPC64 specific class under
>>>>>>> hotspot/agent/src/share/classes/ would have to be implemented (e.g.
>>>>>>> sun/jvm/hotspot/runtime/amd64/AMD64CurrentFrameGuess). Are you
>>>>>>> interested in contributing to this project?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Volker
>>>>>>>
>>>>>>> PS: I cc-ed serviceability-dev because I remember that they started a
>>>>>>> poll a while ago about who is using the SA tools. I'm therefore not
>>>>>>> quite sure what's the current status and what are the future plan for
>>>>>>> these libraries.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jul 3, 2014 at 9:04 PM, Maynard Johnson <mayna...@us.ibm.com>
>>>>>>> wrote:
>>>>>>>> Hi, all,
>>>>>>>> On my Intel laptop, I note that certain jdk9 serviceability tools --
>>>>>>>> jstack, jmap, jsadebugd -- have an option to pass a core file instead
>>>>>>>> of a process ID; for example:
>>>>>>>>
>>>>>>>> $ jstack -h
>>>>>>>> Usage:
>>>>>>>> jstack [-l] <pid>
>>>>>>>> (to connect to running process)
>>>>>>>> jstack -F [-m] [-l] <pid>
>>>>>>>> (to connect to a hung process)
>>>>>>>> jstack [-m] [-l] <executable> <core>
>>>>>>>> (to connect to a core file)
>>>>>>>> jstack [-m] [-l] [server_id@]<remote server IP or hostname>
>>>>>>>> (to connect to a remote debug server)
>>>>>>>>
>>>>>>>> But on my PowerLinux box, the core file option is missing from the
>>>>>>>> usage output. I see that
>>>>>>>> jdk9-dev/jdk/src/share/classes/sun/tools/jstack/JStack.java requires
>>>>>>>> the existence of sun.jvm.hotspot.tools.JStack for the core file option
>>>>>>>> to be made available. On my Intel system, the
>>>>>>>> sun.jvm.hotspot.tools.JStack class is packaged in sa-jdi.jar in
>>>>>>>> <jdk9Dev-install>/jvm/openjdk-1.9.0-internal/lib/. But the sa-jdi.jar
>>>>>>>> is missing on PowerPC. Is there a technical reason for this or is it
>>>>>>>> an oversight?
>>>>>>>>
>>>>>>>> The jsadebugd tool does not run at all on PowerLinux; it gets the
>>>>>>>> following error:
>>>>>>>>
>>>>>>>> Error: Could not find or load main class
>>>>>>>> sun.jvm.hotspot.jdi.SADebugServer
>>>>>>>>
>>>>>>>> On my Intel system, the SADebugServer class is packaged in the
>>>>>>>> sa-jdi.jar mentioned above.
>>>>>>>>
>>>>>>>> I've spent the past day or so looking at makefiles until I'm
>>>>>>>> cross-eyed, but haven't yet found where the issue might be. Any tips
>>>>>>>> would be appreciated.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>> -Maynard
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
