Re: JIT debug symbols and OProfile under Hadoop

2010-06-21 Thread Shrinivas Joshi
Hi All,

I just wanted to check if anybody had a comment on this query.

Thanks,
-Shrinivas

On Wed, Jun 16, 2010 at 9:50 PM, Shrinivas Joshi jshrini...@gmail.comwrote:

 Sorry if this is repeat email for you, I did send this to common-dev list
 as well.

 Hello,
 I am trying to get profiles for a workload running on top of Hadoop 0.20.2
 framework. The workload jars and Hadoop jars have been compiled with debug
 symbols enabled. I could see local variable tables and line number tables in
 the jar class files using javap. I am passing the right value for *
 agentpath* flag pointing to oprofile JVMTI agent and I am also seeing it
 passed correctly to map and reduce JVMs from *ps *output. This is done
 through conf/mapred-site.xml properties. However, in the profiles I am
 unable to see meaningful samples for JITed code; everything is getting
 attributed to anon samples.

 Profiling a test app works just fine with the same JVM that is being used
 by Hadoop. I can see the right debug symbols being attributed in the
 resulting profiling. In fact, I even tried profiling a JVM process that was
 spawned using *ProcessBuilder* API, the way Hadoop spawns its map and
 reduce JVMs, that seem to work fine too.

 Does anybody have any pointers as to what I might be missing here? Has
 anybody successfully collected profiles showing JIT debug symbols for any
 Hadoop workload? I am using OProfile 0.9.6.

 Thanks,

 -Shrinivas



JIT debug symbols and OProfile under Hadoop

2010-06-16 Thread Shrinivas Joshi
Hello,



I am trying to get profiles for a workload running on top of Hadoop 0.20.2
framework. The workload jars and Hadoop jars have been compiled with debug
symbols enabled. I could see local variable tables and line number tables in
the jar class files using javap. I am passing the right value for *agentpath
* flag pointing to oprofile JVMTI agent and I am also seeing it passed
correctly to map and reduce JVMs from *ps *output. This is done through
conf/mapred-site.xml properties. However, in the profiles I am unable to see
meaningful samples for JITed code; everything is getting attributed to anon
samples.



Profiling a test app works just fine with the same JVM that is being used by
Hadoop. I can see the right debug symbols being attributed in the resulting
profiling. In fact, I even tried profiling a JVM process that was spawned
using *ProcessBuilder* API, the way Hadoop spawns its map and reduce JVMs,
that seem to work fine too.



Does anybody have any pointers as to what I might be missing here? Has
anybody successfully collected profiles showing JIT debug symbols for any
Hadoop workload? I am using OProfile 0.9.6.



Thanks,

-Shrinivas