Hi, Has anyone successfully used Java Flight Recorder (JFR) with Spark Streaming on Oracle Java 8? JFR works for me on batch jobs but not with Streaming.
I'm running my streaming job on Amazon EMR. I have enabled Java Flight Recorder (JFR) to profile CPU usage. But at the end of the job, the JFR output files get deleted and are not available. This is the error in the JFR trace logs: [jfr][DEBUG][880.590] Release chunk /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_40_45_8823_0.jfr.part ref count now 0 [jfr][DEBUG][880.590] Repository chunk /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_40_45_8823_0.jfr.part deleted. [jfr][DEBUG][880.590] Repository chunk /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_40_45_8823_0.jfr deleted. [jfr][INFO ][880.593] Stopped recording [id=1, name=dump-on-exit-clone-of-0, start=Wed Feb 19 17:26:06 UTC 2020, end=Wed Feb 19 17:40:45 UTC 2020] [jfr][ERROR][880.593] Could not copy to /tmp/hotspot-pid-8823-id-0-2020_02_19_17_40_45.jfr [jfr][ERROR][880.593] java.io.FileNotFoundException:No chunks [jfr][ERROR][880.593] oracle.jrockit.jfr.ChunksChannel.<init>(ChunksChannel.java:26) [jfr][ERROR][880.593] oracle.jrockit.jfr.Recording.copyTo(Recording.java:404) [jfr][ERROR][880.593] oracle.jrockit.jfr.JFRImpl.dumpOnExit(JFRImpl.java:802) [jfr][ERROR][880.593] oracle.jrockit.jfr.JFRImpl.dumpOnExit(JFRImpl.java:789) [jfr][ERROR][880.593] oracle.jrockit.jfr.JFRImpl.destroy(JFRImpl.java:767) [jfr][ERROR][880.593] oracle.jrockit.jfr.JFR$2.run(JFR.java:208) [jfr][ERROR][880.593] java.lang.Thread.run(Thread.java:748) [jfr][ERROR][880.593] Could not delete /tmp/hotspot-pid-8823-id-0-2020_02_19_17_40_45.jfr [jfr][ERROR][880.593] java.io.FileNotFoundException:No chunks [jfr][ERROR][880.593] oracle.jrockit.jfr The rest of the trace can be found in the attachment (stdout.txt). Also, I've not changed the default value of " spark.streaming.stopGracefullyOnShutdown" (its default value is false) The Spark driver and executor JVMs are launched with the following options: -XX:+UnlockCommercialFeatures -XX:+FlightRecorder -XX:FlightRecorderOptions=defaultrecording=true,disk=true,repository=/tmp,maxage=2h,dumponexit=true,dumponexitpath=/tmp,loglevel=trace Pramod
[jfr][INFO ][0.000] JFR log level set. Log level now at [TRACE] [jfr][TRACE][0.001] Jfr::initialize_stage_one completed [jfr][TRACE][0.093] Loaded JFR library [jfr][INFO ][0.125] Using /mnt/tmp/2020_02_19_17_26_04_8823 as Flight Recorder repository. [jfr][TRACE][0.157] Default settings loaded. [jfr][TRACE][0.164] VMJFR created. [jfr][TRACE][0.239] classFileLoadHook called for java/lang/Throwable [jfr][TRACE][0.239] instrumenting java.lang.Throwable [jfr][TRACE][0.245] classFileLoadHook called for java/lang/Error [jfr][TRACE][0.245] instrumenting java.lang.Error [jfr][TRACE][0.247] RetransformClasses successful [jfr][TRACE][0.270] Processing instrumentation class: class oracle.jrockit.jfr.FileInputStreamInstrumentor [jfr][TRACE][0.275] Inliner processing method read()I [jfr][TRACE][0.275] MethodCallInliner: targetMethod=oracle.jrockit.jfr.FileInputStreamInstrumentor.read()I [jfr][TRACE][0.275] Inlining call to read()I [jfr][TRACE][0.275] Inlining done [jfr][TRACE][0.275] Inlining call to read()I [jfr][TRACE][0.275] Inlining done [jfr][TRACE][0.275] Inliner processing method read([B)I [jfr][TRACE][0.275] MethodCallInliner: targetMethod=oracle.jrockit.jfr.FileInputStreamInstrumentor.read([B)I [jfr][TRACE][0.276] Inlining call to read([B)I [jfr][TRACE][0.276] Inlining done [jfr][TRACE][0.276] Inlining call to read([B)I [jfr][TRACE][0.276] Inlining done [jfr][TRACE][0.276] Inliner processing method read([BII)I [jfr][TRACE][0.276] MethodCallInliner: targetMethod=oracle.jrockit.jfr.FileInputStreamInstrumentor.read([BII)I [jfr][TRACE][0.276] Inlining call to read([BII)I [jfr][TRACE][0.276] Inlining done [jfr][TRACE][0.276] Inlining call to read([BII)I [jfr][TRACE][0.276] Inlining done [jfr][TRACE][0.279] Deleting read([B)I [jfr][TRACE][0.279] Deleting read()I [jfr][TRACE][0.279] Deleting read([BII)I [jfr][TRACE][0.280] Copying method: read()I [jfr][TRACE][0.280] with mapper: {oracle/jrockit/jfr/FileInputStreamInstrumentor=java/io/FileInputStream} [jfr][TRACE][0.282] Copying method: read([B)I [jfr][TRACE][0.282] with mapper: {oracle/jrockit/jfr/FileInputStreamInstrumentor=java/io/FileInputStream} [jfr][TRACE][0.282] Copying method: read([BII)I [jfr][TRACE][0.282] with mapper: {oracle/jrockit/jfr/FileInputStreamInstrumentor=java/io/FileInputStream} [jfr][TRACE][0.284] Processing instrumentation class: class oracle.jrockit.jfr.FileOutputStreamInstrumentor [jfr][TRACE][0.285] Inliner processing method write(I)V [jfr][TRACE][0.285] MethodCallInliner: targetMethod=oracle.jrockit.jfr.FileOutputStreamInstrumentor.write(I)V [jfr][TRACE][0.285] Inlining call to write(I)V [jfr][TRACE][0.285] Inlining done [jfr][TRACE][0.285] Inlining call to write(I)V [jfr][TRACE][0.285] Inlining done [jfr][TRACE][0.286] Inliner processing method write([B)V [jfr][TRACE][0.286] MethodCallInliner: targetMethod=oracle.jrockit.jfr.FileOutputStreamInstrumentor.write([B)V [jfr][TRACE][0.286] Inlining call to write([B)V [jfr][TRACE][0.286] Inlining done [jfr][TRACE][0.286] Inlining call to write([B)V [jfr][TRACE][0.286] Inlining done [jfr][TRACE][0.286] Inliner processing method write([BII)V [jfr][TRACE][0.286] MethodCallInliner: targetMethod=oracle.jrockit.jfr.FileOutputStreamInstrumentor.write([BII)V [jfr][TRACE][0.286] Inlining call to write([BII)V [jfr][TRACE][0.286] Inlining done [jfr][TRACE][0.286] Inlining call to write([BII)V [jfr][TRACE][0.286] Inlining done [jfr][TRACE][0.287] Deleting write([B)V [jfr][TRACE][0.287] Deleting write([BII)V [jfr][TRACE][0.287] Deleting write(I)V [jfr][TRACE][0.288] Copying method: write(I)V [jfr][TRACE][0.288] with mapper: {oracle/jrockit/jfr/FileOutputStreamInstrumentor=java/io/FileOutputStream} [jfr][TRACE][0.289] Copying method: write([B)V [jfr][TRACE][0.289] with mapper: {oracle/jrockit/jfr/FileOutputStreamInstrumentor=java/io/FileOutputStream} [jfr][TRACE][0.289] Copying method: write([BII)V [jfr][TRACE][0.289] with mapper: {oracle/jrockit/jfr/FileOutputStreamInstrumentor=java/io/FileOutputStream} [jfr][TRACE][0.291] Processing instrumentation class: class oracle.jrockit.jfr.RandomAccessFileInstrumentor [jfr][TRACE][0.294] Inliner processing method read()I [jfr][TRACE][0.294] MethodCallInliner: targetMethod=oracle.jrockit.jfr.RandomAccessFileInstrumentor.read()I [jfr][TRACE][0.294] Inlining call to read()I [jfr][TRACE][0.294] Inlining done [jfr][TRACE][0.294] Inlining call to read()I [jfr][TRACE][0.294] Inlining done [jfr][TRACE][0.294] Inliner processing method read([B)I [jfr][TRACE][0.294] MethodCallInliner: targetMethod=oracle.jrockit.jfr.RandomAccessFileInstrumentor.read([B)I [jfr][TRACE][0.294] Inlining call to read([B)I [jfr][TRACE][0.294] Inlining done [jfr][TRACE][0.294] Inlining call to read([B)I [jfr][TRACE][0.294] Inlining done [jfr][TRACE][0.294] Inliner processing method read([BII)I [jfr][TRACE][0.294] MethodCallInliner: targetMethod=oracle.jrockit.jfr.RandomAccessFileInstrumentor.read([BII)I [jfr][TRACE][0.294] Inlining call to read([BII)I [jfr][TRACE][0.294] Inlining done [jfr][TRACE][0.294] Inlining call to read([BII)I [jfr][TRACE][0.294] Inlining done [jfr][TRACE][0.295] Inliner processing method write(I)V [jfr][TRACE][0.295] MethodCallInliner: targetMethod=oracle.jrockit.jfr.RandomAccessFileInstrumentor.write(I)V [jfr][TRACE][0.295] Inlining call to write(I)V [jfr][TRACE][0.295] Inlining done [jfr][TRACE][0.295] Inlining call to write(I)V [jfr][TRACE][0.295] Inlining done [jfr][TRACE][0.295] Inliner processing method write([B)V [jfr][TRACE][0.295] MethodCallInliner: targetMethod=oracle.jrockit.jfr.RandomAccessFileInstrumentor.write([B)V [jfr][TRACE][0.295] Inlining call to write([B)V [jfr][TRACE][0.295] Inlining done [jfr][TRACE][0.295] Inlining call to write([B)V [jfr][TRACE][0.295] Inlining done [jfr][TRACE][0.295] Inliner processing method write([BII)V [jfr][TRACE][0.295] MethodCallInliner: targetMethod=oracle.jrockit.jfr.RandomAccessFileInstrumentor.write([BII)V [jfr][TRACE][0.295] Inlining call to write([BII)V [jfr][TRACE][0.295] Inlining done [jfr][TRACE][0.295] Inlining call to write([BII)V [jfr][TRACE][0.295] Inlining done [jfr][TRACE][0.296] Deleting write([BII)V [jfr][TRACE][0.296] Deleting write(I)V [jfr][TRACE][0.296] Deleting write([B)V [jfr][TRACE][0.296] Deleting read([BII)I [jfr][TRACE][0.296] Deleting read([B)I [jfr][TRACE][0.296] Deleting read()I [jfr][TRACE][0.299] Copying method: read()I [jfr][TRACE][0.299] with mapper: {oracle/jrockit/jfr/RandomAccessFileInstrumentor=java/io/RandomAccessFile} [jfr][TRACE][0.300] Copying method: read([B)I [jfr][TRACE][0.300] with mapper: {oracle/jrockit/jfr/RandomAccessFileInstrumentor=java/io/RandomAccessFile} [jfr][TRACE][0.300] Copying method: read([BII)I [jfr][TRACE][0.300] with mapper: {oracle/jrockit/jfr/RandomAccessFileInstrumentor=java/io/RandomAccessFile} [jfr][TRACE][0.301] Copying method: write(I)V [jfr][TRACE][0.301] with mapper: {oracle/jrockit/jfr/RandomAccessFileInstrumentor=java/io/RandomAccessFile} [jfr][TRACE][0.302] Copying method: write([B)V [jfr][TRACE][0.302] with mapper: {oracle/jrockit/jfr/RandomAccessFileInstrumentor=java/io/RandomAccessFile} [jfr][TRACE][0.303] Copying method: write([BII)V [jfr][TRACE][0.303] with mapper: {oracle/jrockit/jfr/RandomAccessFileInstrumentor=java/io/RandomAccessFile} [jfr][TRACE][0.305] Processing instrumentation class: class oracle.jrockit.jfr.FileChannelImplInstrumentor [jfr][TRACE][0.312] Inliner processing method read(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.312] MethodCallInliner: targetMethod=oracle.jrockit.jfr.FileChannelImplInstrumentor.read(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.312] Inlining call to read(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.313] Inlining done [jfr][TRACE][0.313] Inlining call to read(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.313] Inlining done [jfr][TRACE][0.313] Inliner processing method read(Ljava/nio/ByteBuffer;J)I [jfr][TRACE][0.313] MethodCallInliner: targetMethod=oracle.jrockit.jfr.FileChannelImplInstrumentor.read(Ljava/nio/ByteBuffer;J)I [jfr][TRACE][0.313] Inlining call to read(Ljava/nio/ByteBuffer;J)I [jfr][TRACE][0.313] Inlining done [jfr][TRACE][0.313] Inlining call to read(Ljava/nio/ByteBuffer;J)I [jfr][TRACE][0.313] Inlining done [jfr][TRACE][0.313] Inliner processing method read([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.313] MethodCallInliner: targetMethod=oracle.jrockit.jfr.FileChannelImplInstrumentor.read([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.313] Inlining call to read([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.313] Inlining done [jfr][TRACE][0.314] Inlining call to read([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.314] Inlining done [jfr][TRACE][0.314] Inliner processing method write(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.314] MethodCallInliner: targetMethod=oracle.jrockit.jfr.FileChannelImplInstrumentor.write(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.314] Inlining call to write(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.314] Inlining done [jfr][TRACE][0.314] Inlining call to write(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.314] Inlining done [jfr][TRACE][0.314] Inliner processing method write(Ljava/nio/ByteBuffer;J)I [jfr][TRACE][0.314] MethodCallInliner: targetMethod=oracle.jrockit.jfr.FileChannelImplInstrumentor.write(Ljava/nio/ByteBuffer;J)I [jfr][TRACE][0.314] Inlining call to write(Ljava/nio/ByteBuffer;J)I [jfr][TRACE][0.315] Inlining done [jfr][TRACE][0.315] Inlining call to write(Ljava/nio/ByteBuffer;J)I [jfr][TRACE][0.315] Inlining done [jfr][TRACE][0.315] Inliner processing method write([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.315] MethodCallInliner: targetMethod=oracle.jrockit.jfr.FileChannelImplInstrumentor.write([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.315] Inlining call to write([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.315] Inlining done [jfr][TRACE][0.315] Inlining call to write([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.315] Inlining done [jfr][TRACE][0.319] Deleting write(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.319] Deleting write(Ljava/nio/ByteBuffer;J)I [jfr][TRACE][0.319] Deleting write([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.319] Deleting read(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.319] Deleting read([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.319] Deleting read(Ljava/nio/ByteBuffer;J)I [jfr][TRACE][0.330] Copying method: read(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.330] with mapper: {oracle/jrockit/jfr/FileChannelImplInstrumentor=sun/nio/ch/FileChannelImpl} [jfr][TRACE][0.333] Copying method: read(Ljava/nio/ByteBuffer;J)I [jfr][TRACE][0.333] with mapper: {oracle/jrockit/jfr/FileChannelImplInstrumentor=sun/nio/ch/FileChannelImpl} [jfr][TRACE][0.334] Copying method: read([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.334] with mapper: {oracle/jrockit/jfr/FileChannelImplInstrumentor=sun/nio/ch/FileChannelImpl} [jfr][TRACE][0.338] Copying method: write(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.338] with mapper: {oracle/jrockit/jfr/FileChannelImplInstrumentor=sun/nio/ch/FileChannelImpl} [jfr][TRACE][0.341] Copying method: write(Ljava/nio/ByteBuffer;J)I [jfr][TRACE][0.341] with mapper: {oracle/jrockit/jfr/FileChannelImplInstrumentor=sun/nio/ch/FileChannelImpl} [jfr][TRACE][0.343] Copying method: write([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.343] with mapper: {oracle/jrockit/jfr/FileChannelImplInstrumentor=sun/nio/ch/FileChannelImpl} [jfr][TRACE][0.351] Processing instrumentation class: class oracle.jrockit.jfr.SocketInputStreamInstrumentor [jfr][TRACE][0.353] Inliner processing method read([BIII)I [jfr][TRACE][0.353] MethodCallInliner: targetMethod=oracle.jrockit.jfr.SocketInputStreamInstrumentor.read([BIII)I [jfr][TRACE][0.353] Inlining call to read([BIII)I [jfr][TRACE][0.353] Inlining done [jfr][TRACE][0.353] Inlining call to read([BIII)I [jfr][TRACE][0.353] Inlining done [jfr][TRACE][0.354] Deleting read([BIII)I [jfr][TRACE][0.354] Copying method: read([BIII)I [jfr][TRACE][0.354] with mapper: {oracle/jrockit/jfr/SocketInputStreamInstrumentor=java/net/SocketInputStream, oracle/jrockit/jfr/SocketInputStreamInstrumentor$InetAddressHolder=java/net/InetAddress$InetAddressHolder, oracle/jrockit/jfr/SocketInputStreamInstrumentor$AbstractPlainSocketImpl=java/net/AbstractPlainSocketImpl, oracle/jrockit/jfr/SocketInputStreamInstrumentor$InetAddress=java/net/InetAddress} [jfr][TRACE][0.361] Processing instrumentation class: class oracle.jrockit.jfr.SocketOutputStreamInstrumentor [jfr][TRACE][0.362] Inliner processing method socketWrite([BII)V [jfr][TRACE][0.362] MethodCallInliner: targetMethod=oracle.jrockit.jfr.SocketOutputStreamInstrumentor.socketWrite([BII)V [jfr][TRACE][0.362] Inlining call to socketWrite([BII)V [jfr][TRACE][0.362] Inlining done [jfr][TRACE][0.362] Inlining call to socketWrite([BII)V [jfr][TRACE][0.363] Inlining done [jfr][TRACE][0.363] Deleting socketWrite([BII)V [jfr][TRACE][0.363] Copying method: socketWrite([BII)V [jfr][TRACE][0.363] with mapper: {oracle/jrockit/jfr/SocketOutputStreamInstrumentor$InetAddressHolder=java/net/InetAddress$InetAddressHolder, oracle/jrockit/jfr/SocketOutputStreamInstrumentor=java/net/SocketOutputStream, oracle/jrockit/jfr/SocketOutputStreamInstrumentor$AbstractPlainSocketImpl=java/net/AbstractPlainSocketImpl, oracle/jrockit/jfr/SocketOutputStreamInstrumentor$InetAddress=java/net/InetAddress} [jfr][TRACE][0.368] Processing instrumentation class: class oracle.jrockit.jfr.SocketChannelImplInstrumentor [jfr][TRACE][0.373] Inliner processing method read(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.373] MethodCallInliner: targetMethod=oracle.jrockit.jfr.SocketChannelImplInstrumentor.read(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.373] Inlining call to read(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.374] Inlining done [jfr][TRACE][0.374] Inlining call to read(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.374] Inlining done [jfr][TRACE][0.374] Inliner processing method read([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.374] MethodCallInliner: targetMethod=oracle.jrockit.jfr.SocketChannelImplInstrumentor.read([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.374] Inlining call to read([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.375] Inlining done [jfr][TRACE][0.375] Inlining call to read([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.375] Inlining done [jfr][TRACE][0.375] Inliner processing method write(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.375] MethodCallInliner: targetMethod=oracle.jrockit.jfr.SocketChannelImplInstrumentor.write(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.375] Inlining call to write(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.376] Inlining done [jfr][TRACE][0.376] Inlining call to write(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.376] Inlining done [jfr][TRACE][0.376] Inliner processing method write([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.376] MethodCallInliner: targetMethod=oracle.jrockit.jfr.SocketChannelImplInstrumentor.write([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.376] Inlining call to write([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.376] Inlining done [jfr][TRACE][0.376] Inlining call to write([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.377] Inlining done [jfr][TRACE][0.378] Deleting write(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.378] Deleting write([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.378] Deleting read(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.378] Deleting read([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.390] Copying method: read(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.390] with mapper: {oracle/jrockit/jfr/SocketChannelImplInstrumentor=sun/nio/ch/SocketChannelImpl} [jfr][TRACE][0.396] Copying method: read([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.396] with mapper: {oracle/jrockit/jfr/SocketChannelImplInstrumentor=sun/nio/ch/SocketChannelImpl} [jfr][TRACE][0.403] Copying method: write(Ljava/nio/ByteBuffer;)I [jfr][TRACE][0.403] with mapper: {oracle/jrockit/jfr/SocketChannelImplInstrumentor=sun/nio/ch/SocketChannelImpl} [jfr][TRACE][0.409] Copying method: write([Ljava/nio/ByteBuffer;II)J [jfr][TRACE][0.409] with mapper: {oracle/jrockit/jfr/SocketChannelImplInstrumentor=sun/nio/ch/SocketChannelImpl} [jfr][TRACE][0.425] Shutdown hook registered [jfr][TRACE][0.425] Java started. Took 334 ms [jfr][TRACE][0.426] Starting up default recording [jfr][DEBUG][0.426] Bufferthread started (0x228c) [jfr][INFO ][1.935] Starting recording [id=0, name=HotSpot default, start=Wed Feb 19 17:26:06 UTC 2020, maxAge=7200000ms] [jfr][DEBUG][1.936] First recording starting... [jfr][TRACE][1.936] Created chunk path /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_26_06_8823_0.jfr [jfr][TRACE][1.941] Binding recording [id=0, name=HotSpot default, start=Wed Feb 19 17:26:06 UTC 2020, maxAge=7200000ms] [jfr][TRACE][1.941] Default recording started [jfr][TRACE][1.941] Finished starting default recording [jfr][TRACE][1.941] Jfr::initialize_stage_two completed [jfr][TRACE][155.630] Filled global buffer. Total full global buffers 1 [jfr][DEBUG][155.640] Wrote 1 full global buffer(s) of 458576 B of data to chunk. [jfr][TRACE][287.089] Filled global buffer. Total full global buffers 1 [jfr][DEBUG][287.090] Wrote 1 full global buffer(s) of 459678 B of data to chunk. [jfr][TRACE][421.275] Filled global buffer. Total full global buffers 1 [jfr][DEBUG][421.275] Wrote 1 full global buffer(s) of 460393 B of data to chunk. [jfr][TRACE][575.360] Filled global buffer. Total full global buffers 1 [jfr][DEBUG][575.360] Wrote 1 full global buffer(s) of 460354 B of data to chunk. [jfr][TRACE][717.271] Filled global buffer. Total full global buffers 1 [jfr][DEBUG][717.573] Wrote 1 full global buffer(s) of 462769 B of data to chunk. [jfr][DEBUG][880.473] Shutdown hook: destroy java [jfr][INFO ][880.473] Dump on exit: recording 0 'HotSpot default' [jfr][INFO ][880.475] Starting recording [id=1, name=dump-on-exit-clone-of-0, start=Wed Feb 19 17:26:06 UTC 2020] [jfr][DEBUG][880.475] Recording starting, issuing buffer rotation... [jfr][TRACE][880.484] Filled global buffer. Total full global buffers 1 [jfr][DEBUG][880.485] Wrote 1 full global buffer(s) of 458055 B of data to chunk. [jfr][TRACE][880.564] Created chunk path /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_40_45_8823_0.jfr [jfr][WARN ][880.565] VMJFR.finishChunk ignored an exception [jfr][WARN ][880.565] java.io.IOException:No such file or directory [jfr][WARN ][880.565] java.io.UnixFileSystem.createFileExclusively(Native Method) [jfr][WARN ][880.566] java.io.File.createTempFile(File.java:2024) [jfr][WARN ][880.566] java.io.File.createTempFile(File.java:2070) [jfr][WARN ][880.566] oracle.jrockit.jfr.RepositoryChunk$1.run(RepositoryChunk.java:76) [jfr][WARN ][880.566] oracle.jrockit.jfr.RepositoryChunk$1.run(RepositoryChunk.java:73) [jfr][WARN ][880.566] java.security.AccessController.doPrivileged(Native Method) [jfr][WARN ][880.566] java.security.AccessController.doPrivileged(AccessController.java:713) [jfr][WARN ][880.566] oracle.jrockit.jfr.RepositoryChunk.finish(RepositoryChunk.java:73) [jfr][WARN ][880.566] oracle.jrockit.jfr.RepositoryChunk.finish(RepositoryChunk.java:57) [jfr][WARN ][880.566] oracle.jrockit.jfr.VMJFR.finishChunk(VMJFR.java:357) [jfr][DEBUG][880.566] Release chunk /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_26_06_8823_0.jfr.part ref count now 0 [jfr][DEBUG][880.568] Repository chunk /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_26_06_8823_0.jfr.part deleted. [jfr][DEBUG][880.568] Repository chunk /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_26_06_8823_0.jfr deleted. [jfr][DEBUG][880.570] Recording stopping, issuing buffer rotation... [jfr][TRACE][880.589] Created chunk path /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_40_45_8823_1.jfr [jfr][WARN ][880.590] VMJFR.finishChunk ignored an exception [jfr][WARN ][880.590] java.io.IOException:No such file or directory [jfr][WARN ][880.590] java.io.UnixFileSystem.createFileExclusively(Native Method) [jfr][WARN ][880.590] java.io.File.createTempFile(File.java:2024) [jfr][WARN ][880.590] java.io.File.createTempFile(File.java:2070) [jfr][WARN ][880.590] oracle.jrockit.jfr.RepositoryChunk$1.run(RepositoryChunk.java:76) [jfr][WARN ][880.590] oracle.jrockit.jfr.RepositoryChunk$1.run(RepositoryChunk.java:73) [jfr][WARN ][880.590] java.security.AccessController.doPrivileged(Native Method) [jfr][WARN ][880.590] java.security.AccessController.doPrivileged(AccessController.java:713) [jfr][WARN ][880.590] oracle.jrockit.jfr.RepositoryChunk.finish(RepositoryChunk.java:73) [jfr][WARN ][880.590] oracle.jrockit.jfr.RepositoryChunk.finish(RepositoryChunk.java:57) [jfr][WARN ][880.590] oracle.jrockit.jfr.VMJFR.finishChunk(VMJFR.java:357) [jfr][DEBUG][880.590] Release chunk /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_40_45_8823_0.jfr.part ref count now 0 [jfr][DEBUG][880.590] Repository chunk /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_40_45_8823_0.jfr.part deleted. [jfr][DEBUG][880.590] Repository chunk /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_40_45_8823_0.jfr deleted. [jfr][INFO ][880.593] Stopped recording [id=1, name=dump-on-exit-clone-of-0, start=Wed Feb 19 17:26:06 UTC 2020, end=Wed Feb 19 17:40:45 UTC 2020] [jfr][ERROR][880.593] Could not copy to /tmp/hotspot-pid-8823-id-0-2020_02_19_17_40_45.jfr [jfr][ERROR][880.593] java.io.FileNotFoundException:No chunks [jfr][ERROR][880.593] oracle.jrockit.jfr.ChunksChannel.<init>(ChunksChannel.java:26) [jfr][ERROR][880.593] oracle.jrockit.jfr.Recording.copyTo(Recording.java:404) [jfr][ERROR][880.593] oracle.jrockit.jfr.JFRImpl.dumpOnExit(JFRImpl.java:802) [jfr][ERROR][880.593] oracle.jrockit.jfr.JFRImpl.dumpOnExit(JFRImpl.java:789) [jfr][ERROR][880.593] oracle.jrockit.jfr.JFRImpl.destroy(JFRImpl.java:767) [jfr][ERROR][880.593] oracle.jrockit.jfr.JFR$2.run(JFR.java:208) [jfr][ERROR][880.593] java.lang.Thread.run(Thread.java:748) [jfr][ERROR][880.593] Could not delete /tmp/hotspot-pid-8823-id-0-2020_02_19_17_40_45.jfr [jfr][ERROR][880.593] java.io.FileNotFoundException:No chunks [jfr][ERROR][880.593] oracle.jrockit.jfr.ChunksChannel.<init>(ChunksChannel.java:26) [jfr][ERROR][880.593] oracle.jrockit.jfr.Recording.copyTo(Recording.java:404) [jfr][ERROR][880.593] oracle.jrockit.jfr.JFRImpl.dumpOnExit(JFRImpl.java:802) [jfr][ERROR][880.593] oracle.jrockit.jfr.JFRImpl.dumpOnExit(JFRImpl.java:789) [jfr][ERROR][880.593] oracle.jrockit.jfr.JFRImpl.destroy(JFRImpl.java:767) [jfr][ERROR][880.593] oracle.jrockit.jfr.JFR$2.run(JFR.java:208) [jfr][ERROR][880.593] java.lang.Thread.run(Thread.java:748) [jfr][WARN ][880.593] Could not dump recording 0 'HotSpot default' on exit [jfr][WARN ][880.593] java.io.FileNotFoundException:No chunks [jfr][WARN ][880.593] oracle.jrockit.jfr.ChunksChannel.<init>(ChunksChannel.java:26) [jfr][WARN ][880.593] oracle.jrockit.jfr.Recording.copyTo(Recording.java:404) [jfr][WARN ][880.593] oracle.jrockit.jfr.JFRImpl.dumpOnExit(JFRImpl.java:802) [jfr][WARN ][880.593] oracle.jrockit.jfr.JFRImpl.dumpOnExit(JFRImpl.java:789) [jfr][WARN ][880.593] oracle.jrockit.jfr.JFRImpl.destroy(JFRImpl.java:767) [jfr][WARN ][880.593] oracle.jrockit.jfr.JFR$2.run(JFR.java:208) [jfr][WARN ][880.593] java.lang.Thread.run(Thread.java:748) [jfr][DEBUG][880.593] Last recording stopping... [jfr][WARN ][880.608] VMJFR.finishChunk ignored an exception [jfr][WARN ][880.608] java.io.IOException:No such file or directory [jfr][WARN ][880.608] java.io.UnixFileSystem.createFileExclusively(Native Method) [jfr][WARN ][880.608] java.io.File.createTempFile(File.java:2024) [jfr][WARN ][880.608] java.io.File.createTempFile(File.java:2070) [jfr][WARN ][880.608] oracle.jrockit.jfr.RepositoryChunk$1.run(RepositoryChunk.java:76) [jfr][WARN ][880.608] oracle.jrockit.jfr.RepositoryChunk$1.run(RepositoryChunk.java:73) [jfr][WARN ][880.608] java.security.AccessController.doPrivileged(Native Method) [jfr][WARN ][880.608] java.security.AccessController.doPrivileged(AccessController.java:713) [jfr][WARN ][880.608] oracle.jrockit.jfr.RepositoryChunk.finish(RepositoryChunk.java:73) [jfr][WARN ][880.608] oracle.jrockit.jfr.RepositoryChunk.finish(RepositoryChunk.java:57) [jfr][WARN ][880.608] oracle.jrockit.jfr.VMJFR.finishChunk(VMJFR.java:357) [jfr][DEBUG][880.608] Release chunk /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_40_45_8823_1.jfr.part ref count now 0 [jfr][DEBUG][880.608] Repository chunk /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_40_45_8823_1.jfr.part deleted. [jfr][DEBUG][880.608] Repository chunk /mnt/tmp/2020_02_19_17_26_04_8823/2020_02_19_17_40_45_8823_1.jfr deleted. [jfr][INFO ][880.609] Stopped recording [id=0, name=HotSpot default, start=Wed Feb 19 17:26:06 UTC 2020, end=Wed Feb 19 17:40:45 UTC 2020, maxAge=7200000ms] [jfr][INFO ][880.609] Flight Recorder repository deleted [jfr][DEBUG][880.609] Shutdown hook: shutdown native [jfr][DEBUG][880.609] Bufferthread stopped.
--------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org