*Issue Summary:* Framework reboot due to native crash in zygote crash with
SIGSEGV error.
*Reproduction:* Long duration test with multiple apps and reproduction
rate – 1/100.
*Description:*
Below is the tombstone for zygote:
Line 56603:07-08 10:19:39.605 26565 26565 F DEBUG : Build
fingerprint:
------------------------------------------------------------------
Line 56604: 07-08 10:19:39.605 26565 26565 F DEBUG : Revision: '0'
Line 56605: 07-08 10:19:39.605 26565 26565 F DEBUG : ABI: 'arm'
Line 56608: 07-08 10:19:39.606 26565 26565 F DEBUG : pid: 652,
tid: 26546, name: HeapTaskDaemon >>> zygote <<<
Line 56609: 07-08 10:19:39.606 26565 26565 F DEBUG : signal 11
(SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x130
Line 56610: 07-08 10:19:39.606 26565 26565 F DEBUG : Cause: null
pointer dereference
Line 56611: 07-08 10:19:39.606 26565 26565 F DEBUG : r0
dec0b000 r1 5616eb84 r2 00000007 r3 00000000
Line 56612: 07-08 10:19:39.606 26565 26565 F DEBUG : r4
00000130 r5 cd7929c8 r6 00000000 r7 e4fdc380
Line 56613: 07-08 10:19:39.606 26565 26565 F DEBUG : r8
cd7924bc r9 00000000 r10 00000002 r11 00008a0c
Line 56614: 07-08 10:19:39.607 26565 26565 F DEBUG : ip
e49b8974 sp cd7924b0 lr e45e1507 pc e45ec316
Line 56712: 07-08 10:19:39.730 26565 26565 F DEBUG :
Line 56713: 07-08 10:19:39.730 26565 26565 F DEBUG : backtrace:
Line 56715: 07-08 10:19:39.730 26565 26565 F DEBUG : #00 pc
000aa316 /system/lib/libart.so (art::TimingLogger::Reset()+106)
Line 56716: 07-08 10:19:39.730 26565 26565 F DEBUG : #01 pc
0016663b /system/lib/libart.so
(art::gc::collector::GarbageCollector::Run(art::gc::GcCause, bool)+178)
Line 56717: 07-08 10:19:39.730 26565 26565 F DEBUG : #02 pc
0018035d /system/lib/libart.so
(art::gc::Heap::CollectGarbageInternal(art::gc::collector::GcType,
art::gc::GcCause, bool)+2420)
Line 56718: 07-08 10:19:39.730 26565 26565 F DEBUG : #03 pc
0018dbeb /system/lib/libart.so (art::gc::Heap::ConcurrentGC(art::Thread*,
art::gc::GcCause, bool)+182)
Line 56719: 07-08 10:19:39.730 26565 26565 F DEBUG : #04 pc
00191b11 /system/lib/libart.so
(art::gc::Heap::ConcurrentGCTask::Run(art::Thread*)+20)
Line 56720: 07-08 10:19:39.730 26565 26565 F DEBUG : #05 pc
001aa957 /system/lib/libart.so
(art::gc::TaskProcessor::RunAllTasks(art::Thread*)+34)
Line 56721: 07-08 10:19:39.731 26565 26565 F DEBUG : #06 pc
0007463b /system/framework/arm/boot-core-libart.oat (offset 0x73000)
(dalvik.system.VMRuntime.clampGrowthLimit [DEDUPED]+74)
Line 56722: 07-08 10:19:39.731 26565 26565 F DEBUG : #07 pc
0014a85d /system/framework/arm/boot-core-libart.oat (offset 0x73000)
(java.lang.Daemons$HeapTaskDaemon.runInternal+172)
Line 56723: 07-08 10:19:39.731 26565 26565 F DEBUG : #08 pc
000ec963 /system/framework/arm/boot-core-libart.oat (offset 0x73000)
(java.lang.Daemons$Daemon.run+66)
Line 56724: 07-08 10:19:39.731 26565 26565 F DEBUG : #09 pc
002151b1 /system/framework/arm/boot-core-oj.oat (offset 0x106000)
(java.lang.Thread.run+64)
Line 56725: 07-08 10:19:39.731 26565 26565 F DEBUG : #10 pc
00411575 /system/lib/libart.so (art_quick_invoke_stub_internal+68)
Line 56726: 07-08 10:19:39.731 26565 26565 F DEBUG : #11 pc
003eb045 /system/lib/libart.so (art_quick_invoke_stub+224)
Line 56727: 07-08 10:19:39.731 26565 26565 F DEBUG : #12 pc
000a183d /system/lib/libart.so (art::ArtMethod::Invoke(art::Thread*,
unsigned int*, unsigned int, art::JValue*, char const*)+136)
Line 56728: 07-08 10:19:39.731 26565 26565 F DEBUG : #13 pc
003498d5 /system/lib/libart.so (art::(anonymous
namespace)::InvokeWithArgArray(art::ScopedObjectAccessAlreadyRunnable
const&, art::ArtMethod*, art::(anonymous namespace)::ArgArray*,
art::JValue*, char const*)+52)
Line 56729: 07-08 10:19:39.731 26565 26565 F DEBUG : #14 pc
0034a62d /system/lib/libart.so
(art::InvokeVirtualOrInterfaceWithJValues(art::ScopedObjectAccessAlreadyRunnable
const&, _jobject*, _jmethodID*, jvalue*)+320)
Line 56730: 07-08 10:19:39.731 26565 26565 F DEBUG : #15 pc
0036d0a3 /system/lib/libart.so (art::Thread::CreateCallback(void*)+866)
Line 56731: 07-08 10:19:39.731 26565 26565 F DEBUG : #16 pc
00072dcd /system/lib/libc.so (__pthread_start(void*)+22)
Line 56732: 07-08 10:19:39.731 26565 26565 F DEBUG : #17 pc
0001e3b1 /system/lib/libc.so (__start_thread+24)
One more observation is that we saw few app crashes prior to zygote crash
in the path of zygote forking these apps.
pid: 17395, tid: 17395, name: o.android.imoi >>> zygote <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x4
Cause: null pointer dereference
r0 00000000 r1 8b148311 r2 00000000 r3 00000000
r4 e4bdc424 r5 e4bdc420 r6 e4b97a64 r7 e4bdc3c8
r8 e4bc8000 r9 e4bdc448 r10 00003000 r11 00000003
ip 000000ff sp ffbe1230 lr e41e7dd5 pc e41e7de8
backtrace:
#00 pc 000a8de8 /system/lib/libart.so
(art::CumulativeLogger::Reset()+68)
#01 pc 00166901 /system/lib/libart.so
(art::gc::collector::GarbageCollector:: ()+192)
#02 pc 0017d853 /system/lib/libart.so
(art::gc::Heap::ResetGcPerformanceInfo()+34)
#03 pc 003570db /system/lib/libart.so
(art::Runtime::InitNonZygoteOrPostFork(_JNIEnv*, bool,
art::Runtime::NativeBridgeAction, char const*, bool)+74)
#04 pc 002e8fb7 /system/lib/libart.so
(art::ZygoteHooks_nativePostForkChild(_JNIEnv*, _jclass*, long long, int,
unsigned char, unsigned char, _jstring*)+3146)
#05 pc 00074c63 /system/framework/arm/boot-core-libart.oat (offset
0x73000) (dalvik.system.ZygoteHooks.nativePostForkChild+154)
#06 pc 000eba15 /system/framework/arm/boot-core-libart.oat (offset
0x73000) (dalvik.system.ZygoteHooks.postForkChild+68)
#07 pc 00ba0ab9 /system/framework/arm/boot-framework.oat (offset
0x393000) (com.android.internal.os.Zygote.callPostForkChildHooks+80)
#08 pc 00412975 /system/lib/libart.so
(art_quick_invoke_stub_internal+68)
#09 pc 003eaec7 /system/lib/libart.so
(art_quick_invoke_static_stub+222)
#10 pc 000a184f /system/lib/libart.so
(art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int,
art::JValue*, char const*)+154)
#11 pc 00349655 /system/lib/libart.so (art::(anonymous
namespace)::InvokeWithArgArray(art::ScopedObjectAccessAlreadyRunnable
const&, art::ArtMethod*, art::(anonymous namespace)::ArgArray*,
art::JValue*, char const*)+52)
#12 pc 0034947f /system/lib/libart.so
(art::InvokeWithVarArgs(art::ScopedObjectAccessAlreadyRunnable const&,
_jobject*, _jmethodID*, std::__va_list)+310)
#13 pc 00290219 /system/lib/libart.so
(art::JNI::CallStaticVoidMethodV(_JNIEnv*, _jclass*, _jmethodID*,
std::__va_list)+444)
#14 pc 0006e579 /system/lib/libandroid_runtime.so
(_JNIEnv::CallStaticVoidMethod(_jclass*, _jmethodID*, ...)+28)
#15 pc 0011c2ed /system/lib/libandroid_runtime.so ((anonymous
namespace)::ForkAndSpecializeCommon(_JNIEnv*, unsigned int, unsigned int,
_jintArray*, int, _jobjectArray*, long long, long long, int, _jstring*,
_jstring*, bool, _jintArray*, _jintArray*, bool, _jstring*, _jstring*)+4052)
#16 pc 0011ab37 /system/lib/libandroid_runtime.so
(android::com_android_internal_os_Zygote_nativeForkAndSpecialize(_JNIEnv*,
_jclass*, int, int, _jintArray*, int, _jobjectArray*, int, _jstring*,
_jstring*, _jintArray*, _jintArray*, unsigned char, _jstring*,
_jstring*)+470)
#17 pc 003b8ba3 /system/framework/arm/boot-framework.oat (offset
0x393000) (com.android.internal.os.Zygote.nativeForkAndSpecialize+338)
#18 pc 00ba3a8b /system/framework/arm/boot-framework.oat (offset
0x393000) (com.android.internal.os.ZygoteConnection.processOneCommand+1450)
#19 pc 00ba7a5b /system/framework/arm/boot-framework.oat (offset
0x393000) (com.android.internal.os.ZygoteServer.runSelectLoop+770)
#20 pc 00ba5269 /system/framework/arm/boot-framework.oat (offset
0x393000) (com.android.internal.os.ZygoteInit.main+1696)
#21 pc 00412975 /system/lib/libart.so
(art_quick_invoke_stub_internal+68)
#22 pc 003eaec7 /system/lib/libart.so
(art_quick_invoke_static_stub+222)
#23 pc 000a184f /system/lib/libart.so
(art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int,
art::JValue*, char const*)+154)
#24 pc 00349655 /system/lib/libart.so (art::(anonymous
namespace)::InvokeWithArgArray(art::ScopedObjectAccessAlreadyRunnable
const&, art::ArtMethod*, art::(anonymous namespace)::ArgArray*,
art::JValue*, char const*)+52)
#25 pc 0034947f /system/lib/libart.so
(art::InvokeWithVarArgs(art::ScopedObjectAccessAlreadyRunnable const&,
_jobject*, _jmethodID*, std::__va_list)+310)
#26 pc 00290219 /system/lib/libart.so
(art::JNI::CallStaticVoidMethodV(_JNIEnv*, _jclass*, _jmethodID*,
std::__va_list)+444)
#27 pc 0006e579 /system/lib/libandroid_runtime.so
(_JNIEnv::CallStaticVoidMethod(_jclass*, _jmethodID*, ...)+28)
#28 pc 0007073b /system/lib/libandroid_runtime.so
(android::AndroidRuntime::start(char const*,
android::Vector<android::String8> const&, bool)+462)
#29 pc 00001c8f /system/bin/app_process32 (main+1122)
#30 pc 000a2245 /system/lib/libc.so (__libc_init+48)
#31 pc 000017eb /system/bin/app_process32 (_start_main+38)
#32 pc 000000c4 <unknown>
*Analysis:*
Loaded coredump in GDB:
#0 art::TimingLogger::Reset (this=0x130) at
art/runtime/base/timing_logger.cc:148
No locals.
#1 0xe46a863e in Reset (this=0x120, gc_cause=<optimized out>,
clear_soft_references=<optimized out>) at
art/runtime/gc/collector/garbage_collector.cc:49
No locals.
#2 art::gc::collector::GarbageCollector::Run (this=0xe4fdc380,
gc_cause=art::gc::kGcCauseBackground, clear_soft_references=true) at
art/runtime/gc/collector/garbage_collector.cc:92
start_time = 151785656375586
current_iteration = 0x120
end_time = <optimized out>
self = <optimized out>
#3 0xe46c2360 in art::gc::Heap::CollectGarbageInternal (this=0xe4f3c800,
gc_type=art::gc::collector::kGcTypeFull,
gc_cause=art::gc::kGcCauseBackground, clear_soft_references=<optimized out>)
at art/runtime/gc/heap.cc:2648
runtime = 0xe4f3c400
self = 0xe42acc00
collector = 0xe4fdc380
#4 0xe46cfbee in art::gc::Heap::ConcurrentGC (this=0xe4f3c800,
self=0xe42acc00, cause=art::gc::kGcCauseBackground, force_full=<optimized
out>) at art/runtime/gc/heap.cc:3675
next_gc_type = art::gc::collector::kGcTypeSticky
tid = 26546
#5 0xe46d3b14 in art::gc::Heap::ConcurrentGCTask::Run (this=<optimized
out>, self=0x5616eb84) at art/runtime/gc/heap.cc:3620
heap = 0xe4f3c800
#6 0xe46ec958 in art::gc::TaskProcessor::RunAllTasks (this=0xe4f30200,
self=0xe42acc00) at art/runtime/gc/task_processor.cc:129
task = 0xdec08000
#7 0x720bc63c in ?? ()
>From here, we see that at frame 2, current_iteration = 0x120 is holding
invalid address, which is a member of garbage collector class, see below
code for reference.
In file -- art/runtime/gc/collector/garbage_collector.cc
91 Iteration* current_iteration = GetCurrentIteration();
92 current_iteration->Reset(gc_cause, clear_soft_references);
429 const collector::Iteration* GetCurrentGcIteration() const {
430 return ¤t_gc_iteration_;
431 }
1254 collector::Iteration current_gc_iteration_;
And we see the collector object being zeroed out, which seems to be the
reason for our crash.
gdb) f 2
#2 art::gc::collector::GarbageCollector::Run (this=0xe4fdc380,
gc_cause=art::gc::kGcCauseBackground, clear_soft_references=true) at
art/runtime/gc/collector/garbage_collector.cc:92
92 in art/runtime/gc/collector/garbage_collector.cc
(gdb) x/100 this
0xe4fdc380: 0 0 0 0
0xe4fdc390: 0 0 0 0
0xe4fdc3a0: 0 0 0 0
0xe4fdc3b0: 0 0 0 0
0xe4fdc3c0: 0 0 0 0
0xe4fdc3d0: 0 0 0 0
0xe4fdc3e0: 0 0 0 0
0xe4fdc3f0: 0 0 0 0
0xe4fdc400: 0 0 0 0
0xe4fdc410: 0 0 0 0
0xe4fdc420: 0 0 0 0
0xe4fdc430: 0 0 0 0
0xe4fdc440: 0 0 0 0
0xe4fdc450: 0 0 0 0
0xe4fdc460: 0 0 0 0
0xe4fdc470: 0 0 0 0
0xe4fdc480: 0 0 0 0
0xe4fdc490: 0 0 0 0
0xe4fdc4a0: 0 0 0 0
0xe4fdc4b0: 0 0 0 0
0xe4fdc4c0: 0 0 0 0
0xe4fdc4d0: 0 0 0 0
0xe4fdc4e0: 0 0 0 0
0xe4fdc4f0: 0 0 0 0
The app crashes seen prior to this zygote crash also seem be to due to
similar reason, collector object being NULL.
Can you please help to provide any debug suggestions/ share any similar
instances of this issue ?
Debug approaches we pursued:
We have internally tried to use ASAN and malloc_debug to check is such
corruptions can be caught.
Unfortunately, after enabling malloc_debug, issue was not reproducible.
And with ASAN enablement, device runs slow, and results in other unrelated
issues.
Regards,
Deepika
--
You received this message because you are subscribed to the Google Groups
"Android Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to android-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-developers/ebe1f5f3-fcf0-488e-b087-378dbf7313de%40googlegroups.com.
