Hi Siegmar,
mpiexec and java run as distinct processes. Your JRE message
says java process raises SEGV. So you should trace the java
process, not the mpiexec process. And more, your JRE message
says the crash happened outside the Java Virtual Machine in
native code. So usual Java program debugger is useless.
You should trace native code part of the java process.
Unfortunately I don't know how to debug such one.
The log file output by JRE may help you.
> # An error report file with more information is saved as:
> # /home/fd1026/work/skripte/master/parallel/prog/mpi/java/hs_err_pid13080.log
Regards,
Takahiro
> Hi,
>
> I installed openmpi-dev-124-g91e9686 on Solaris 10 Sparc with
> gcc-4.9.1 to track down the error with my small Java program.
> I started single stepping in orterun.c at line 1081 and
> continued until I got the segmentation fault. I get
> "jdata = 0x0" in version openmpi-1.8.2a1r31804, which is the
> last one which works with Java in my environment, while I get
> "jdata = 0x100125250" in this version. Unfortunately I don't
> know which files or variables are important to look at. Perhaps
> somebody can look at the following lines of code and tell me,
> which information I should provide to solve the problem. I know
> that Solaris isn't any longer on your list of supported systems,
> but perhaps we can get it working again, if you tell me what
> you need and I do the debugging.
>
> /usr/local/gdb-7.6.1_64_gcc/bin/gdb mpiexec
> GNU gdb (GDB) 7.6.1
> ...
> (gdb) run -np 1 java InitFinalizeMain
> Starting program: /usr/local/openmpi-1.9.0_64_gcc/bin/mpiexec \
> -np 1 java InitFinalizeMain
> [Thread debugging using libthread_db enabled]
> [New Thread 1 (LWP 1)]
> [New LWP 2 ]
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0xffffffff7ea3c7f0, pid=13064, tid=2
> ...
> [LWP 2 exited]
> [New Thread 2 ]
> [Switching to Thread 1 (LWP 1)]
> sol_thread_fetch_registers: td_ta_map_id2thr: no thread can be
> found to satisfy query
> (gdb) thread 1
> [Switching to thread 1 (LWP 1 )]
> #0 0xffffffff7f6173d0 in rtld_db_dlactivity () from /usr/lib/sparcv9/ld.so.1
> (gdb) b orterun.c:1081
> Breakpoint 1 at 0x1000070dc: file
> ../../../../openmpi-dev-124-g91e9686/orte/tools/orterun/orterun.c, line 1081.
> (gdb) r
> The program being debugged has been started already.
> Start it from the beginning? (y or n) y
>
> Starting program: /usr/local/openmpi-1.9.0_64_gcc/bin/mpiexec -np 1 java
> InitFinalizeMain
> [Thread debugging using libthread_db enabled]
> [New Thread 1 (LWP 1)]
> [New LWP 2 ]
> [Switching to Thread 1 (LWP 1)]
>
> Breakpoint 1, orterun (argc=5, argv=0xffffffff7fffe0d8)
> at ../../../../openmpi-dev-124-g91e9686/orte/tools/orterun/orterun.c:1081
> 1081 rc = orte_plm.spawn(jdata);
> (gdb) print jdata
> $1 = (orte_job_t *) 0x100125250
> (gdb) s
> rsh_launch (jdata=0x100125250)
> at
> ../../../../../openmpi-dev-124-g91e9686/orte/mca/plm/rsh/plm_rsh_module.c:876
> 876 if (ORTE_FLAG_TEST(jdata, ORTE_JOB_FLAG_RESTART)) {
> (gdb) s
> 881 ORTE_ACTIVATE_JOB_STATE(jdata, ORTE_JOB_STATE_INIT);
> (gdb)
> orte_util_print_name_args (name=0x100118380 <orte_process_info+104>)
> at ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:122
> 122 if (NULL == name) {
> (gdb)
> 142 job = orte_util_print_jobids(name->jobid);
> (gdb)
> orte_util_print_jobids (job=2502885376) at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:170
> 170 ptr = get_print_name_buffer();
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:92
> 92 if (!fns_init) {
> (gdb)
> 101 ret = opal_tsd_getspecific(print_args_tsd_key, (void**)&ptr);
> (gdb)
> opal_tsd_getspecific (key=1, valuep=0xffffffff7fffd990)
> at ../../openmpi-dev-124-g91e9686/opal/threads/tsd.h:163
> 163 *valuep = pthread_getspecific(key);
> (gdb)
> 164 return OPAL_SUCCESS;
> (gdb)
> 165 }
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:102
> 102 if (OPAL_SUCCESS != ret) return NULL;
> (gdb)
> 104 if (NULL == ptr) {
> (gdb)
> 113 return (orte_print_args_buffers_t*) ptr;
> (gdb)
> 114 }
> (gdb)
> orte_util_print_jobids (job=2502885376) at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:172
> 172 if (NULL == ptr) {
> (gdb)
> 178 if (ORTE_PRINT_NAME_ARG_NUM_BUFS == ptr->cntr) {
> (gdb)
> 182 if (ORTE_JOBID_INVALID == job) {
> (gdb)
> 184 } else if (ORTE_JOBID_WILDCARD == job) {
> (gdb)
> 187 tmp1 = ORTE_JOB_FAMILY((unsigned long)job);
> (gdb)
> 188 tmp2 = ORTE_LOCAL_JOBID((unsigned long)job);
> (gdb)
> 189 snprintf(ptr->buffers[ptr->cntr++],
> (gdb)
> 193 return ptr->buffers[ptr->cntr-1];
> (gdb)
> 194 }
> (gdb)
> orte_util_print_name_args (name=0x100118380 <orte_process_info+104>)
> at ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:143
> 143 vpid = orte_util_print_vpids(name->vpid);
> (gdb)
> orte_util_print_vpids (vpid=0) at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:260
> 260 ptr = get_print_name_buffer();
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:92
> 92 if (!fns_init) {
> (gdb)
> 101 ret = opal_tsd_getspecific(print_args_tsd_key, (void**)&ptr);
> (gdb)
> opal_tsd_getspecific (key=1, valuep=0xffffffff7fffd9a0)
> at ../../openmpi-dev-124-g91e9686/opal/threads/tsd.h:163
> 163 *valuep = pthread_getspecific(key);
> (gdb)
> 164 return OPAL_SUCCESS;
> (gdb)
> 165 }
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:102
> 102 if (OPAL_SUCCESS != ret) return NULL;
> (gdb)
> 104 if (NULL == ptr) {
> (gdb)
> 113 return (orte_print_args_buffers_t*) ptr;
> (gdb)
> 114 }
> (gdb)
> orte_util_print_vpids (vpid=0) at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:262
> 262 if (NULL == ptr) {
> (gdb)
> 268 if (ORTE_PRINT_NAME_ARG_NUM_BUFS == ptr->cntr) {
> (gdb)
> 272 if (ORTE_VPID_INVALID == vpid) {
> (gdb)
> 274 } else if (ORTE_VPID_WILDCARD == vpid) {
> (gdb)
> 277 snprintf(ptr->buffers[ptr->cntr++],
> (gdb)
> 281 return ptr->buffers[ptr->cntr-1];
> (gdb)
> 282 }
> (gdb)
> orte_util_print_name_args (name=0x100118380 <orte_process_info+104>)
> at ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:146
> 146 ptr = get_print_name_buffer();
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:92
> 92 if (!fns_init) {
> (gdb)
> 101 ret = opal_tsd_getspecific(print_args_tsd_key, (void**)&ptr);
> (gdb)
> opal_tsd_getspecific (key=1, valuep=0xffffffff7fffda60)
> at ../../openmpi-dev-124-g91e9686/opal/threads/tsd.h:163
> 163 *valuep = pthread_getspecific(key);
> (gdb)
> 164 return OPAL_SUCCESS;
> (gdb)
> 165 }
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:102
> 102 if (OPAL_SUCCESS != ret) return NULL;
> (gdb)
> 104 if (NULL == ptr) {
> (gdb)
> 113 return (orte_print_args_buffers_t*) ptr;
> (gdb)
> 114 }
> (gdb)
> orte_util_print_name_args (name=0x100118380 <orte_process_info+104>)
> at ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:148
> 148 if (NULL == ptr) {
> (gdb)
> 154 if (ORTE_PRINT_NAME_ARG_NUM_BUFS == ptr->cntr) {
> (gdb)
> 158 snprintf(ptr->buffers[ptr->cntr++],
> (gdb)
> 162 return ptr->buffers[ptr->cntr-1];
> (gdb)
> 163 }
> (gdb)
> orte_util_print_jobids (job=4294967295) at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:170
> 170 ptr = get_print_name_buffer();
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:92
> 92 if (!fns_init) {
> (gdb)
> 101 ret = opal_tsd_getspecific(print_args_tsd_key, (void**)&ptr);
> (gdb)
> opal_tsd_getspecific (key=1, valuep=0xffffffff7fffda60)
> at ../../openmpi-dev-124-g91e9686/opal/threads/tsd.h:163
> 163 *valuep = pthread_getspecific(key);
> (gdb)
> 164 return OPAL_SUCCESS;
> (gdb)
> 165 }
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:102
> 102 if (OPAL_SUCCESS != ret) return NULL;
> (gdb)
> 104 if (NULL == ptr) {
> (gdb)
> 113 return (orte_print_args_buffers_t*) ptr;
> (gdb)
> 114 }
> (gdb)
> orte_util_print_jobids (job=4294967295) at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:172
> 172 if (NULL == ptr) {
> (gdb)
> 178 if (ORTE_PRINT_NAME_ARG_NUM_BUFS == ptr->cntr) {
> (gdb)
> 182 if (ORTE_JOBID_INVALID == job) {
> (gdb)
> 183 snprintf(ptr->buffers[ptr->cntr++],
> ORTE_PRINT_NAME_ARGS_MAX_SIZE, "[INVALID]");
> (gdb)
> 193 return ptr->buffers[ptr->cntr-1];
> (gdb)
> 194 }
> (gdb)
> orte_job_state_to_str (state=1) at
> ../../openmpi-dev-124-g91e9686/orte/util/error_strings.c:217
> 217 switch(state) {
> (gdb)
> 221 return "PENDING INIT";
> (gdb)
> 317 }
> (gdb)
> opal_output_verbose (level=1, output_id=0,
> format=0xffffffff7f14dd98 <orte_job_states>
> "\336\257\276\355\336\257\276\355")
> at ../../../openmpi-dev-124-g91e9686/opal/util/output.c:373
> 373 va_start(arglist, format);
> (gdb)
> 369 {
> (gdb)
> 370 if (output_id >= 0 && output_id < OPAL_OUTPUT_MAX_STREAMS &&
> (gdb)
> 377 }
> (gdb)
> orte_state_base_activate_job_state (jdata=0x100125250, state=1)
> at
> ../../../../openmpi-dev-124-g91e9686/orte/mca/state/base/state_base_fns.c:33
> 33 opal_list_item_t *itm, *any=NULL, *error=NULL;
> (gdb)
> 37 for (itm = opal_list_get_first(&orte_job_states);
> (gdb)
> opal_list_get_first (list=0xffffffff7f14dd98 <orte_job_states>)
> at ../../../../openmpi-dev-124-g91e9686/opal/class/opal_list.h:320
> 320 opal_list_item_t* item =
> (opal_list_item_t*)list->opal_list_sentinel.opal_list_next;
> (gdb)
> 324 assert(1 == item->opal_list_item_refcount);
> (gdb)
> 325 assert( list == item->opal_list_item_belong_to );
> (gdb)
> 328 return item;
> (gdb)
> 329 }
> (gdb)
> orte_state_base_activate_job_state (jdata=0x100125250, state=1)
> at
> ../../../../openmpi-dev-124-g91e9686/orte/mca/state/base/state_base_fns.c:38
> 38 itm != opal_list_get_end(&orte_job_states);
> (gdb)
> opal_list_get_end (list=0xffffffff7f14dd98 <orte_job_states>)
> at ../../../../openmpi-dev-124-g91e9686/opal/class/opal_list.h:399
> 399 return &(list->opal_list_sentinel);
> (gdb)
> 400 }
> (gdb)
> orte_state_base_activate_job_state (jdata=0x100125250, state=1)
> at
> ../../../../openmpi-dev-124-g91e9686/orte/mca/state/base/state_base_fns.c:37
> 37 for (itm = opal_list_get_first(&orte_job_states);
> (gdb)
> 40 s = (orte_state_t*)itm;
> (gdb)
> 41 if (s->job_state == ORTE_JOB_STATE_ANY) {
> (gdb)
> 45 if (s->job_state == ORTE_JOB_STATE_ERROR) {
> (gdb)
> 48 if (s->job_state == state) {
> (gdb)
> 49 OPAL_OUTPUT_VERBOSE((1,
> orte_state_base_framework.framework_output,
> (gdb)
> orte_util_print_name_args (name=0x100118380 <orte_process_info+104>)
> at ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:122
> 122 if (NULL == name) {
> (gdb)
> 142 job = orte_util_print_jobids(name->jobid);
> (gdb)
> orte_util_print_jobids (job=2502885376) at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:170
> 170 ptr = get_print_name_buffer();
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:92
> 92 if (!fns_init) {
> (gdb)
> 101 ret = opal_tsd_getspecific(print_args_tsd_key, (void**)&ptr);
> (gdb)
> opal_tsd_getspecific (key=1, valuep=0xffffffff7fffd880)
> at ../../openmpi-dev-124-g91e9686/opal/threads/tsd.h:163
> 163 *valuep = pthread_getspecific(key);
> (gdb)
> 164 return OPAL_SUCCESS;
> (gdb)
> 165 }
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:102
> 102 if (OPAL_SUCCESS != ret) return NULL;
> (gdb)
> 104 if (NULL == ptr) {
> (gdb)
> 113 return (orte_print_args_buffers_t*) ptr;
> (gdb)
> 114 }
> (gdb)
> orte_util_print_jobids (job=2502885376) at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:172
> 172 if (NULL == ptr) {
> (gdb)
> 178 if (ORTE_PRINT_NAME_ARG_NUM_BUFS == ptr->cntr) {
> (gdb)
> 182 if (ORTE_JOBID_INVALID == job) {
> (gdb)
> 184 } else if (ORTE_JOBID_WILDCARD == job) {
> (gdb)
> 187 tmp1 = ORTE_JOB_FAMILY((unsigned long)job);
> (gdb)
> 188 tmp2 = ORTE_LOCAL_JOBID((unsigned long)job);
> (gdb)
> 189 snprintf(ptr->buffers[ptr->cntr++],
> (gdb)
> 193 return ptr->buffers[ptr->cntr-1];
> (gdb)
> 194 }
> (gdb)
> orte_util_print_name_args (name=0x100118380 <orte_process_info+104>)
> at ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:143
> 143 vpid = orte_util_print_vpids(name->vpid);
> (gdb)
> orte_util_print_vpids (vpid=0) at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:260
> 260 ptr = get_print_name_buffer();
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:92
> 92 if (!fns_init) {
> (gdb)
> 101 ret = opal_tsd_getspecific(print_args_tsd_key, (void**)&ptr);
> (gdb)
> opal_tsd_getspecific (key=1, valuep=0xffffffff7fffd890)
> at ../../openmpi-dev-124-g91e9686/opal/threads/tsd.h:163
> 163 *valuep = pthread_getspecific(key);
> (gdb)
> 164 return OPAL_SUCCESS;
> (gdb)
> 165 }
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:102
> 102 if (OPAL_SUCCESS != ret) return NULL;
> (gdb)
> 104 if (NULL == ptr) {
> (gdb)
> 113 return (orte_print_args_buffers_t*) ptr;
> (gdb)
> 114 }
> (gdb)
> orte_util_print_vpids (vpid=0) at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:262
> 262 if (NULL == ptr) {
> (gdb)
> 268 if (ORTE_PRINT_NAME_ARG_NUM_BUFS == ptr->cntr) {
> (gdb)
> 272 if (ORTE_VPID_INVALID == vpid) {
> (gdb)
> 274 } else if (ORTE_VPID_WILDCARD == vpid) {
> (gdb)
> 277 snprintf(ptr->buffers[ptr->cntr++],
> (gdb)
> 281 return ptr->buffers[ptr->cntr-1];
> (gdb)
> 282 }
> (gdb)
> orte_util_print_name_args (name=0x100118380 <orte_process_info+104>)
> at ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:146
> 146 ptr = get_print_name_buffer();
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:92
> 92 if (!fns_init) {
> (gdb)
> 101 ret = opal_tsd_getspecific(print_args_tsd_key, (void**)&ptr);
> (gdb)
> opal_tsd_getspecific (key=1, valuep=0xffffffff7fffd950)
> at ../../openmpi-dev-124-g91e9686/opal/threads/tsd.h:163
> 163 *valuep = pthread_getspecific(key);
> (gdb)
> 164 return OPAL_SUCCESS;
> (gdb)
> 165 }
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:102
> 102 if (OPAL_SUCCESS != ret) return NULL;
> (gdb)
> 104 if (NULL == ptr) {
> (gdb)
> 113 return (orte_print_args_buffers_t*) ptr;
> (gdb)
> 114 }
> (gdb)
> orte_util_print_name_args (name=0x100118380 <orte_process_info+104>)
> at ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:148
> 148 if (NULL == ptr) {
> (gdb)
> 154 if (ORTE_PRINT_NAME_ARG_NUM_BUFS == ptr->cntr) {
> (gdb)
> 158 snprintf(ptr->buffers[ptr->cntr++],
> (gdb)
> 162 return ptr->buffers[ptr->cntr-1];
> (gdb)
> 163 }
> (gdb)
> orte_util_print_jobids (job=4294967295) at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:170
> 170 ptr = get_print_name_buffer();
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:92
> 92 if (!fns_init) {
> (gdb)
> 101 ret = opal_tsd_getspecific(print_args_tsd_key, (void**)&ptr);
> (gdb)
> opal_tsd_getspecific (key=1, valuep=0xffffffff7fffd950)
> at ../../openmpi-dev-124-g91e9686/opal/threads/tsd.h:163
> 163 *valuep = pthread_getspecific(key);
> (gdb)
> 164 return OPAL_SUCCESS;
> (gdb)
> 165 }
> (gdb)
> get_print_name_buffer () at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:102
> 102 if (OPAL_SUCCESS != ret) return NULL;
> (gdb)
> 104 if (NULL == ptr) {
> (gdb)
> 113 return (orte_print_args_buffers_t*) ptr;
> (gdb)
> 114 }
> (gdb)
> orte_util_print_jobids (job=4294967295) at
> ../../openmpi-dev-124-g91e9686/orte/util/name_fns.c:172
> 172 if (NULL == ptr) {
> (gdb)
> 178 if (ORTE_PRINT_NAME_ARG_NUM_BUFS == ptr->cntr) {
> (gdb)
> 182 if (ORTE_JOBID_INVALID == job) {
> (gdb)
> 183 snprintf(ptr->buffers[ptr->cntr++],
> ORTE_PRINT_NAME_ARGS_MAX_SIZE, "[INVALID]");
> (gdb)
> 193 return ptr->buffers[ptr->cntr-1];
> (gdb)
> 194 }
> (gdb)
> orte_job_state_to_str (state=1) at
> ../../openmpi-dev-124-g91e9686/orte/util/error_strings.c:217
> 217 switch(state) {
> (gdb)
> 221 return "PENDING INIT";
> (gdb)
> 317 }
> (gdb)
> opal_output_verbose (level=1, output_id=-1, format=0x1 <Address 0x1 out of
> bounds>)
> at ../../../openmpi-dev-124-g91e9686/opal/util/output.c:373
> 373 va_start(arglist, format);
> (gdb)
> 369 {
> (gdb)
> 370 if (output_id >= 0 && output_id < OPAL_OUTPUT_MAX_STREAMS &&
> (gdb)
> 377 }
> (gdb)
> orte_state_base_activate_job_state (jdata=0x100125250, state=1)
> at
> ../../../../openmpi-dev-124-g91e9686/orte/mca/state/base/state_base_fns.c:54
> 54 if (NULL == s->cbfunc) {
> (gdb)
> 62 caddy = OBJ_NEW(orte_state_caddy_t);
> (gdb)
> opal_obj_new_debug (type=0xffffffff7f14c7d8 <orte_state_caddy_t_class>,
> file=0xffffffff7f034c08
> "../../../../openmpi-dev-124-g91e9686/orte/mca/state/base/state_base_fns.c",
> line=62) at ../../../../openmpi-dev-124-g91e9686/opal/class/opal_object.h:249
> 249 opal_object_t* object = opal_obj_new(type);
> (gdb)
> opal_obj_new (cls=0xffffffff7f14c7d8 <orte_state_caddy_t_class>)
> at ../../../../openmpi-dev-124-g91e9686/opal/class/opal_object.h:465
> 465 assert(cls->cls_sizeof >= sizeof(opal_object_t));
> (gdb)
> 470 object = (opal_object_t *) malloc(cls->cls_sizeof);
> (gdb)
> 472 if (0 == cls->cls_initialized) {
> (gdb)
> 473 opal_class_initialize(cls);
> (gdb)
> opal_class_initialize (cls=0xffffffff7f14c7d8 <orte_state_caddy_t_class>)
> at ../../openmpi-dev-124-g91e9686/opal/class/opal_object.c:79
> 79 assert(cls);
> (gdb)
> 84 if (1 == cls->cls_initialized) {
> (gdb)
> 87 opal_atomic_lock(&class_lock);
> (gdb)
> opal_atomic_lock (lock=0xffffffff7ee89bf0 <class_lock>)
> at ../../openmpi-dev-124-g91e9686/opal/include/opal/sys/atomic_impl.h:397
> 397 while( !opal_atomic_cmpset_acq_32( &(lock->u.lock),
> (gdb)
> opal_atomic_cmpset_acq_32 (addr=0xffffffff7ee89bf0 <class_lock>, oldval=0,
> newval=1)
> at
> ../../openmpi-dev-124-g91e9686/opal/include/opal/sys/sparcv9/atomic.h:107
> 107 rc = opal_atomic_cmpset_32(addr, oldval, newval);
> (gdb)
> opal_atomic_cmpset_32 (addr=0xffffffff7ee89bf0 <class_lock>, oldval=0,
> newval=1)
> at
> ../../openmpi-dev-124-g91e9686/opal/include/opal/sys/sparcv9/atomic.h:93
> 93 int32_t ret = newval;
> (gdb)
> 95 __asm__ __volatile__("casa [%1] " ASI_P ", %2, %0"
> (gdb)
> 98 return (ret == oldval);
> (gdb)
> 99 }
> (gdb)
> opal_atomic_cmpset_acq_32 (addr=0xffffffff7ee89bf0 <class_lock>, oldval=0,
> newval=1)
> at
> ../../openmpi-dev-124-g91e9686/opal/include/opal/sys/sparcv9/atomic.h:108
> 108 opal_atomic_rmb();
> (gdb)
> opal_atomic_rmb () at
> ../../openmpi-dev-124-g91e9686/opal/include/opal/sys/sparcv9/atomic.h:63
> 63 MEMBAR("#LoadLoad");
> (gdb)
> 64 }
> (gdb)
> opal_atomic_cmpset_acq_32 (addr=0xffffffff7ee89bf0 <class_lock>, oldval=0,
> newval=1)
> at
> ../../openmpi-dev-124-g91e9686/opal/include/opal/sys/sparcv9/atomic.h:110
> 110 return rc;
> (gdb)
> 111 }
> (gdb)
> opal_atomic_lock (lock=0xffffffff7ee89bf0 <class_lock>)
> at ../../openmpi-dev-124-g91e9686/opal/include/opal/sys/atomic_impl.h:403
> 403 }
> (gdb)
> opal_class_initialize (cls=0xffffffff7f14c7d8 <orte_state_caddy_t_class>)
> at ../../openmpi-dev-124-g91e9686/opal/class/opal_object.c:93
> 93 if (1 == cls->cls_initialized) {
> (gdb)
> 103 cls->cls_depth = 0;
> (gdb)
> 104 cls_construct_array_count = 0;
> (gdb)
> 105 cls_destruct_array_count = 0;
> (gdb)
> 106 for (c = cls; c; c = c->cls_parent) {
> (gdb)
> 107 if( NULL != c->cls_construct ) {
> (gdb)
> 108 cls_construct_array_count++;
> (gdb)
> 110 if( NULL != c->cls_destruct ) {
> (gdb)
> 111 cls_destruct_array_count++;
> (gdb)
> 113 cls->cls_depth++;
> (gdb)
> 106 for (c = cls; c; c = c->cls_parent) {
> (gdb)
> 107 if( NULL != c->cls_construct ) {
> (gdb)
> 110 if( NULL != c->cls_destruct ) {
> (gdb)
> 113 cls->cls_depth++;
> (gdb)
> 106 for (c = cls; c; c = c->cls_parent) {
> (gdb)
> 122 (void (**)(opal_object_t*))malloc((cls_construct_array_count +
> (gdb)
> 123 cls_destruct_array_count +
> 2)
> *
> (gdb)
> 122 (void (**)(opal_object_t*))malloc((cls_construct_array_count +
> (gdb)
> 121 cls->cls_construct_array =
> (gdb)
> 125 if (NULL == cls->cls_construct_array) {
> (gdb)
> 130 cls->cls_construct_array + cls_construct_array_count + 1;
> (gdb)
> 129 cls->cls_destruct_array =
> (gdb)
> 136 cls_construct_array = cls->cls_construct_array +
> cls_construct_array_count;
> (gdb)
> 137 cls_destruct_array = cls->cls_destruct_array;
> (gdb)
> 139 c = cls;
> (gdb)
> 140 *cls_construct_array = NULL; /* end marker for the constructors
> */
> (gdb)
> 141 for (i = 0; i < cls->cls_depth; i++) {
> (gdb)
> 142 if( NULL != c->cls_construct ) {
> (gdb)
> 143 --cls_construct_array;
> (gdb)
> 144 *cls_construct_array = c->cls_construct;
> (gdb)
> 146 if( NULL != c->cls_destruct ) {
> (gdb)
> 147 *cls_destruct_array = c->cls_destruct;
> (gdb)
> 148 cls_destruct_array++;
> (gdb)
> 150 c = c->cls_parent;
> (gdb)
> 141 for (i = 0; i < cls->cls_depth; i++) {
> (gdb)
> 142 if( NULL != c->cls_construct ) {
> (gdb)
> 146 if( NULL != c->cls_destruct ) {
> (gdb)
> 150 c = c->cls_parent;
> (gdb)
> 141 for (i = 0; i < cls->cls_depth; i++) {
> (gdb)
> 152 *cls_destruct_array = NULL; /* end marker for the destructors */
> (gdb)
> 154 cls->cls_initialized = 1;
> (gdb)
> 155 save_class(cls);
> (gdb)
> save_class (cls=0xffffffff7f14c7d8 <orte_state_caddy_t_class>)
> at ../../openmpi-dev-124-g91e9686/opal/class/opal_object.c:188
> 188 if (num_classes >= max_classes) {
> (gdb)
> 189 expand_array();
> (gdb)
> expand_array () at ../../openmpi-dev-124-g91e9686/opal/class/opal_object.c:201
> 201 max_classes += increment;
> (gdb)
> 202 classes = (void**)realloc(classes, sizeof(opal_class_t*) *
> max_classes);
> (gdb)
> 203 if (NULL == classes) {
> (gdb)
> 207 for (i = num_classes; i < max_classes; ++i) {
> (gdb)
> 208 classes[i] = NULL;
> (gdb)
> 207 for (i = num_classes; i < max_classes; ++i) {
> (gdb)
> 208 classes[i] = NULL;
> (gdb)
> 207 for (i = num_classes; i < max_classes; ++i) {
> (gdb)
> 208 classes[i] = NULL;
> (gdb)
> 207 for (i = num_classes; i < max_classes; ++i) {
> (gdb)
> 208 classes[i] = NULL;
> (gdb)
> 207 for (i = num_classes; i < max_classes; ++i) {
> (gdb)
> 208 classes[i] = NULL;
> (gdb)
> 207 for (i = num_classes; i < max_classes; ++i) {
> (gdb)
> 208 classes[i] = NULL;
> (gdb)
> 207 for (i = num_classes; i < max_classes; ++i) {
> (gdb)
> 208 classes[i] = NULL;
> (gdb)
> 207 for (i = num_classes; i < max_classes; ++i) {
> (gdb)
> 208 classes[i] = NULL;
> (gdb)
> 207 for (i = num_classes; i < max_classes; ++i) {
> (gdb)
> 208 classes[i] = NULL;
> (gdb)
> 207 for (i = num_classes; i < max_classes; ++i) {
> (gdb)
> 208 classes[i] = NULL;
> (gdb)
> 207 for (i = num_classes; i < max_classes; ++i) {
> (gdb)
> 210 }
> (gdb)
> save_class (cls=0xffffffff7f14c7d8 <orte_state_caddy_t_class>)
> at ../../openmpi-dev-124-g91e9686/opal/class/opal_object.c:192
> 192 classes[num_classes] = cls->cls_construct_array;
> (gdb)
> 193 ++num_classes;
> (gdb)
> 194 }
> (gdb)
> opal_class_initialize (cls=0xffffffff7f14c7d8 <orte_state_caddy_t_class>)
> at ../../openmpi-dev-124-g91e9686/opal/class/opal_object.c:159
> 159 opal_atomic_unlock(&class_lock);
> (gdb)
> opal_atomic_unlock (lock=0xffffffff7ee89bf0 <class_lock>)
> at ../../openmpi-dev-124-g91e9686/opal/include/opal/sys/atomic_impl.h:409
> 409 opal_atomic_wmb();
> (gdb)
> opal_atomic_wmb () at
> ../../openmpi-dev-124-g91e9686/opal/include/opal/sys/sparcv9/atomic.h:69
> 69 MEMBAR("#StoreStore");
> (gdb)
> 70 }
> (gdb)
> opal_atomic_unlock (lock=0xffffffff7ee89bf0 <class_lock>)
> at ../../openmpi-dev-124-g91e9686/opal/include/opal/sys/atomic_impl.h:410
> 410 lock->u.lock=OPAL_ATOMIC_UNLOCKED;
> (gdb)
> 411 }
> (gdb)
> opal_class_initialize (cls=0xffffffff7f14c7d8 <orte_state_caddy_t_class>)
> at ../../openmpi-dev-124-g91e9686/opal/class/opal_object.c:160
> 160 }
> (gdb)
> opal_obj_new (cls=0xffffffff7f14c7d8 <orte_state_caddy_t_class>)
> at ../../../../openmpi-dev-124-g91e9686/opal/class/opal_object.h:475
> 475 if (NULL != object) {
> (gdb)
> 476 object->obj_class = cls;
> (gdb)
> 477 object->obj_reference_count = 1;
> (gdb)
> 478 opal_obj_run_constructors(object);
> (gdb)
> opal_obj_run_constructors (object=0x1001bfcf0)
> at ../../../../openmpi-dev-124-g91e9686/opal/class/opal_object.h:420
> 420 assert(NULL != object->obj_class);
> (gdb)
> 422 cls_construct = object->obj_class->cls_construct_array;
> (gdb)
> 423 while( NULL != *cls_construct ) {
> (gdb)
> 424 (*cls_construct)(object);
> (gdb)
> orte_state_caddy_construct (caddy=0x1001bfcf0)
> at
> ../../../../openmpi-dev-124-g91e9686/orte/mca/state/base/state_base_frame.c:84
> 84 memset(&caddy->ev, 0, sizeof(opal_event_t));
> (gdb)
> 85 caddy->jdata = NULL;
> (gdb)
> 86 }
> (gdb)
> opal_obj_run_constructors (object=0x1001bfcf0)
> at ../../../../openmpi-dev-124-g91e9686/opal/class/opal_object.h:425
> 425 cls_construct++;
> (gdb)
> 423 while( NULL != *cls_construct ) {
> (gdb)
> 427 }
> (gdb)
> opal_obj_new (cls=0xffffffff7f14c7d8 <orte_state_caddy_t_class>)
> at ../../../../openmpi-dev-124-g91e9686/opal/class/opal_object.h:480
> 480 return object;
> (gdb)
> 481 }
> (gdb)
> opal_obj_new_debug (type=0xffffffff7f14c7d8 <orte_state_caddy_t_class>,
> file=0xffffffff7f034c08
> "../../../../openmpi-dev-124-g91e9686/orte/mca/state/base/state_base_fns.c",
> line=62) at ../../../../openmpi-dev-124-g91e9686/opal/class/opal_object.h:250
> 250 object->obj_magic_id = OPAL_OBJ_MAGIC_ID;
> (gdb)
> 251 object->cls_init_file_name = file;
> (gdb)
> 252 object->cls_init_lineno = line;
> (gdb)
> 253 return object;
> (gdb)
> 254 }
> (gdb)
> orte_state_base_activate_job_state (jdata=0x100125250, state=1)
> at
> ../../../../openmpi-dev-124-g91e9686/orte/mca/state/base/state_base_fns.c:63
> 63 if (NULL != jdata) {
> (gdb)
> 64 caddy->jdata = jdata;
> (gdb)
> 65 caddy->job_state = state;
> (gdb)
> 66 OBJ_RETAIN(jdata);
> (gdb)
> opal_obj_update (inc=1, object=0x100125250)
> at ../../../../openmpi-dev-124-g91e9686/opal/class/opal_object.h:497
> 497 return opal_atomic_add_32(&(object->obj_reference_count), inc);
> (gdb)
> opal_atomic_add_32 (addr=0x100125260, delta=1)
> at
> ../../../../openmpi-dev-124-g91e9686/opal/include/opal/sys/atomic_impl.h:63
> 63 oldval = *addr;
> (gdb)
> 64 } while (0 == opal_atomic_cmpset_32(addr, oldval, oldval + delta));
> (gdb)
> opal_atomic_cmpset_32 (addr=0x100125260, oldval=1, newval=2)
> at
> ../../../../openmpi-dev-124-g91e9686/opal/include/opal/sys/sparcv9/atomic.h:93
> 93 int32_t ret = newval;
> (gdb)
> 95 __asm__ __volatile__("casa [%1] " ASI_P ", %2, %0"
> (gdb)
> 98 return (ret == oldval);
> (gdb)
> 99 }
> (gdb)
> opal_atomic_add_32 (addr=0x100125260, delta=1)
> at
> ../../../../openmpi-dev-124-g91e9686/opal/include/opal/sys/atomic_impl.h:65
> 65 return (oldval + delta);
> (gdb)
> 66 }
> (gdb)
> orte_state_base_activate_job_state (jdata=0x100125250, state=1)
> at
> ../../../../openmpi-dev-124-g91e9686/orte/mca/state/base/state_base_fns.c:66
> 66 OBJ_RETAIN(jdata);
> (gdb)
> 68 opal_event_set(orte_event_base, &caddy->ev, -1,
> OPAL_EV_WRITE, s->cbfunc, caddy);
> (gdb)
> 69 opal_event_set_priority(&caddy->ev, s->priority);
> (gdb)
> 70 opal_event_active(&caddy->ev, OPAL_EV_WRITE, 1);
> (gdb)
> 71 return;
> (gdb)
> 105 }
> (gdb)
> rsh_launch (jdata=0x100125250)
> at
> ../../../../../openmpi-dev-124-g91e9686/orte/mca/plm/rsh/plm_rsh_module.c:883
> 883 return ORTE_SUCCESS;
> (gdb)
> 884 }
> (gdb)
> orterun (argc=5, argv=0xffffffff7fffe0d8)
> at ../../../../openmpi-dev-124-g91e9686/orte/tools/orterun/orterun.c:1084
> 1084 while (orte_event_base_active) {
> (gdb)
> 1085 opal_event_loop(orte_event_base, OPAL_EVLOOP_ONCE);
> (gdb)
> 1084 while (orte_event_base_active) {
> (gdb)
> 1085 opal_event_loop(orte_event_base, OPAL_EVLOOP_ONCE);
> (gdb)
> 1084 while (orte_event_base_active) {
> (gdb)
> 1085 opal_event_loop(orte_event_base, OPAL_EVLOOP_ONCE);
> (gdb)
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0xffffffff7ea3c7f0, pid=13080, tid=2
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0-b132) (build 1.8.0-b132)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.0-b70 mixed mode
> solaris-sparc
> compressed oops)
> # Problematic frame:
> # 1084 while (orte_event_base_active) {
> (gdb)
> 1085 opal_event_loop(orte_event_base, OPAL_EVLOOP_ONCE);
> (gdb)
> C [libc.so.1+0x3c7f0] strlen+0x50
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /home/fd1026/work/skripte/master/parallel/prog/mpi/java/hs_err_pid13080.log
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.sun.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
> --------------------------------------------------------------------------
> mpiexec noticed that process rank 0 with PID 0 on node tyr exited on signal 6
> (Abort).
> --------------------------------------------------------------------------
> 1084 while (orte_event_base_active) {
> (gdb)
> 1089 orte_odls.kill_local_procs(NULL);
> (gdb)
>
>
> Thank you very much for any help in advance.
>
> Kind regards
>
> Siegmar
