Hi Nysal,
This is what I was wondering, it hdr->tag was expected to be null or not. I'll
soon send a valgrind output to the list, hoping this could help to locate an
invalid
memory access allowing to understand why reg->cbfunc / hdr->tag are null.
Do you think that a thread race condition could explain the hdr->tag value ?
Thanks for your help,
Eloi
On Monday 16 August 2010 20:46:39 Nysal Jan wrote:
> The value of hdr->tag seems wrong.
>
> In ompi/mca/pml/ob1/pml_ob1_hdr.h
> #define MCA_PML_OB1_HDR_TYPE_MATCH (MCA_BTL_TAG_PML + 1)
> #define MCA_PML_OB1_HDR_TYPE_RNDV (MCA_BTL_TAG_PML + 2)
> #define MCA_PML_OB1_HDR_TYPE_RGET (MCA_BTL_TAG_PML + 3)
> #define MCA_PML_OB1_HDR_TYPE_ACK (MCA_BTL_TAG_PML + 4)
> #define MCA_PML_OB1_HDR_TYPE_NACK (MCA_BTL_TAG_PML + 5)
> #define MCA_PML_OB1_HDR_TYPE_FRAG (MCA_BTL_TAG_PML + 6)
> #define MCA_PML_OB1_HDR_TYPE_GET (MCA_BTL_TAG_PML + 7)
> #define MCA_PML_OB1_HDR_TYPE_PUT (MCA_BTL_TAG_PML + 8)
> #define MCA_PML_OB1_HDR_TYPE_FIN (MCA_BTL_TAG_PML + 9)
>
> and in ompi/mca/btl/btl.h
> #define MCA_BTL_TAG_PML 0x40
>
> So hdr->tag should be a value >= 65
> Since the tag is incorrect you are not getting the proper callback function
> pointer and hence the SEGV.
> I'm not sure at this point as to why you are getting an invalid/corrupt
> message header ?
>
> --Nysal
>
> On Tue, Aug 10, 2010 at 7:45 PM, Eloi Gaudry <e...@fft.be> wrote:
> > Hi,
> >
> > sorry, i just forgot to add the values of the function parameters:
> > (gdb) print reg->cbdata
> > $1 = (void *) 0x0
> > (gdb) print openib_btl->super
> > $2 = {btl_component = 0x2b341edd7380, btl_eager_limit = 12288,
> > btl_rndv_eager_limit = 12288, btl_max_send_size = 65536,
> > btl_rdma_pipeline_send_length = 1048576,
> >
> > btl_rdma_pipeline_frag_size = 1048576, btl_min_rdma_pipeline_size =
> >
> > 1060864, btl_exclusivity = 1024, btl_latency = 10, btl_bandwidth = 800,
> > btl_flags = 310,
> >
> > btl_add_procs = 0x2b341eb8ee47 <mca_btl_openib_add_procs>, btl_del_procs
> > =
> >
> > 0x2b341eb90156 <mca_btl_openib_del_procs>, btl_register = 0, btl_finalize
> > = 0x2b341eb93186 <mca_btl_openib_finalize>,
> >
> > btl_alloc = 0x2b341eb90a3e <mca_btl_openib_alloc>, btl_free =
> >
> > 0x2b341eb91400 <mca_btl_openib_free>, btl_prepare_src = 0x2b341eb91813
> > <mca_btl_openib_prepare_src>,
> >
> > btl_prepare_dst = 0x2b341eb91f2e <mca_btl_openib_prepare_dst>, btl_send
> > =
> >
> > 0x2b341eb94517 <mca_btl_openib_send>, btl_sendi = 0x2b341eb9340d
> > <mca_btl_openib_sendi>,
> >
> > btl_put = 0x2b341eb94660 <mca_btl_openib_put>, btl_get = 0x2b341eb94c4e
> >
> > <mca_btl_openib_get>, btl_dump = 0x2b341acd45cb <mca_btl_base_dump>,
> > btl_mpool = 0xf3f4110,
> >
> > btl_register_error = 0x2b341eb90565 <mca_btl_openib_register_error_cb>,
> >
> > btl_ft_event = 0x2b341eb952e7 <mca_btl_openib_ft_event>}
> > (gdb) print hdr->tag
> > $3 = 0 '\0'
> > (gdb) print des
> > $4 = (mca_btl_base_descriptor_t *) 0xf4a6700
> > (gdb) print reg->cbfunc
> > $5 = (mca_btl_base_module_recv_cb_fn_t) 0
> >
> > Eloi
> >
> > On Tuesday 10 August 2010 16:04:08 Eloi Gaudry wrote:
> > > Hi,
> > >
> > > Here is the output of a core file generated during a segmentation fault
> > > observed during a collective call (using openib):
> > >
> > > #0 0x0000000000000000 in ?? ()
> > > (gdb) where
> > > #0 0x0000000000000000 in ?? ()
> > > #1 0x00002aedbc4e05f4 in btl_openib_handle_incoming
> > > (openib_btl=0x1902f9b0, ep=0x1908a1c0, frag=0x190d9700, byte_len=18) at
> > > btl_openib_component.c:2881 #2 0x00002aedbc4e25e2 in handle_wc
> > > (device=0x19024ac0, cq=0, wc=0x7ffff279ce90) at
> > > btl_openib_component.c:3178 #3 0x00002aedbc4e2e9d in poll_device
> > > (device=0x19024ac0, count=2) at btl_openib_component.c:3318 #4
> > > 0x00002aedbc4e34b8 in progress_one_device (device=0x19024ac0) at
> > > btl_openib_component.c:3426 #5 0x00002aedbc4e3561 in
> > > btl_openib_component_progress () at btl_openib_component.c:3451 #6
> > > 0x00002aedb8b22ab8 in opal_progress () at runtime/opal_progress.c:207
> > > #7 0x00002aedb859f497 in opal_condition_wait (c=0x2aedb888ccc0,
> > > m=0x2aedb888cd20) at ../opal/threads/condition.h:99 #8
> >
> > 0x00002aedb859fa31
> >
> > > in ompi_request_default_wait_all (count=2, requests=0x7ffff279d0e0,
> > > statuses=0x0) at request/req_wait.c:262 #9 0x00002aedbd7559ad in
> > > ompi_coll_tuned_allreduce_intra_recursivedoubling (sbuf=0x7ffff279d444,
> > > rbuf=0x7ffff279d440, count=1, dtype=0x6788220, op=0x6787a20,
> > > comm=0x19d81ff0, module=0x19d82b20) at coll_tuned_allreduce.c:223
> > > #10 0x00002aedbd7514f7 in ompi_coll_tuned_allreduce_intra_dec_fixed
> > > (sbuf=0x7ffff279d444, rbuf=0x7ffff279d440, count=1, dtype=0x6788220,
> > > op=0x6787a20, comm=0x19d81ff0, module=0x19d82b20) at
> > > coll_tuned_decision_fixed.c:63
> > > #11 0x00002aedb85c7792 in PMPI_Allreduce (sendbuf=0x7ffff279d444,
> > > recvbuf=0x7ffff279d440, count=1, datatype=0x6788220, op=0x6787a20,
> > > comm=0x19d81ff0) at pallreduce.c:102 #12 0x0000000004387dbf in
> > > FEMTown::MPI::Allreduce (sendbuf=0x7ffff279d444,
> > > recvbuf=0x7ffff279d440, count=1, datatype=0x6788220, op=0x6787a20,
> > > comm=0x19d81ff0) at stubs.cpp:626 #13 0x0000000004058be8 in
> > > FEMTown::Domain::align (itf=
> >
> > {<FEMTown::Boost::shared_base_ptr<FEMTown::Domain::Interface>>
> >
> > > = {_vptr.shared_base_ptr = 0x7ffff279d620, ptr_ = {px = 0x199942a4, pn
> > > = {pi_ = 0x6}}}, <No data fields>}) at interface.cpp:371
> > > #14 0x00000000040cb858 in
> >
> > FEMTown::Field::detail::align_itfs_and_neighbhors
> >
> > > (dim=2, set={px = 0x7ffff279d780, pn = {pi_ = 0x2f279d640}},
> > > check_info=@0x7ffff279d7f0) at check.cpp:63 #15 0x00000000040cbfa8 in
> > > FEMTown::Field::align_elements (set={px = 0x7ffff279d950, pn = {pi_ =
> > > 0x66e08d0}}, check_info=@0x7ffff279d7f0) at check.cpp:159 #16
> > > 0x00000000039acdd4 in PyField_align_elements (self=0x0,
> > > args=0x2aaab0765050, kwds=0x19d2e950) at check.cpp:31 #17
> > > 0x0000000001fbf76d in FEMTown::Main::ExErrCatch<_object* (*)(_object*,
> > > _object*, _object*)>::exec<_object> (this=0x7ffff279dc20, s=0x0,
> > > po1=0x2aaab0765050, po2=0x19d2e950) at
> > > /home/qa/svntop/femtown/modules/main/py/exception.hpp:463
> > > #18 0x00000000039acc82 in PyField_align_elements_ewrap (self=0x0,
> > > args=0x2aaab0765050, kwds=0x19d2e950) at check.cpp:39 #19
> > > 0x00000000044093a0 in PyEval_EvalFrameEx (f=0x19b52e90,
> > > throwflag=<value optimized out>) at Python/ceval.c:3921 #20
> > > 0x000000000440aae9 in PyEval_EvalCodeEx (co=0x2aaab754ad50,
> > > globals=<value optimized out>, locals=<value optimized out>, args=0x3,
> > > argcount=1, kws=0x19ace4a0, kwcount=2, defs=0x2aaab75e4800,
> > > defcount=2, closure=0x0) at
> > > Python/ceval.c:2968
> > > #21 0x0000000004408f58 in PyEval_EvalFrameEx (f=0x19ace2d0,
> > > throwflag=<value optimized out>) at Python/ceval.c:3802 #22
> > > 0x000000000440aae9 in PyEval_EvalCodeEx (co=0x2aaab7550120,
> >
> > globals=<value
> >
> > > optimized out>, locals=<value optimized out>, args=0x7, argcount=1,
> > > kws=0x19acc418, kwcount=3, defs=0x2aaab759e958, defcount=6,
> > > closure=0x0) at Python/ceval.c:2968
> > > #23 0x0000000004408f58 in PyEval_EvalFrameEx (f=0x19acc1c0,
> > > throwflag=<value optimized out>) at Python/ceval.c:3802 #24
> > > 0x000000000440aae9 in PyEval_EvalCodeEx (co=0x2aaab8b5e738,
> >
> > globals=<value
> >
> > > optimized out>, locals=<value optimized out>, args=0x6, argcount=1,
> > > kws=0x19abd328, kwcount=5, defs=0x2aaab891b7e8, defcount=3,
> > > closure=0x0) at Python/ceval.c:2968
> > > #25 0x0000000004408f58 in PyEval_EvalFrameEx (f=0x19abcea0,
> > > throwflag=<value optimized out>) at Python/ceval.c:3802 #26
> > > 0x000000000440aae9 in PyEval_EvalCodeEx (co=0x2aaab3eb4198,
> >
> > globals=<value
> >
> > > optimized out>, locals=<value optimized out>, args=0xb, argcount=1,
> > > kws=0x19a89df0, kwcount=10, defs=0x0, defcount=0, closure=0x0) at
> > > Python/ceval.c:2968
> > > #27 0x0000000004408f58 in PyEval_EvalFrameEx (f=0x19a89c40,
> > > throwflag=<value optimized out>) at Python/ceval.c:3802 #28
> > > 0x000000000440aae9 in PyEval_EvalCodeEx (co=0x2aaab3eb4288,
> >
> > globals=<value
> >
> > > optimized out>, locals=<value optimized out>, args=0x1, argcount=0,
> > > kws=0x19a89330, kwcount=0, defs=0x2aaab8b66668, defcount=1,
> > > closure=0x0) at Python/ceval.c:2968
> > > #29 0x0000000004408f58 in PyEval_EvalFrameEx (f=0x19a891b0,
> > > throwflag=<value optimized out>) at Python/ceval.c:3802 #30
> > > 0x000000000440aae9 in PyEval_EvalCodeEx (co=0x2aaab8b6a738,
> >
> > globals=<value
> >
> > > optimized out>, locals=<value optimized out>, args=0x0, argcount=0,
> > > kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at
> > > Python/ceval.c:2968
> > > #31 0x000000000440ac02 in PyEval_EvalCode (co=0x1902f9b0, globals=0x0,
> > > locals=0x190d9700) at Python/ceval.c:522 #32 0x000000000442853c in
> > > PyRun_StringFlags (str=0x192fd3d8 "DIRECT.Actran.main()", start=<value
> > > optimized out>, globals=0x192213d0, locals=0x192213d0, flags=0x0) at
> > > Python/pythonrun.c:1335 #33 0x0000000004429690 in
> > > PyRun_SimpleStringFlags (command=0x192fd3d8 "DIRECT.Actran.main()",
> > > flags=0x0) at
> > > Python/pythonrun.c:957 #34 0x0000000001fa1cf9 in
> > > FEMTown::Python::FEMPy::run_application (this=0x7ffff279f650) at
> > > fempy.cpp:873 #35 0x000000000434ce99 in FEMTown::Main::Batch::run
> > > (this=0x7ffff279f650) at batch.cpp:374 #36 0x0000000001f9aa25 in main
> > > (argc=8, argv=0x7ffff279fa48) at main.cpp:10 (gdb) f 1
> > > #1 0x00002aedbc4e05f4 in btl_openib_handle_incoming
> > > (openib_btl=0x1902f9b0, ep=0x1908a1c0, frag=0x190d9700, byte_len=18) at
> > > btl_openib_component.c:2881 2881 reg->cbfunc(
> >
> > > &openib_btl->super, hdr->tag, des, reg->cbdata ); Current language:
> > auto;
> >
> > > currently c
> > > (gdb)
> > > #1 0x00002aedbc4e05f4 in btl_openib_handle_incoming
> > > (openib_btl=0x1902f9b0, ep=0x1908a1c0, frag=0x190d9700, byte_len=18) at
> > > btl_openib_component.c:2881 2881 reg->cbfunc(
> > > &openib_btl->super, hdr->tag, des, reg->cbdata ); (gdb) l
> > > 2876
> > > 2877 if(OPAL_LIKELY(!(is_credit_msg = is_credit_message(frag))))
> > > { 2878 /* call registered callback */
> > > 2879 mca_btl_active_message_callback_t* reg;
> > > 2880 reg = mca_btl_base_active_message_trigger + hdr->tag;
> > > 2881 reg->cbfunc( &openib_btl->super, hdr->tag, des,
> >
> > reg->cbdata
> >
> > > ); 2882 if(MCA_BTL_OPENIB_RDMA_FRAG(frag)) {
> > > 2883 cqp = (hdr->credits >> 11) & 0x0f;
> > > 2884 hdr->credits &= 0x87ff;
> > > 2885 } else {
> > >
> > > Regards,
> > > Eloi
> > >
> > > On Friday 16 July 2010 16:01:02 Eloi Gaudry wrote:
> > > > Hi Edgar,
> > > >
> > > > The only difference I could observed was that the segmentation fault
> > > > appeared sometimes later during the parallel computation.
> > > >
> > > > I'm running out of idea here. I wish I could use the "--mca coll
> > > > tuned" with "--mca self,sm,tcp" so that I could check that the issue
> > > > is not somehow limited to the tuned collective routines.
> > > >
> > > > Thanks,
> > > > Eloi
> > > >
> > > > On Thursday 15 July 2010 17:24:24 Edgar Gabriel wrote:
> > > > > On 7/15/2010 10:18 AM, Eloi Gaudry wrote:
> > > > > > hi edgar,
> > > > > >
> > > > > > thanks for the tips, I'm gonna try this option as well. the
> > > > > > segmentation fault i'm observing always happened during a
> >
> > collective
> >
> > > > > > communication indeed... does it basically switch all collective
> > > > > > communication to basic mode, right ?
> > > > > >
> > > > > > sorry for my ignorance, but what's a NCA ?
> > > > >
> > > > > sorry, I meant to type HCA (InifinBand networking card)
> > > > >
> > > > > Thanks
> > > > > Edgar
> > > > >
> > > > > > thanks,
> > > > > > éloi
> > > > > >
> > > > > > On Thursday 15 July 2010 16:20:54 Edgar Gabriel wrote:
> > > > > >> you could try first to use the algorithms in the basic module,
> >
> > e.g.
> >
> > > > > >> mpirun -np x --mca coll basic ./mytest
> > > > > >>
> > > > > >> and see whether this makes a difference. I used to observe
> >
> > sometimes
> >
> > > > > >> a (similar ?) problem in the openib btl triggered from the tuned
> > > > > >> collective component, in cases where the ofed libraries were
> > > > > >> installed but no NCA was found on a node. It used to work
> > > > > >> however with the basic component.
> > > > > >>
> > > > > >> Thanks
> > > > > >> Edgar
> > > > > >>
> > > > > >> On 7/15/2010 3:08 AM, Eloi Gaudry wrote:
> > > > > >>> hi Rolf,
> > > > > >>>
> > > > > >>> unfortunately, i couldn't get rid of that annoying segmentation
> > > > > >>> fault when selecting another bcast algorithm. i'm now going to
> > > > > >>> replace MPI_Bcast with a naive implementation (using MPI_Send
> > > > > >>> and MPI_Recv) and see if that helps.
> > > > > >>>
> > > > > >>> regards,
> > > > > >>> éloi
> > > > > >>>
> > > > > >>> On Wednesday 14 July 2010 10:59:53 Eloi Gaudry wrote:
> > > > > >>>> Hi Rolf,
> > > > > >>>>
> > > > > >>>> thanks for your input. You're right, I miss the
> > > > > >>>> coll_tuned_use_dynamic_rules option.
> > > > > >>>>
> > > > > >>>> I'll check if I the segmentation fault disappears when using
> > > > > >>>> the basic bcast linear algorithm using the proper command
> > > > > >>>> line you provided.
> > > > > >>>>
> > > > > >>>> Regards,
> > > > > >>>> Eloi
> > > > > >>>>
> > > > > >>>> On Tuesday 13 July 2010 20:39:59 Rolf vandeVaart wrote:
> > > > > >>>>> Hi Eloi:
> > > > > >>>>> To select the different bcast algorithms, you need to add an
> > > > > >>>>> extra mca parameter that tells the library to use dynamic
> > > > > >>>>> selection. --mca coll_tuned_use_dynamic_rules 1
> > > > > >>>>>
> > > > > >>>>> One way to make sure you are typing this in correctly is to
> > > > > >>>>> use it with ompi_info. Do the following:
> > > > > >>>>> ompi_info -mca coll_tuned_use_dynamic_rules 1 --param coll
> > > > > >>>>>
> > > > > >>>>> You should see lots of output with all the different
> > > > > >>>>> algorithms that can be selected for the various collectives.
> > > > > >>>>> Therefore, you need this:
> > > > > >>>>>
> > > > > >>>>> --mca coll_tuned_use_dynamic_rules 1 --mca
> > > > > >>>>> coll_tuned_bcast_algorithm 1
> > > > > >>>>>
> > > > > >>>>> Rolf
> > > > > >>>>>
> > > > > >>>>> On 07/13/10 11:28, Eloi Gaudry wrote:
> > > > > >>>>>> Hi,
> > > > > >>>>>>
> > > > > >>>>>> I've found that "--mca coll_tuned_bcast_algorithm 1" allowed
> >
> > to
> >
> > > > > >>>>>> switch to the basic linear algorithm. Anyway whatever the
> > > > > >>>>>> algorithm used, the segmentation fault remains.
> > > > > >>>>>>
> > > > > >>>>>> Does anyone could give some advice on ways to diagnose the
> >
> > issue
> >
> > > > > >>>>>> I'm facing ?
> > > > > >>>>>>
> > > > > >>>>>> Regards,
> > > > > >>>>>> Eloi
> > > > > >>>>>>
> > > > > >>>>>> On Monday 12 July 2010 10:53:58 Eloi Gaudry wrote:
> > > > > >>>>>>> Hi,
> > > > > >>>>>>>
> > > > > >>>>>>> I'm focusing on the MPI_Bcast routine that seems to
> > > > > >>>>>>> randomly segfault when using the openib btl. I'd like to
> > > > > >>>>>>> know if there is any way to make OpenMPI switch to a
> > > > > >>>>>>> different algorithm than the default one being selected
> > > > > >>>>>>> for MPI_Bcast.
> > > > > >>>>>>>
> > > > > >>>>>>> Thanks for your help,
> > > > > >>>>>>> Eloi
> > > > > >>>>>>>
> > > > > >>>>>>> On Friday 02 July 2010 11:06:52 Eloi Gaudry wrote:
> > > > > >>>>>>>> Hi,
> > > > > >>>>>>>>
> > > > > >>>>>>>> I'm observing a random segmentation fault during an
> >
> > internode
> >
> > > > > >>>>>>>> parallel computation involving the openib btl and
> > > > > >>>>>>>> OpenMPI-1.4.2 (the same issue can be observed with
> > > > > >>>>>>>> OpenMPI-1.3.3).
> > > > > >>>>>>>>
> > > > > >>>>>>>> mpirun (Open MPI) 1.4.2
> > > > > >>>>>>>> Report bugs to http://www.open-mpi.org/community/help/
> > > > > >>>>>>>> [pbn08:02624] *** Process received signal ***
> > > > > >>>>>>>> [pbn08:02624] Signal: Segmentation fault (11)
> > > > > >>>>>>>> [pbn08:02624] Signal code: Address not mapped (1)
> > > > > >>>>>>>> [pbn08:02624] Failing at address: (nil)
> > > > > >>>>>>>> [pbn08:02624] [ 0] /lib64/libpthread.so.0
> > > > > >>>>>>>> [0x349540e4c0] [pbn08:02624] *** End of error message
> > > > > >>>>>>>> ***
> > > > > >>>>>>>> sh: line 1: 2624 Segmentation fault
> >
> > \/share\/hpc3\/actran_suite\/Actran_11\.0\.rc2\.41872\/RedHatE
> >
> > > > > >>>>>>>> L\ -5 \/ x 86 _6 4\ /bin\/actranpy_mp
> >
> > '--apl=/share/hpc3/actran_suite/Actran_11.0.rc2.41872/RedHatEL
> >
> > > > > >>>>>>>> -5 /x 86 _ 64 /A c tran_11.0.rc2.41872'
> >
> > '--inputfile=/work/st25652/LSF_130073_0_47696_0/Case1_3Dreal_m
> >
> > > > > >>>>>>>> 4_ n2 .d a t'
> > > > > >>>>>>>> '--scratch=/scratch/st25652/LSF_130073_0_47696_0/scratch'
> > > > > >>>>>>>> '--mem=3200' '--threads=1' '--errorlevel=FATAL'
> >
> > '--t_max=0.1'
> >
> > > > > >>>>>>>> '--parallel=domain'
> > > > > >>>>>>>>
> > > > > >>>>>>>> If I choose not to use the openib btl (by using --mca btl
> > > > > >>>>>>>> self,sm,tcp on the command line, for instance), I don't
> > > > > >>>>>>>> encounter any problem and the parallel computation runs
> > > > > >>>>>>>> flawlessly.
> > > > > >>>>>>>>
> > > > > >>>>>>>> I would like to get some help to be able:
> > > > > >>>>>>>> - to diagnose the issue I'm facing with the openib btl
> > > > > >>>>>>>> - understand why this issue is observed only when using
> > > > > >>>>>>>> the openib btl and not when using self,sm,tcp
> > > > > >>>>>>>>
> > > > > >>>>>>>> Any help would be very much appreciated.
> > > > > >>>>>>>>
> > > > > >>>>>>>> The outputs of ompi_info and the configure scripts of
> >
> > OpenMPI
> >
> > > > > >>>>>>>> are enclosed to this email, and some information on the
> > > > > >>>>>>>> infiniband drivers as well.
> > > > > >>>>>>>>
> > > > > >>>>>>>> Here is the command line used when launching a parallel
> > > > > >>>>>>>> computation
> > > > > >>>>>>>>
> > > > > >>>>>>>> using infiniband:
> > > > > >>>>>>>> path_to_openmpi/bin/mpirun -np $NPROCESS --hostfile
> > > > > >>>>>>>> host.list --mca
> > > > > >>>>>>>>
> > > > > >>>>>>>> btl openib,sm,self,tcp --display-map --verbose --version
> > > > > >>>>>>>> --mca mpi_warn_on_fork 0 --mca
> > > > > >>>>>>>> btl_openib_want_fork_support
> >
> > 0
> >
> > > > > >>>>>>>> [...]
> > > > > >>>>>>>>
> > > > > >>>>>>>> and the command line used if not using infiniband:
> > > > > >>>>>>>> path_to_openmpi/bin/mpirun -np $NPROCESS --hostfile
> > > > > >>>>>>>> host.list --mca
> > > > > >>>>>>>>
> > > > > >>>>>>>> btl self,sm,tcp --display-map --verbose --version --mca
> > > > > >>>>>>>> mpi_warn_on_fork 0 --mca btl_openib_want_fork_support 0
> >
> > [...]
> >
> > > > > >>>>>>>> Thanks,
> > > > > >>>>>>>> Eloi
> > > > > >>>>>>
> > > > > >>>>>> _______________________________________________
> > > > > >>>>>> users mailing list
> > > > > >>>>>> us...@open-mpi.org
> > > > > >>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
> > --
> >
> >
> > Eloi Gaudry
> >
> > Free Field Technologies
> > Company Website: http://www.fft.be
> > Company Phone: +32 10 487 959
> >
> > _______________________________________________
> > users mailing list
> > us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
--
Eloi Gaudry
Free Field Technologies
Company Website: http://www.fft.be
Company Phone: +32 10 487 959
