Technically speaking, converting a 64 bits to a big endian representation requires the swap of the 2 32 bits parts. So the correct approach would have been: uint64_t htonll(uint64_t v) { return ((((uint64_t)ntohl(n)) << 32 | (uint64_t)ntohl(n >> 32)); }
George. On Tue, Aug 5, 2014 at 5:52 AM, Ralph Castain <r...@open-mpi.org> wrote: > FWIW: that's exactly how we do it in ORTE > > On Aug 4, 2014, at 10:25 PM, Gilles Gouaillardet < > gilles.gouaillar...@iferc.org> wrote: > > George, > > i confirm there was a problem when running on an heterogeneous cluster, > this is now fixed in r32425. > > i am not convinced i chose the most elegant way to achieve the desired > result ... > could you please double check this commit ? > > Thanks, > > Gilles > > On 2014/08/02 0:14, George Bosilca wrote: > > Gilles, > > The design of the BTL move was to let the opal_process_name_t be agnostic to > what is stored inside, and all accesses should be done through the provided > accessors. Thus, big endian or little endian doesn’t make a difference, as > long as everything goes through the accessors. > > I’m skeptical about the support of heterogeneous environments in the current > code, so I didn’t pay much attention to handling the case in the TCP BTL. But > in case we do care it is enough to make the 2 macros point to something > meaningful instead of being empty (bswap_64 or something). > > George. > > On Aug 1, 2014, at 06:52 , Gilles Gouaillardet > <gilles.gouaillar...@iferc.org> <gilles.gouaillar...@iferc.org> wrote: > > > George and Ralph, > > i am very confused whether there is an issue or not. > > > anyway, today Paul and i ran basic tests on big endian machines and did not > face any issue related to big endianness. > > so i made my homework, digged into the code, and basically, > opal_process_name_t is used as an orte_process_name_t. > for example, in ompi_proc_init : > > OMPI_CAST_ORTE_NAME(&proc->super.proc_name)->jobid = OMPI_PROC_MY_NAME->jobid; > OMPI_CAST_ORTE_NAME(&proc->super.proc_name)->vpid = i; > > and with > > #define OMPI_CAST_ORTE_NAME(a) ((orte_process_name_t*)(a)) > > so as long as an opal_process_name_t is used as an orte_process_name_t, there > is no problem, > regardless the endianness of the homogenous cluster we are running on. > > for the sake of readability (and for being pedantic too ;-) ) in r32357, > &proc_temp->super.proc_name > could be replaced with > OMPI_CAST_ORTE_NAME(&proc_temp->super.proc_name) > > > > That being said, in btl/tcp, i noticed : > > in mca_btl_tcp_component_recv_handler : > > opal_process_name_t guid; > [...] > /* recv the process identifier */ > retval = recv(sd, (char *)&guid, sizeof(guid), 0); > if(retval != sizeof(guid)) { > CLOSE_THE_SOCKET(sd); > return; > } > OPAL_PROCESS_NAME_NTOH(guid); > > and in mca_btl_tcp_endpoint_send_connect_ack : > > /* send process identifier to remote endpoint */ > opal_process_name_t guid = btl_proc->proc_opal->proc_name; > > OPAL_PROCESS_NAME_HTON(guid); > if(mca_btl_tcp_endpoint_send_blocking(btl_endpoint, &guid, sizeof(guid)) > != > > and with > > #define OPAL_PROCESS_NAME_NTOH(guid) > #define OPAL_PROCESS_NAME_HTON(guid) > > > i had no time yet to test yet, but for now, i can only suspect : > - there will be an issue with the tcp btl on an heterogeneous cluster > - for this case, the fix is to have a different version of the > OPAL_PROCESS_NAME_xTOy > on little endian arch if heterogeneous mode is supported. > > > > does that make sense ? > > Cheers, > > Gilles > > > On 2014/07/31 1:29, George Bosilca wrote: > > The underlying structure changed, so a little bit of fiddling is normal. > Instead of using a field in the ompi_proc_t you are now using a field down > in opal_proc_t, a field that simply cannot have the same type as before > (orte_process_name_t). > > George. > > > > On Wed, Jul 30, 2014 at 12:19 PM, Ralph Castain <r...@open-mpi.org> > <r...@open-mpi.org> wrote: > > > George - my point was that we regularly tested using the method in that > routine, and now we have to do something a little different. So it is an > "issue" in that we have to make changes across the code base to ensure we > do things the "new" way, that's all > > On Jul 30, 2014, at 9:17 AM, George Bosilca <bosi...@icl.utk.edu> > <bosi...@icl.utk.edu> wrote: > > No, this is not going to be an issue if the opal_identifier_t is used > correctly (aka only via the exposed accessors). > > George. > > > > On Wed, Jul 30, 2014 at 12:09 PM, Ralph Castain <r...@open-mpi.org> > <r...@open-mpi.org> wrote: > > > Yeah, my fix won't work for big endian machines - this is going to be an > issue across the code base now, so we'll have to troll and fix it. I was > doing the minimal change required to fix the trunk in the meantime. > > On Jul 30, 2014, at 9:06 AM, George Bosilca <bosi...@icl.utk.edu> > <bosi...@icl.utk.edu> wrote: > > Yes. opal_process_name_t has basically no meaning by itself, it is a 64 > bits storage location used by the upper layer to save some local key that > can be later used to extract information. Calling the OPAL level compare > function might be a better fit there. > > George. > > > > On Wed, Jul 30, 2014 at 11:50 AM, Gilles Gouaillardet > <gilles.gouaillar...@gmail.com> wrote: > > > Ralph, > > was it really that simple ? > > proc_temp->super.proc_name has type opal_process_name_t : > typedef opal_identifier_t opal_process_name_t; > typedef uint64_t opal_identifier_t; > > *but* > > item_ptr->peer has type orte_process_name_t : > struct orte_process_name_t { > orte_jobid_t jobid; > orte_vpid_t vpid; > }; > > bottom line, is r32357 still valid on a big endian arch ? > > Cheers, > > Gilles > > > On Wed, Jul 30, 2014 at 11:49 PM, Ralph Castain <r...@open-mpi.org> > <r...@open-mpi.org> > wrote: > > > I just fixed this one - all that was required was an ampersand as the > name was being passed into the function instead of a pointer to the name > > r32357 > > On Jul 30, 2014, at 7:43 AM, Gilles GOUAILLARDET > <gilles.gouaillar...@gmail.com> wrote: > > Rolf, > > r32353 can be seen as a suspect... > Even if it is correct, it might have exposed the bug discussed in #4815 > even more (e.g. we hit the bug 100% after the fix) > > does the attached patch to #4815 fixes the problem ? > > If yes, and if you see this issue as a showstopper, feel free to commit > it and drop a note to #4815 > ( I am afk until tomorrow) > > Cheers, > > Gilles > > Rolf vandeVaart <rvandeva...@nvidia.com> <rvandeva...@nvidia.com> wrote: > > Just an FYI that my trunk version (r32355) does not work at all anymore > if I do not include "--mca coll ^ml". Here is a stack trace from the > ibm/pt2pt/send test running on a single node. > > > > (gdb) where > > #0 0x00007f6c0d1321d0 in ?? () > > #1 <signal handler called> > > #2 0x00007f6c183abd52 in orte_util_compare_name_fields (fields=15 > '\017', name1=0x192350001, name2=0xbaf76c) at ../../orte/util/name_fns.c:522 > > #3 0x00007f6c0bea17be in bcol_basesmuma_smcm_allgather_connection > (sm_bcol_module=0x7f6bf3b68040, module=0xb3d200, peer_list=0x7f6c0c0a6748, > back_files=0x7f6bf3ffd6c8, > > comm=0x6037a0, input=..., base_fname=0x7f6c0bea2606 > "sm_payload_mem_", map_all=false) at > ../../../../../ompi/mca/bcol/basesmuma/bcol_basesmuma_smcm.c:237 > > #4 0x00007f6c0be98307 in bcol_basesmuma_bank_init_opti > (payload_block=0xbc0f60, data_offset=64, bcol_module=0x7f6bf3b68040, > reg_data=0xba28c0) > > at > ../../../../../ompi/mca/bcol/basesmuma/bcol_basesmuma_buf_mgmt.c:302 > > #5 0x00007f6c0cced386 in mca_coll_ml_register_bcols > (ml_module=0xba5c40) at ../../../../../ompi/mca/coll/ml/coll_ml_module.c:510 > > #6 0x00007f6c0cced68f in ml_module_memory_initialization > (ml_module=0xba5c40) at ../../../../../ompi/mca/coll/ml/coll_ml_module.c:558 > > #7 0x00007f6c0ccf06b1 in ml_discover_hierarchy (ml_module=0xba5c40) at > ../../../../../ompi/mca/coll/ml/coll_ml_module.c:1539 > > #8 0x00007f6c0ccf4e0b in mca_coll_ml_comm_query (comm=0x6037a0, > priority=0x7fffe7991b58) at > ../../../../../ompi/mca/coll/ml/coll_ml_module.c:2963 > > #9 0x00007f6c18cc5b09 in query_2_0_0 (component=0x7f6c0cf50940, > comm=0x6037a0, priority=0x7fffe7991b58, module=0x7fffe7991b90) > > at ../../../../ompi/mca/coll/base/coll_base_comm_select.c:372 > > #10 0x00007f6c18cc5ac8 in query (component=0x7f6c0cf50940, > comm=0x6037a0, priority=0x7fffe7991b58, module=0x7fffe7991b90) > > at ../../../../ompi/mca/coll/base/coll_base_comm_select.c:355 > > #11 0x00007f6c18cc59d2 in check_one_component (comm=0x6037a0, > component=0x7f6c0cf50940, module=0x7fffe7991b90) > > at ../../../../ompi/mca/coll/base/coll_base_comm_select.c:317 > > #12 0x00007f6c18cc5818 in check_components (components=0x7f6c18f46ef0, > comm=0x6037a0) at ../../../../ompi/mca/coll/base/coll_base_comm_select.c:281 > > #13 0x00007f6c18cbe3c9 in mca_coll_base_comm_select (comm=0x6037a0) at > ../../../../ompi/mca/coll/base/coll_base_comm_select.c:117 > > #14 0x00007f6c18c52301 in ompi_mpi_init (argc=1, argv=0x7fffe79924c8, > requested=0, provided=0x7fffe79922e8) at > ../../ompi/runtime/ompi_mpi_init.c:918 > > #15 0x00007f6c18c86e92 in PMPI_Init (argc=0x7fffe799234c, > argv=0x7fffe7992340) at pinit.c:84 > > #16 0x0000000000401056 in main (argc=1, argv=0x7fffe79924c8) at > send.c:32 > > (gdb) up > > #1 <signal handler called> > > (gdb) up > > #2 0x00007f6c183abd52 in orte_util_compare_name_fields (fields=15 > '\017', name1=0x192350001, name2=0xbaf76c) at ../../orte/util/name_fns.c:522 > > 522 if (name1->jobid < name2->jobid) { > > (gdb) print name1 > > $1 = (const orte_process_name_t *) 0x192350001 > > (gdb) print *name1 > > Cannot access memory at address 0x192350001 > > (gdb) print name2 > > $2 = (const orte_process_name_t *) 0xbaf76c > > (gdb) print *name2 > > $3 = {jobid = 2452946945, vpid = 1} > > (gdb) > > > > > > > > -----Original Message----- > From: devel [mailto:devel-boun...@open-mpi.org <devel-boun...@open-mpi.org> > > <devel-boun...@open-mpi.org> <devel-boun...@open-mpi.org>] On Behalf Of > Gilles > > > Gouaillardet > Sent: Wednesday, July 30, 2014 2:16 AM > To: Open MPI Developers > Subject: Re: [OMPI devel] trunk compilation errors in jenkins > George, > #4815 is indirectly related to the move : > in bcol/basesmuma, we used to compare ompi_process_name_t, and now > we (try to) compare an ompi_process_name_t and an opal_process_name_t > (which causes a glory SIGSEGV) > i proposed a temporary patch which is both broken and unelegant, could > > you > > > please advise a correct solution ? > Cheers, > Gilles > On 2014/07/27 7:37, George Bosilca wrote: > > If you have any issue with the move, I’ll be happy to help and/or > > support > > > you on your last move toward a completely generic BTL. To facilitate > > your > > > work I exposed a minimalistic set of OMPI information at the OPAL > > level. Take > > > a look at opal/util/proc.h for more info, but please try not to expose > > more. > > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: http://www.open- > > <http://www.open-mpi.org/community/lists/devel/2014/07/15348.php> > <http://www.open-mpi.org/community/lists/devel/2014/07/15348.php> > > mpi.org/community/lists/devel/2014/07/15348.php > > <http://www.open-mpi.org/community/lists/devel/2014/07/15348.php> > <http://www.open-mpi.org/community/lists/devel/2014/07/15348.php> > ------------------------------ > This email message is for the sole use of the intended recipient(s) > and may contain confidential information. Any unauthorized review, use, > disclosure or distribution is prohibited. If you are not the intended > recipient, please contact the sender by reply email and destroy all copies > of the original message. > ------------------------------ > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this > post:http://www.open-mpi.org/community/lists/devel/2014/07/15355.php > > > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this > post:http://www.open-mpi.org/community/lists/devel/2014/07/15356.php > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this > post:http://www.open-mpi.org/community/lists/devel/2014/07/15363.php > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this > post:http://www.open-mpi.org/community/lists/devel/2014/07/15364.php > > > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this > post:http://www.open-mpi.org/community/lists/devel/2014/07/15365.php > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this > post:http://www.open-mpi.org/community/lists/devel/2014/07/15366.php > > > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this > post:http://www.open-mpi.org/community/lists/devel/2014/07/15367.php > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/07/15368.php > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/08/15446.php > > > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/08/15454.php > > > _______________________________________________ > devel mailing list > de...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/08/15509.php > > > > _______________________________________________ > devel mailing list > de...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/08/15514.php >