Am 11.11.2014 um 02:25 schrieb Ralph Castain: > Another thing you can do is (a) ensure you built with —enable-debug, and then > (b) run it with -mca oob_base_verbose 100 (without the tcp_if_include > option) so we can watch the connection handshake and see what it is doing. > The —hetero-nodes will have not affect here and can be ignored.
Done. It really tries to connect to the outside interface of the headnode. But being there a firewall or not: the nodes have no clue how to reach 137.248.0.0 - they have no gateway to this network at all. It tries so independent from the internal or external name of the headnode given in the machinefile - I hit ^C then. I attached the output of Open MPI 1.8.1 for this setup too. -- Reuti
Wed Nov 12 16:43:12 CET 2014 [annemarie:01246] mca: base: components_register: registering oob components [annemarie:01246] mca: base: components_register: found loaded component tcp [annemarie:01246] mca: base: components_register: component tcp register function successful [annemarie:01246] mca: base: components_open: opening oob components [annemarie:01246] mca: base: components_open: found loaded component tcp [annemarie:01246] mca: base: components_open: component tcp open function successful [annemarie:01246] mca:oob:select: checking available component tcp [annemarie:01246] mca:oob:select: Querying component [tcp] [annemarie:01246] oob:tcp: component_available called [annemarie:01246] WORKING INTERFACE 1 KERNEL INDEX 1 FAMILY: V4 [annemarie:01246] [[37241,0],0] oob:tcp:init rejecting loopback interface lo [annemarie:01246] WORKING INTERFACE 2 KERNEL INDEX 2 FAMILY: V4 [annemarie:01246] [[37241,0],0] oob:tcp:init adding 137.248.x.y to our list of V4 connections [annemarie:01246] WORKING INTERFACE 3 KERNEL INDEX 3 FAMILY: V4 [annemarie:01246] [[37241,0],0] oob:tcp:init adding 192.168.154.30 to our list of V4 connections [annemarie:01246] WORKING INTERFACE 4 KERNEL INDEX 5 FAMILY: V4 [annemarie:01246] [[37241,0],0] oob:tcp:init adding 192.168.154.187 to our list of V4 connections [annemarie:01246] [[37241,0],0] TCP STARTUP [annemarie:01246] [[37241,0],0] attempting to bind to IPv4 port 0 [annemarie:01246] [[37241,0],0] assigned IPv4 port 53661 [annemarie:01246] mca:oob:select: Adding component to end [annemarie:01246] mca:oob:select: Found 1 active transports [node28:05663] mca: base: components_register: registering oob components [node28:05663] mca: base: components_register: found loaded component tcp [node28:05663] mca: base: components_register: component tcp register function successful [node28:05663] mca: base: components_open: opening oob components [node28:05663] mca: base: components_open: found loaded component tcp [node28:05663] mca: base: components_open: component tcp open function successful [node28:05663] mca:oob:select: checking available component tcp [node28:05663] mca:oob:select: Querying component [tcp] [node28:05663] oob:tcp: component_available called [node28:05663] WORKING INTERFACE 1 KERNEL INDEX 1 FAMILY: V4 [node28:05663] [[37241,0],1] oob:tcp:init rejecting loopback interface lo [node28:05663] WORKING INTERFACE 2 KERNEL INDEX 2 FAMILY: V4 [node28:05663] [[37241,0],1] oob:tcp:init adding 192.168.154.28 to our list of V4 connections [node28:05663] WORKING INTERFACE 3 KERNEL INDEX 3 FAMILY: V4 [node28:05663] [[37241,0],1] oob:tcp:init adding 192.168.154.98 to our list of V4 connections [node28:05663] [[37241,0],1] TCP STARTUP [node28:05663] [[37241,0],1] attempting to bind to IPv4 port 0 [node28:05663] [[37241,0],1] assigned IPv4 port 45802 [node28:05663] mca:oob:select: Adding component to end [node28:05663] mca:oob:select: Found 1 active transports [node28:05663] [[37241,0],1]: set_addr to uri 2440626176.0;tcp://137.248.x.y,192.168.154.30,192.168.154.187:53661 [node28:05663] [[37241,0],1]:set_addr checking if peer [[37241,0],0] is reachable via component tcp [node28:05663] [[37241,0],1] oob:tcp: working peer [[37241,0],0] address tcp://137.248.x.y,192.168.154.30,192.168.154.187:53661 [node28:05663] [[37241,0],1] PASSING ADDR 137.248.x.y TO MODULE [node28:05663] [[37241,0],1]:tcp set addr for peer [[37241,0],0] [node28:05663] [[37241,0],1] PASSING ADDR 192.168.154.30 TO MODULE [node28:05663] [[37241,0],1]:tcp set addr for peer [[37241,0],0] [node28:05663] [[37241,0],1] PASSING ADDR 192.168.154.187 TO MODULE [node28:05663] [[37241,0],1]:tcp set addr for peer [[37241,0],0] [node28:05663] [[37241,0],1]: peer [[37241,0],0] is reachable via component tcp [node28:05663] [[37241,0],1] OOB_SEND: rml_oob_send.c:199 [node28:05663] [[37241,0],1]:tcp:processing set_peer cmd [node28:05663] [[37241,0],1] SET_PEER ADDING PEER [[37241,0],0] [node28:05663] [[37241,0],1] set_peer: peer [[37241,0],0] is listening on net 137.248.x.y port 53661 [node28:05663] [[37241,0],1]:tcp:processing set_peer cmd [node28:05663] [[37241,0],1] set_peer: peer [[37241,0],0] is listening on net 192.168.154.30 port 53661 [node28:05663] [[37241,0],1]:tcp:processing set_peer cmd [node28:05663] [[37241,0],1] set_peer: peer [[37241,0],0] is listening on net 192.168.154.187 port 53661 [node28:05663] [[37241,0],1] oob:base:send to target [[37241,0],0] [node28:05663] [[37241,0],1] oob:tcp:send_nb to peer [[37241,0],0]:10 [node28:05663] [[37241,0],1] tcp:send_nb to peer [[37241,0],0] [node28:05663] [[37241,0],1]:[oob_tcp.c:478] post send to [[37241,0],0] [node28:05663] [[37241,0],1]:[oob_tcp.c:415] processing send to peer [[37241,0],0]:10 [node28:05663] [[37241,0],1]:[oob_tcp.c:449] queue pending to [[37241,0],0] [node28:05663] [[37241,0],1] tcp:send_nb: initiating connection to [[37241,0],0] [node28:05663] [[37241,0],1]:[oob_tcp.c:463] connect to [[37241,0],0] [node28:05663] [[37241,0],1] orte_tcp_peer_try_connect: attempting to connect to proc [[37241,0],0] [node28:05663] [[37241,0],1] oob:tcp:peer creating socket to [[37241,0],0] [node28:05663] [[37241,0],1] orte_tcp_peer_try_connect: attempting to connect to proc [[37241,0],0] on socket 9 [node28:05663] [[37241,0],1] orte_tcp_peer_try_connect: attempting to connect to proc [[37241,0],0] on 137.248.x.y:53661 - 0 retries [node28:05663] [[37241,0],1] waiting for connect completion to [[37241,0],0] - activating send event ^CKilled by signal 2. [annemarie:01246] [[37241,0],0] OOB_SEND: rml_oob_send.c:199 [annemarie:01246] [[37241,0],0] oob:base:send to target [[37241,0],1] [annemarie:01246] [[37241,0],0] oob:base:send unknown peer [[37241,0],1] [annemarie:01246] [[37241,0],0] oob:tcp:send_nb to peer [[37241,0],1]:15 [annemarie:01246] [[37241,0],0] tcp:send_nb to peer [[37241,0],1] [annemarie:01246] [[37241,0],0]:[oob_tcp.c:478] post send to [[37241,0],1] [annemarie:01246] [[37241,0],0]:[oob_tcp.c:415] processing send to peer [[37241,0],1]:15 [annemarie:01246] [[37241,0],0]:[oob_tcp.c:430] hop [[37241,0],1] unknown [annemarie:01246] [[37241,0],0]:[oob_tcp.c:432] post no route to [[37241,0],1] [annemarie:01246] [[37241,0],0] tcp:no route called for peer [[37241,0],1] [annemarie:01246] [[37241,0],0] TCP SHUTDOWN [annemarie:01246] mca: base: close: component tcp closed [annemarie:01246] mca: base: close: unloading component tcp
reuti@annemarie:~> date; mpiexec --mca oob_base_verbose 100 --hostfile machines ./mpihello; dateWed Nov 12 17:23:46 CET 2014 [annemarie:10704] mca: base: components_register: registering oob components [annemarie:10704] mca: base: components_register: found loaded component tcp [annemarie:10704] mca: base: components_register: component tcp register function successful [annemarie:10704] mca: base: components_open: opening oob components [annemarie:10704] mca: base: components_open: found loaded component tcp [annemarie:10704] mca: base: components_open: component tcp open function successful [annemarie:10704] mca:oob:select: checking available component tcp [annemarie:10704] mca:oob:select: Querying component [tcp] [annemarie:10704] oob:tcp: component_available called [annemarie:10704] WORKING INTERFACE 1 KERNEL INDEX 1 FAMILY: V4 [annemarie:10704] [[48247,0],0] oob:tcp:init rejecting loopback interface lo [annemarie:10704] WORKING INTERFACE 2 KERNEL INDEX 2 FAMILY: V4 [annemarie:10704] [[48247,0],0] oob:tcp:init creating module for V4 address on interface eth0 [annemarie:10704] [[48247,0],0] creating OOB-TCP module for interface eth0 [annemarie:10704] [[48247,0],0] oob:tcp:init adding 137.248.x.y to our list of V4 connections [annemarie:10704] WORKING INTERFACE 3 KERNEL INDEX 3 FAMILY: V4 [annemarie:10704] [[48247,0],0] oob:tcp:init creating module for V4 address on interface eth1 [annemarie:10704] [[48247,0],0] creating OOB-TCP module for interface eth1 [annemarie:10704] [[48247,0],0] oob:tcp:init adding 192.168.154.30 to our list of V4 connections [annemarie:10704] WORKING INTERFACE 4 KERNEL INDEX 5 FAMILY: V4 [annemarie:10704] [[48247,0],0] oob:tcp:init creating module for V4 address on interface eth3 [annemarie:10704] [[48247,0],0] creating OOB-TCP module for interface eth3 [annemarie:10704] [[48247,0],0] oob:tcp:init adding 192.168.154.187 to our list of V4 connections [annemarie:10704] [[48247,0],0] TCP STARTUP [annemarie:10704] [[48247,0],0] attempting to bind to IPv4 port 0 [annemarie:10704] [[48247,0],0] assigned IPv4 port 34790 [annemarie:10704] mca:oob:select: Adding component to end [annemarie:10704] mca:oob:select: Found 1 active transports [node28:09503] mca: base: components_register: registering oob components [node28:09503] mca: base: components_register: found loaded component tcp [node28:09503] mca: base: components_register: component tcp register function successful [node28:09503] mca: base: components_open: opening oob components [node28:09503] mca: base: components_open: found loaded component tcp [node28:09503] mca: base: components_open: component tcp open function successful [node28:09503] mca:oob:select: checking available component tcp [node28:09503] mca:oob:select: Querying component [tcp] [node28:09503] oob:tcp: component_available called [node28:09503] WORKING INTERFACE 1 KERNEL INDEX 1 FAMILY: V4 [node28:09503] [[48247,0],1] oob:tcp:init rejecting loopback interface lo [node28:09503] WORKING INTERFACE 2 KERNEL INDEX 2 FAMILY: V4 [node28:09503] [[48247,0],1] oob:tcp:init creating module for V4 address on interface eth0 [node28:09503] [[48247,0],1] creating OOB-TCP module for interface eth0 [node28:09503] [[48247,0],1] oob:tcp:init adding 192.168.154.28 to our list of V4 connections [node28:09503] WORKING INTERFACE 3 KERNEL INDEX 3 FAMILY: V4 [node28:09503] [[48247,0],1] oob:tcp:init creating module for V4 address on interface eth1 [node28:09503] [[48247,0],1] creating OOB-TCP module for interface eth1 [node28:09503] [[48247,0],1] oob:tcp:init adding 192.168.154.98 to our list of V4 connections [node28:09503] [[48247,0],1] TCP STARTUP [node28:09503] [[48247,0],1] attempting to bind to IPv4 port 0 [node28:09503] [[48247,0],1] assigned IPv4 port 37727 [node28:09503] mca:oob:select: Adding component to end [node28:09503] mca:oob:select: Found 1 active transports [annemarie:10704] [[48247,0],0] mca_oob_tcp_listen_thread: new connection: (12, 0) 192.168.154.28:54525 [node28:09503] [[48247,0],1]: set_addr to uri 3161915392.0;tcp://137.248.x.y,192.168.154.30,192.168.154.187:34790 [node28:09503] [[48247,0],1]:set_addr checking if peer [[48247,0],0] is reachable via component tcp [node28:09503] [[48247,0],1] oob:tcp: working peer [[48247,0],0] address tcp://137.248.x.y,192.168.154.30,192.168.154.187:34790 [node28:09503] [[48247,0],1] UNFOUND KERNEL INDEX -13 FOR ADDRESS 137.248.x.y [node28:09503] [[48247,0],1] PEER [[48247,0],0] MAY BE REACHABLE USING MODULE AT KINDEX 2 INTERFACE eth0 [node28:09503] [[48247,0],1] PASSING ADDR 192.168.154.30 TO INTERFACE eth0 AT KERNEL INDEX 2 [node28:09503] [[48247,0],1]:tcp set addr for peer [[48247,0],0] [node28:09503] [[48247,0],1] UNFOUND KERNEL INDEX -13 FOR ADDRESS 192.168.154.187 [node28:09503] [[48247,0],1]: peer [[48247,0],0] is reachable via component tcp [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1]:tcp:processing set_peer cmd for interface eth0 [node28:09503] [[48247,0],1] SET_PEER ADDING PEER [[48247,0],0] [node28:09503] [[48247,0],1] set_peer: peer [[48247,0],0] is listening on interface eth0 net 192.168.154.30 port 34790 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:10 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:10 [node28:09503] [[48247,0],1]:[oob_tcp.c:476] queue pending to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_nb: initiating connection to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:490] connect to [[48247,0],0] [node28:09503] [[48247,0],1] orte_tcp_peer_try_connect: attempting to connect to proc [[48247,0],0] via interface eth0 [node28:09503] [[48247,0],1] oob:tcp:peer creating socket to [[48247,0],0] [node28:09503] [[48247,0],1] orte_tcp_peer_try_connect: attempting to connect to proc [[48247,0],0] via interface eth0 on socket 9 [node28:09503] [[48247,0],1] orte_tcp_peer_try_connect: attempting to connect to proc [[48247,0],0] on 192.168.154.30:34790 - 0 retries [node28:09503] [[48247,0],1] waiting for connect completion to [[48247,0],0] - activating send event [annemarie:10704] [[48247,0],0] connection_handler: working connection (12, 0) 192.168.154.28:54525 [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler CONNECTING [node28:09503] [[48247,0],1]:tcp:complete_connect called for peer [[48247,0],0] on socket 9 [node28:09503] [[48247,0],1] tcp_peer_complete_connect: sending ack to [[48247,0],0] [node28:09503] [[48247,0],1] SEND CONNECT ACK [node28:09503] [[48247,0],1] send blocking of 40 bytes to socket 9 [node28:09503] [[48247,0],1] connect-ack sent to socket 9 [node28:09503] [[48247,0],1] tcp_peer_complete_connect: setting read event on connection to [[48247,0],0] [annemarie:10704] [[48247,0],0] accept_connection: 192.168.154.28:54525 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called [annemarie:10704] [[48247,0],0] RECV CONNECT ACK FROM UNKNOWN ON SOCKET 12 [annemarie:10704] [[48247,0],0] waiting for connect ack from UNKNOWN [annemarie:10704] [[48247,0],0] connect ack received from UNKNOWN [annemarie:10704] [[48247,0],0] connect-ack recvd from UNKNOWN [annemarie:10704] [[48247,0],0] mca_oob_tcp_recv_connect: connection from new peer [annemarie:10704] [[48247,0],0] connect-ack header from [[48247,0],1] is okay [annemarie:10704] [[48247,0],0] waiting for connect ack from [[48247,0],1] [annemarie:10704] [[48247,0],0] connect ack received from [[48247,0],1] [annemarie:10704] [[48247,0],0] connect-ack version from [[48247,0],1] matches ours [annemarie:10704] [[48247,0],0] connect-ack [[48247,0],1] authenticated [annemarie:10704] [[48247,0],0] tcp:peer_accept called for peer [[48247,0],1] in state UNKNOWN on socket 12 [annemarie:10704] [[48247,0],0] SEND CONNECT ACK [annemarie:10704] [[48247,0],0] send blocking of 40 bytes to socket 12 [annemarie:10704] [[48247,0],0] connect-ack sent to socket 12 [annemarie:10704] [[48247,0],0]-[[48247,0],1] tcp_peer_connected on socket 12 [annemarie:10704] [[48247,0],0]-[[48247,0],1] accepted: 192.168.154.30 - 192.168.154.28 nodelay 0 sndbuf 262144 rcvbuf 262144 flags 00000802 [annemarie:10704] [[48247,0],0] tcp:set_module called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,0],0] [node28:09503] [[48247,0],1] RECV CONNECT ACK FROM [[48247,0],0] ON SOCKET 9 [node28:09503] [[48247,0],1] waiting for connect ack from [[48247,0],0] [node28:09503] [[48247,0],1] connect ack received from [[48247,0],0] [node28:09503] [[48247,0],1] connect-ack recvd from [[48247,0],0] [node28:09503] [[48247,0],1] connect-ack header from [[48247,0],0] is okay [node28:09503] [[48247,0],1] waiting for connect ack from [[48247,0],0] [node28:09503] [[48247,0],1] connect ack received from [[48247,0],0] [node28:09503] [[48247,0],1] connect-ack version from [[48247,0],0] matches ours [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 28708 [node28:09503] [[48247,0],1] connect-ack [[48247,0],0] authenticated [node28:09503] [[48247,0],1]-[[48247,0],0] tcp_peer_connected on socket 9 [node28:09503] [[48247,0],1]-[[48247,0],0] connected: 192.168.154.28 - 192.168.154.30 nodelay 0 sndbuf 262144 rcvbuf 262144 flags 00000802 [node28:09503] [[48247,0],1]:tcp:recv:handler starting send/recv events [node28:09503] [[48247,0],1] tcp:set_module called for peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 28708 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 28708 BYTES FOR DEST [[48247,0],0] TAG 10 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0]: set_addr to uri 3161915392.1;tcp://192.168.154.28,192.168.154.98:37727 [annemarie:10704] [[48247,0],0]:set_addr checking if peer [[48247,0],1] is reachable via component tcp [annemarie:10704] [[48247,0],0] oob:tcp: working peer [[48247,0],1] address tcp://192.168.154.28,192.168.154.98:37727 [annemarie:10704] [[48247,0],0] PEER [[48247,0],1] MAY BE REACHABLE USING MODULE AT KINDEX 3 INTERFACE eth1 [annemarie:10704] [[48247,0],0] PASSING ADDR 192.168.154.28 TO INTERFACE eth1 AT KERNEL INDEX 3 [annemarie:10704] [[48247,0],0]:tcp set addr for peer [[48247,0],1] [annemarie:10704] [[48247,0],0] UNFOUND KERNEL INDEX -13 FOR ADDRESS 192.168.154.98 [annemarie:10704] [[48247,0],0]: peer [[48247,0],1] is reachable via component tcp [annemarie:10704] [[48247,0],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10704] [[48247,0],0]:tcp:processing set_peer cmd for interface eth1 [annemarie:10704] [[48247,0],0] set_peer: peer [[48247,0],1] is listening on interface eth1 net 192.168.154.28 port 37727 [annemarie:10704] [[48247,0],0] oob:base:send to target [[48247,0],1] [annemarie:10704] [[48247,0],0] oob:base:send known transport for peer [[48247,0],1] [annemarie:10704] [[48247,0],0] oob:tcp:send_nb to peer [[48247,0],1]:1 [annemarie:10704] [[48247,0],0] tcp:send_nb to peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:508] post send to [[48247,0],1] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:442] processing send to peer [[48247,0],1]:1 [annemarie:10704] [[48247,0],0] tcp:send_nb: already connected to [[48247,0],1] - queueing for send [annemarie:10704] [[48247,0],0]:[oob_tcp.c:469] queue send to [[48247,0],1] [annemarie:10704] [[48247,0],0] tcp:send_handler called to send to peer [[48247,0],1] [annemarie:10704] [[48247,0],0] tcp:send_handler SENDING TO [[48247,0],1] [annemarie:10704] [[48247,0],0] MESSAGE SEND COMPLETE TO [[48247,0],1] OF 119 BYTES ON SOCKET 12 [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler allocate new recv msg [node28:09503] [[48247,0],1]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]:tcp:recv:handler allocate data region of size 119 [annemarie:10704] [[48247,0],0]: set_addr to uri 3161915392.0;tcp://137.248.x.y,192.168.154.30,192.168.154.187:34790 [annemarie:10704] [[48247,0],0]:set_addr peer [[48247,0],0] is me [annemarie:10704] [[48247,0],0]: set_addr to uri 3161915392.1;tcp://192.168.154.28,192.168.154.98:37727 [annemarie:10704] [[48247,0],0]:set_addr checking if peer [[48247,0],1] is reachable via component tcp [annemarie:10704] [[48247,0],0] oob:tcp: working peer [[48247,0],1] address tcp://192.168.154.28,192.168.154.98:37727 [annemarie:10704] [[48247,0],0] PEER [[48247,0],1] MAY BE REACHABLE USING MODULE AT KINDEX 3 INTERFACE eth1 [annemarie:10704] [[48247,0],0] PASSING ADDR 192.168.154.28 TO INTERFACE eth1 AT KERNEL INDEX 3 [annemarie:10704] [[48247,0],0]:tcp set addr for peer [[48247,0],1] [annemarie:10704] [[48247,0],0] UNFOUND KERNEL INDEX -13 FOR ADDRESS 192.168.154.98 [annemarie:10704] [[48247,0],0]: peer [[48247,0],1] is reachable via component tcp [annemarie:10704] [[48247,0],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10704] [[48247,0],0]:tcp:processing set_peer cmd for interface eth1 [annemarie:10704] [[48247,0],0] oob:base:send to target [[48247,0],1] [annemarie:10704] [[48247,0],0] oob:base:send known transport for peer [[48247,0],1] [annemarie:10704] [[48247,0],0] oob:tcp:send_nb to peer [[48247,0],1]:15 [annemarie:10704] [[48247,0],0] tcp:send_nb to peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:508] post send to [[48247,0],1] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:442] processing send to peer [[48247,0],1]:15 [annemarie:10704] [[48247,0],0] tcp:send_nb: already connected to [[48247,0],1] - queueing for send [annemarie:10704] [[48247,0],0]:[oob_tcp.c:469] queue send to [[48247,0],1] [annemarie:10704] [[48247,0],0] tcp:send_handler called to send to peer [[48247,0],1] [annemarie:10704] [[48247,0],0] tcp:send_handler SENDING TO [[48247,0],1] [annemarie:10704] [[48247,0],0] MESSAGE SEND COMPLETE TO [[48247,0],1] OF 1048 BYTES ON SOCKET 12 [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1] RECVD COMPLETE MESSAGE FROM [[48247,0],0] (ORIGIN [[48247,0],0]) OF 119 BYTES FOR DEST [[48247,0],1] TAG 1 [node28:09503] [[48247,0],1] DELIVERING TO RML [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler allocate new recv msg [node28:09503] [[48247,0],1]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]:tcp:recv:handler allocate data region of size 1048 [node28:09503] [[48247,0],1] RECVD COMPLETE MESSAGE FROM [[48247,0],0] (ORIGIN [[48247,0],0]) OF 1048 BYTES FOR DEST [[48247,0],1] TAG 15 [node28:09503] [[48247,0],1] DELIVERING TO RML [node28:09503] [[48247,0],1]: set_addr to uri 3161915392.0;tcp://137.248.x.y,192.168.154.30,192.168.154.187:34790 [node28:09503] [[48247,0],1]:set_addr checking if peer [[48247,0],0] is reachable via component tcp [node28:09503] [[48247,0],1] oob:tcp: working peer [[48247,0],0] address tcp://137.248.x.y,192.168.154.30,192.168.154.187:34790 [node28:09503] [[48247,0],1] UNFOUND KERNEL INDEX -13 FOR ADDRESS 137.248.x.y [node28:09503] [[48247,0],1] PEER [[48247,0],0] MAY BE REACHABLE USING MODULE AT KINDEX 2 INTERFACE eth0 [node28:09503] [[48247,0],1] PASSING ADDR 192.168.154.30 TO INTERFACE eth0 AT KERNEL INDEX 2 [node28:09503] [[48247,0],1]:tcp set addr for peer [[48247,0],0] [node28:09503] [[48247,0],1] UNFOUND KERNEL INDEX -13 FOR ADDRESS 192.168.154.187 [node28:09503] [[48247,0],1]: peer [[48247,0],0] is reachable via component tcp [node28:09503] [[48247,0],1]: set_addr to uri 3161915392.1;tcp://192.168.154.28,192.168.154.98:37727 [node28:09503] [[48247,0],1]:set_addr peer [[48247,0],1] is me [node28:09503] [[48247,0],1]:tcp:processing set_peer cmd for interface eth0 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:5 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:5 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 73 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 73 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 73 BYTES FOR DEST [[48247,0],0] TAG 5 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10709] mca: base: components_register: registering oob components [annemarie:10709] mca: base: components_register: found loaded component tcp [annemarie:10709] mca: base: components_register: component tcp register function successful [annemarie:10709] mca: base: components_open: opening oob components [annemarie:10709] mca: base: components_open: found loaded component tcp [annemarie:10709] mca: base: components_open: component tcp open function successful [annemarie:10709] mca:oob:select: checking available component tcp [annemarie:10709] mca:oob:select: Querying component [tcp] [annemarie:10709] oob:tcp: component_available called [annemarie:10709] WORKING INTERFACE 1 KERNEL INDEX 1 FAMILY: V4 [annemarie:10709] [[48247,1],0] oob:tcp:init rejecting loopback interface lo [annemarie:10709] WORKING INTERFACE 2 KERNEL INDEX 2 FAMILY: V4 [annemarie:10709] [[48247,1],0] oob:tcp:init creating module for V4 address on interface eth0 [annemarie:10709] [[48247,1],0] creating OOB-TCP module for interface eth0 [annemarie:10709] [[48247,1],0] oob:tcp:init adding 137.248.x.y to our list of V4 connections [annemarie:10709] WORKING INTERFACE 3 KERNEL INDEX 3 FAMILY: V4 [annemarie:10709] [[48247,1],0] oob:tcp:init creating module for V4 address on interface eth1 [annemarie:10709] [[48247,1],0] creating OOB-TCP module for interface eth1 [annemarie:10709] [[48247,1],0] oob:tcp:init adding 192.168.154.30 to our list of V4 connections [annemarie:10709] WORKING INTERFACE 4 KERNEL INDEX 5 FAMILY: V4 [annemarie:10709] [[48247,1],0] oob:tcp:init creating module for V4 address on interface eth3 [annemarie:10709] [[48247,1],0] creating OOB-TCP module for interface eth3 [annemarie:10709] [[48247,1],0] oob:tcp:init adding 192.168.154.187 to our list of V4 connections [annemarie:10709] [[48247,1],0] TCP STARTUP [annemarie:10709] [[48247,1],0] attempting to bind to IPv4 port 0 [annemarie:10709] [[48247,1],0] assigned IPv4 port 48617 [annemarie:10709] mca:oob:select: Adding component to end [annemarie:10709] mca:oob:select: Found 1 active transports [annemarie:10709] [[48247,1],0]: set_addr to uri 3161915392.0;tcp://137.248.x.y,192.168.154.30,192.168.154.187:34790 [annemarie:10709] [[48247,1],0]:set_addr checking if peer [[48247,0],0] is reachable via component tcp [annemarie:10709] [[48247,1],0] oob:tcp: working peer [[48247,0],0] address tcp://137.248.x.y,192.168.154.30,192.168.154.187:34790 [annemarie:10709] [[48247,1],0] PEER [[48247,0],0] MAY BE REACHABLE USING MODULE AT KINDEX 2 INTERFACE eth0 [annemarie:10709] [[48247,1],0] PASSING ADDR 137.248.x.y TO INTERFACE eth0 AT KERNEL INDEX 2 [annemarie:10709] [[48247,1],0]:tcp set addr for peer [[48247,0],0] [annemarie:10709] [[48247,1],0] PEER [[48247,0],0] MAY BE REACHABLE USING MODULE AT KINDEX 3 INTERFACE eth1 [annemarie:10709] [[48247,1],0] PASSING ADDR 192.168.154.30 TO INTERFACE eth1 AT KERNEL INDEX 3 [annemarie:10709] [[48247,1],0]:tcp set addr for peer [[48247,0],0] [annemarie:10709] [[48247,1],0] PEER [[48247,0],0] MAY BE REACHABLE USING MODULE AT KINDEX 5 INTERFACE eth3 [annemarie:10709] [[48247,1],0] PASSING ADDR 192.168.154.187 TO INTERFACE eth3 AT KERNEL INDEX 5 [annemarie:10709] [[48247,1],0]:tcp set addr for peer [[48247,0],0] [annemarie:10709] [[48247,1],0]: peer [[48247,0],0] is reachable via component tcp [annemarie:10709] [[48247,1],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10709] [[48247,1],0]:tcp:processing set_peer cmd for interface eth0 [annemarie:10709] [[48247,1],0] SET_PEER ADDING PEER [[48247,0],0] [annemarie:10709] [[48247,1],0] set_peer: peer [[48247,0],0] is listening on interface eth0 net 137.248.x.y port 34790 [annemarie:10709] [[48247,1],0]:tcp:processing set_peer cmd for interface eth1 [annemarie:10709] [[48247,1],0] SET_PEER ADDING PEER [[48247,0],0] [annemarie:10709] [[48247,1],0] set_peer: peer [[48247,0],0] is listening on interface eth1 net 192.168.154.30 port 34790 [annemarie:10709] [[48247,1],0]:tcp:processing set_peer cmd for interface eth3 [annemarie:10709] [[48247,1],0] SET_PEER ADDING PEER [[48247,0],0] [annemarie:10709] [[48247,1],0] set_peer: peer [[48247,0],0] is listening on interface eth3 net 192.168.154.187 port 34790 [annemarie:10709] [[48247,1],0] oob:base:send to target [[48247,0],0] [annemarie:10709] [[48247,1],0] oob:tcp:send_nb to peer [[48247,0],0]:1 [annemarie:10709] [[48247,1],0] tcp:send_nb to peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:[oob_tcp.c:508] post send to [[48247,0],0] [annemarie:10709] [[48247,1],0]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:1 [annemarie:10709] [[48247,1],0]:[oob_tcp.c:476] queue pending to [[48247,0],0] [annemarie:10709] [[48247,1],0] tcp:send_nb: initiating connection to [[48247,0],0] [annemarie:10709] [[48247,1],0]:[oob_tcp.c:490] connect to [[48247,0],0] [annemarie:10709] [[48247,1],0] orte_tcp_peer_try_connect: attempting to connect to proc [[48247,0],0] via interface eth0 [annemarie:10709] [[48247,1],0] oob:tcp:peer creating socket to [[48247,0],0] [annemarie:10709] [[48247,1],0] orte_tcp_peer_try_connect: attempting to connect to proc [[48247,0],0] via interface eth0 on socket 11 [annemarie:10709] [[48247,1],0] orte_tcp_peer_try_connect: attempting to connect to proc [[48247,0],0] on 137.248.x.y:34790 - 0 retries [annemarie:10704] [[48247,0],0] mca_oob_tcp_listen_thread: new connection: (14, 11) 137.248.x.y:43308 [annemarie:10704] [[48247,0],0] connection_handler: working connection (14, 11) 137.248.x.y:43308 [annemarie:10704] [[48247,0],0] accept_connection: 137.248.x.y:43308 [annemarie:10709] [[48247,1],0] waiting for connect completion to [[48247,0],0] - activating send event [annemarie:10709] [[48247,1],0] tcp:send_handler called to send to peer [[48247,0],0] [annemarie:10709] [[48247,1],0] tcp:send_handler CONNECTING [annemarie:10709] [[48247,1],0]:tcp:complete_connect called for peer [[48247,0],0] on socket 11 [annemarie:10709] [[48247,1],0] tcp_peer_complete_connect: sending ack to [[48247,0],0] [annemarie:10709] [[48247,1],0] SEND CONNECT ACK [annemarie:10704] [[48247,0],0]:tcp:recv:handler called [annemarie:10704] [[48247,0],0] RECV CONNECT ACK FROM UNKNOWN ON SOCKET 14 [annemarie:10704] [[48247,0],0] waiting for connect ack from UNKNOWN [annemarie:10704] [[48247,0],0] connect ack received from UNKNOWN [annemarie:10704] [[48247,0],0] connect-ack recvd from UNKNOWN [annemarie:10704] [[48247,0],0] mca_oob_tcp_recv_connect: connection from new peer [annemarie:10704] [[48247,0],0] connect-ack header from [[48247,1],0] is okay [annemarie:10704] [[48247,0],0] waiting for connect ack from [[48247,1],0] [annemarie:10704] [[48247,0],0] connect ack received from [[48247,1],0] [annemarie:10704] [[48247,0],0] connect-ack version from [[48247,1],0] matches ours [annemarie:10704] [[48247,0],0] connect-ack [[48247,1],0] authenticated [annemarie:10704] [[48247,0],0] tcp:peer_accept called for peer [[48247,1],0] in state UNKNOWN on socket 14 [annemarie:10704] [[48247,0],0] SEND CONNECT ACK [annemarie:10704] [[48247,0],0] send blocking of 40 bytes to socket 14 [annemarie:10704] [[48247,0],0] connect-ack sent to socket 14 [annemarie:10704] [[48247,0],0]-[[48247,1],0] tcp_peer_connected on socket 14 [annemarie:10704] [[48247,0],0]-[[48247,1],0] accepted: 137.248.x.y - 137.248.x.y nodelay 0 sndbuf 262144 rcvbuf 262144 flags 00000802 [annemarie:10709] [[48247,1],0] send blocking of 40 bytes to socket 11 [annemarie:10709] [[48247,1],0] connect-ack sent to socket 11 [annemarie:10709] [[48247,1],0] tcp_peer_complete_connect: setting read event on connection to [[48247,0],0] [annemarie:10704] [[48247,0],0] tcp:set_module called for peer [[48247,1],0] [annemarie:10709] [[48247,1],0]:tcp:recv:handler called for peer [[48247,0],0] [annemarie:10709] [[48247,1],0] RECV CONNECT ACK FROM [[48247,0],0] ON SOCKET 11 [annemarie:10709] [[48247,1],0] waiting for connect ack from [[48247,0],0] [annemarie:10709] [[48247,1],0] connect ack received from [[48247,0],0] [annemarie:10709] [[48247,1],0] connect-ack recvd from [[48247,0],0] [annemarie:10709] [[48247,1],0] connect-ack header from [[48247,0],0] is okay [annemarie:10709] [[48247,1],0] waiting for connect ack from [[48247,0],0] [annemarie:10709] [[48247,1],0] connect ack received from [[48247,0],0] [annemarie:10709] [[48247,1],0] connect-ack version from [[48247,0],0] matches ours [annemarie:10709] [[48247,1],0] connect-ack [[48247,0],0] authenticated [annemarie:10709] [[48247,1],0]-[[48247,0],0] tcp_peer_connected on socket 11 [annemarie:10709] [[48247,1],0]-[[48247,0],0] connected: 137.248.x.y - 137.248.x.y nodelay 0 sndbuf 262144 rcvbuf 262144 flags 00000802 [annemarie:10709] [[48247,1],0]:tcp:recv:handler starting send/recv events [annemarie:10709] [[48247,1],0] tcp:set_module called for peer [[48247,0],0] [annemarie:10709] [[48247,1],0] tcp:send_handler called to send to peer [[48247,0],0] [annemarie:10709] [[48247,1],0] tcp:send_handler SENDING TO [[48247,0],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 96 [annemarie:10709] [[48247,1],0] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 96 BYTES ON SOCKET 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 178 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 509 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 178 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 178 BYTES FOR DEST [[48247,0],0] TAG 2 [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 226 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 709 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 196 BYTES ON SOCKET 9 [node28:09586] mca: base: components_register: registering oob components [node28:09586] mca: base: components_register: found loaded component tcp [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 509 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 509 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 226 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 226 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] mca: base: components_register: component tcp register function successful [node28:09586] mca: base: components_open: opening oob components [node28:09586] mca: base: components_open: found loaded component tcp [node28:09586] mca: base: components_open: component tcp open function successful [node28:09586] mca:oob:select: checking available component tcp [node28:09586] mca:oob:select: Querying component [tcp] [node28:09586] oob:tcp: component_available called [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09586] WORKING INTERFACE 1 KERNEL INDEX 1 FAMILY: V4 [node28:09586] [[48247,1],1] oob:tcp:init rejecting loopback interface lo [node28:09586] WORKING INTERFACE 2 KERNEL INDEX 2 FAMILY: V4 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 544 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] connection_event_handler: working connection (11, 11) 192.168.154.28:47928 [node28:09503] [[48247,0],1] accept_connection: 192.168.154.28:47928 [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 1791 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 632 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1]:tcp:recv:handler called [node28:09503] [[48247,0],1] RECV CONNECT ACK FROM UNKNOWN ON SOCKET 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 709 [node28:09503] [[48247,0],1] waiting for connect ack from UNKNOWN [node28:09503] [[48247,0],1] connect ack received from UNKNOWN [node28:09503] [[48247,0],1] connect-ack recvd from UNKNOWN [node28:09503] [[48247,0],1] mca_oob_tcp_recv_connect: connection from new peer [node28:09503] [[48247,0],1] connect-ack header from [[48247,1],1] is okay [node28:09503] [[48247,0],1] waiting for connect ack from [[48247,1],1] [node28:09503] [[48247,0],1] connect ack received from [[48247,1],1] [node28:09503] [[48247,0],1] connect-ack version from [[48247,1],1] matches ours [node28:09503] [[48247,0],1] connect-ack [[48247,1],1] authenticated [node28:09503] [[48247,0],1] tcp:peer_accept called for peer [[48247,1],1] in state UNKNOWN on socket 11 [node28:09503] [[48247,0],1] SEND CONNECT ACK [node28:09503] [[48247,0],1] send blocking of 40 bytes to socket 11 [node28:09503] [[48247,0],1] connect-ack sent to socket 11 [node28:09503] [[48247,0],1]-[[48247,1],1] tcp_peer_connected on socket 11 [node28:09503] [[48247,0],1]-[[48247,1],1] accepted: 192.168.154.28 - 192.168.154.28 nodelay 0 sndbuf 262144 rcvbuf 262144 flags 00000802 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] tcp:set_module called for peer [[48247,1],1] [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 709 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,1],1] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler allocate new recv msg [node28:09503] [[48247,0],1]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]:tcp:recv:handler allocate data region of size 80 [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 394 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 1364 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 196 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 196 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1] oob:tcp:init creating module for V4 address on interface eth0 [node28:09586] [[48247,1],1] creating OOB-TCP module for interface eth0 [node28:09586] [[48247,1],1] oob:tcp:init adding 192.168.154.28 to our list of V4 connections [node28:09586] WORKING INTERFACE 3 KERNEL INDEX 3 FAMILY: V4 [node28:09586] [[48247,1],1] oob:tcp:init creating module for V4 address on interface eth1 [node28:09586] [[48247,1],1] creating OOB-TCP module for interface eth1 [node28:09586] [[48247,1],1] oob:tcp:init adding 192.168.154.98 to our list of V4 connections [node28:09586] [[48247,1],1] TCP STARTUP [node28:09586] [[48247,1],1] attempting to bind to IPv4 port 0 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 544 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 544 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1] assigned IPv4 port 52014 [node28:09586] mca:oob:select: Adding component to end [node28:09586] mca:oob:select: Found 1 active transports [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 1791 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 1791 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1]: set_addr to uri 3161915392.1;tcp://192.168.154.28,192.168.154.98:37727 [node28:09586] [[48247,1],1]:set_addr checking if peer [[48247,0],1] is reachable via component tcp [node28:09586] [[48247,1],1] oob:tcp: working peer [[48247,0],1] address tcp://192.168.154.28,192.168.154.98:37727 [node28:09586] [[48247,1],1] PEER [[48247,0],1] MAY BE REACHABLE USING MODULE AT KINDEX 2 INTERFACE eth0 [node28:09586] [[48247,1],1] PASSING ADDR 192.168.154.28 TO INTERFACE eth0 AT KERNEL INDEX 2 [node28:09586] [[48247,1],1]:tcp set addr for peer [[48247,0],1] [node28:09586] [[48247,1],1] PEER [[48247,0],1] MAY BE REACHABLE USING MODULE AT KINDEX 3 INTERFACE eth1 [node28:09586] [[48247,1],1] PASSING ADDR 192.168.154.98 TO INTERFACE eth1 AT KERNEL INDEX 3 [node28:09586] [[48247,1],1]:tcp set addr for peer [[48247,0],1] [node28:09586] [[48247,1],1]: peer [[48247,0],1] is reachable via component tcp [node28:09586] [[48247,1],1] OOB_SEND: rml_oob_send.c:199 [node28:09586] [[48247,1],1]:tcp:processing set_peer cmd for interface eth0 [node28:09586] [[48247,1],1] SET_PEER ADDING PEER [[48247,0],1] [node28:09586] [[48247,1],1] set_peer: peer [[48247,0],1] is listening on interface eth0 net 192.168.154.28 port 37727 [node28:09586] [[48247,1],1]:tcp:processing set_peer cmd for interface eth1 [node28:09586] [[48247,1],1] SET_PEER ADDING PEER [[48247,0],1] [node28:09586] [[48247,1],1] set_peer: peer [[48247,0],1] is listening on interface eth1 net 192.168.154.98 port 37727 [node28:09586] [[48247,1],1] oob:base:send to target [[48247,0],1] [node28:09586] [[48247,1],1] oob:tcp:send_nb to peer [[48247,0],1]:1 [node28:09586] [[48247,1],1] tcp:send_nb to peer [[48247,0],1] [node28:09586] [[48247,1],1]:[oob_tcp.c:508] post send to [[48247,0],1] [node28:09586] [[48247,1],1]:[oob_tcp.c:442] processing send to peer [[48247,0],1]:1 [node28:09586] [[48247,1],1]:[oob_tcp.c:476] queue pending to [[48247,0],1] [node28:09586] [[48247,1],1] tcp:send_nb: initiating connection to [[48247,0],1] [node28:09586] [[48247,1],1]:[oob_tcp.c:490] connect to [[48247,0],1] [node28:09586] [[48247,1],1] orte_tcp_peer_try_connect: attempting to connect to proc [[48247,0],1] via interface eth0 [node28:09586] [[48247,1],1] oob:tcp:peer creating socket to [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 632 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 632 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1] orte_tcp_peer_try_connect: attempting to connect to proc [[48247,0],1] via interface eth0 on socket 11 [node28:09586] [[48247,1],1] orte_tcp_peer_try_connect: attempting to connect to proc [[48247,0],1] on 192.168.154.28:37727 - 0 retries [node28:09586] [[48247,1],1] waiting for connect completion to [[48247,0],1] - activating send event [node28:09586] [[48247,1],1] tcp:send_handler called to send to peer [[48247,0],1] [node28:09586] [[48247,1],1] tcp:send_handler CONNECTING [node28:09586] [[48247,1],1]:tcp:complete_connect called for peer [[48247,0],1] on socket 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 394 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 394 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1] tcp_peer_complete_connect: sending ack to [[48247,0],1] [node28:09586] [[48247,1],1] SEND CONNECT ACK [node28:09586] [[48247,1],1] send blocking of 40 bytes to socket 11 [node28:09586] [[48247,1],1] connect-ack sent to socket 11 [node28:09586] [[48247,1],1] tcp_peer_complete_connect: setting read event on connection to [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 1364 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 1364 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1]:tcp:recv:handler called for peer [[48247,0],1] [node28:09586] [[48247,1],1] RECV CONNECT ACK FROM [[48247,0],1] ON SOCKET 11 [node28:09586] [[48247,1],1] waiting for connect ack from [[48247,0],1] [node28:09586] [[48247,1],1] connect ack received from [[48247,0],1] [node28:09586] [[48247,1],1] connect-ack recvd from [[48247,0],1] [node28:09586] [[48247,1],1] connect-ack header from [[48247,0],1] is okay [node28:09586] [[48247,1],1] waiting for connect ack from [[48247,0],1] [node28:09586] [[48247,1],1] connect ack received from [[48247,0],1] [node28:09586] [[48247,1],1] connect-ack version from [[48247,0],1] matches ours [node28:09586] [[48247,1],1] connect-ack [[48247,0],1] authenticated [node28:09586] [[48247,1],1]-[[48247,0],1] tcp_peer_connected on socket 11 [node28:09586] [[48247,1],1]-[[48247,0],1] connected: 192.168.154.28 - 192.168.154.28 nodelay 0 sndbuf 262144 rcvbuf 262144 flags 00000802 [node28:09586] [[48247,1],1]:tcp:recv:handler starting send/recv events [node28:09586] [[48247,1],1] tcp:set_module called for peer [[48247,0],1] [node28:09586] [[48247,1],1] tcp:send_handler called to send to peer [[48247,0],1] [node28:09586] [[48247,1],1] tcp:send_handler SENDING TO [[48247,0],1] [node28:09586] [[48247,1],1] MESSAGE SEND COMPLETE TO [[48247,0],1] OF 80 BYTES ON SOCKET 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,1],0] (ORIGIN [[48247,1],0]) OF 96 BYTES FOR DEST [[48247,0],0] TAG 1 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10704] [[48247,0],0] oob:base:send to target [[48247,1],0] [annemarie:10704] [[48247,0],0] oob:base:send known transport for peer [[48247,1],0] [annemarie:10704] [[48247,0],0] oob:tcp:send_nb to peer [[48247,1],0]:20 [annemarie:10704] [[48247,0],0] tcp:send_nb to peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:508] post send to [[48247,1],0] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:442] processing send to peer [[48247,1],0]:20 [annemarie:10704] [[48247,0],0] tcp:send_nb: already connected to [[48247,1],0] - queueing for send [annemarie:10704] [[48247,0],0]:[oob_tcp.c:469] queue send to [[48247,1],0] [annemarie:10704] [[48247,0],0] tcp:send_handler called to send to peer [[48247,1],0] [annemarie:10704] [[48247,0],0] tcp:send_handler SENDING TO [[48247,1],0] [annemarie:10704] [[48247,0],0] MESSAGE SEND COMPLETE TO [[48247,1],0] OF 11876 BYTES ON SOCKET 14 [annemarie:10709] [[48247,1],0]:tcp:recv:handler called for peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:tcp:recv:handler CONNECTED [annemarie:10709] [[48247,1],0]:tcp:recv:handler allocate new recv msg [annemarie:10709] [[48247,1],0]:tcp:recv:handler read hdr [annemarie:10709] [[48247,1],0]:tcp:recv:handler allocate data region of size 11876 [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,1],1] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1] RECVD COMPLETE MESSAGE FROM [[48247,1],1] (ORIGIN [[48247,1],1]) OF 80 BYTES FOR DEST [[48247,0],1] TAG 1 [node28:09503] [[48247,0],1] DELIVERING TO RML [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,1],1] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,1],1] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,1],1]:20 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,1],1] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,1],1] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,1],1]:20 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,1],1] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,1],1] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,1],1] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,1],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 366 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 366 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,1],1] OF 28935 BYTES ON SOCKET 11 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 366 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:5 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:5 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 123 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 123 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 123 BYTES FOR DEST [[48247,0],0] TAG 5 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1]:tcp:recv:handler called for peer [[48247,0],1] [node28:09586] [[48247,1],1]:tcp:recv:handler CONNECTED [node28:09586] [[48247,1],1]:tcp:recv:handler allocate new recv msg [node28:09586] [[48247,1],1]:tcp:recv:handler read hdr [node28:09586] [[48247,1],1]:tcp:recv:handler allocate data region of size 28935 [annemarie:10709] [[48247,1],0]:tcp:recv:handler called for peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:tcp:recv:handler CONNECTED [annemarie:10709] [[48247,1],0] RECVD COMPLETE MESSAGE FROM [[48247,0],0] (ORIGIN [[48247,0],0]) OF 11876 BYTES FOR DEST [[48247,1],0] TAG 20 [annemarie:10709] [[48247,1],0] DELIVERING TO RML [annemarie:10709] [[48247,1],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10709] [[48247,1],0] oob:base:send to target [[48247,0],0] [annemarie:10709] [[48247,1],0] oob:base:send known transport for peer [[48247,0],0] [annemarie:10709] [[48247,1],0] oob:tcp:send_nb to peer [[48247,0],0]:30 [annemarie:10709] [[48247,1],0] tcp:send_nb to peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:[oob_tcp.c:508] post send to [[48247,0],0] [annemarie:10709] [[48247,1],0]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:30 [annemarie:10709] [[48247,1],0] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [annemarie:10709] [[48247,1],0]:[oob_tcp.c:469] queue send to [[48247,0],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 174 [annemarie:10709] [[48247,1],0] tcp:send_handler called to send to peer [[48247,0],0] [annemarie:10709] [[48247,1],0] tcp:send_handler SENDING TO [[48247,0],0] [annemarie:10709] [[48247,1],0] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 174 BYTES ON SOCKET 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 162 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 162 [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 216 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 162 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 216 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 216 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1]:tcp:recv:handler called for peer [[48247,0],1] [node28:09586] [[48247,1],1]:tcp:recv:handler CONNECTED [node28:09586] [[48247,1],1] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 28935 BYTES FOR DEST [[48247,1],1] TAG 20 [node28:09586] [[48247,1],1] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 528 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,1],1] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]:tcp:recv:handler allocate data region of size 150 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 528 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 448 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 528 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1] OOB_SEND: rml_oob_send.c:199 [node28:09586] [[48247,1],1] oob:base:send to target [[48247,0],1] [node28:09586] [[48247,1],1] oob:base:send known transport for peer [[48247,0],1] [node28:09586] [[48247,1],1] oob:tcp:send_nb to peer [[48247,0],1]:30 [node28:09586] [[48247,1],1] tcp:send_nb to peer [[48247,0],1] [node28:09586] [[48247,1],1]:[oob_tcp.c:508] post send to [[48247,0],1] [node28:09586] [[48247,1],1]:[oob_tcp.c:442] processing send to peer [[48247,0],1]:30 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 448 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 448 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1] tcp:send_nb: already connected to [[48247,0],1] - queueing for send [node28:09586] [[48247,1],1]:[oob_tcp.c:469] queue send to [[48247,0],1] [node28:09586] [[48247,1],1] tcp:send_handler called to send to peer [[48247,0],1] [node28:09586] [[48247,1],1] tcp:send_handler SENDING TO [[48247,0],1] [node28:09586] [[48247,1],1] MESSAGE SEND COMPLETE TO [[48247,0],1] OF 150 BYTES ON SOCKET 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,1],0] (ORIGIN [[48247,1],0]) OF 174 BYTES FOR DEST [[48247,0],0] TAG 30 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,1],1] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1] RECVD COMPLETE MESSAGE FROM [[48247,1],1] (ORIGIN [[48247,1],1]) OF 150 BYTES FOR DEST [[48247,0],1] TAG 30 [node28:09503] [[48247,0],1] DELIVERING TO RML [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:32 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 172 [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:32 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 172 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 172 BYTES FOR DEST [[48247,0],0] TAG 32 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10704] [[48247,0],0] oob:base:send to target [[48247,0],1] [annemarie:10704] [[48247,0],0] oob:base:send known transport for peer [[48247,0],1] [annemarie:10704] [[48247,0],0] oob:tcp:send_nb to peer [[48247,0],1]:15 [annemarie:10704] [[48247,0],0] tcp:send_nb to peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:508] post send to [[48247,0],1] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:442] processing send to peer [[48247,0],1]:15 [annemarie:10704] [[48247,0],0] tcp:send_nb: already connected to [[48247,0],1] - queueing for send [annemarie:10704] [[48247,0],0]:[oob_tcp.c:469] queue send to [[48247,0],1] [annemarie:10704] [[48247,0],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10704] [[48247,0],0] oob:base:send to target [[48247,1],0] [annemarie:10704] [[48247,0],0] oob:base:send known transport for peer [[48247,1],0] [annemarie:10704] [[48247,0],0] oob:tcp:send_nb to peer [[48247,1],0]:30 [annemarie:10704] [[48247,0],0] tcp:send_nb to peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:508] post send to [[48247,1],0] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:442] processing send to peer [[48247,1],0]:30 [annemarie:10704] [[48247,0],0] tcp:send_nb: already connected to [[48247,1],0] - queueing for send [annemarie:10704] [[48247,0],0]:[oob_tcp.c:469] queue send to [[48247,1],0] [annemarie:10704] [[48247,0],0] tcp:send_handler called to send to peer [[48247,0],1] [annemarie:10704] [[48247,0],0] tcp:send_handler SENDING TO [[48247,0],1] [annemarie:10704] [[48247,0],0] MESSAGE SEND COMPLETE TO [[48247,0],1] OF 344 BYTES ON SOCKET 12 [annemarie:10704] [[48247,0],0] tcp:send_handler called to send to peer [[48247,1],0] [annemarie:10704] [[48247,0],0] tcp:send_handler SENDING TO [[48247,1],0] [annemarie:10704] [[48247,0],0] MESSAGE SEND COMPLETE TO [[48247,1],0] OF 314 BYTES ON SOCKET 14 [annemarie:10709] [[48247,1],0]:tcp:recv:handler called for peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:tcp:recv:handler CONNECTED [annemarie:10709] [[48247,1],0]:tcp:recv:handler allocate new recv msg [annemarie:10709] [[48247,1],0]:tcp:recv:handler read hdr [annemarie:10709] [[48247,1],0]:tcp:recv:handler allocate data region of size 314 [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler allocate new recv msg [node28:09503] [[48247,0],1]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]:tcp:recv:handler allocate data region of size 344 [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1] RECVD COMPLETE MESSAGE FROM [[48247,0],0] (ORIGIN [[48247,0],0]) OF 344 BYTES FOR DEST [[48247,0],1] TAG 15 [node28:09503] [[48247,0],1] DELIVERING TO RML [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,1],1] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,1],1] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,1],1]:30 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,1],1] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,1],1] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,1],1]:30 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,1],1] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,1],1] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,1],1] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,1],1] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,1],1] OF 314 BYTES ON SOCKET 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 364 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 364 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 346 BYTES ON SOCKET 9 [annemarie:10709] [[48247,1],0]:tcp:recv:handler called for peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:tcp:recv:handler CONNECTED [annemarie:10709] [[48247,1],0] RECVD COMPLETE MESSAGE FROM [[48247,0],0] (ORIGIN [[48247,0],0]) OF 314 BYTES FOR DEST [[48247,1],0] TAG 30 [annemarie:10709] [[48247,1],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 364 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 346 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 346 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1]:tcp:recv:handler called for peer [[48247,0],1] [node28:09586] [[48247,1],1]:tcp:recv:handler CONNECTED [node28:09586] [[48247,1],1]:tcp:recv:handler allocate new recv msg [node28:09586] [[48247,1],1]:tcp:recv:handler read hdr [node28:09586] [[48247,1],1]:tcp:recv:handler allocate data region of size 314 [node28:09586] [[48247,1],1]:tcp:recv:handler called for peer [[48247,0],1] [node28:09586] [[48247,1],1]:tcp:recv:handler CONNECTED [node28:09586] [[48247,1],1] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 314 BYTES FOR DEST [[48247,1],1] TAG 30 [node28:09586] [[48247,1],1] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 945 [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,1],1] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler allocate new recv msg [node28:09503] [[48247,0],1]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]:tcp:recv:handler allocate data region of size 10 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 945 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 945 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 10 [annemarie:10709] [[48247,1],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10709] [[48247,1],0] oob:base:send to target [[48247,0],0] [annemarie:10709] [[48247,1],0] oob:base:send known transport for peer [[48247,0],0] [annemarie:10709] [[48247,1],0] oob:tcp:send_nb to peer [[48247,0],0]:30 [annemarie:10709] [[48247,1],0] tcp:send_nb to peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:[oob_tcp.c:508] post send to [[48247,0],0] [annemarie:10709] [[48247,1],0]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:30 [annemarie:10709] [[48247,1],0] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [annemarie:10709] [[48247,1],0]:[oob_tcp.c:469] queue send to [[48247,0],0] [annemarie:10709] [[48247,1],0] tcp:send_handler called to send to peer [[48247,0],0] [annemarie:10709] [[48247,1],0] tcp:send_handler SENDING TO [[48247,0],0] [node28:09586] [[48247,1],1] OOB_SEND: rml_oob_send.c:199 [node28:09586] [[48247,1],1] oob:base:send to target [[48247,0],1] [node28:09586] [[48247,1],1] oob:base:send known transport for peer [[48247,0],1] [node28:09586] [[48247,1],1] oob:tcp:send_nb to peer [[48247,0],1]:30 [node28:09586] [[48247,1],1] tcp:send_nb to peer [[48247,0],1] [node28:09586] [[48247,1],1]:[oob_tcp.c:508] post send to [[48247,0],1] [node28:09586] [[48247,1],1]:[oob_tcp.c:442] processing send to peer [[48247,0],1]:30 [node28:09586] [[48247,1],1] tcp:send_nb: already connected to [[48247,0],1] - queueing for send [node28:09586] [[48247,1],1]:[oob_tcp.c:469] queue send to [[48247,0],1] [node28:09586] [[48247,1],1] tcp:send_handler called to send to peer [[48247,0],1] [node28:09586] [[48247,1],1] tcp:send_handler SENDING TO [[48247,0],1] [node28:09586] [[48247,1],1] MESSAGE SEND COMPLETE TO [[48247,0],1] OF 10 BYTES ON SOCKET 11 [annemarie:10709] [[48247,1],0] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 10 BYTES ON SOCKET 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 32 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,1],0] [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,1],1] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1] RECVD COMPLETE MESSAGE FROM [[48247,1],1] (ORIGIN [[48247,1],1]) OF 10 BYTES FOR DEST [[48247,0],1] TAG 30 [node28:09503] [[48247,0],1] DELIVERING TO RML [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:32 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:32 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 32 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,1],0] (ORIGIN [[48247,1],0]) OF 10 BYTES FOR DEST [[48247,0],0] TAG 30 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 32 BYTES FOR DEST [[48247,0],0] TAG 32 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10704] [[48247,0],0] oob:base:send to target [[48247,0],1] [annemarie:10704] [[48247,0],0] oob:base:send known transport for peer [[48247,0],1] [annemarie:10704] [[48247,0],0] oob:tcp:send_nb to peer [[48247,0],1]:15 [annemarie:10704] [[48247,0],0] tcp:send_nb to peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:508] post send to [[48247,0],1] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:442] processing send to peer [[48247,0],1]:15 [annemarie:10704] [[48247,0],0] tcp:send_nb: already connected to [[48247,0],1] - queueing for send [annemarie:10704] [[48247,0],0]:[oob_tcp.c:469] queue send to [[48247,0],1] [annemarie:10704] [[48247,0],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10704] [[48247,0],0] oob:base:send to target [[48247,1],0] [annemarie:10704] [[48247,0],0] oob:base:send known transport for peer [[48247,1],0] [annemarie:10704] [[48247,0],0] oob:tcp:send_nb to peer [[48247,1],0]:30 [annemarie:10704] [[48247,0],0] tcp:send_nb to peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:508] post send to [[48247,1],0] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:442] processing send to peer [[48247,1],0]:30 [annemarie:10704] [[48247,0],0] tcp:send_nb: already connected to [[48247,1],0] - queueing for send [annemarie:10704] [[48247,0],0]:[oob_tcp.c:469] queue send to [[48247,1],0] [annemarie:10704] [[48247,0],0] tcp:send_handler called to send to peer [[48247,0],1] [annemarie:10704] [[48247,0],0] tcp:send_handler SENDING TO [[48247,0],1] [annemarie:10704] [[48247,0],0] MESSAGE SEND COMPLETE TO [[48247,0],1] OF 40 BYTES ON SOCKET 12 [annemarie:10704] [[48247,0],0] tcp:send_handler called to send to peer [[48247,1],0] [annemarie:10704] [[48247,0],0] tcp:send_handler SENDING TO [[48247,1],0] [annemarie:10704] [[48247,0],0] MESSAGE SEND COMPLETE TO [[48247,1],0] OF 10 BYTES ON SOCKET 14 [annemarie:10709] [[48247,1],0]:tcp:recv:handler called for peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:tcp:recv:handler CONNECTED [annemarie:10709] [[48247,1],0]:tcp:recv:handler allocate new recv msg [annemarie:10709] [[48247,1],0]:tcp:recv:handler read hdr [annemarie:10709] [[48247,1],0]:tcp:recv:handler allocate data region of size 10 [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler allocate new recv msg [node28:09503] [[48247,0],1]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]:tcp:recv:handler allocate data region of size 40 [annemarie:10709] [[48247,1],0]:tcp:recv:handler called for peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:tcp:recv:handler CONNECTED [annemarie:10709] [[48247,1],0] RECVD COMPLETE MESSAGE FROM [[48247,0],0] (ORIGIN [[48247,0],0]) OF 10 BYTES FOR DEST [[48247,1],0] TAG 30 [annemarie:10709] [[48247,1],0] DELIVERING TO RML [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1] RECVD COMPLETE MESSAGE FROM [[48247,0],0] (ORIGIN [[48247,0],0]) OF 40 BYTES FOR DEST [[48247,0],1] TAG 15 [node28:09503] [[48247,0],1] DELIVERING TO RML [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,1],1] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,1],1] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,1],1]:30 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,1],1] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,1],1] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,1],1]:30 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,1],1] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,1],1] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,1],1] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,1],1] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,1],1] OF 10 BYTES ON SOCKET 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 363 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 363 BYTES ON SOCKET 9 Total: 2 Universe: 2 Hello World from Node 0. [annemarie:10709] [[48247,1],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10709] [[48247,1],0] oob:base:send to target [[48247,0],0] [annemarie:10709] [[48247,1],0] oob:base:send known transport for peer [[48247,0],0] [annemarie:10709] [[48247,1],0] oob:tcp:send_nb to peer [[48247,0],0]:30 [annemarie:10709] [[48247,1],0] tcp:send_nb to peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:[oob_tcp.c:508] post send to [[48247,0],0] [annemarie:10709] [[48247,1],0]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:30 [annemarie:10709] [[48247,1],0] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [annemarie:10709] [[48247,1],0]:[oob_tcp.c:469] queue send to [[48247,0],0] [annemarie:10709] [[48247,1],0] tcp:send_handler called to send to peer [[48247,0],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 10 [annemarie:10709] [[48247,1],0] tcp:send_handler SENDING TO [[48247,0],0] [annemarie:10709] [[48247,1],0] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 10 BYTES ON SOCKET 11 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 345 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,1],0] (ORIGIN [[48247,1],0]) OF 10 BYTES FOR DEST [[48247,0],0] TAG 30 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 363 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 345 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 345 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1]:tcp:recv:handler called for peer [[48247,0],1] [node28:09586] [[48247,1],1]:tcp:recv:handler CONNECTED [node28:09586] [[48247,1],1]:tcp:recv:handler allocate new recv msg [node28:09586] [[48247,1],1]:tcp:recv:handler read hdr [node28:09586] [[48247,1],1]:tcp:recv:handler allocate data region of size 10 [node28:09586] [[48247,1],1]:tcp:recv:handler called for peer [[48247,0],1] [node28:09586] [[48247,1],1]:tcp:recv:handler CONNECTED [node28:09586] [[48247,1],1] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 10 BYTES FOR DEST [[48247,1],1] TAG 30 [node28:09586] [[48247,1],1] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 55 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 55 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 55 BYTES ON SOCKET 9 Hello World from Node 1. [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 945 [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,1],1] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler allocate new recv msg [node28:09503] [[48247,0],1]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]:tcp:recv:handler allocate data region of size 10 [node28:09503] [[48247,0],1] RECVD COMPLETE MESSAGE FROM [[48247,1],1] (ORIGIN [[48247,1],1]) OF 10 BYTES FOR DEST [[48247,0],1] TAG 30 [node28:09503] [[48247,0],1] DELIVERING TO RML [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:32 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:32 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 945 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 32 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 945 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1] OOB_SEND: rml_oob_send.c:199 [node28:09586] [[48247,1],1] oob:base:send to target [[48247,0],1] [node28:09586] [[48247,1],1] oob:base:send known transport for peer [[48247,0],1] [node28:09586] [[48247,1],1] oob:tcp:send_nb to peer [[48247,0],1]:30 [node28:09586] [[48247,1],1] tcp:send_nb to peer [[48247,0],1] [node28:09586] [[48247,1],1]:[oob_tcp.c:508] post send to [[48247,0],1] [node28:09586] [[48247,1],1]:[oob_tcp.c:442] processing send to peer [[48247,0],1]:30 [node28:09586] [[48247,1],1] tcp:send_nb: already connected to [[48247,0],1] - queueing for send [node28:09586] [[48247,1],1]:[oob_tcp.c:469] queue send to [[48247,0],1] [node28:09586] [[48247,1],1] tcp:send_handler called to send to peer [[48247,0],1] [node28:09586] [[48247,1],1] tcp:send_handler SENDING TO [[48247,0],1] [node28:09586] [[48247,1],1] MESSAGE SEND COMPLETE TO [[48247,0],1] OF 10 BYTES ON SOCKET 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 32 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 32 BYTES FOR DEST [[48247,0],0] TAG 32 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10704] [[48247,0],0] oob:base:send to target [[48247,0],1] [annemarie:10704] [[48247,0],0] oob:base:send known transport for peer [[48247,0],1] [annemarie:10704] [[48247,0],0] oob:tcp:send_nb to peer [[48247,0],1]:15 [annemarie:10704] [[48247,0],0] tcp:send_nb to peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:508] post send to [[48247,0],1] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:442] processing send to peer [[48247,0],1]:15 [annemarie:10704] [[48247,0],0] tcp:send_nb: already connected to [[48247,0],1] - queueing for send [annemarie:10704] [[48247,0],0]:[oob_tcp.c:469] queue send to [[48247,0],1] [annemarie:10704] [[48247,0],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10704] [[48247,0],0] oob:base:send to target [[48247,1],0] [annemarie:10704] [[48247,0],0] oob:base:send known transport for peer [[48247,1],0] [annemarie:10704] [[48247,0],0] oob:tcp:send_nb to peer [[48247,1],0]:30 [annemarie:10704] [[48247,0],0] tcp:send_nb to peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:508] post send to [[48247,1],0] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:442] processing send to peer [[48247,1],0]:30 [annemarie:10704] [[48247,0],0] tcp:send_nb: already connected to [[48247,1],0] - queueing for send [annemarie:10704] [[48247,0],0]:[oob_tcp.c:469] queue send to [[48247,1],0] [annemarie:10704] [[48247,0],0] tcp:send_handler called to send to peer [[48247,0],1] [annemarie:10704] [[48247,0],0] tcp:send_handler SENDING TO [[48247,0],1] [annemarie:10704] [[48247,0],0] MESSAGE SEND COMPLETE TO [[48247,0],1] OF 40 BYTES ON SOCKET 12 [annemarie:10704] [[48247,0],0] tcp:send_handler called to send to peer [[48247,1],0] [annemarie:10704] [[48247,0],0] tcp:send_handler SENDING TO [[48247,1],0] [annemarie:10704] [[48247,0],0] MESSAGE SEND COMPLETE TO [[48247,1],0] OF 10 BYTES ON SOCKET 14 [annemarie:10709] [[48247,1],0]:tcp:recv:handler called for peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:tcp:recv:handler CONNECTED [annemarie:10709] [[48247,1],0]:tcp:recv:handler allocate new recv msg [annemarie:10709] [[48247,1],0]:tcp:recv:handler read hdr [annemarie:10709] [[48247,1],0]:tcp:recv:handler allocate data region of size 10 [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler allocate new recv msg [node28:09503] [[48247,0],1]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]:tcp:recv:handler allocate data region of size 40 [annemarie:10709] [[48247,1],0]:tcp:recv:handler called for peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:tcp:recv:handler CONNECTED [annemarie:10709] [[48247,1],0] RECVD COMPLETE MESSAGE FROM [[48247,0],0] (ORIGIN [[48247,0],0]) OF 10 BYTES FOR DEST [[48247,1],0] TAG 30 [annemarie:10709] [[48247,1],0] DELIVERING TO RML [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1] RECVD COMPLETE MESSAGE FROM [[48247,0],0] (ORIGIN [[48247,0],0]) OF 40 BYTES FOR DEST [[48247,0],1] TAG 15 [node28:09503] [[48247,0],1] DELIVERING TO RML [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,1],1] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,1],1] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,1],1]:30 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,1],1] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,1],1] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,1],1]:30 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,1],1] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,1],1] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,1],1] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,1],1] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,1],1] OF 10 BYTES ON SOCKET 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 363 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 363 BYTES ON SOCKET 9 [annemarie:10709] [[48247,1],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10709] [[48247,1],0] oob:base:send to target [[48247,0],0] [annemarie:10709] [[48247,1],0] oob:base:send known transport for peer [[48247,0],0] [annemarie:10709] [[48247,1],0] oob:tcp:send_nb to peer [[48247,0],0]:1 [annemarie:10709] [[48247,1],0] tcp:send_nb to peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:[oob_tcp.c:508] post send to [[48247,0],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 8 [annemarie:10709] [[48247,1],0]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:1 [annemarie:10709] [[48247,1],0] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [annemarie:10709] [[48247,1],0]:[oob_tcp.c:469] queue send to [[48247,0],0] [annemarie:10709] [[48247,1],0] tcp:send_handler called to send to peer [[48247,0],0] [annemarie:10709] [[48247,1],0] tcp:send_handler SENDING TO [[48247,0],0] [annemarie:10709] [[48247,1],0] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 8 BYTES ON SOCKET 11 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 345 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,1],0] (ORIGIN [[48247,1],0]) OF 8 BYTES FOR DEST [[48247,0],0] TAG 1 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10704] [[48247,0],0] oob:base:send to target [[48247,1],0] [annemarie:10704] [[48247,0],0] oob:base:send known transport for peer [[48247,1],0] [annemarie:10704] [[48247,0],0] oob:tcp:send_nb to peer [[48247,1],0]:20 [annemarie:10704] [[48247,0],0] tcp:send_nb to peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:508] post send to [[48247,1],0] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:442] processing send to peer [[48247,1],0]:20 [annemarie:10704] [[48247,0],0] tcp:send_nb: already connected to [[48247,1],0] - queueing for send [annemarie:10704] [[48247,0],0]:[oob_tcp.c:469] queue send to [[48247,1],0] [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,1],1] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler allocate new recv msg [node28:09503] [[48247,0],1]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]:tcp:recv:handler allocate data region of size 8 [annemarie:10704] [[48247,0],0] tcp:send_handler called to send to peer [[48247,1],0] [annemarie:10704] [[48247,0],0] tcp:send_handler SENDING TO [[48247,1],0] [annemarie:10704] [[48247,0],0] MESSAGE SEND COMPLETE TO [[48247,1],0] OF 0 BYTES ON SOCKET 14 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 363 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1]:tcp:recv:handler called for peer [[48247,0],1] [node28:09586] [[48247,1],1]:tcp:recv:handler CONNECTED [node28:09586] [[48247,1],1]:tcp:recv:handler allocate new recv msg [node28:09586] [[48247,1],1]:tcp:recv:handler read hdr [node28:09586] [[48247,1],1]:tcp:recv:handler allocate data region of size 10 [annemarie:10709] [[48247,1],0]:tcp:recv:handler called for peer [[48247,0],0] [annemarie:10709] [[48247,1],0]:tcp:recv:handler CONNECTED [annemarie:10709] [[48247,1],0]:tcp:recv:handler allocate new recv msg [annemarie:10709] [[48247,1],0]:tcp:recv:handler read hdr [annemarie:10709] [[48247,1],0] RECVD ZERO-BYTE MESSAGE FROM [[48247,0],0] for tag 20 [annemarie:10709] [[48247,1],0] RECVD COMPLETE MESSAGE FROM [[48247,0],0] (ORIGIN [[48247,0],0]) OF 0 BYTES FOR DEST [[48247,1],0] TAG 20 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 345 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 345 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [node28:09503] [[48247,0],1] RECVD COMPLETE MESSAGE FROM [[48247,1],1] (ORIGIN [[48247,1],1]) OF 8 BYTES FOR DEST [[48247,0],1] TAG 1 [node28:09503] [[48247,0],1] DELIVERING TO RML [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send to target [[48247,1],1] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,1],1] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,1],1]:20 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,1],1] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,1],1] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,1],1]:20 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,1],1] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,1],1] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 942 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 942 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 942 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1]:tcp:recv:handler called for peer [[48247,0],1] [node28:09586] [[48247,1],1]:tcp:recv:handler CONNECTED [node28:09586] [[48247,1],1] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 10 BYTES FOR DEST [[48247,1],1] TAG 30 [node28:09586] [[48247,1],1] DELIVERING TO RML [annemarie:10709] [[48247,1],0] DELIVERING TO RML [annemarie:10709] [[48247,1],0] TCP SHUTDOWN [annemarie:10709] [[48247,1],0] RELEASING PEER OBJ [[48247,0],0] [annemarie:10709] [[48247,1],0] CLOSING SOCKET 11 [annemarie:10709] [[48247,1],0] RELEASING PEER OBJ [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,1],1] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,1],1] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,1],1] OF 0 BYTES ON SOCKET 11 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 550 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 72 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 550 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 550 BYTES FOR DEST [[48247,0],0] TAG 2 [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,1],1] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler allocate new recv msg [node28:09503] [[48247,0],1]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]-[[48247,1],1] mca_oob_tcp_msg_recv: peer closed connection [node28:09503] [[48247,0],1] tcp_peer_close for [[48247,1],1] sd 11 state CONNECTED [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 139 BYTES ON SOCKET 9 [node28:09503] [[48247,0],1] tcp:lost connection called for peer [[48247,1],1] [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:2 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 203 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]-[[48247,1],0] mca_oob_tcp_msg_recv: peer closed connection [annemarie:10704] [[48247,0],0] tcp_peer_close for [[48247,1],0] sd 14 state CONNECTED [annemarie:10704] [[48247,0],0] tcp:lost connection called for peer [[48247,1],0] [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 72 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 72 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1] OOB_SEND: rml_oob_send.c:199 [node28:09586] [[48247,1],1] oob:base:send to target [[48247,0],1] [node28:09586] [[48247,1],1] oob:base:send known transport for peer [[48247,0],1] [node28:09586] [[48247,1],1] oob:tcp:send_nb to peer [[48247,0],1]:1 [node28:09586] [[48247,1],1] tcp:send_nb to peer [[48247,0],1] [node28:09586] [[48247,1],1]:[oob_tcp.c:508] post send to [[48247,0],1] [node28:09586] [[48247,1],1]:[oob_tcp.c:442] processing send to peer [[48247,0],1]:1 [node28:09586] [[48247,1],1] tcp:send_nb: already connected to [[48247,0],1] - queueing for send [node28:09586] [[48247,1],1]:[oob_tcp.c:469] queue send to [[48247,0],1] [node28:09586] [[48247,1],1] tcp:send_handler called to send to peer [[48247,0],1] [node28:09586] [[48247,1],1] tcp:send_handler SENDING TO [[48247,0],1] [node28:09586] [[48247,1],1] MESSAGE SEND COMPLETE TO [[48247,0],1] OF 8 BYTES ON SOCKET 11 [annemarie:10709] [[48247,1],0] RELEASING PEER OBJ [[48247,0],0] [annemarie:10709] mca: base: close: component tcp closed [annemarie:10709] mca: base: close: unloading component tcp [node28:09586] [[48247,1],1]:tcp:recv:handler called for peer [[48247,0],1] [node28:09586] [[48247,1],1]:tcp:recv:handler CONNECTED [node28:09586] [[48247,1],1]:tcp:recv:handler allocate new recv msg [node28:09586] [[48247,1],1]:tcp:recv:handler read hdr [node28:09586] [[48247,1],1] RECVD ZERO-BYTE MESSAGE FROM [[48247,0],1] for tag 20 [node28:09586] [[48247,1],1] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 0 BYTES FOR DEST [[48247,1],1] TAG 20 [node28:09586] [[48247,1],1] DELIVERING TO RML [node28:09586] [[48247,1],1] TCP SHUTDOWN [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 139 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 139 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1] RELEASING PEER OBJ [[48247,0],1] [node28:09586] [[48247,1],1] CLOSING SOCKET 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 203 [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 203 BYTES FOR DEST [[48247,0],0] TAG 2 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [node28:09586] [[48247,1],1] RELEASING PEER OBJ [[48247,0],1] [node28:09586] mca: base: close: component tcp closed [node28:09586] mca: base: close: unloading component tcp [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1] OOB_SEND: rml_oob_send.c:199 [node28:09503] [[48247,0],1] oob:base:send to target [[48247,0],0] [node28:09503] [[48247,0],1] oob:base:send known transport for peer [[48247,0],0] [node28:09503] [[48247,0],1] oob:tcp:send_nb to peer [[48247,0],0]:5 [node28:09503] [[48247,0],1] tcp:send_nb to peer [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:508] post send to [[48247,0],0] [node28:09503] [[48247,0],1]:[oob_tcp.c:442] processing send to peer [[48247,0],0]:5 [node28:09503] [[48247,0],1] tcp:send_nb: already connected to [[48247,0],0] - queueing for send [node28:09503] [[48247,0],1]:[oob_tcp.c:469] queue send to [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler called to send to peer [[48247,0],0] [node28:09503] [[48247,0],1] tcp:send_handler SENDING TO [[48247,0],0] [node28:09503] [[48247,0],1] MESSAGE SEND COMPLETE TO [[48247,0],0] OF 73 BYTES ON SOCKET 9 [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate data region of size 73 [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0] RECVD COMPLETE MESSAGE FROM [[48247,0],1] (ORIGIN [[48247,0],1]) OF 73 BYTES FOR DEST [[48247,0],0] TAG 5 [annemarie:10704] [[48247,0],0] DELIVERING TO RML [annemarie:10704] [[48247,0],0] OOB_SEND: rml_oob_send.c:199 [annemarie:10704] [[48247,0],0] oob:base:send to target [[48247,0],1] [annemarie:10704] [[48247,0],0] oob:base:send known transport for peer [[48247,0],1] [annemarie:10704] [[48247,0],0] oob:tcp:send_nb to peer [[48247,0],1]:15 [annemarie:10704] [[48247,0],0] tcp:send_nb to peer [[48247,0],1] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:508] post send to [[48247,0],1] [annemarie:10704] [[48247,0],0]:[oob_tcp.c:442] processing send to peer [[48247,0],1]:15 [annemarie:10704] [[48247,0],0] tcp:send_nb: already connected to [[48247,0],1] - queueing for send [annemarie:10704] [[48247,0],0]:[oob_tcp.c:469] queue send to [[48247,0],1] [annemarie:10704] [[48247,0],0] tcp:send_handler called to send to peer [[48247,0],1] [annemarie:10704] [[48247,0],0] tcp:send_handler SENDING TO [[48247,0],1] [annemarie:10704] [[48247,0],0] MESSAGE SEND COMPLETE TO [[48247,0],1] OF 8 BYTES ON SOCKET 12 [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1]:tcp:recv:handler allocate new recv msg [node28:09503] [[48247,0],1]:tcp:recv:handler read hdr [node28:09503] [[48247,0],1]:tcp:recv:handler allocate data region of size 8 [node28:09503] [[48247,0],1]:tcp:recv:handler called for peer [[48247,0],0] [node28:09503] [[48247,0],1]:tcp:recv:handler CONNECTED [node28:09503] [[48247,0],1] RECVD COMPLETE MESSAGE FROM [[48247,0],0] (ORIGIN [[48247,0],0]) OF 8 BYTES FOR DEST [[48247,0],1] TAG 15 [node28:09503] [[48247,0],1] DELIVERING TO RML [annemarie:10704] [[48247,0],0]:tcp:recv:handler called for peer [[48247,0],1] [node28:09503] [[48247,0],1] TCP SHUTDOWN [node28:09503] [[48247,0],1] RELEASING PEER OBJ [[48247,0],0] [node28:09503] [[48247,0],1] CLOSING SOCKET 9 [node28:09503] [[48247,0],1] RELEASING PEER OBJ [[48247,1],1] [node28:09503] [[48247,0],1] CLOSING SOCKET 11 [annemarie:10704] [[48247,0],0]:tcp:recv:handler CONNECTED [annemarie:10704] [[48247,0],0]:tcp:recv:handler allocate new recv msg [annemarie:10704] [[48247,0],0]:tcp:recv:handler read hdr [annemarie:10704] [[48247,0],0]-[[48247,0],1] mca_oob_tcp_msg_recv: peer closed connection [node28:09503] mca: base: close: component tcp closed [node28:09503] mca: base: close: unloading component tcp [annemarie:10704] [[48247,0],0] tcp_peer_close for [[48247,0],1] sd 12 state CONNECTED [annemarie:10704] [[48247,0],0] tcp:lost connection called for peer [[48247,0],1] [annemarie:10704] [[48247,0],0] TCP SHUTDOWN [annemarie:10704] [[48247,0],0] RELEASING PEER OBJ [[48247,1],0] [annemarie:10704] [[48247,0],0] CLOSING SOCKET 14 [annemarie:10704] [[48247,0],0] RELEASING PEER OBJ [[48247,0],1] [annemarie:10704] [[48247,0],0] CLOSING SOCKET 12 [annemarie:10704] mca: base: close: component tcp closed [annemarie:10704] mca: base: close: unloading component tcp Wed Nov 12 17:23:48 CET 2014
