Re: [OMPI users] [OMPI USERS] Jumbo frames

2017-05-05 Thread Barrett, Brian via users 
But in many ways, it’s also not helpful to change the MTU from Open MPI.  It 
sounds like you made a bunch of changes all at once; I’d break them down and 
build up.  MTU is a very system-level configuration.  Use a tcp transmission 
test (iperf, etc.) to make sure TCP connections work between the nodes.  Once 
that’s working, you can start with Open MPI.  While Open MPI doesn’t have a way 
to set MTU, it can adjust how big the messages it passes to write() are, which 
will result in the same thing if the system is well configured.  In particular, 
you can start with moving around the eager frag limit, although that can have 
impact on memory consumption.

But, as I said, first thing is to get your operating system and networking gear 
set up properly.  It sounds like you’re not quite there yet, but it’s doubtful 
that this list will be the place to get help.

Brian

On May 5, 2017, at 7:41 AM, George Bosilca 
mailto:bosi...@icl.utk.edu>> wrote:

"ompi_info --param btl tcp -l 9" will give you all the TCP options. 
Unfortunately, OMPI does not support programatically changing the value of the 
MTU.

  George.

PS: We would be happy to receive contributions from the community.


On Fri, May 5, 2017 at 10:29 AM, Alberto Ortiz 
mailto:alberto.orti...@gmail.com>> wrote:
I am using version 1.10.6 on archlinux.
The option I should pass to mpirun should then be "-mca btl_tcp_mtu 13000"? 
Just to be sure.
Thank you,
Alberto

El 5 may. 2017 16:26, "r...@open-mpi.org" 
mailto:r...@open-mpi.org>> escribió:
If you are looking to use TCP packets, then you want to set the send/recv 
buffer size in the TCP btl, not the openib one, yes?

Also, what version of OMPI are you using?

> On May 5, 2017, at 7:16 AM, Alberto Ortiz 
> mailto:alberto.orti...@gmail.com>> wrote:
>
> Hi,
> I have a program running with openMPI over a network using a gigabit switch. 
> This switch supports jumbo frames up to 13.000 bytes, so, in order to test 
> and see if it would be faster communicating with this frame lengths, I am 
> trying to use them with my program. I have set the MTU in each node to be 
> 13.000 but when running the program it doesn't even initiate, it gets 
> blocked. I have tried different lengths from 1.500 up to 13.000 but it 
> doesn't work with any length.
>
> I have searched and only found that I have to set OMPI with "-mca 
> btl_openib_ib_mtu 13000" or the length to be used, but I don't seem to get it 
> working.
>
> Which are the steps to get OMPI to use larger TCP packets length? Is it 
> possible to reach 13000 bytes instead of the standard 1500?
>
> Thank yo in advance,
> Alberto
> ___
> users mailing list
> users@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/users

___
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users

___
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users

___
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users

___
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users

Re: [OMPI users] OMPI users] [OMPI USERS] Jumbo frames

2017-05-05 Thread Gilles Gouaillardet 
Alberto,

Are you saying the program hang even without jumbo frame (aka 1500 MTU) ?
At first, make sure there is no firewall running, and then you can try
mpirun --mca btl tcp,vader,self --mca oob_tcp_if_include eth0 --mca 
btl_tcp_if_include eth0 ...
(Replace eth0 with the interface name you want to use)

Cheers,

Gilles

Alberto Ortiz  wrote:
>I am using version 1.10.6 on archlinux.
>
>The option I should pass to mpirun should then be "-mca btl_tcp_mtu 13000"? 
>Just to be sure.
>
>Thank you,
>
>Alberto
>
>
>El 5 may. 2017 16:26, "r...@open-mpi.org"  escribió:
>
>If you are looking to use TCP packets, then you want to set the send/recv 
>buffer size in the TCP btl, not the openib one, yes?
>
>Also, what version of OMPI are you using?
>
>> On May 5, 2017, at 7:16 AM, Alberto Ortiz  wrote:
>>
>> Hi,
>> I have a program running with openMPI over a network using a gigabit switch. 
>> This switch supports jumbo frames up to 13.000 bytes, so, in order to test 
>> and see if it would be faster communicating with this frame lengths, I am 
>> trying to use them with my program. I have set the MTU in each node to be 
>> 13.000 but when running the program it doesn't even initiate, it gets 
>> blocked. I have tried different lengths from 1.500 up to 13.000 but it 
>> doesn't work with any length.
>>
>> I have searched and only found that I have to set OMPI with "-mca 
>> btl_openib_ib_mtu 13000" or the length to be used, but I don't seem to get 
>> it working.
>>
>> Which are the steps to get OMPI to use larger TCP packets length? Is it 
>> possible to reach 13000 bytes instead of the standard 1500?
>>
>> Thank yo in advance,
>> Alberto
>> ___
>> users mailing list
>> users@lists.open-mpi.org
>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>
>___
>users mailing list
>users@lists.open-mpi.org
>https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>
___
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users

Re: [OMPI users] [OMPI USERS] Jumbo frames

2017-05-05 Thread George Bosilca 
"ompi_info --param btl tcp -l 9" will give you all the TCP options.
Unfortunately, OMPI does not support programatically changing the value of
the MTU.

  George.

PS: We would be happy to receive contributions from the community.


On Fri, May 5, 2017 at 10:29 AM, Alberto Ortiz 
wrote:

> I am using version 1.10.6 on archlinux.
> The option I should pass to mpirun should then be "-mca btl_tcp_mtu
> 13000"? Just to be sure.
> Thank you,
> Alberto
>
> El 5 may. 2017 16:26, "r...@open-mpi.org"  escribió:
>
>> If you are looking to use TCP packets, then you want to set the send/recv
>> buffer size in the TCP btl, not the openib one, yes?
>>
>> Also, what version of OMPI are you using?
>>
>> > On May 5, 2017, at 7:16 AM, Alberto Ortiz 
>> wrote:
>> >
>> > Hi,
>> > I have a program running with openMPI over a network using a gigabit
>> switch. This switch supports jumbo frames up to 13.000 bytes, so, in order
>> to test and see if it would be faster communicating with this frame
>> lengths, I am trying to use them with my program. I have set the MTU in
>> each node to be 13.000 but when running the program it doesn't even
>> initiate, it gets blocked. I have tried different lengths from 1.500 up to
>> 13.000 but it doesn't work with any length.
>> >
>> > I have searched and only found that I have to set OMPI with "-mca
>> btl_openib_ib_mtu 13000" or the length to be used, but I don't seem to get
>> it working.
>> >
>> > Which are the steps to get OMPI to use larger TCP packets length? Is it
>> possible to reach 13000 bytes instead of the standard 1500?
>> >
>> > Thank yo in advance,
>> > Alberto
>> > ___
>> > users mailing list
>> > users@lists.open-mpi.org
>> > https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>
>> ___
>> users mailing list
>> users@lists.open-mpi.org
>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>
>
> ___
> users mailing list
> users@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>
___
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users

Re: [OMPI users] [OMPI USERS] Jumbo frames

2017-05-05 Thread Alberto Ortiz 
I am using version 1.10.6 on archlinux.
The option I should pass to mpirun should then be "-mca btl_tcp_mtu 13000"?
Just to be sure.
Thank you,
Alberto

El 5 may. 2017 16:26, "r...@open-mpi.org"  escribió:

> If you are looking to use TCP packets, then you want to set the send/recv
> buffer size in the TCP btl, not the openib one, yes?
>
> Also, what version of OMPI are you using?
>
> > On May 5, 2017, at 7:16 AM, Alberto Ortiz 
> wrote:
> >
> > Hi,
> > I have a program running with openMPI over a network using a gigabit
> switch. This switch supports jumbo frames up to 13.000 bytes, so, in order
> to test and see if it would be faster communicating with this frame
> lengths, I am trying to use them with my program. I have set the MTU in
> each node to be 13.000 but when running the program it doesn't even
> initiate, it gets blocked. I have tried different lengths from 1.500 up to
> 13.000 but it doesn't work with any length.
> >
> > I have searched and only found that I have to set OMPI with "-mca
> btl_openib_ib_mtu 13000" or the length to be used, but I don't seem to get
> it working.
> >
> > Which are the steps to get OMPI to use larger TCP packets length? Is it
> possible to reach 13000 bytes instead of the standard 1500?
> >
> > Thank yo in advance,
> > Alberto
> > ___
> > users mailing list
> > users@lists.open-mpi.org
> > https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>
> ___
> users mailing list
> users@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>
___
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users

Re: [OMPI users] [OMPI USERS] Jumbo frames

2017-05-05 Thread r...@open-mpi.org 
If you are looking to use TCP packets, then you want to set the send/recv 
buffer size in the TCP btl, not the openib one, yes?

Also, what version of OMPI are you using?

> On May 5, 2017, at 7:16 AM, Alberto Ortiz  wrote:
> 
> Hi,
> I have a program running with openMPI over a network using a gigabit switch. 
> This switch supports jumbo frames up to 13.000 bytes, so, in order to test 
> and see if it would be faster communicating with this frame lengths, I am 
> trying to use them with my program. I have set the MTU in each node to be 
> 13.000 but when running the program it doesn't even initiate, it gets 
> blocked. I have tried different lengths from 1.500 up to 13.000 but it 
> doesn't work with any length.
> 
> I have searched and only found that I have to set OMPI with "-mca 
> btl_openib_ib_mtu 13000" or the length to be used, but I don't seem to get it 
> working.
> 
> Which are the steps to get OMPI to use larger TCP packets length? Is it 
> possible to reach 13000 bytes instead of the standard 1500?
> 
> Thank yo in advance,
> Alberto
> ___
> users mailing list
> users@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/users

___
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users

[OMPI users] [OMPI USERS] Jumbo frames

2017-05-05 Thread Alberto Ortiz 
Hi,
I have a program running with openMPI over a network using a gigabit
switch. This switch supports jumbo frames up to 13.000 bytes, so, in order
to test and see if it would be faster communicating with this frame
lengths, I am trying to use them with my program. I have set the MTU in
each node to be 13.000 but when running the program it doesn't even
initiate, it gets blocked. I have tried different lengths from 1.500 up to
13.000 but it doesn't work with any length.

I have searched and only found that I have to set OMPI with "-mca
btl_openib_ib_mtu 13000" or the length to be used, but I don't seem to get
it working.

Which are the steps to get OMPI to use larger TCP packets length? Is it
possible to reach 13000 bytes instead of the standard 1500?

Thank yo in advance,
Alberto
___
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users

Re: [OMPI users] Jumbo frames

2006-08-25 Thread Caird, Andrew J 
Massimiliano,

It should work automatically, but I have seen instances where switches
or Ethernet cards can't support the full 9000 bytes per frame, and we've
had to go as low as 6000 bytes to get consistent performance.  It seems
like everyone's interpretation of what the 9000 bytes is for is a little
different.

Does it work with the defaults 1500byte setting?  You might try
increasing in smaller steps to see where it stops working.

Good luck.
--andrew


> -Original Message-
> From: users-boun...@open-mpi.org 
> [mailto:users-boun...@open-mpi.org] On Behalf Of Massimiliano Fatica
> Sent: Friday, August 25, 2006 1:30 AM
> To: us...@open-mpi.org
> Subject: [OMPI users] Jumbo frames
> 
> Hi,
> I am trying to use Jumbo frames but mpirun will not start the job.
> I am using OpenMPI v1.1 shipped with the latest Rocks (4.2).
> Ifconfig is reporting that all the NIC on the cluster are 
> using an MTU of 9000 and the switch (HP Procurve) should be 
> able to use Jumbo frames.
> 
> Is there any special flag I need to pass to mpirun or a 
> configuration file I need to edit?
> 
> Thanks
> Massimiliano
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>

[OMPI users] Jumbo frames

2006-08-25 Thread Massimiliano Fatica 

Hi,
I am trying to use Jumbo frames but mpirun will not start the job.
I am using OpenMPI v1.1 shipped with the latest Rocks (4.2).
Ifconfig is reporting that all the NIC on the cluster are using an MTU
of 9000 and the switch (HP Procurve) should be able to use Jumbo
frames.

Is there any special flag I need to pass to mpirun or a configuration
file I need to edit?

Thanks
Massimiliano