Re: [OMPI users] cluster with iOS or Android devices?

[Ralph Castain]

The connection on/off the iPad looks like an Ethernet port, I believe - but you 
should check that. Alternatively, you can send/recv across the wifi connection. 
No idea of the relative speeds, but you should be able to google that data.

On Nov 30, 2012, at 3:35 PM, shiny knight  wrote:

> I totally get your point Jeff, and thanks for pointing it out...this is an 
> aspect that I didn't consider yet.
> 
> Power should not be an issue, since the devices are plugged in. Now I need to 
> evaluate exactly how much power I can pull while the device is connected to 
> the computer, compared to the power needed to run a process at 100% CPU load. 
> Running over batteries is absolutely out of question I guess; my calculations 
> should go on for at least a couple of hours, so I doubt that I can run a 
> small device with batteries and accomplish my objectives.
> 
> Is there any info about how the I/O on the iPad and iPhone works? So I can 
> have an idea about what I can run on that cable and for how long. As you 
> pointed out, the main issue will be syncing processes...wifi may be feasible 
> but would be slower I guess (without specs is hard to even make assumptions).
> 
> No worries, you are talking about things that has to be evaluated; I am just 
> exploring an alternate use of old hardware; which may result in not being 
> convenient at all in the end. So any comments helps :)
> 
> I will focus on calculating what you suggested. Theoretically the dual core 
> Apple processors should be powerful enough to give some sort of performance 
> boost, but I am new to ARM so I don't really know too much about their 
> structure and pipeline, so I may be totally wrong.
> 
> -lou
> 
> 
> On Nov 30, 2012, at 7:35 AM, Jeff Squyres wrote:
> 
>> Not to throw cold water on this, but I think the canonical problem cited 
>> with doing distributed computations on mobile devices is the power 
>> requirement.  Meaning: if the devices are running on battery, you're really 
>> not going to get much computation out of them.  
>> 
>> And if you have them plugged in, you have a potential IO issue (i.e., how to 
>> get the input onto the device and the output out of the device).  You 
>> probably only have 802.11g (maybe 802.11n?) wifi available, and you might 
>> have to deal with a LOT of I/O.  Meaning: you might need to restrict this 
>> work to applications that are compute-heavy but IO-light.  But then again, 
>> you're dealing with small, "slow" processors, so compute-heavy problems on 
>> such processors might not do so well.  Or, more precisely, you might get 
>> much more compute efficiency with traditional "big" HPC servers.
>> 
>> Don't get me wrong; I'm not trying to say this is a bad idea.  I'm just 
>> saying that it's worth doing some back-of-the-envelope calculations before 
>> you spend a lot of effort on porting code to mobile platforms.
>> 
>> For example, here's some interesting data points that would be good to 
>> calculate:
>> 
>> 1. How many (pick your favorite mobile device; say -- iPhone 5) would it 
>> take to equal the power of one cheap Intel Sandy Bridge-based server with 16 
>> cores?  Compare things like max sustained FLOPS and IOPS (integer ops, not 
>> IO ops), RAM sizes, etc.
>> 
>> 2. What's the procurement cost differential between 1 Intel Sandy 
>> Bridge-based server and N iPhone 5s?  What's the operational cost 
>> differential?
>> 
>> 
>> 
>> On Nov 30, 2012, at 10:25 AM, Ralph Castain wrote:
>> 
>>> Just an FYI: xgrid is no longer being distributed or supported.
>>> 
>>> I'd start by first building OMPI against the iOS simulator in Xcode. You 
>>> may run into some issues with the atomics that will need addressing, and 
>>> there may be other issues with syntax and header file locations. Best to 
>>> resolve those first.
>>> 
>>> Once you get that to build, you can test running several procs on a single 
>>> iPad. If you have older iPads, I'm not sure that will work as they don't 
>>> multi-task. But might be worth a try.
>>> 
>>> You'll then need to find a way to launch the processes across iPads. I 
>>> don't know if ssh will work, so you may have to devise a new plm module. I 
>>> can advise as you go.
>>> 
>>> FWIW: I have an iPad 1 and iOS development kit, so I can potentially help 
>>> with problems.
>>> 
>>> 
>>> On Nov 29, 2012, at 10:16 PM, shiny knight  wrote:
>>> 
 Thanks for all your replies.
 
 As now I have access to 3 iOS devices and 1 Android, so if possible I 
 would be oriented to pursue more the iOS route.
 
 So it seems that there is not yet a simple way to do so on these devices 
 (Thanks for the paper posted Dominik); I will have to look deeper in that 
 project that you mentioned and wait for some official release (at least 
 for the Android side)
 
 I may install linux distro on a virtual machine; mostly I work on OSX so 
 it should not be that bad (OSX allows me to work 

Re: [OMPI users] OpenMPI-1.6.3 & MXM

[Joseph Farran]

Konz,

For whatever it is worth, I am in the same boat.

I have CentOS 6.3, trying to compile OpenMPI 1.6.3 with the mxm from Mellanox 
and it fails.

Also, the Mellanox OFED ( MLNX_OFED_LINUX-1.5.3-3.1.0-rhel6.3-x86_64 ) does not 
work either.

Mellanox really needs to step in here and help out.

I have a cluster full of Mellanox products and I hate to think we chose the 
wrong Infiniband vendor.

Joseph


On 11/30/2012 12:33 PM, Konz, Jeffrey (SSA Solution Centers) wrote:


I tried building the latest OpenMPI-1.6.3 with MXM support and got this error:

make[2]: Entering directory `Src/openmpi-1.6.3/ompi/mca/mtl/mxm'

  CC mtl_mxm.lo

  CC mtl_mxm_cancel.lo

  CC mtl_mxm_component.lo

  CC mtl_mxm_endpoint.lo

  CC mtl_mxm_probe.lo

  CC mtl_mxm_recv.lo

  CC mtl_mxm_send.lo

mtl_mxm_send.c: In function 'ompi_mtl_mxm_send':

mtl_mxm_send.c:96: error: 'mxm_wait_t' undeclared (first use in this function)

mtl_mxm_send.c:96: error: (Each undeclared identifier is reported only once

mtl_mxm_send.c:96: error: for each function it appears in.)

mtl_mxm_send.c:96: error: expected ';' before 'wait'

mtl_mxm_send.c:104: error: 'MXM_REQ_FLAG_BLOCKING' undeclared (first use in 
this function)

mtl_mxm_send.c:118: error: 'MXM_REQ_FLAG_SEND_SYNC' undeclared (first use in 
this function)

mtl_mxm_send.c:134: error: 'wait' undeclared (first use in this function)

mtl_mxm_send.c: In function 'ompi_mtl_mxm_isend':

mtl_mxm_send.c:183: error: 'MXM_REQ_FLAG_SEND_SYNC' undeclared (first use in 
this function)

make[2]: *** [mtl_mxm_send.lo] Error 1

Our OFED is 1.5.3 and our MXM version is 1.0.601.

Thanks,

-Jeff

/**/

/* Jeff Konz  jeffrey.k...@hp.com */

/* Solutions Architect   HPC Benchmarking */

/* Americas Shared Solutions Architecture (SSA)   */

/* Hewlett-Packard Company*/

/* Office: 248-491-7480  Mobile: 248-345-6857 */

/**/



___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] cluster with iOS or Android devices?

[shiny knight]

I totally get your point Jeff, and thanks for pointing it out...this is an 
aspect that I didn't consider yet.

Power should not be an issue, since the devices are plugged in. Now I need to 
evaluate exactly how much power I can pull while the device is connected to the 
computer, compared to the power needed to run a process at 100% CPU load. 
Running over batteries is absolutely out of question I guess; my calculations 
should go on for at least a couple of hours, so I doubt that I can run a small 
device with batteries and accomplish my objectives.

Is there any info about how the I/O on the iPad and iPhone works? So I can have 
an idea about what I can run on that cable and for how long. As you pointed 
out, the main issue will be syncing processes...wifi may be feasible but would 
be slower I guess (without specs is hard to even make assumptions).

No worries, you are talking about things that has to be evaluated; I am just 
exploring an alternate use of old hardware; which may result in not being 
convenient at all in the end. So any comments helps :)

I will focus on calculating what you suggested. Theoretically the dual core 
Apple processors should be powerful enough to give some sort of performance 
boost, but I am new to ARM so I don't really know too much about their 
structure and pipeline, so I may be totally wrong.

-lou


On Nov 30, 2012, at 7:35 AM, Jeff Squyres wrote:

> Not to throw cold water on this, but I think the canonical problem cited with 
> doing distributed computations on mobile devices is the power requirement.  
> Meaning: if the devices are running on battery, you're really not going to 
> get much computation out of them.  
> 
> And if you have them plugged in, you have a potential IO issue (i.e., how to 
> get the input onto the device and the output out of the device).  You 
> probably only have 802.11g (maybe 802.11n?) wifi available, and you might 
> have to deal with a LOT of I/O.  Meaning: you might need to restrict this 
> work to applications that are compute-heavy but IO-light.  But then again, 
> you're dealing with small, "slow" processors, so compute-heavy problems on 
> such processors might not do so well.  Or, more precisely, you might get much 
> more compute efficiency with traditional "big" HPC servers.
> 
> Don't get me wrong; I'm not trying to say this is a bad idea.  I'm just 
> saying that it's worth doing some back-of-the-envelope calculations before 
> you spend a lot of effort on porting code to mobile platforms.
> 
> For example, here's some interesting data points that would be good to 
> calculate:
> 
> 1. How many (pick your favorite mobile device; say -- iPhone 5) would it take 
> to equal the power of one cheap Intel Sandy Bridge-based server with 16 
> cores?  Compare things like max sustained FLOPS and IOPS (integer ops, not IO 
> ops), RAM sizes, etc.
> 
> 2. What's the procurement cost differential between 1 Intel Sandy 
> Bridge-based server and N iPhone 5s?  What's the operational cost 
> differential?
> 
> 
> 
> On Nov 30, 2012, at 10:25 AM, Ralph Castain wrote:
> 
>> Just an FYI: xgrid is no longer being distributed or supported.
>> 
>> I'd start by first building OMPI against the iOS simulator in Xcode. You may 
>> run into some issues with the atomics that will need addressing, and there 
>> may be other issues with syntax and header file locations. Best to resolve 
>> those first.
>> 
>> Once you get that to build, you can test running several procs on a single 
>> iPad. If you have older iPads, I'm not sure that will work as they don't 
>> multi-task. But might be worth a try.
>> 
>> You'll then need to find a way to launch the processes across iPads. I don't 
>> know if ssh will work, so you may have to devise a new plm module. I can 
>> advise as you go.
>> 
>> FWIW: I have an iPad 1 and iOS development kit, so I can potentially help 
>> with problems.
>> 
>> 
>> On Nov 29, 2012, at 10:16 PM, shiny knight  wrote:
>> 
>>> Thanks for all your replies.
>>> 
>>> As now I have access to 3 iOS devices and 1 Android, so if possible I would 
>>> be oriented to pursue more the iOS route.
>>> 
>>> So it seems that there is not yet a simple way to do so on these devices 
>>> (Thanks for the paper posted Dominik); I will have to look deeper in that 
>>> project that you mentioned and wait for some official release (at least for 
>>> the Android side)
>>> 
>>> I may install linux distro on a virtual machine; mostly I work on OSX so it 
>>> should not be that bad (OSX allows me to work with both Android and iOS 
>>> hassle free; that's why I had the thought to use my devices for MPI).
>>> 
>>> Beatty: My idea is to use the devices only when plugged in; I was reading a 
>>> paper about how to use MPI and dynamically change the number of nodes 
>>> attached, while crunching data for a process. So it would be possible to 
>>> add and remove nodes on the fly, and was trying to apply it to a portable 
>>> device 

Re: [OMPI users] cluster with iOS or Android devices?

[shiny knight]

Thanks a lot for the pointers Ralph.

So I just check out the source for OMPI and build it in Xcode with target iOS? 
Sounds pretty straight forward. I will probably have to deal with errors but it 
seems that you did it already and it should not be that hard (I am still 
learning many things).

My iPad runs 5.1 so should be fine with the multitasking side. My first 
objective will be to run just one process on the iPad, sent by the main 
computer; then it will be interesting to explore a way to let the various 
device to communicate.

Much appreciated the help!

-lou


On Nov 30, 2012, at 7:25 AM, Ralph Castain wrote:

> Just an FYI: xgrid is no longer being distributed or supported.
> 
> I'd start by first building OMPI against the iOS simulator in Xcode. You may 
> run into some issues with the atomics that will need addressing, and there 
> may be other issues with syntax and header file locations. Best to resolve 
> those first.
> 
> Once you get that to build, you can test running several procs on a single 
> iPad. If you have older iPads, I'm not sure that will work as they don't 
> multi-task. But might be worth a try.
> 
> You'll then need to find a way to launch the processes across iPads. I don't 
> know if ssh will work, so you may have to devise a new plm module. I can 
> advise as you go.
> 
> FWIW: I have an iPad 1 and iOS development kit, so I can potentially help 
> with problems.
> 
> 
> On Nov 29, 2012, at 10:16 PM, shiny knight  wrote:
> 
>> Thanks for all your replies.
>> 
>> As now I have access to 3 iOS devices and 1 Android, so if possible I would 
>> be oriented to pursue more the iOS route.
>> 
>> So it seems that there is not yet a simple way to do so on these devices 
>> (Thanks for the paper posted Dominik); I will have to look deeper in that 
>> project that you mentioned and wait for some official release (at least for 
>> the Android side)
>> 
>> I may install linux distro on a virtual machine; mostly I work on OSX so it 
>> should not be that bad (OSX allows me to work with both Android and iOS 
>> hassle free; that's why I had the thought to use my devices for MPI).
>> 
>> Beatty: My idea is to use the devices only when plugged in; I was reading a 
>> paper about how to use MPI and dynamically change the number of nodes 
>> attached, while crunching data for a process. So it would be possible to add 
>> and remove nodes on the fly, and was trying to apply it to a portable device 
>> (http://www.cs.rpi.edu/~szymansk/papers/ppam05.pdf) before realizing that 
>> there is no MPI implementation for them.
>> 
>> I would never envision a system where a user has a device in his pocket that 
>> is actually doing "something" behind is back...mine was a simple issue with 
>> having devices sitting on my desk, which I use to test my apps, and I could 
>> use these devices in a more productive way, while I have them tethered to my 
>> main machine (which is the main server where MPI development is done).
>> 
>> Would you mind elaborate on the approach that you mentioned? I never used 
>> Xgrid, so I am not sure about how your solution would work.
>> 
>> Thanks!
>> 
>> Lou
>> 
>> 
>> On Nov 29, 2012, at 4:14 PM, Beatty, Daniel D CIV NAVAIR, 474300D wrote:
>> 
>>> Greetings Ladies and gentlemen,
>>> There is one alternative approach and this a psuedo-cloud based MPI.  The
>>> idea is that MPI node list is adjusted via the cloud similar to the way
>>> Xgrid's Bonjour used to do it for Xgrid.
>>> 
>>> In this case, it is applying an MPI notion to the OpenCL codelets.  There
>>> are obvious issues with security, battery life, etc.  There is considerable
>>> room for discussion as far expectations.  Do jobs run free if the device is
>>> plugged in?  If the device in the pocket, can the user switch to power
>>> conservation/ cooler pockets?  What constitutes fairness?  Do owners have a
>>> right to be biased in judgement?   These are tough questions that I think I
>>> will have to provide fair assurances for.  After all, everyone likes to
>>> think they are control of what they put in their pocket.
>>> 
>>> V/R,
>>> Dan
>>> 
>>> 
>>> On 11/28/12 3:06 PM, "Dominik Goeddeke"
>>>  wrote:
>>> 
 shameless plug: 
 http://www.mathematik.tu-dortmund.de/~goeddeke/pubs/pdf/Goeddeke_2012_EEV.pdf
 
 In the MontBlanc project (www.montblanc-project.eu), a lot of folks from
 all around Europe look into exactly this. Together with a few
 colleagues, we have been honoured to get access to an early prototype
 system. The runs for the paper above (accepted in JCP as of last week)
 have been carried out with MPICH2 back in June, but OpenMPI also worked
 flawlessly except for some issues with SLURM integration at the time we
 did those tests.
 
 The bottom line is: The prototype machine (128 Tegra2's) ran standard
 ubuntu, and since Android is essentially Linux, it should not be to
 hard to 

Re: [OMPI users] MPI_Allreduce with F90 Handles

[Jeff Squyres]

On Nov 30, 2012, at 2:04 PM, Shane Hart wrote:

> I've attached a small sample program that demonstrates the problem.  You can 
> toggle working/non-working behaviour by toggling commenting on line 27.

Thanks!  I got swamped this week, but I'll try to look at it next week 
(although with the Forum meeting next week, it might have to wait until the 
week after :-( ).

> I've tried to open a bug report, but the system isn't letting me register for 
> Trac:
>  
> Trac detected an internal error:
> KeyError: 'recaptcha_challenge_field'

Weird.  I've forwarded this on to our sysadmin to see what's going on there.

I'll file a ticket and CC you.

>  
> Shane
>  
> On Friday, November 30, 2012 10:35:57 AM users-requ...@open-mpi.org wrote:
> All,
>  
> I have a Fortran code that works quite well with OpenMPI 1.4.3 where I create 
> a handle using:
>  
> call MPI_TYPE_CREATE_F90_INTEGER(9, COMM_INT4, ierror)
>  
> and then do a reduction with:
>  
> call MPI_ALLREDUCE(send_buffer, buffer, count, COMM_INT4, MPI_SUM, 
> communicator,  ierror)
>  
> However, this fails with OpenMPI 1.6.3 stating:
>  
> An error occurred in MPI_Allreduce: the reduction operation MPI_SUM is not 
> defined for non-intrinsic datatypes (attempted with datatype named "Dup 
> MPI_INT")
>  
> The MPI standard states that MPI_SUM works with Fortran integers, and Fortran 
> integers are defined in Section 5.9.2 as:
>  
> MPI_INTEGER, MPI_AINT, MPI_OFFSET,
> *and handles returned from
> MPI_TYPE_CREATE_F90_INTEGER*,
> and if available: MPI_INTEGER1,
> MPI_INTEGER2, MPI_INTEGER4,
> MPI_INTEGER8, MPI_INTEGER16
>  
> (emphasis mine)
>  
> However the manual page on MPI_Reduce for OpenMPI only states that Fortran 
> integers are:
>  
> Fortran integer:  MPI_INTEGER
> Does OpenMPI not support using MPI_SUM with Fortran integer handles?  I'm 
> hoping that it's just an oversight as it used to work...
>  
> System: Fedora 17
> Compilers: GCC 4.7.2
> OpenMPI's tested: 1.4.3 (works), 1.5.4 (doesn't work), 1.6.3 (doesn't work)
>  
> Thanks for any insight,
> Shane
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

[OMPI users] OpenMPI-1.6.3 & MXM

[Konz, Jeffrey (SSA Solution Centers)]

I tried building the latest OpenMPI-1.6.3 with MXM support and got this error:



make[2]: Entering directory `Src/openmpi-1.6.3/ompi/mca/mtl/mxm'

  CC mtl_mxm.lo

  CC mtl_mxm_cancel.lo

  CC mtl_mxm_component.lo

  CC mtl_mxm_endpoint.lo

  CC mtl_mxm_probe.lo

  CC mtl_mxm_recv.lo

  CC mtl_mxm_send.lo

mtl_mxm_send.c: In function 'ompi_mtl_mxm_send':

mtl_mxm_send.c:96: error: 'mxm_wait_t' undeclared (first use in this function)

mtl_mxm_send.c:96: error: (Each undeclared identifier is reported only once

mtl_mxm_send.c:96: error: for each function it appears in.)

mtl_mxm_send.c:96: error: expected ';' before 'wait'

mtl_mxm_send.c:104: error: 'MXM_REQ_FLAG_BLOCKING' undeclared (first use in 
this function)

mtl_mxm_send.c:118: error: 'MXM_REQ_FLAG_SEND_SYNC' undeclared (first use in 
this function)

mtl_mxm_send.c:134: error: 'wait' undeclared (first use in this function)

mtl_mxm_send.c: In function 'ompi_mtl_mxm_isend':

mtl_mxm_send.c:183: error: 'MXM_REQ_FLAG_SEND_SYNC' undeclared (first use in 
this function)

make[2]: *** [mtl_mxm_send.lo] Error 1





Our OFED is 1.5.3 and our MXM version is 1.0.601.



Thanks,



-Jeff



/**/

/* Jeff Konz  jeffrey.k...@hp.com */

/* Solutions Architect   HPC Benchmarking */

/* Americas Shared Solutions Architecture (SSA)   */

/* Hewlett-Packard Company*/

/* Office: 248-491-7480  Mobile: 248-345-6857 */

/**/

Re: [OMPI users] MPI_Allreduce with F90 Handles

[Shane Hart]

I've attached a small sample program that demonstrates the problem.  You can 
toggle working/non-working behaviour by toggling commenting on line 27.

I've tried to open a bug report, but the system isn't letting me register for 
Trac:

Trac detected an internal error:
KeyError: 'recaptcha_challenge_field'


Shane

On Friday, November 30, 2012 10:35:57 AM users-requ...@open-mpi.org wrote:
All,
 
I have a Fortran code that works quite well with OpenMPI 1.4.3 where I create 
a handle using:
 
call MPI_TYPE_CREATE_F90_INTEGER(9, COMM_INT4, ierror)
 
and then do a reduction with:
 
call MPI_ALLREDUCE(send_buffer, buffer, count, COMM_INT4, MPI_SUM, 
communicator,  
ierror)
 
However, this fails with OpenMPI 1.6.3 stating:
 
An error occurred in MPI_Allreduce: the reduction operation MPI_SUM is not 
defined for non-intrinsic datatypes (attempted with datatype named "Dup 
MPI_INT")
 
The MPI standard states that MPI_SUM works with Fortran integers, and Fortran 
integers are defined in Section 5.9.2 as:
 
MPI_INTEGER, MPI_AINT, MPI_OFFSET,
*and handles returned from
MPI_TYPE_CREATE_F90_INTEGER*,
and if available: MPI_INTEGER1,
MPI_INTEGER2, MPI_INTEGER4,
MPI_INTEGER8, MPI_INTEGER16
 
(emphasis mine)
 
However the manual page on MPI_Reduce for OpenMPI only states that Fortran 
integers are:
 
Fortran integer:  MPI_INTEGER
Does OpenMPI not support using MPI_SUM with Fortran integer handles?  I'm 
hoping that it's just an oversight as it used to work...
 
System: Fedora 17
Compilers: GCC 4.7.2
OpenMPI's tested: 1.4.3 (works), 1.5.4 (doesn't work), 1.6.3 (doesn't work)
 
Thanks for any insight,
Shane
program mpi_sum_test
implicit none

include 'mpif.h'

integer, parameter :: my_int_kind = selected_int_kind(9)
integer(kind=my_int_kind) :: my_ints
integer(kind=my_int_kind) :: my_ints_sum

integer :: test_fortran_integer_handle
integer :: test_fortran_integer_size
integer :: ierror
integer :: node_num
integer :: num_nodes

call mpi_init(ierror)
call mpi_comm_rank(mpi_comm_world, node_num, ierror)
call mpi_comm_size(mpi_comm_world, num_nodes, ierror)

call mpi_type_create_f90_integer(9, test_fortran_integer_handle, ierror)
test_fortran_integer_size = sizeof(my_ints)
if (node_num == 0) write (*,*) 'integer handle created as: ', test_fortran_integer_handle, ' mpi_int is ', mpi_int

! *** An error occurred in MPI_Reduce: the reduction operation MPI_SUM is not defined for non-intrinsic datatypes
! (attempted with datatype named "Dup MPI_INT")
! Uncomment next line to make this program work (since MPI reports my handle is a duplicate of MPI_INT we can just use that):
!test_fortran_integer_handle = mpi_int

my_ints = int(node_num+1, kind=my_int_kind)

call mpi_reduce(my_ints, my_ints_sum, 1, test_fortran_integer_handle, mpi_sum, 0, mpi_comm_world, ierror)

if (node_num == 0) write (*,*) 'Sum = ', my_ints_sum

call mpi_finalize(ierror)

end program mpi_sum_test

Re: [OMPI users] BLCR + Qlogic infiniband

[Josh Hursey]

The openib BTL and BLCR support in Open MPI were working about a year ago
(when I last checked). The psm BTL is not supported at the moment though.

>From the error, I suspect that we are not fully closing the openib btl
driver before the checkpoint thus when we try to restart it is looking for
a resource that is no longer present. I created a ticket for us to
investigate further if you want to follow it:
  https://svn.open-mpi.org/trac/ompi/ticket/3417

Unfortunately, I do not know who is currently supporting that code path (I
might pick it back up at some point, but cannot promise anything in the
near future). But I will keep an eye on the ticket and see what I can do.
If it is what I think it is, then it should not take too much work to get
it working again.

-- Josh

On Wed, Nov 28, 2012 at 5:14 AM, William Hay  wrote:

> I'm trying to build openmpi with support for BLCR plus qlogic infiniband
> (plus grid engine).  Everything seems to compile OK and checkpoints are
> taken but whenever I try to restore a checkpoint I get the following error:
> - do_mmap(, 2aaab18c7000, 1000, ...) failed:
> ffea
> - mmap failed: /dev/ipath
> - thaw_threads returned error, aborting. -22
> - thaw_threads returned error, aborting. -22
> Restart failed: Invalid argument
>
> This occurs whether I specify psm or openib as the btl.
>
> This looks like the sort of thing I would expect to be handled by the blcr
> supporting code in openmpi.  So I guess I have a couple ofquestions.
> 1)Are Infiniband and BLCR support in openmpi compatible?
> 2)Are there any special tricks necessary to get them working together.
>
>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>



-- 
Joshua Hursey
Assistant Professor of Computer Science
University of Wisconsin-La Crosse
http://cs.uwlax.edu/~jjhursey

Re: [OMPI users] MCA crs: none (MCA v2.0, API v2.0, Component v1.6.3)

[Josh Hursey]

Can you send the config.log and some of the other information described on:
  http://www.open-mpi.org/community/help/

-- Josh

On Wed, Nov 14, 2012 at 6:01 PM, Ifeanyi  wrote:

> Hi all,
>
> I got this message when I issued this command:
>
> root@node1:/home/abolap# ompi_info | grep crs
>  MCA crs: none (MCA v2.0, API v2.0, Component v1.6.3)
>
> The installation looks okay and I have reinstalled but still got the same
> issue.
>
> When I searched for the solution I found out that this is a bug which Josh
> has filed (https://svn.open-mpi.org/trac/ompi/ticket/2097) but I cannot
> see the solution or workaround.
>
> This is the initial post -
> http://www.digipedia.pl/usenet/thread/11269/6087/#post6031
>
> Please assist.
>
> Regards,
> Ifeanyi
>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>



-- 
Joshua Hursey
Assistant Professor of Computer Science
University of Wisconsin-La Crosse
http://cs.uwlax.edu/~jjhursey

Re: [OMPI users] (no subject)

[Josh Hursey]

Pramoda,

That paper was exploring an application of a proposed extension to the MPI
standard for fault tolerance purposes. By default this proposed interface
is not provided by Open MPI. We have created a prototype version of Open
MPI that includes this extension, and it can be found at the following
website:
  http://fault-tolerance.org/

You should look at the interfaces in the new proposal (ULFM Specification)
since MPI_Comm_validate_rank is no longer part of the proposal. You can get
the same functionality through some of the new interfaces that replace it.
There are some examples on that website, and in the proposal that should
help you as well.

Best,
Josh

On Mon, Nov 19, 2012 at 8:59 AM, sri pramoda  wrote:

>  Dear Sir,
> I am Pramoda,PG scholar from Jadavpur Univesity,India.
>  I've gone through a paper "Building a Fault Tolerant MPI Application:
>  A Ring Communication Example".In this I found MPI_Comm_validate_rank
> command.
>  But I didn't found this command in mpi. Hence I request you to please
> send methe implementation of this command.
> Thank you,
>   Pramoda.
>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>



-- 
Joshua Hursey
Assistant Professor of Computer Science
University of Wisconsin-La Crosse
http://cs.uwlax.edu/~jjhursey

Re: [OMPI users] cluster with iOS or Android devices?

[Jeff Squyres]

Not to throw cold water on this, but I think the canonical problem cited with 
doing distributed computations on mobile devices is the power requirement.  
Meaning: if the devices are running on battery, you're really not going to get 
much computation out of them.  

And if you have them plugged in, you have a potential IO issue (i.e., how to 
get the input onto the device and the output out of the device).  You probably 
only have 802.11g (maybe 802.11n?) wifi available, and you might have to deal 
with a LOT of I/O.  Meaning: you might need to restrict this work to 
applications that are compute-heavy but IO-light.  But then again, you're 
dealing with small, "slow" processors, so compute-heavy problems on such 
processors might not do so well.  Or, more precisely, you might get much more 
compute efficiency with traditional "big" HPC servers.

Don't get me wrong; I'm not trying to say this is a bad idea.  I'm just saying 
that it's worth doing some back-of-the-envelope calculations before you spend a 
lot of effort on porting code to mobile platforms.

For example, here's some interesting data points that would be good to 
calculate:

1. How many (pick your favorite mobile device; say -- iPhone 5) would it take 
to equal the power of one cheap Intel Sandy Bridge-based server with 16 cores?  
Compare things like max sustained FLOPS and IOPS (integer ops, not IO ops), RAM 
sizes, etc.

2. What's the procurement cost differential between 1 Intel Sandy Bridge-based 
server and N iPhone 5s?  What's the operational cost differential?



On Nov 30, 2012, at 10:25 AM, Ralph Castain wrote:

> Just an FYI: xgrid is no longer being distributed or supported.
> 
> I'd start by first building OMPI against the iOS simulator in Xcode. You may 
> run into some issues with the atomics that will need addressing, and there 
> may be other issues with syntax and header file locations. Best to resolve 
> those first.
> 
> Once you get that to build, you can test running several procs on a single 
> iPad. If you have older iPads, I'm not sure that will work as they don't 
> multi-task. But might be worth a try.
> 
> You'll then need to find a way to launch the processes across iPads. I don't 
> know if ssh will work, so you may have to devise a new plm module. I can 
> advise as you go.
> 
> FWIW: I have an iPad 1 and iOS development kit, so I can potentially help 
> with problems.
> 
> 
> On Nov 29, 2012, at 10:16 PM, shiny knight  wrote:
> 
>> Thanks for all your replies.
>> 
>> As now I have access to 3 iOS devices and 1 Android, so if possible I would 
>> be oriented to pursue more the iOS route.
>> 
>> So it seems that there is not yet a simple way to do so on these devices 
>> (Thanks for the paper posted Dominik); I will have to look deeper in that 
>> project that you mentioned and wait for some official release (at least for 
>> the Android side)
>> 
>> I may install linux distro on a virtual machine; mostly I work on OSX so it 
>> should not be that bad (OSX allows me to work with both Android and iOS 
>> hassle free; that's why I had the thought to use my devices for MPI).
>> 
>> Beatty: My idea is to use the devices only when plugged in; I was reading a 
>> paper about how to use MPI and dynamically change the number of nodes 
>> attached, while crunching data for a process. So it would be possible to add 
>> and remove nodes on the fly, and was trying to apply it to a portable device 
>> (http://www.cs.rpi.edu/~szymansk/papers/ppam05.pdf) before realizing that 
>> there is no MPI implementation for them.
>> 
>> I would never envision a system where a user has a device in his pocket that 
>> is actually doing "something" behind is back...mine was a simple issue with 
>> having devices sitting on my desk, which I use to test my apps, and I could 
>> use these devices in a more productive way, while I have them tethered to my 
>> main machine (which is the main server where MPI development is done).
>> 
>> Would you mind elaborate on the approach that you mentioned? I never used 
>> Xgrid, so I am not sure about how your solution would work.
>> 
>> Thanks!
>> 
>> Lou
>> 
>> 
>> On Nov 29, 2012, at 4:14 PM, Beatty, Daniel D CIV NAVAIR, 474300D wrote:
>> 
>>> Greetings Ladies and gentlemen,
>>> There is one alternative approach and this a psuedo-cloud based MPI.  The
>>> idea is that MPI node list is adjusted via the cloud similar to the way
>>> Xgrid's Bonjour used to do it for Xgrid.
>>> 
>>> In this case, it is applying an MPI notion to the OpenCL codelets.  There
>>> are obvious issues with security, battery life, etc.  There is considerable
>>> room for discussion as far expectations.  Do jobs run free if the device is
>>> plugged in?  If the device in the pocket, can the user switch to power
>>> conservation/ cooler pockets?  What constitutes fairness?  Do owners have a
>>> right to be biased in judgement?   These are tough questions that I think I
>>> will have to provide fair 

Re: [OMPI users] cluster with iOS or Android devices?

[Ralph Castain]

Just an FYI: xgrid is no longer being distributed or supported.

I'd start by first building OMPI against the iOS simulator in Xcode. You may 
run into some issues with the atomics that will need addressing, and there may 
be other issues with syntax and header file locations. Best to resolve those 
first.

Once you get that to build, you can test running several procs on a single 
iPad. If you have older iPads, I'm not sure that will work as they don't 
multi-task. But might be worth a try.

You'll then need to find a way to launch the processes across iPads. I don't 
know if ssh will work, so you may have to devise a new plm module. I can advise 
as you go.

FWIW: I have an iPad 1 and iOS development kit, so I can potentially help with 
problems.


On Nov 29, 2012, at 10:16 PM, shiny knight  wrote:

> Thanks for all your replies.
> 
> As now I have access to 3 iOS devices and 1 Android, so if possible I would 
> be oriented to pursue more the iOS route.
> 
> So it seems that there is not yet a simple way to do so on these devices 
> (Thanks for the paper posted Dominik); I will have to look deeper in that 
> project that you mentioned and wait for some official release (at least for 
> the Android side)
> 
> I may install linux distro on a virtual machine; mostly I work on OSX so it 
> should not be that bad (OSX allows me to work with both Android and iOS 
> hassle free; that's why I had the thought to use my devices for MPI).
> 
> Beatty: My idea is to use the devices only when plugged in; I was reading a 
> paper about how to use MPI and dynamically change the number of nodes 
> attached, while crunching data for a process. So it would be possible to add 
> and remove nodes on the fly, and was trying to apply it to a portable device 
> (http://www.cs.rpi.edu/~szymansk/papers/ppam05.pdf) before realizing that 
> there is no MPI implementation for them.
> 
> I would never envision a system where a user has a device in his pocket that 
> is actually doing "something" behind is back...mine was a simple issue with 
> having devices sitting on my desk, which I use to test my apps, and I could 
> use these devices in a more productive way, while I have them tethered to my 
> main machine (which is the main server where MPI development is done).
> 
> Would you mind elaborate on the approach that you mentioned? I never used 
> Xgrid, so I am not sure about how your solution would work.
> 
> Thanks!
> 
> Lou
> 
> 
> On Nov 29, 2012, at 4:14 PM, Beatty, Daniel D CIV NAVAIR, 474300D wrote:
> 
>> Greetings Ladies and gentlemen,
>> There is one alternative approach and this a psuedo-cloud based MPI.  The
>> idea is that MPI node list is adjusted via the cloud similar to the way
>> Xgrid's Bonjour used to do it for Xgrid.
>> 
>> In this case, it is applying an MPI notion to the OpenCL codelets.  There
>> are obvious issues with security, battery life, etc.  There is considerable
>> room for discussion as far expectations.  Do jobs run free if the device is
>> plugged in?  If the device in the pocket, can the user switch to power
>> conservation/ cooler pockets?  What constitutes fairness?  Do owners have a
>> right to be biased in judgement?   These are tough questions that I think I
>> will have to provide fair assurances for.  After all, everyone likes to
>> think they are control of what they put in their pocket.
>> 
>> V/R,
>> Dan
>> 
>> 
>> On 11/28/12 3:06 PM, "Dominik Goeddeke"
>>  wrote:
>> 
>>> shameless plug: 
>>> http://www.mathematik.tu-dortmund.de/~goeddeke/pubs/pdf/Goeddeke_2012_EEV.pdf
>>> 
>>> In the MontBlanc project (www.montblanc-project.eu), a lot of folks from
>>> all around Europe look into exactly this. Together with a few
>>> colleagues, we have been honoured to get access to an early prototype
>>> system. The runs for the paper above (accepted in JCP as of last week)
>>> have been carried out with MPICH2 back in June, but OpenMPI also worked
>>> flawlessly except for some issues with SLURM integration at the time we
>>> did those tests.
>>> 
>>> The bottom line is: The prototype machine (128 Tegra2's) ran standard
>>> ubuntu, and since Android is essentially Linux, it should not be to
>>> hard to get the system you envision up and running, Shiny Knight.
>>> 
>>> Cheers,
>>> 
>>> Dominik
>>> 
>>> 
>>> On 11/29/2012 12:00 AM, Vincent Diepeveen wrote:
 You might want to post in beowulf mailing list see cc
 and you want to install linux of course.
 
 OpenFabrics releases openmpi, yet it only works at a limited number of
 distributions - most important is having
 the correct kernel (usually old kernel).
 
 I'm gonna try get it to work at debian soon.
 
 
 
 On Nov 28, 2012, at 11:50 PM, shiny knight wrote:
 
> I was looking for some info about MPI port on iOS or Android devices.
> 
> I have some old devices that may result useful, if I could be able to
> include 

Re: [OMPI users] cluster with iOS or Android devices?

[shiny knight]

Thanks for all your replies.

As now I have access to 3 iOS devices and 1 Android, so if possible I would be 
oriented to pursue more the iOS route.

So it seems that there is not yet a simple way to do so on these devices 
(Thanks for the paper posted Dominik); I will have to look deeper in that 
project that you mentioned and wait for some official release (at least for the 
Android side)

I may install linux distro on a virtual machine; mostly I work on OSX so it 
should not be that bad (OSX allows me to work with both Android and iOS hassle 
free; that's why I had the thought to use my devices for MPI).

Beatty: My idea is to use the devices only when plugged in; I was reading a 
paper about how to use MPI and dynamically change the number of nodes attached, 
while crunching data for a process. So it would be possible to add and remove 
nodes on the fly, and was trying to apply it to a portable device 
(http://www.cs.rpi.edu/~szymansk/papers/ppam05.pdf) before realizing that there 
is no MPI implementation for them.

I would never envision a system where a user has a device in his pocket that is 
actually doing "something" behind is back...mine was a simple issue with having 
devices sitting on my desk, which I use to test my apps, and I could use these 
devices in a more productive way, while I have them tethered to my main machine 
(which is the main server where MPI development is done).

Would you mind elaborate on the approach that you mentioned? I never used 
Xgrid, so I am not sure about how your solution would work.

Thanks!

Lou


On Nov 29, 2012, at 4:14 PM, Beatty, Daniel D CIV NAVAIR, 474300D wrote:

> Greetings Ladies and gentlemen,
> There is one alternative approach and this a psuedo-cloud based MPI.  The
> idea is that MPI node list is adjusted via the cloud similar to the way
> Xgrid's Bonjour used to do it for Xgrid.
> 
> In this case, it is applying an MPI notion to the OpenCL codelets.  There
> are obvious issues with security, battery life, etc.  There is considerable
> room for discussion as far expectations.  Do jobs run free if the device is
> plugged in?  If the device in the pocket, can the user switch to power
> conservation/ cooler pockets?  What constitutes fairness?  Do owners have a
> right to be biased in judgement?   These are tough questions that I think I
> will have to provide fair assurances for.  After all, everyone likes to
> think they are control of what they put in their pocket.
> 
> V/R,
> Dan
> 
> 
> On 11/28/12 3:06 PM, "Dominik Goeddeke"
>  wrote:
> 
>> shameless plug: 
>> http://www.mathematik.tu-dortmund.de/~goeddeke/pubs/pdf/Goeddeke_2012_EEV.pdf
>> 
>> In the MontBlanc project (www.montblanc-project.eu), a lot of folks from
>> all around Europe look into exactly this. Together with a few
>> colleagues, we have been honoured to get access to an early prototype
>> system. The runs for the paper above (accepted in JCP as of last week)
>> have been carried out with MPICH2 back in June, but OpenMPI also worked
>> flawlessly except for some issues with SLURM integration at the time we
>> did those tests.
>> 
>> The bottom line is: The prototype machine (128 Tegra2's) ran standard
>> ubuntu, and since Android is essentially Linux, it should not be to
>> hard to get the system you envision up and running, Shiny Knight.
>> 
>> Cheers,
>> 
>> Dominik
>> 
>> 
>> On 11/29/2012 12:00 AM, Vincent Diepeveen wrote:
>>> You might want to post in beowulf mailing list see cc
>>> and you want to install linux of course.
>>> 
>>> OpenFabrics releases openmpi, yet it only works at a limited number of
>>> distributions - most important is having
>>> the correct kernel (usually old kernel).
>>> 
>>> I'm gonna try get it to work at debian soon.
>>> 
>>> 
>>> 
>>> On Nov 28, 2012, at 11:50 PM, shiny knight wrote:
>>> 
 I was looking for some info about MPI port on iOS or Android devices.
 
 I have some old devices that may result useful, if I could be able to
 include them in my computation scheme.
 
 OpenCL runs on iOS and Android, so I was wondering if there is any
 way to have an old iPhone/phone or iPad/tablet to run MPI.
 
 Tried to look everywhere, but I didn't find anything that says that
 it is possible, nor I've found any practical example.
 
 Thanks!
 ___
 users mailing list
 us...@open-mpi.org
 http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> 
>>> ___
>>> users mailing list
>>> us...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>> 
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users