[Yade-users] [Question #647852]: Using openmp in a python script.
New question #647852 on Yade: https://answers.launchpad.net/yade/+question/647852 Hello, I currently run Yade by importing it in a python script as explained in [1]. However I am unable to utilize more than 1 core(for yade) in my simulations, I've tried the method described in [2], and I don't see any difference and I've also tried setting 'ompThreads=(N)' in the engines and I see no effect. (There are around 50k particles in the simulation) I run yade in parallel with another application under MPI as follows : mpiexec -n 1 python yade : - n 12 app2 Is there a way I could get openmp working in such a situation? (I had disabled LINSOLV, LBMFLOW,BLAS during compilation as suggested in here [3]) [1] https://yade-dem.org/doc/user.html#importing-yade-in-other-python-applications [2]https://answers.launchpad.net/yade/+question/271679 [3]https://answers.launchpad.net/yade/+question/247865 Thanks, deepak -- You received this question notification because your team yade-users is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #647852]: Using openmp in a python script.
Question #647852 on Yade changed: https://answers.launchpad.net/yade/+question/647852 Status: Open => Answered Gary Pekmezi proposed the following answer: The first thing I notice, is that there is only 1 mpi process for python, so only 1 core. I don't believe there will be any speedup with only one core beyond maybe 2-3 simultaneous processes. One other thing to consider, is setting the environment variable OMP_NUM_THREADS before running the mpiexec command. -- You received this question notification because your team yade-users is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #647852]: Using openmp in a python script.
Question #647852 on Yade changed: https://answers.launchpad.net/yade/+question/647852 Chareyre proposed the following answer: Hello Deepak, did you try a parallel 'hello world' with mpi nesting openmp on the same system? Maybe even without python. I would make sure this one is as expected before diving into more complex situations. B Le 13 juil. 2017 5:18 PM, "Deepak" a écrit : > New question #647852 on Yade: > https://answers.launchpad.net/yade/+question/647852 > > Hello, > > I currently run Yade by importing it in a python script as explained in > [1]. > However I am unable to utilize more than 1 core(for yade) in my > simulations, I've tried the method described in [2], and I don't see any > difference and I've also tried setting 'ompThreads=(N)' in the engines and > I see no effect. (There are around 50k particles in the simulation) > > I run yade in parallel with another application under MPI as follows : > mpiexec -n 1 python yade : - n 12 app2 > > Is there a way I could get openmp working in such a situation? > > (I had disabled LINSOLV, LBMFLOW,BLAS during compilation as suggested in > here [3]) > > [1] https://yade-dem.org/doc/user.html#importing-yade-in-other- > python-applications > [2]https://answers.launchpad.net/yade/+question/271679 > [3]https://answers.launchpad.net/yade/+question/247865 > > Thanks, > deepak > > -- > You received this question notification because your team yade-users is > an answer contact for Yade. > > ___ > Mailing list: https://launchpad.net/~yade-users > Post to : yade-users@lists.launchpad.net > Unsubscribe : https://launchpad.net/~yade-users > More help : https://help.launchpad.net/ListHelp > > -- You received this question notification because your team yade-users is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #647852]: Using openmp in a python script.
Question #647852 on Yade changed: https://answers.launchpad.net/yade/+question/647852 Status: Answered => Open Deepak is still having a problem: Hello Bruno, So I did a few tests with YADE and python (like calling OpenMP codes inside python with MPI), and they work quite well. I noticed that the OpenMP performance goes low when one tries to get some data by looping through YADE objects (before Newton Integrator): for e.g: def myfunc(self): pvel= []; prot = [] for b in O.bodies: pvel.append(b.state.vel); prot.append(b.state.angVel) # another function (parallelized with openmp (tested independently with python and mpi)) fx = calcforce(pvel, prot) # now add forces to the bodies, this also reduces the performance.. for i, b in enumerate(O.bodies:): O.forces.addF(b.id,Vector3(f[i][0],f[i][1],f[i][2])) so I re-phrase my question: Is there a possible way on how I could loop through the objects and addforces every timestep without compromising on the performance? A very crude working example is here: https://pastebin.com/J5FsRWQe (taken from examples/concrete/triax-test, see that last few lines of adding forces) Hi Gary, each mpi process can spawn an openmp thread, so I am quite sure it's not about the number of processes set in mpi (I tested this too). Thanks, deepak -- You received this question notification because your team yade-users is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #647852]: Using openmp in a python script.
Question #647852 on Yade changed: https://answers.launchpad.net/yade/+question/647852 Status: Open => Answered Gary Pekmezi proposed the following answer: Deepak, I think you will need to load a multiprocessing library first, then you can loop through Yade objects using a multiprocessing pool [1] instead of the single-threaded sequential for loop you are currently using. For example, instead of: for b in O.bodies: pvel.append(b.state.vel); prot.append(b.state.angVel) you would do something like (please note it's just a sample, I haven't tested it): import multiprocessing as mp def appVel(i): pvel[i] = (O.bodies)[i].state.vel prot[i] = (O.bodies)[i].state.angVel if __name__=="__main__": numBods = len(O.bodies) pvel = mp.Array('d',range(numBods)) prot = mp.Array('d',range(numBods)) p=mp.Pool(numThreads) p.map(appVel,range(0,numBods)) [1] https://docs.python.org/2/library/multiprocessing.html -- You received this question notification because your team yade-users is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #647852]: Using openmp in a python script.
Question #647852 on Yade changed: https://answers.launchpad.net/yade/+question/647852 Bruno Chareyre proposed the following answer: >so I re-phrase my question: Is there a possible way on how I could loop through the objects and addforces every timestep without compromising on the performance? Does it mean that the question is unrelated to OpenMP, and that the real problem is the inefficiency of python loops (not a surprise)? A cure could be to implement the loops in c++, i.e. give a python array to a yade module and let it loop (in c++) through the data and set values. Bruno -- You received this question notification because your team yade-users is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp