Dear Wolfgang, Thank you for the response.
> Paras: > I'm not sure any of us have experience with Amesos:SuperLU, so I'm not > sure > anyone will know right away what the problem may be. > > I was wondering if, while writing the wrappers and testing them out, someone managed to figure out the requisite combination of installation time options. > But here are a couple of questions: > * What happens if you run the program with just two MPI jobs on one > machine? > In that case, you can watch what the two programs are doing by having > 'top' > run in a separate window. > I ran a job with a direct solver from the beginning on one node with 72 processes (see job107396-flops graph) and it is evident that all except a few cores do a similar amount of flops. I guess some of the processes are meant to "coordinate the work" and not do much actual computation. > * How do you distribute the matrix and right hand side? Are they both > fully > distributed? > I use BlockSpasrseMatrix and BlockVector objects available through TrilinosWrappers. > * Is the solution you get correct? > I do not have an exact solution for comparison, but the simulation results, both visual as well as global quantities like force, energy etc., seem to indicate that the physics is captured correctly. Thus, I claim that the solution is correct. > * If the answer to the last question is yes, then either Amesos or SuperLU > is > apparently copying the data of the linear system from all other processes > to > just one process that then solves the linear system. It might be useful to > take a debugger, running with just two MPI processes, to step into the > Amesos > routines to see if you get to a place where that is happening, and then to > read the code in that place to see what flags need to be set to make sure > the > solution really does happen in a distributed way. > > One would probably need to debug the code while running on at least two nodes . I do not have much experience with debugging an MPI code. Will try to learn more about this. Best regards, Paras -- The deal.II project is located at http://www.dealii.org/ For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en --- You received this message because you are subscribed to the Google Groups "deal.II User Group" group. To unsubscribe from this group and stop receiving emails from it, send an email to dealii+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/CAEU6zmSe8LDUeRAa%3DrLuJkA_0oSeAbg5M1iZxisvmiEC%2BYgp1g%40mail.gmail.com.
