petscvariables: hardwired build dir instead of install dir
On Mon, Mar 24, 2008 at 10:53 PM, Barry Smith bsmith at mcs.anl.gov wrote: On Mar 24, 2008, at 10:42 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 10:38 PM, Barry Smith bsmith at mcs.anl.gov wrote: On Mar 24, 2008, at 10:25 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 10:14 PM, Barry Smith bsmith at mcs.anl.gov wrote: Matt, The sed is so trivial it is silly to even think about replacing it with python! I did not realize until after reading Lisandro's email What does that have to do with anything? If its so trivial, then it won't take any time at all. This is at least the third time I have had to fool with this sed stuff (I already reported that sed bug two months ago). I do not want to do it again. Is there any justification, except inertia, for keeping that in sed? Not having hundreds of dinky little python scripts lying around that do the same thing as Unix utilities is a good reason. If you write the entire install: rule in python that would be great, then you could start on some of the other rules in conf/rules I am only objecting to replacing Unix one-liners with python one liners. Fine, but you just asked for a lot more Python then 1 line to check which form of the -i flag is on the machine. Yes, but that is the autoconf model! You wanted the autoconf model, not me! You first tried it with autoconf, not possible, so you wrote a better than autoconf in python. I never asked anyone to make a conf system, I was perfectly happy requiring people to write the correct flags directly into petscconf :-). What bothers me about one-line python scripts to fix Unix weirdness is now you have two models (some things you fix by running config/configure.py and figuring things out and some things you fix by replacing Unix with python). One thing you should know about me is I HATE HATE HATE HATE using two models for doing something at the same time (this is why I hate Mathematica and PERL, they each support about 20 different programming models that can be used together). So now we have reached the root if the issue; in my mind if you introduce a new model approach to solving some problem you toss the old, you don't use them both together! I am all for consistency. However I have a different interpretation of what constitutes the model. I thought the model was 1) Test a set of tools, e.g. compilers, make, ... 2) Customize the use of each based upon our tests 3) Use the set of tools required for each build task I think I am proposing a decision for 3), namely replacing one tool with another. We already do this, for instance when we link with a linker or a compiler. I think we should eliminate all uses of sed with python. Notice that this is not wasted work, as it will survive when eventually everything is replaced by Python. Matt Barry Matt Barry Matt that the sed -i option behaved differently on different systems. Barry On Mar 24, 2008, at 10:07 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 9:50 PM, Barry Smith bsmith at mcs.anl.gov wrote: On Mar 24, 2008, at 10:57 AM, Lisandro Dalcin wrote: Barry, things are still broken. I think that at some point we have to review the 'install:' target more carefully. First, the 'sed' command i being called in a wrong way. This is not true; the sed is being called correctly. The problem is that -i is not a standard sed option and different systems gnu and freebsd treat it differently. freebsd requires a space between the -i and the suffix; gnu has no space; gnu also allows the use of -i to indicate no backup while freebsd expects -i Your patch works on POS gnu systems, but is broken on far superior Apple MacOS X systems! :-) Matt you need to add a config/configure.py test to detect the type of sed -i it is. I totally disagree. We should ditch all this crap, and just write nice, PORTABLE Python code. I will do it. I just need someone to explain what this sed is doing. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting
petscvariables: hardwired build dir instead of install dir
On Mar 24, 2008, at 11:26 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 10:53 PM, Barry Smith bsmith at mcs.anl.gov wrote: On Mar 24, 2008, at 10:42 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 10:38 PM, Barry Smith bsmith at mcs.anl.gov wrote: On Mar 24, 2008, at 10:25 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 10:14 PM, Barry Smith bsmith at mcs.anl.gov wrote: Matt, The sed is so trivial it is silly to even think about replacing it with python! I did not realize until after reading Lisandro's email What does that have to do with anything? If its so trivial, then it won't take any time at all. This is at least the third time I have had to fool with this sed stuff (I already reported that sed bug two months ago). I do not want to do it again. Is there any justification, except inertia, for keeping that in sed? Not having hundreds of dinky little python scripts lying around that do the same thing as Unix utilities is a good reason. If you write the entire install: rule in python that would be great, then you could start on some of the other rules in conf/rules I am only objecting to replacing Unix one-liners with python one liners. Fine, but you just asked for a lot more Python then 1 line to check which form of the -i flag is on the machine. Yes, but that is the autoconf model! You wanted the autoconf model, not me! You first tried it with autoconf, not possible, so you wrote a better than autoconf in python. I never asked anyone to make a conf system, I was perfectly happy requiring people to write the correct flags directly into petscconf :-). What bothers me about one-line python scripts to fix Unix weirdness is now you have two models (some things you fix by running config/configure.py and figuring things out and some things you fix by replacing Unix with python). One thing you should know about me is I HATE HATE HATE HATE using two models for doing something at the same time (this is why I hate Mathematica and PERL, they each support about 20 different programming models that can be used together). So now we have reached the root if the issue; in my mind if you introduce a new model approach to solving some problem you toss the old, you don't use them both together! I am all for consistency. However I have a different interpretation of what constitutes the model. I thought the model was 1) Test a set of tools, e.g. compilers, make, ... 2) Customize the use of each based upon our tests 3) Use the set of tools required for each build task I think I am proposing a decision for 3), namely replacing one tool with another. We already do this, for instance when we link with a linker or a compiler. I think we should eliminate all uses of sed with python. I think that is silly; sed is a perfectly fine tool, like all unix utilities it just has some whacked behavior, that in this case can be easily fixed by having a SED_INPLACE macro. Why is SED so special to be replaced by python, why not cp or all the other things that we have config macros for? Notice that this is not wasted work, as it will survive when eventually everything is replaced by Python. It is because when the install: rule is replaced by python it will not be a line by line replacement so your python sed script will not be needed (it would just make the python script more cumbersome.) I think a make rule by rule change to python is a far better way to go then replacing individual lines in the rules with python scripts and then later trying to merge those individual line python scripts together. Barry Matt Barry Matt Barry Matt that the sed -i option behaved differently on different systems. Barry On Mar 24, 2008, at 10:07 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 9:50 PM, Barry Smith bsmith at mcs.anl.gov wrote: On Mar 24, 2008, at 10:57 AM, Lisandro Dalcin wrote: Barry, things are still broken. I think that at some point we have to review the 'install:' target more carefully. First, the 'sed' command i being called in a wrong way. This is not true; the sed is being called correctly. The problem is that -i is not a standard sed option and different systems gnu and freebsd treat it differently. freebsd requires a space between the -i and the suffix; gnu has no space; gnu also allows the use of -i to indicate no backup while freebsd expects -i Your patch works on POS gnu systems, but is broken on far superior Apple MacOS X systems! :-) Matt you need to add a config/configure.py test to detect the type of sed -i it is. I totally disagree. We should ditch all this crap, and just write nice, PORTABLE Python code. I will do it. I just need someone to explain what this sed is doing. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any
ML with OpenMPI
On Sat 2008-03-22 10:19, Lisandro Dalcin wrote:
Give a try. When using MPICH2, PETSc just passes
--with-mpi=PATH_TO_MPI and ML get it right. Perhaps ML have some
trouble with OpenMPI, I've never tried. If you built OpenMPI yourself
and with shared libs, do not forget to define LD_LIBRARY_PATH to point
to the dir with the OpenMPI libs. If not, some configure test of ML
could fail, and then MPI is assumed to be absent.
It turns out I was chasing this in entirely the wrong direction. ML was
configured just fine and was correctly using MPI, but we had not defined
HAVE_CONFIG_H so ml_common.h was not setting all the variables that depend on
ml_config.h. In particular, ml_config.h sets HAVE_MPI correctly, but the
following is in ml_config.h:
#ifdef HAVE_CONFIG_H
...
#ifdef HAVE_MPI
#ifndef ML_MPI
#define ML_MPI
#endif
#endif
...
#endif /*ifdef HAVE_CONFIG_H*/
Indeed, adding -DHAVE_CONFIG_H to CFLAGS in src/ksp/pc/impls/ml/makefile fixes
the problem (and the manual include of ml_config.h in ml.c becomes unnecessary).
That is, the patch below makes everything work correctly.
Jed
diff -r 2ae11e456aa7 src/ksp/pc/impls/ml/makefile
--- a/src/ksp/pc/impls/ml/makefile Fri Mar 21 17:33:24 2008 -0500
+++ b/src/ksp/pc/impls/ml/makefile Tue Mar 25 08:35:12 2008 +0100
@@ -5,7 +5,7 @@
ALL: lib
-CFLAGS = ${ML_INCLUDE}
+CFLAGS = ${ML_INCLUDE} -DHAVE_CONFIG_H
FFLAGS =
SOURCEC = ml.c
SOURCEF =
diff -r 2ae11e456aa7 src/ksp/pc/impls/ml/ml.c
--- a/src/ksp/pc/impls/ml/ml.c Fri Mar 21 17:33:24 2008 -0500
+++ b/src/ksp/pc/impls/ml/ml.c Tue Mar 25 08:35:12 2008 +0100
@@ -10,7 +10,6 @@
#include math.h
EXTERN_C_BEGIN
-#include ml_config.h
#include ml_include.h
EXTERN_C_END
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL:
http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20080325/525e21f4/attachment.pgp
ML with OpenMPI
However, I still do not understand why I never had this problem. Jed,
you built ML yourself, or were you letting PETSc to automatically
download and build it? Or perhaps I did not noticed the problem
because of MPICH2?
On 3/25/08, Barry Smith bsmith at mcs.anl.gov wrote:
I have pushed this fix to petsc-dev
Thank you for figuring this out,
Barry
On Mar 25, 2008, at 2:38 AM, Jed Brown wrote:
On Sat 2008-03-22 10:19, Lisandro Dalcin wrote:
Give a try. When using MPICH2, PETSc just passes
--with-mpi=PATH_TO_MPI and ML get it right. Perhaps ML have some
trouble with OpenMPI, I've never tried. If you built OpenMPI yourself
and with shared libs, do not forget to define LD_LIBRARY_PATH to
point
to the dir with the OpenMPI libs. If not, some configure test of ML
could fail, and then MPI is assumed to be absent.
It turns out I was chasing this in entirely the wrong direction. ML
was
configured just fine and was correctly using MPI, but we had not
defined
HAVE_CONFIG_H so ml_common.h was not setting all the variables that
depend on
ml_config.h. In particular, ml_config.h sets HAVE_MPI correctly,
but the
following is in ml_config.h:
#ifdef HAVE_CONFIG_H
...
#ifdef HAVE_MPI
#ifndef ML_MPI
#define ML_MPI
#endif
#endif
...
#endif /*ifdef HAVE_CONFIG_H*/
Indeed, adding -DHAVE_CONFIG_H to CFLAGS in src/ksp/pc/impls/ml/
makefile fixes
the problem (and the manual include of ml_config.h in ml.c becomes
unnecessary).
That is, the patch below makes everything work correctly.
Jed
diff -r 2ae11e456aa7 src/ksp/pc/impls/ml/makefile
--- a/src/ksp/pc/impls/ml/makefile Fri Mar 21 17:33:24 2008 -0500
+++ b/src/ksp/pc/impls/ml/makefile Tue Mar 25 08:35:12 2008 +0100
@@ -5,7 +5,7 @@
ALL: lib
-CFLAGS = ${ML_INCLUDE}
+CFLAGS = ${ML_INCLUDE} -DHAVE_CONFIG_H
FFLAGS =
SOURCEC = ml.c
SOURCEF =
diff -r 2ae11e456aa7 src/ksp/pc/impls/ml/ml.c
--- a/src/ksp/pc/impls/ml/ml.c Fri Mar 21 17:33:24 2008 -0500
+++ b/src/ksp/pc/impls/ml/ml.c Tue Mar 25 08:35:12 2008 +0100
@@ -10,7 +10,6 @@
#include math.h
EXTERN_C_BEGIN
-#include ml_config.h
#include ml_include.h
EXTERN_C_END
--
Lisandro Dalc?n
---
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
ML with OpenMPI
Well, then that would mean that I was using ML through PETSc in PARALLEL runs with no MPI support !!! Do you believe that scenario is possible? Looking at ML configure script and generated makefiles, in them there is a line saying DEFS = -DHAVE_CONFIG_H Do you have that line? Next, this $(DEFS) is included in compiler command definition. Additionally, I did $ nm -C libml.a | grep MPI and undefined references to the MPI functions appered as expected. Sorry about my insinstence, but I believe we need to figure out what's exactly going on. On 3/25/08, Jed Brown jed at 59a2.org wrote: I let PETSc build it for me. I think you did not notice the problem because MPICH2 defines MPI_Comm to be an int which happens to be the same as used by ML in their dummy MPI so there is no type mismatch. From the contents of ml_common.h, it looks like you would still run into trouble if you were using optional features of ML. The reason I like OpenMPI is exactly this stronger type checking and that it seems to crash sooner when I have a bug. Jed -- Lisandro Dalc?n --- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594
ML with OpenMPI
On Tue 2008-03-25 12:11, Lisandro Dalcin wrote: Well, then that would mean that I was using ML through PETSc in PARALLEL runs with no MPI support !!! Do you believe that scenario is possible? No, ML was built correctly. The build output has -DHAVE_CONFIG_H on every build line. What *is* happening is that the headers included by ksp/pc/impls/ml/ml.c were essentially for a non-MPI build because HAVE_CONFIG_H was not defined. That is, including ml_config.h defines the autoconf'd macros (like HAVE_MPI) and ml_common.h uses them to set ML-local macros (like ML_MPI) *only* if HAVE_CONFIG_H is defined. So when we include ml_include.h without defining HAVE_CONFIG_H, we see the interface for a default (non-MPI) build. This interface is (apparently) the same as an MPI build with MPICH2, but not with OpenMPI. Since the library was built with MPI, there was no dangerous type casting, and you were using it with MPI, there was no problem. When using OpenMPI, it sees a conflict between the ML's dummy MPI interface and OpenMPI's because ML and MPICH2 use MPI_Comm = int while OpenMPI uses an opaque pointer value. Looking at ML configure script and generated makefiles, in them there is a line saying DEFS = -DHAVE_CONFIG_H Do you have that line? Next, this $(DEFS) is included in compiler command definition. Additionally, I did $ nm -C libml.a | grep MPI and undefined references to the MPI functions appered as expected. Sorry about my insinstence, but I believe we need to figure out what's exactly going on. No problem. I agree it is important. Does my explanation above make sense to you? Jed -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20080325/6e972319/attachment.pgp
ML with OpenMPI
OK. Now all is clear to me. Sorry about my confusion. So I have to conclude that ML machinery for including headers is a bit broken, I think. Many thanks for your explanation. On 3/25/08, Jed Brown jed at 59a2.org wrote: No, ML was built correctly. The build output has -DHAVE_CONFIG_H on every build line. What *is* happening is that the headers included by ksp/pc/impls/ml/ml.c were essentially for a non-MPI build because HAVE_CONFIG_H was not defined. That is, including ml_config.h defines the autoconf'd macros (like HAVE_MPI) and ml_common.h uses them to set ML-local macros (like ML_MPI) *only* if HAVE_CONFIG_H is defined. So when we include ml_include.h without defining HAVE_CONFIG_H, we see the interface for a default (non-MPI) build. This interface is (apparently) the same as an MPI build with MPICH2, but not with OpenMPI. Since the library was built with MPI, there was no dangerous type casting, and you were using it with MPI, there was no problem. When using OpenMPI, it sees a conflict between the ML's dummy MPI interface and OpenMPI's because ML and MPICH2 use MPI_Comm = int while OpenMPI uses an opaque pointer value. Looking at ML configure script and generated makefiles, in them there is a line saying DEFS = -DHAVE_CONFIG_H Do you have that line? Next, this $(DEFS) is included in compiler command definition. Additionally, I did $ nm -C libml.a | grep MPI and undefined references to the MPI functions appered as expected. Sorry about my insinstence, but I believe we need to figure out what's exactly going on. No problem. I agree it is important. Does my explanation above make sense to you? Jed -- Lisandro Dalc?n --- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594
ML with OpenMPI
I think our usage of ML is broken. We use #include ml_config.h in ml.c. However looks like ml requires HAVE_CONFIG_H defined for other things aswell. So the correct fix is to change the above line to: #define HAVE_CONFIG_H Satish On Tue, 25 Mar 2008, Lisandro Dalcin wrote: OK. Now all is clear to me. Sorry about my confusion. So I have to conclude that ML machinery for including headers is a bit broken, I think. Many thanks for your explanation. On 3/25/08, Jed Brown jed at 59a2.org wrote: No, ML was built correctly. The build output has -DHAVE_CONFIG_H on every build line. What *is* happening is that the headers included by ksp/pc/impls/ml/ml.c were essentially for a non-MPI build because HAVE_CONFIG_H was not defined. That is, including ml_config.h defines the autoconf'd macros (like HAVE_MPI) and ml_common.h uses them to set ML-local macros (like ML_MPI) *only* if HAVE_CONFIG_H is defined. So when we include ml_include.h without defining HAVE_CONFIG_H, we see the interface for a default (non-MPI) build. This interface is (apparently) the same as an MPI build with MPICH2, but not with OpenMPI. Since the library was built with MPI, there was no dangerous type casting, and you were using it with MPI, there was no problem. When using OpenMPI, it sees a conflict between the ML's dummy MPI interface and OpenMPI's because ML and MPICH2 use MPI_Comm = int while OpenMPI uses an opaque pointer value. Looking at ML configure script and generated makefiles, in them there is a line saying DEFS = -DHAVE_CONFIG_H Do you have that line? Next, this $(DEFS) is included in compiler command definition. Additionally, I did $ nm -C libml.a | grep MPI and undefined references to the MPI functions appered as expected. Sorry about my insinstence, but I believe we need to figure out what's exactly going on. No problem. I agree it is important. Does my explanation above make sense to you? Jed
