petscvariables: hardwired build dir instead of install dir
you do a config/configure.py with a --prefix then the make then the make install and see what goes wrong? Barry with -- Lisandro Dalc?n --- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 -- Lisandro Dalc?n --- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 -- Lisandro Dalc?n --- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 -- next part -- A non-text attachment was scrubbed... Name: make_install.diff Type: text/x-patch Size: 1504 bytes Desc: not available URL: http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20080324/312cd99c/attachment.bin
petscvariables: hardwired build dir instead of install dir
If someone explains to me in words what that sed crap is doing, I will
rewrite it in python.
Thanks,
Matt
On Mon, Mar 24, 2008 at 10:57 AM, Lisandro Dalcin dalcinl at gmail.com wrote:
Barry, things are still broken. I think that at some point we have to
review the 'install:' target more carefully.
First, the 'sed' command i being called in a wrong way.
Second, ALL files under $PETSC_ARCH/conf are being copied, I'm not
sure if all them are needed (logs, RDict.db, configure.py, etc) when
you 'install' PETSc in a central, system location.
Finally, 'sed' trickery is done under all files under 'conf/' and that
can be dangerous.
I attach a hg diff fixing some of that issues. But I still believe
that this have to be more carefully reviewed. Unfortunately, I will
not be able to do that myself for, let say, three month from now.
By using the patch in the attached file, I was able to install PETSc
and next build petsc4py. However, this does not necesarily means that
things are all OK. I already do some trickery in the build system of
petsc4py in order to 'fix' the problems with the stuff in PETSc
makefiles since version 2.3.2 until latest petsc-dev.
Regards,
On 3/22/08, Barry Smith bsmith at mcs.anl.gov wrote:
Lisandro
The rule was a total mess; I think I have fixed it now, please try
again
and let me know how it goes.
Thanks for reporting the problem,
Barry
On Mar 22, 2008, at 1:18 PM, Lisandro Dalcin wrote:
OK, it seems my stderr were being sent I do not know were... I get the
following (BTW, I was not aware of the 'sed' trick, but I was not
seeing the error)
[root at trantor petsc-dev]# make install
Installing PETSc at /usr/local/petsc/dev/linux-gnu
sed: can't read s?${PETSC_DIR}?TMP_INSTALL_DIR?g: No such file or
directory
sed: can't read s?TMP_INSTALL_DIR?/usr/local/petsc/dev/linux-gnu?g: No
such file or directory
sed: can't read s?/${PETSC_ARCH}??g: No such file or directory
making shared libraries in /usr/local/petsc/dev/linux-gnu//lib
building libpetsc.so
building libpetscvec.so
building libpetscmat.so
building libpetscdm.so
building libpetscksp.so
building libpetscsnes.so
building libpetscts.so
building libpetsccontrib.so
sh/bash: PETSC_DIR=/usr/local/petsc/dev/linux-gnu; export PETSC_DIR
sh/bash: unset PETSC_ARCH
csh/tcsh: setenv PETSC_DIR /usr/local/petsc/dev/linux-gnu
csh/tcsh: unsetenv PETSC_ARCH
Then do make test to verify correct install
And then the 'petscvariables' still have the build dir in many places.
So the 'sed' command is not working for me. I do not know way, I've
never had the time to learn regexps :-(.
BTW, I'm on a Fedora 6 box.
On 3/22/08, Barry Smith bsmith at mcs.anl.gov wrote:
Ok, I think I understand your concern,
On Mar 22, 2008, at 9:00 AM, Lisandro Dalcin wrote:
On 3/22/08, Barry Smith bsmith at mcs.anl.gov wrote:
Do you mean that it lists, for example,
SUPERLU_DIST_LIB = -Wl,-rpath,/Users/bsmith/Src/petsc-dev/arch-icc-
superlu_dist/lib -L/Users/bsmith/Src/petsc-dev/arch-icc-
superlu_dist/
lib -lsuperlu_dist_2.2
instead of
SUPERLU_DIST_LIB = -Wl,-rpath,$PETSC_DIR/arch-icc-superlu_dist/
lib -L
$PETSC_DIR/arch-icc-superlu_dist/lib -lsuperlu_dist_2.2
exactly that, as an example, I've just built petsc-dev, passing as
prefix the following '--prefix=/usr/local/petsc/2.3.3/linux-
gnu' (yes,
I still want to have multiconfig installations of petsc in a central
location, so I add the PETSC_ARCH to the prefix)
But the 'petscvariables' file stills says, for example
ML_INCLUDE = -I. -I/usr/local/mpich2/1.0.6/include
-I/usr/local/mpich2/1.0.6/include
-I/repos/hg/petsc/petsc-dev/linux-gnu/include
This is because the place we want ML to install itself has be
passed in as the true path, it cannot be sent as a symbolic PETSC_DIR
since configure of the subpackage doesn't know PETSC_DIR
The last include dir is the directory were I've built PETSc. I would
love to see that include as -I${PETSC_DIR}/${PETSC_ARCH}/include.
And
now, if external packages got installed in $PETSC_DIR/$PETSC_ARCH,
we
perhaps could just put nothing, as that location is always taken
into
accout for PETSc itself.
The real problem: if I remove the build dir, the 'petscvariables'
point to locations that no longer exists, This is dangerous. I
already
had trouble in the past with the old 'bmake' based system. Is
there a
chance to 'fix' this, or I'm missing something?
make intall: see the rule in makefile is suppose to replace all
uses of the
petsc-dir
petscvariables: hardwired build dir instead of install dir
On 3/24/08, Matthew Knepley knepley at gmail.com wrote:
If someone explains to me in words what that sed crap is doing, I will
rewrite it in python.
It does not pay the effort i think. The 'sed' regexps were OK, what
was wrong was the command line arguments being passed to 'sed'; look
at the patch I've attached before, there is an empty string after the
'-i' flag, and then 'sed' thinks it is the search/replace command,
this empty str have to be removed. In short, the sed crap is actually
a typo.
However, Matt. I still think that the complete install process have to
be reviewed a bit. Just doing cp -fr somedir/* is not always the
best thing to do, specially as the code is changing and new stuff is
being added. And worse than that is to 'sed' in all files inside a
dir.
I'm pretty sure that the vast majority of PETSc users never 'install'
PETSc in system locations (perhaps I'm the only one). I'm completely
out of time, but at some point I'll review all this; further, I'll
develop RPM specs for building source RPM's for PETSc. When one does
this, all the mistakes in a software build system are easily
discovered.
Thanks,
Matt
On Mon, Mar 24, 2008 at 10:57 AM, Lisandro Dalcin dalcinl at gmail.com
wrote:
Barry, things are still broken. I think that at some point we have to
review the 'install:' target more carefully.
First, the 'sed' command i being called in a wrong way.
Second, ALL files under $PETSC_ARCH/conf are being copied, I'm not
sure if all them are needed (logs, RDict.db, configure.py, etc) when
you 'install' PETSc in a central, system location.
Finally, 'sed' trickery is done under all files under 'conf/' and that
can be dangerous.
I attach a hg diff fixing some of that issues. But I still believe
that this have to be more carefully reviewed. Unfortunately, I will
not be able to do that myself for, let say, three month from now.
By using the patch in the attached file, I was able to install PETSc
and next build petsc4py. However, this does not necesarily means that
things are all OK. I already do some trickery in the build system of
petsc4py in order to 'fix' the problems with the stuff in PETSc
makefiles since version 2.3.2 until latest petsc-dev.
Regards,
On 3/22/08, Barry Smith bsmith at mcs.anl.gov wrote:
Lisandro
The rule was a total mess; I think I have fixed it now, please try
again
and let me know how it goes.
Thanks for reporting the problem,
Barry
On Mar 22, 2008, at 1:18 PM, Lisandro Dalcin wrote:
OK, it seems my stderr were being sent I do not know were... I get
the
following (BTW, I was not aware of the 'sed' trick, but I was not
seeing the error)
[root at trantor petsc-dev]# make install
Installing PETSc at /usr/local/petsc/dev/linux-gnu
sed: can't read s?${PETSC_DIR}?TMP_INSTALL_DIR?g: No such file or
directory
sed: can't read s?TMP_INSTALL_DIR?/usr/local/petsc/dev/linux-gnu?g:
No
such file or directory
sed: can't read s?/${PETSC_ARCH}??g: No such file or directory
making shared libraries in /usr/local/petsc/dev/linux-gnu//lib
building libpetsc.so
building libpetscvec.so
building libpetscmat.so
building libpetscdm.so
building libpetscksp.so
building libpetscsnes.so
building libpetscts.so
building libpetsccontrib.so
sh/bash: PETSC_DIR=/usr/local/petsc/dev/linux-gnu; export PETSC_DIR
sh/bash: unset PETSC_ARCH
csh/tcsh: setenv PETSC_DIR /usr/local/petsc/dev/linux-gnu
csh/tcsh: unsetenv PETSC_ARCH
Then do make test to verify correct install
And then the 'petscvariables' still have the build dir in many
places.
So the 'sed' command is not working for me. I do not know way, I've
never had the time to learn regexps :-(.
BTW, I'm on a Fedora 6 box.
On 3/22/08, Barry Smith bsmith at mcs.anl.gov wrote:
Ok, I think I understand your concern,
On Mar 22, 2008, at 9:00 AM, Lisandro Dalcin wrote:
On 3/22/08, Barry Smith bsmith at mcs.anl.gov wrote:
Do you mean that it lists, for example,
SUPERLU_DIST_LIB =
-Wl,-rpath,/Users/bsmith/Src/petsc-dev/arch-icc-
superlu_dist/lib -L/Users/bsmith/Src/petsc-dev/arch-icc-
superlu_dist/
lib -lsuperlu_dist_2.2
instead of
SUPERLU_DIST_LIB = -Wl,-rpath,$PETSC_DIR/arch-icc-superlu_dist/
lib -L
$PETSC_DIR/arch-icc-superlu_dist/lib -lsuperlu_dist_2.2
exactly that, as an example, I've just built petsc-dev, passing as
prefix the following '--prefix=/usr/local/petsc/2.3.3/linux-
gnu' (yes,
I still want to have multiconfig installations of petsc in a
central
petscvariables: hardwired build dir instead of install dir
On Mar 24, 2008, at 10:57 AM, Lisandro Dalcin wrote:
Barry, things are still broken. I think that at some point we have to
review the 'install:' target more carefully.
First, the 'sed' command i being called in a wrong way.
This is not true; the sed is being called correctly. The problem
is that -i
is not a standard sed option and different systems gnu and freebsd treat
it differently. freebsd requires a space between the -i and the suffix;
gnu has no space; gnu also allows the use of -i to indicate no backup
while freebsd expects -i
Your patch works on POS gnu systems, but is broken on far superior
Apple MacOS X systems! :-)
Matt you need to add a config/configure.py test to detect the
type of sed -i it is.
Second, ALL files under $PETSC_ARCH/conf are being copied, I'm not
sure if all them are needed (logs, RDict.db, configure.py, etc) when
you 'install' PETSc in a central, system location.
I like copying everything because they tell exactly what build was
done
with what options. But if it is being dumped into a generic conf
directory
yes it will be confusing, but then the files rules, variables and base
are confusing because they are not marked as PETSc related
Finally, 'sed' trickery is done under all files under 'conf/' and that
can be dangerous.
Yes, if it is a generic conf directory it is kind of a problem.
Barry
I attach a hg diff fixing some of that issues. But I still believe
that this have to be more carefully reviewed. Unfortunately, I will
not be able to do that myself for, let say, three month from now.
By using the patch in the attached file, I was able to install PETSc
and next build petsc4py. However, this does not necesarily means that
things are all OK. I already do some trickery in the build system of
petsc4py in order to 'fix' the problems with the stuff in PETSc
makefiles since version 2.3.2 until latest petsc-dev.
Regards,
On 3/22/08, Barry Smith bsmith at mcs.anl.gov wrote:
Lisandro
The rule was a total mess; I think I have fixed it now, please try
again
and let me know how it goes.
Thanks for reporting the problem,
Barry
On Mar 22, 2008, at 1:18 PM, Lisandro Dalcin wrote:
OK, it seems my stderr were being sent I do not know were... I get
the
following (BTW, I was not aware of the 'sed' trick, but I was not
seeing the error)
[root at trantor petsc-dev]# make install
Installing PETSc at /usr/local/petsc/dev/linux-gnu
sed: can't read s?${PETSC_DIR}?TMP_INSTALL_DIR?g: No such file or
directory
sed: can't read s?TMP_INSTALL_DIR?/usr/local/petsc/dev/linux-gnu?
g: No
such file or directory
sed: can't read s?/${PETSC_ARCH}??g: No such file or directory
making shared libraries in /usr/local/petsc/dev/linux-gnu//lib
building libpetsc.so
building libpetscvec.so
building libpetscmat.so
building libpetscdm.so
building libpetscksp.so
building libpetscsnes.so
building libpetscts.so
building libpetsccontrib.so
sh/bash: PETSC_DIR=/usr/local/petsc/dev/linux-gnu; export PETSC_DIR
sh/bash: unset PETSC_ARCH
csh/tcsh: setenv PETSC_DIR /usr/local/petsc/dev/linux-gnu
csh/tcsh: unsetenv PETSC_ARCH
Then do make test to verify correct install
And then the 'petscvariables' still have the build dir in many
places.
So the 'sed' command is not working for me. I do not know way, I've
never had the time to learn regexps :-(.
BTW, I'm on a Fedora 6 box.
On 3/22/08, Barry Smith bsmith at mcs.anl.gov wrote:
Ok, I think I understand your concern,
On Mar 22, 2008, at 9:00 AM, Lisandro Dalcin wrote:
On 3/22/08, Barry Smith bsmith at mcs.anl.gov wrote:
Do you mean that it lists, for example,
SUPERLU_DIST_LIB = -Wl,-rpath,/Users/bsmith/Src/petsc-dev/arch-
icc-
superlu_dist/lib -L/Users/bsmith/Src/petsc-dev/arch-icc-
superlu_dist/
lib -lsuperlu_dist_2.2
instead of
SUPERLU_DIST_LIB = -Wl,-rpath,$PETSC_DIR/arch-icc-superlu_dist/
lib -L
$PETSC_DIR/arch-icc-superlu_dist/lib -lsuperlu_dist_2.2
exactly that, as an example, I've just built petsc-dev, passing as
prefix the following '--prefix=/usr/local/petsc/2.3.3/linux-
gnu' (yes,
I still want to have multiconfig installations of petsc in a
central
location, so I add the PETSC_ARCH to the prefix)
But the 'petscvariables' file stills says, for example
ML_INCLUDE = -I. -I/usr/local/mpich2/1.0.6/include
-I/usr/local/mpich2/1.0.6/include
-I/repos/hg/petsc/petsc-dev/linux-gnu/include
This is because the place we want ML to install itself has be
passed in as the true path, it cannot be sent as a symbolic
PETSC_DIR
since configure of the subpackage doesn't know PETSC_DIR
The last include dir is the directory were I've built PETSc. I
would
love to see that include as -I${PETSC_DIR}/${PETSC_ARCH}/include.
And
now, if external packages got installed in $PETSC_DIR/$PETSC_ARCH,
we
perhaps could just put nothing, as that location is always taken
into
accout for PETSc itself.
The real problem: if
petscvariables: hardwired build dir instead of install dir
On Mon, Mar 24, 2008 at 9:50 PM, Barry Smith bsmith at mcs.anl.gov wrote: On Mar 24, 2008, at 10:57 AM, Lisandro Dalcin wrote: Barry, things are still broken. I think that at some point we have to review the 'install:' target more carefully. First, the 'sed' command i being called in a wrong way. This is not true; the sed is being called correctly. The problem is that -i is not a standard sed option and different systems gnu and freebsd treat it differently. freebsd requires a space between the -i and the suffix; gnu has no space; gnu also allows the use of -i to indicate no backup while freebsd expects -i Your patch works on POS gnu systems, but is broken on far superior Apple MacOS X systems! :-) Matt you need to add a config/configure.py test to detect the type of sed -i it is. I totally disagree. We should ditch all this crap, and just write nice, PORTABLE Python code. I will do it. I just need someone to explain what this sed is doing. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
petscvariables: hardwired build dir instead of install dir
Matt, The sed is so trivial it is silly to even think about replacing it with python! I did not realize until after reading Lisandro's email that the sed -i option behaved differently on different systems. Barry On Mar 24, 2008, at 10:07 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 9:50 PM, Barry Smith bsmith at mcs.anl.gov wrote: On Mar 24, 2008, at 10:57 AM, Lisandro Dalcin wrote: Barry, things are still broken. I think that at some point we have to review the 'install:' target more carefully. First, the 'sed' command i being called in a wrong way. This is not true; the sed is being called correctly. The problem is that -i is not a standard sed option and different systems gnu and freebsd treat it differently. freebsd requires a space between the -i and the suffix; gnu has no space; gnu also allows the use of -i to indicate no backup while freebsd expects -i Your patch works on POS gnu systems, but is broken on far superior Apple MacOS X systems! :-) Matt you need to add a config/configure.py test to detect the type of sed -i it is. I totally disagree. We should ditch all this crap, and just write nice, PORTABLE Python code. I will do it. I just need someone to explain what this sed is doing. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
petscvariables: hardwired build dir instead of install dir
On Mon, Mar 24, 2008 at 10:14 PM, Barry Smith bsmith at mcs.anl.gov wrote: Matt, The sed is so trivial it is silly to even think about replacing it with python! I did not realize until after reading Lisandro's email What does that have to do with anything? If its so trivial, then it won't take any time at all. This is at least the third time I have had to fool with this sed stuff (I already reported that sed bug two months ago). I do not want to do it again. Is there any justification, except inertia, for keeping that in sed? Matt that the sed -i option behaved differently on different systems. Barry On Mar 24, 2008, at 10:07 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 9:50 PM, Barry Smith bsmith at mcs.anl.gov wrote: On Mar 24, 2008, at 10:57 AM, Lisandro Dalcin wrote: Barry, things are still broken. I think that at some point we have to review the 'install:' target more carefully. First, the 'sed' command i being called in a wrong way. This is not true; the sed is being called correctly. The problem is that -i is not a standard sed option and different systems gnu and freebsd treat it differently. freebsd requires a space between the -i and the suffix; gnu has no space; gnu also allows the use of -i to indicate no backup while freebsd expects -i Your patch works on POS gnu systems, but is broken on far superior Apple MacOS X systems! :-) Matt you need to add a config/configure.py test to detect the type of sed -i it is. I totally disagree. We should ditch all this crap, and just write nice, PORTABLE Python code. I will do it. I just need someone to explain what this sed is doing. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
petscvariables: hardwired build dir instead of install dir
On Mar 24, 2008, at 10:25 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 10:14 PM, Barry Smith bsmith at mcs.anl.gov wrote: Matt, The sed is so trivial it is silly to even think about replacing it with python! I did not realize until after reading Lisandro's email What does that have to do with anything? If its so trivial, then it won't take any time at all. This is at least the third time I have had to fool with this sed stuff (I already reported that sed bug two months ago). I do not want to do it again. Is there any justification, except inertia, for keeping that in sed? Not having hundreds of dinky little python scripts lying around that do the same thing as Unix utilities is a good reason. If you write the entire install: rule in python that would be great, then you could start on some of the other rules in conf/rules I am only objecting to replacing Unix one-liners with python one liners. Barry Matt that the sed -i option behaved differently on different systems. Barry On Mar 24, 2008, at 10:07 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 9:50 PM, Barry Smith bsmith at mcs.anl.gov wrote: On Mar 24, 2008, at 10:57 AM, Lisandro Dalcin wrote: Barry, things are still broken. I think that at some point we have to review the 'install:' target more carefully. First, the 'sed' command i being called in a wrong way. This is not true; the sed is being called correctly. The problem is that -i is not a standard sed option and different systems gnu and freebsd treat it differently. freebsd requires a space between the -i and the suffix; gnu has no space; gnu also allows the use of -i to indicate no backup while freebsd expects -i Your patch works on POS gnu systems, but is broken on far superior Apple MacOS X systems! :-) Matt you need to add a config/configure.py test to detect the type of sed -i it is. I totally disagree. We should ditch all this crap, and just write nice, PORTABLE Python code. I will do it. I just need someone to explain what this sed is doing. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
petscvariables: hardwired build dir instead of install dir
On Mon, Mar 24, 2008 at 10:38 PM, Barry Smith bsmith at mcs.anl.gov wrote: On Mar 24, 2008, at 10:25 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 10:14 PM, Barry Smith bsmith at mcs.anl.gov wrote: Matt, The sed is so trivial it is silly to even think about replacing it with python! I did not realize until after reading Lisandro's email What does that have to do with anything? If its so trivial, then it won't take any time at all. This is at least the third time I have had to fool with this sed stuff (I already reported that sed bug two months ago). I do not want to do it again. Is there any justification, except inertia, for keeping that in sed? Not having hundreds of dinky little python scripts lying around that do the same thing as Unix utilities is a good reason. If you write the entire install: rule in python that would be great, then you could start on some of the other rules in conf/rules I am only objecting to replacing Unix one-liners with python one liners. Fine, but you just asked for a lot more Python then 1 line to check which form of the -i flag is on the machine. Matt Barry Matt that the sed -i option behaved differently on different systems. Barry On Mar 24, 2008, at 10:07 PM, Matthew Knepley wrote: On Mon, Mar 24, 2008 at 9:50 PM, Barry Smith bsmith at mcs.anl.gov wrote: On Mar 24, 2008, at 10:57 AM, Lisandro Dalcin wrote: Barry, things are still broken. I think that at some point we have to review the 'install:' target more carefully. First, the 'sed' command i being called in a wrong way. This is not true; the sed is being called correctly. The problem is that -i is not a standard sed option and different systems gnu and freebsd treat it differently. freebsd requires a space between the -i and the suffix; gnu has no space; gnu also allows the use of -i to indicate no backup while freebsd expects -i Your patch works on POS gnu systems, but is broken on far superior Apple MacOS X systems! :-) Matt you need to add a config/configure.py test to detect the type of sed -i it is. I totally disagree. We should ditch all this crap, and just write nice, PORTABLE Python code. I will do it. I just need someone to explain what this sed is doing. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
