Bug#573187: transition: mpi-defaults
On Wed, Mar 14, 2012 at 21:34:45 +0100, Julien Cristau wrote: On Fri, Mar 2, 2012 at 00:35:12 +0100, Julien Cristau wrote: Reverse-dependencies of lam left in testing: - apbs; binNMUs scheduled mips failed, the rest are ok. Filed a bug for the mips FTBFS (#663894) Maintainer says it works on another buildd, given back for now. - cctools; fixed in unstable, but FTBFS (#661658) Still unfixed. Blocked by python2.7 FTBFS. - clustalw-mpi; needs a bug filed Filed, and closed. But will need builds on a bunch of archs. Still unfixed. - gerris; FTBFS (#652258) Still unfixed. - ghemical; binNMUs scheduled Failed (because mpqc was still built against lam, I think). Should be given back I guess. Given back now. Still failing with lam references somehow. - hdf5; needs #631019 sorted so libhdf5-lam-1.8.4 can be removed Still true. - libgpiv; FTBFS (#657199) Fixed. - mpqc; FTBFS on mips, given back for now new version FTBFS on armel, #666110. - mrbayes; unbuildable on kfreebsd (#661659, #661663) mrbayes will be handled by removing it on kbsd. - netpipe; #571449 Still unfixed. - rmpi; binNMUs scheduled FTBFS, needs a bug filed. That's #663896. - tessa; fixed in sid Needs a fixed pytables before it can migrate. Still an issue, pytables is blocked by a numexpr bug AIUI. [...] I'd appreciate some help from the mpi maintainers to make progress here. Ping? Cheers, Julien signature.asc Description: Digital signature
Bug#573187: transition: mpi-defaults
On Fri, Mar 2, 2012 at 00:35:12 +0100, Julien Cristau wrote: Reverse-dependencies of lam left in testing: - apbs; binNMUs scheduled mips failed, the rest are ok. Filed a bug for the mips FTBFS (#663894) - cctools; fixed in unstable, but FTBFS (#661658) Still unfixed. - clustalw-mpi; needs a bug filed Filed, and closed. But will need builds on a bunch of archs. Still unfixed. - fftw; binNMUs scheduled All built, looks like mips/mipsel had expired keys so will need a reupload once that's sorted. Fixed. - gerris; FTBFS (#652258) Still unfixed. - ghemical; binNMUs scheduled Failed (because mpqc was still built against lam, I think). Should be given back I guess. Given back now. - hdf5; needs #631019 sorted so libhdf5-lam-1.8.4 can be removed Still true. - hpcc; binNMUs scheduled Will need a reupload on mips once keys are sorted. Fixed. - libghemical; binNMUs scheduled Failed because of #661674 I think. Given back. - libgpiv; FTBFS (#657199) - mpqc; FTBFS on mips, given back for now - mrbayes; unbuildable on kfreebsd (#661659, #661663) All three still unfixed AFAICT. - mumps; in the middle of a SONAME bump (#650642, #659898, #651452, something else with getdp, petsc/slepc's own bumps This one's now gone. - netpipe; #571449 Still unfixed. - parmetis; binNMUs scheduled - regina-normal; binNMUs scheduled Fixed. - rmpi; binNMUs scheduled FTBFS, needs a bug filed. Filed . - starpu; binNMUs scheduled Looks ok modulo buildd key issues. Fixed. - tessa; fixed in sid Needs a fixed pytables before it can migrate. Still an issue, pytables is blocked by a numexpr bug AIUI. - xmpi; #571448, should apparently be removed I'll remove it from testing now. I'd appreciate some help from the mpi maintainers to make progress here. Cheers, Julien signature.asc Description: Digital signature
Bug#573187: transition: mpi-defaults
On Wed, Feb 29, 2012 at 01:18:06 +0100, Julien Cristau wrote: The new mpi-defaults was uploaded to sid in December, here's the current state. Reverse-dependencies of lam left in testing: - apbs; binNMUs scheduled mips failed, the rest are ok. - cctools; fixed in unstable, but FTBFS (#661658) - clustalw-mpi; needs a bug filed Filed, and closed. But will need builds on a bunch of archs. - fftw; binNMUs scheduled All built, looks like mips/mipsel had expired keys so will need a reupload once that's sorted. - gerris; FTBFS (#652258) - ghemical; binNMUs scheduled Failed (because mpqc was still built against lam, I think). Should be given back I guess. - hdf5; needs #631019 sorted so libhdf5-lam-1.8.4 can be removed - hpcc; binNMUs scheduled Will need a reupload on mips once keys are sorted. - libghemical; binNMUs scheduled Failed because of #661674 I think. - libgpiv; FTBFS (#657199) - mpqc; FTBFS on mips, given back for now - mrbayes; unbuildable on kfreebsd (#661659, #661663) - mumps; in the middle of a SONAME bump (#650642, #659898, #651452, something else with getdp, petsc/slepc's own bumps - netpipe; #571449 - parmetis; binNMUs scheduled - regina-normal; binNMUs scheduled Looks ok so far. - rmpi; binNMUs scheduled FTBFS, needs a bug filed. - starpu; binNMUs scheduled Looks ok modulo buildd key issues. - tessa; fixed in sid Needs a fixed pytables before it can migrate. - xmpi; #571448, should apparently be removed Reverse-dependencies of mpich left in testing: - meep; #661661 - netpipe; see above - xmds; fixed in sid xmds now fixed in wheezy as well. Cheers, signature.asc Description: Digital signature
Bug#573187: transition: mpi-defaults
On Wed, Jun 2, 2010 at 19:06:06 +0100, Adam D. Barratt wrote: Hi, On Tue, 2010-03-09 at 17:52 +0100, Manuel Prinz wrote: The current plan is to EOL MPICH and LAM by removing all reverse dependencies on those packages. One big step in this direction would be via mpi-defaults. The current version depends on LAM/MPI for arches where Open MPI is not supported (yet). A package is prepared that will depend on MPICH2 on these arches. After uploading that, binNMUs are needed for the depending packages. How is this transition progressing? I note that there are still 10 open bugs listed at [1]. [1] http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=old-mpi-eol;users=debian-scie...@lists.debian.org The new mpi-defaults was uploaded to sid in December, here's the current state. Reverse-dependencies of lam left in testing: - apbs; binNMUs scheduled - cctools; fixed in unstable, but FTBFS (#661658) - clustalw-mpi; needs a bug filed - fftw; binNMUs scheduled - gerris; FTBFS (#652258) - ghemical; binNMUs scheduled - hdf5; needs #631019 sorted so libhdf5-lam-1.8.4 can be removed - hpcc; binNMUs scheduled - libghemical; binNMUs scheduled - libgpiv; FTBFS (#657199) - mpqc; FTBFS on mips, given back for now - mrbayes; unbuildable on kfreebsd (#661659, #661663) - mumps; in the middle of a SONAME bump (#650642, #659898, #651452, something else with getdp, petsc/slepc's own bumps - netpipe; #571449 - parmetis; binNMUs scheduled - regina-normal; binNMUs scheduled - rmpi; binNMUs scheduled - starpu; binNMUs scheduled - tessa; fixed in sid - xmpi; #571448, should apparently be removed Reverse-dependencies of mpich left in testing: - meep; #661661 - netpipe; see above - xmds; fixed in sid Cheers, Julien signature.asc Description: Digital signature
Bug#573187: transition: mpi-defaults
Hi, On Tue, 2010-03-09 at 17:52 +0100, Manuel Prinz wrote: The current plan is to EOL MPICH and LAM by removing all reverse dependencies on those packages. One big step in this direction would be via mpi-defaults. The current version depends on LAM/MPI for arches where Open MPI is not supported (yet). A package is prepared that will depend on MPICH2 on these arches. After uploading that, binNMUs are needed for the depending packages. How is this transition progressing? I note that there are still 10 open bugs listed at [1]. [1] http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=old-mpi-eol;users=debian-scie...@lists.debian.org Regards, Adam -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
On 08/04/10 at 18:22 -0500, Pavan Balaji wrote: Manuel Prinz wrote: The problem is that a lot of build systems of MPI-using software want the headers in $prefix/include and the libs in $prefix/lib, and on can often only use $prefix to point to the location of MPI-relevant files. The Debian package of MPICH2 uses $prefix/lib and $prefix/include/mpich2 which a lot of build systems do not like or handle correctly. This is why the openmpi package uses --prefix=/usr/lib/openmpi to build and symlinks the libraries to /usr/lib. This way, everything works as expected. Not pretty, but effective. The questions is whether we apply that change to the mpich2 package, as it was for all MPI packages in Debian so far, or fix every broken build system out there. [0] If we choose to do the first approach, what exactly needs to change in the mpich2 source? It looks like this is more of a packaging issue for Debian? Yes, sorry. Mpich2 provides enough configure options for us to put the files where we want in any case. But since there's only a handful of packages failing to build because of that, I still prefer to fix it in the build system of those packages. -- | Lucas Nussbaum | lu...@lucas-nussbaum.net http://www.lucas-nussbaum.net/ | | jabber: lu...@nussbaum.fr GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Am Donnerstag, den 08.04.2010, 18:22 -0500 schrieb Pavan Balaji: If we choose to do the first approach, what exactly needs to change in the mpich2 source? It looks like this is more of a packaging issue for Debian? As Lucas already said, we're already happy with what MPICH2 provides. But thanks for your offer! We were having a discussion about a common ABI in the context of MPI-3. But that working group fizzled out. I'll check on what exactly happened with it in the next meeting (in May). Even if the MPI standard doesn't specify ABI compatibility, I'll talk to the Open MPI folks to see if MPICH2 and Open MPI do whatever the ABI working group was proposing outside of the standard, which might be sufficient for you guys. That would be very cool indeed! I had some chat with Jeff Squyres (Open MPI upstream) about that a while ago. Maybe you could put your heads together on this. Having it in the standard would of course be super cool, as it would allow to use other MPI implementations as well without much hassle.[1] Lucas, for this transition I will to fix the build systems, or at least work around them. But we should get in touch after the release about some MPI mini-policy.[2] I hope to have all patches working by the end of the weekend, hopefully earlier. Best regards, Manuel [1] Not that I'm for using proprietary stuff but I know some our users do care for that. And so we should have that in mind. Getting to the point where BLAS/LAPACK is (well, almost) would save us some serious headaches. ;) [2] There are a few more problems we do not have a solution for yet, but getting rid of LAM and MPICH is one of the major blockers, IMHO. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Manuel Prinz wrote: Am Donnerstag, den 08.04.2010, 18:22 -0500 schrieb Pavan Balaji: We were having a discussion about a common ABI in the context of MPI-3. But that working group fizzled out. I'll check on what exactly happened with it in the next meeting (in May). Even if the MPI standard doesn't specify ABI compatibility, I'll talk to the Open MPI folks to see if MPICH2 and Open MPI do whatever the ABI working group was proposing outside of the standard, which might be sufficient for you guys. That would be very cool indeed! I had some chat with Jeff Squyres (Open MPI upstream) about that a while ago. Maybe you could put your heads together on this. Having it in the standard would of course be super cool, as it would allow to use other MPI implementations as well without much hassle.[1] I'll talk to Jeff. This had come up in another meeting last November and Jeff and I talked about doing this. But we never followed up on it; the problem is that this will require quite a few changes in one or both MPI implementations. More importantly, both MPICH2 and Open MPI have derivative implementations (Intel MPI, IBM MPI, Cray MPI, Microsoft MPI, MVAPICH2, etc., for MPICH2, and Sun MPI (renamed to Oracle MPI) for Open MPI) that we would need to coordinate with. This can potentially be a big change for them. Also, there is some concern amongst MPI developers that we might need to break ABI compatibility once in a while. For example, we recently had to increase the value of MPI_MAX_ERROR_STRING, which breaks ABI compatibility, but was required. So, we need some sort of escape hatch to get around such issues. I'll try to see what can be done, but no promises :-). -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Manuel Prinz wrote: The problem is that a lot of build systems of MPI-using software want the headers in $prefix/include and the libs in $prefix/lib, and on can often only use $prefix to point to the location of MPI-relevant files. The Debian package of MPICH2 uses $prefix/lib and $prefix/include/mpich2 which a lot of build systems do not like or handle correctly. This is why the openmpi package uses --prefix=/usr/lib/openmpi to build and symlinks the libraries to /usr/lib. This way, everything works as expected. Not pretty, but effective. The questions is whether we apply that change to the mpich2 package, as it was for all MPI packages in Debian so far, or fix every broken build system out there. [0] If we choose to do the first approach, what exactly needs to change in the mpich2 source? It looks like this is more of a packaging issue for Debian? Still waiting for the day when MPI standardizes on ABI, not API… But that may be wishful thinking. We were having a discussion about a common ABI in the context of MPI-3. But that working group fizzled out. I'll check on what exactly happened with it in the next meeting (in May). Even if the MPI standard doesn't specify ABI compatibility, I'll talk to the Open MPI folks to see if MPICH2 and Open MPI do whatever the ABI working group was proposing outside of the standard, which might be sufficient for you guys. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
On 06/04/10 at 21:01 -0500, Pavan Balaji wrote: Sorry, never mind my previous email. Lucas Nussbaum wrote: We would like to switch to using mpich2 on the arches where openmpi is not supported, but some packages fail to build from source when using mpich2. It seems that for some of them, it is caused by build systems expecting to find mpi.h in /usr/lib/mpich2/include/ (as it is the case for other MPI implementations), while it is shipped in /usr/include/mpich2/. By default, mpi.h should installed in $prefix/include, which seems to be what you need (prefix = /usr/lib/mpich2). So, unless you give an --includedir option to configure, this should work the way you expect it to. Maybe I'm not understanding the problem correctly here? We are currently using prefix=/usr and includedir=/usr/include/mpich2, so libs get installed directly in /usr/lib, and header files are in /usr/include/mpich2. We could use includedir=/usr/lib/mpich2/include, but I'm still not sure that this is the correct way to solve that problem, instead of fixing the build systems to be more tolerant about the location of MPI header files. -- | Lucas Nussbaum | lu...@lucas-nussbaum.net http://www.lucas-nussbaum.net/ | | jabber: lu...@nussbaum.fr GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Lucas Nussbaum wrote: On 06/04/10 at 21:01 -0500, Pavan Balaji wrote: By default, mpi.h should installed in $prefix/include, which seems to be what you need (prefix = /usr/lib/mpich2). So, unless you give an --includedir option to configure, this should work the way you expect it to. Maybe I'm not understanding the problem correctly here? We are currently using prefix=/usr and includedir=/usr/include/mpich2, so libs get installed directly in /usr/lib, and header files are in /usr/include/mpich2. We could use includedir=/usr/lib/mpich2/include, but I'm still not sure that this is the correct way to solve that problem, instead of fixing the build systems to be more tolerant about the location of MPI header files. Ok, this is the part I didn't understand fully. Is MPICH2's build failing when includedir is specified as a different location? (I just tried that on my machine and it seems to work correctly for me.) Or did you mean some other package depending on MPICH2 is failing? -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Am Mittwoch, den 07.04.2010, 13:12 -0500 schrieb Pavan Balaji:
Ok, this is the part I didn't understand fully. Is MPICH2's build
failing when includedir is specified as a different location? (I just
tried that on my machine and it seems to work correctly for me.) Or did
you mean some other package depending on MPICH2 is failing?
Packages depending on MPICH2 are failing to build. This is not an issue
with MPICH2, but thanks for taking the time to look into it!
The problem is that a lot of build systems of MPI-using software want
the headers in $prefix/include and the libs in $prefix/lib, and on can
often only use $prefix to point to the location of MPI-relevant files.
The Debian package of MPICH2 uses $prefix/lib and $prefix/include/mpich2
which a lot of build systems do not like or handle correctly. This is
why the openmpi package uses --prefix=/usr/lib/openmpi to build and
symlinks the libraries to /usr/lib. This way, everything works as
expected. Not pretty, but effective. The questions is whether we apply
that change to the mpich2 package, as it was for all MPI packages in
Debian so far, or fix every broken build system out there. [0]
Still waiting for the day when MPI standardizes on ABI, not API… But
that may be wishful thinking.
Best regards,
Manuel
[0] Preferably, we would fix the build system. As far as Open MPI is
concerned, the /usr/lib/openmpi/{lib,include} hierarchy was requested by
a user since he had the very same problem.
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Am Montag, den 05.04.2010, 17:22 +0200 schrieb Marc 'HE' Brockschmidt:
So, did you spend this time on doing fixes already? :-)
Yes, I indeed did. Not as much as I'd liked too, but nevertheless.
It would be great to see at least a few patches for the breaking
packages before the new defaults package is uploaded.
I'll just repeat the list of failing packages here and comment on them
group-wise.
Patch available:
* rmpi: patch attached
* blacs-mpi: patch attached
* mumps: Can be rebuild against fixed blacs-mpi
CMake:
* igstk
* kwwidgets
CMake does some magic here which I do not understand yet. These seem to
fail because they look for a library openmpi provides but mpich2
doesn't. (Well, at least not under that name.) We might fix that via a
symlink in mpich2, or I've to dive deeper into CMake.
Wrong location of header file (mpi.h) at configure:
* apbs
* petsc: almost done, minor issue remaining
There may be more packages in this group, the already patched ones are
also of this type. Most upstream build systems seem to expect
a /foo/{include,lib} hierarchy, and you can only pass the location
of /foo to the build system. Some are flexible enough to work around
that (see patches) but I do not see how to do that for apbs easily.
(It's possible for petsc but my patch has some other issue I have to
track down.) This could easily be fixed it mpich2 would provide this
hierarchy, as we already discussed. It would make the patches much
simpler, some packages of these 11 wouldn't even need patching then. I
think we should reconsider the option of doing the change in mpich2.
(Note: MPICH and LAM did provide this hierarchy, too.)
Remaining packages:
* life: Might be fixed with petsc, not sure yet.
* gdcm: Not looked at yet
* pgapack: Not looked at yet
* scalapack: Seems to be of type wrong location, may need no fix
This is the status as of now. I'm working on the remaining patches, but
the proposed change to mpich2 would save us some time here, as it avoids
patching about half of the packages at all. Please let me know what you
think about that.
Best regards,
Manuel
diff -pruN blacs-mpi-1.1.orig//Bmake.inc blacs-mpi-1.1//Bmake.inc
--- blacs-mpi-1.1.orig//Bmake.inc 2010-04-06 16:23:22.0 +
+++ blacs-mpi-1.1//Bmake.inc 2010-04-06 16:22:02.0 +
@@ -47,6 +47,14 @@
# -
# Name and location of the MPI library.
# -
+ifeq ($(MPI),mpich2)
+# for compilation with mpich:
+ MPIdev = ch_p4
+ MPIplat = LINUX
+ MPILIBdir = /usr/lib
+ MPIINCdir = /usr/include/mpi
+ MPILIB = $(MPILIBdir)/libmpich.so $(MPILIBdir)/libmpich.a
+endif
ifeq ($(MPI),mpich)
# for compilation with mpich:
MPIdir = /usr/lib/mpich
diff -pruN blacs-mpi-1.1.orig//debian/rules blacs-mpi-1.1//debian/rules
--- blacs-mpi-1.1.orig//debian/rules 2010-04-06 16:23:22.0 +
+++ blacs-mpi-1.1//debian/rules 2010-04-06 16:22:02.0 +
@@ -12,9 +12,9 @@ build: build-$(BLACS_MPI)
build-openmpi: build-stamp-openmpi
-build-lam: build-stamp-lam
+#build-lam: build-stamp-lam
-build-mpich: build-stamp-mpich
+build-mpich2: build-stamp-mpich2
patch-stamp:
patch -p1 debian/blacs-openmpi.patch
@@ -100,23 +100,23 @@ build-stamp-lam: patch-stamp
touch build-stamp-lam
-build-stamp-mpich: patch-stamp
+build-stamp-mpich2: patch-stamp
dh_testdir
[ -d TESTING/EXE ] || mkdir TESTING/EXE
# next is a clean
BASEDIR=$(topdir) make cleanall
cd TESTING BASEDIR=$(topdir) make clean
# build the static libraries
- BASEDIR=$(topdir) MPI=mpich make mpi
+ BASEDIR=$(topdir) MPI=mpich2 make mpi
# the testing binaries
cd TESTING BASEDIR=$(topdir) \
BTLIBS='$$(BLACSFINIT) $$(BLACSLIB) $$(BLACSFINIT) $$(MPILIB)' \
- BUILD=static MPI=mpich make
+ BUILD=static MPI=mpich2 make
# next is a clean
BASEDIR=$(topdir) make cleanall
cd TESTING BASEDIR=$(topdir) make clean
# build the shared libraries
- BASEDIR=$(topdir) FPIC=-fPIC MPI=mpich make mpi
+ BASEDIR=$(topdir) FPIC=-fPIC MPI=mpich2 make mpi
mkdir -p tmp
set -e ;\
for i in blacs blacsF77init blacsCinit ; do \
@@ -127,19 +127,19 @@ build-stamp-mpich: patch-stamp
do mv $$j tmp/$$(echo $$j | sed 's,C$$,o,g') ;\
done;\
cd .. ;\
- gcc -shared -Wl,-soname=lib$$i-mpich.so.1 -o lib$$i-mpich.so.1.1 \
+ gcc -shared -Wl,-soname=lib$$i-mpich2.so.1 -o lib$$i-mpich2.so.1.1 \
$$(find tmp -name *.o);\
- ln -fs lib$$i-mpich.so.1.1 lib$$i-mpich.so.1 ;\
- ln -fs lib$$i-mpich.so.1 lib$$i-mpich.so ;\
+ ln -fs lib$$i-mpich2.so.1.1 lib$$i-mpich2.so.1 ;\
+ ln -fs lib$$i-mpich2.so.1 lib$$i-mpich2.so ;\
rm -f tmp/tmp/* ; rmdir tmp/tmp ; rm tmp/* ;\
done
rmdir tmp
# the testing binaries
cd TESTING BASEDIR=$(topdir) \
- BTLIBS='-L.. -lblacsF77init-mpich -lblacs-mpich -lblacsF77init-mpich $$(MPILIB)' \
- BUILD=shared MPI=mpich make
+ BTLIBS='-L.. -lblacsF77init-mpich2 -lblacs-mpich2
Bug#573187: transition: mpi-defaults
Hi Pavan,
Could you take a look at http://bugs.debian.org/573187 and at the
message below?
We would like to switch to using mpich2 on the arches where openmpi is
not supported, but some packages fail to build from source when using
mpich2. It seems that for some of them, it is caused by build systems
expecting to find mpi.h in /usr/lib/mpich2/include/ (as it is the case
for other MPI implementations), while it is shipped in
/usr/include/mpich2/.
This seems like something that would be better fixed upstream, even if I
agree that the file location in openmpi is a bit strange. What do you
think?
- Lucas
On 06/04/10 at 19:41 +0200, Manuel Prinz wrote:
Am Montag, den 05.04.2010, 17:22 +0200 schrieb Marc 'HE' Brockschmidt:
So, did you spend this time on doing fixes already? :-)
Yes, I indeed did. Not as much as I'd liked too, but nevertheless.
It would be great to see at least a few patches for the breaking
packages before the new defaults package is uploaded.
I'll just repeat the list of failing packages here and comment on them
group-wise.
Patch available:
* rmpi: patch attached
* blacs-mpi: patch attached
* mumps: Can be rebuild against fixed blacs-mpi
CMake:
* igstk
* kwwidgets
CMake does some magic here which I do not understand yet. These seem to
fail because they look for a library openmpi provides but mpich2
doesn't. (Well, at least not under that name.) We might fix that via a
symlink in mpich2, or I've to dive deeper into CMake.
Wrong location of header file (mpi.h) at configure:
* apbs
* petsc: almost done, minor issue remaining
There may be more packages in this group, the already patched ones are
also of this type. Most upstream build systems seem to expect
a /foo/{include,lib} hierarchy, and you can only pass the location
of /foo to the build system. Some are flexible enough to work around
that (see patches) but I do not see how to do that for apbs easily.
(It's possible for petsc but my patch has some other issue I have to
track down.) This could easily be fixed it mpich2 would provide this
hierarchy, as we already discussed. It would make the patches much
simpler, some packages of these 11 wouldn't even need patching then. I
think we should reconsider the option of doing the change in mpich2.
(Note: MPICH and LAM did provide this hierarchy, too.)
Remaining packages:
* life: Might be fixed with petsc, not sure yet.
* gdcm: Not looked at yet
* pgapack: Not looked at yet
* scalapack: Seems to be of type wrong location, may need no fix
This is the status as of now. I'm working on the remaining patches, but
the proposed change to mpich2 would save us some time here, as it avoids
patching about half of the packages at all. Please let me know what you
think about that.
Best regards,
Manuel
--
| Lucas Nussbaum
| lu...@lucas-nussbaum.net http://www.lucas-nussbaum.net/ |
| jabber: lu...@nussbaum.fr GPG: 1024D/023B3F4F |
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Hi Lucas,
I'm a little lost trying to find the platform where the build is
failing. Can you point me to the exact output?
Thanks,
-- Pavan
Lucas Nussbaum wrote:
Hi Pavan,
Could you take a look at http://bugs.debian.org/573187 and at the
message below?
We would like to switch to using mpich2 on the arches where openmpi is
not supported, but some packages fail to build from source when using
mpich2. It seems that for some of them, it is caused by build systems
expecting to find mpi.h in /usr/lib/mpich2/include/ (as it is the case
for other MPI implementations), while it is shipped in
/usr/include/mpich2/.
This seems like something that would be better fixed upstream, even if I
agree that the file location in openmpi is a bit strange. What do you
think?
- Lucas
On 06/04/10 at 19:41 +0200, Manuel Prinz wrote:
Am Montag, den 05.04.2010, 17:22 +0200 schrieb Marc 'HE' Brockschmidt:
So, did you spend this time on doing fixes already? :-)
Yes, I indeed did. Not as much as I'd liked too, but nevertheless.
It would be great to see at least a few patches for the breaking
packages before the new defaults package is uploaded.
I'll just repeat the list of failing packages here and comment on them
group-wise.
Patch available:
* rmpi: patch attached
* blacs-mpi: patch attached
* mumps: Can be rebuild against fixed blacs-mpi
CMake:
* igstk
* kwwidgets
CMake does some magic here which I do not understand yet. These seem to
fail because they look for a library openmpi provides but mpich2
doesn't. (Well, at least not under that name.) We might fix that via a
symlink in mpich2, or I've to dive deeper into CMake.
Wrong location of header file (mpi.h) at configure:
* apbs
* petsc: almost done, minor issue remaining
There may be more packages in this group, the already patched ones are
also of this type. Most upstream build systems seem to expect
a /foo/{include,lib} hierarchy, and you can only pass the location
of /foo to the build system. Some are flexible enough to work around
that (see patches) but I do not see how to do that for apbs easily.
(It's possible for petsc but my patch has some other issue I have to
track down.) This could easily be fixed it mpich2 would provide this
hierarchy, as we already discussed. It would make the patches much
simpler, some packages of these 11 wouldn't even need patching then. I
think we should reconsider the option of doing the change in mpich2.
(Note: MPICH and LAM did provide this hierarchy, too.)
Remaining packages:
* life: Might be fixed with petsc, not sure yet.
* gdcm: Not looked at yet
* pgapack: Not looked at yet
* scalapack: Seems to be of type wrong location, may need no fix
This is the status as of now. I'm working on the remaining patches, but
the proposed change to mpich2 would save us some time here, as it avoids
patching about half of the packages at all. Please let me know what you
think about that.
Best regards,
Manuel
--
Pavan Balaji
http://www.mcs.anl.gov/~balaji
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Sorry, never mind my previous email. Lucas Nussbaum wrote: We would like to switch to using mpich2 on the arches where openmpi is not supported, but some packages fail to build from source when using mpich2. It seems that for some of them, it is caused by build systems expecting to find mpi.h in /usr/lib/mpich2/include/ (as it is the case for other MPI implementations), while it is shipped in /usr/include/mpich2/. By default, mpi.h should installed in $prefix/include, which seems to be what you need (prefix = /usr/lib/mpich2). So, unless you give an --includedir option to configure, this should work the way you expect it to. Maybe I'm not understanding the problem correctly here? -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Manuel Prinz man...@debian.org writes: Am Dienstag, den 23.03.2010, 19:30 +0100 schrieb Lucas Nussbaum: Fixing that my providing symlinks might cause subtle failures later, when the package is actually used and thinks that the libs are in some place while they aren't. I do not see why it should. It's only relevant a compile time. The real libs are in /usr/lib, where the linker will find them later, and rpath is not involved. (The openmpi package has used that setup since forever.) Anyway, I'm OK with patching the rules file as well. Time spent on arguing can be better spent elsewhere. ;) So, did you spend this time on doing fixes already? :-) It would be great to see at least a few patches for the breaking packages before the new defaults package is uploaded. Marc -- BOFH #226: A star wars satellite accidently blew up the WAN. pgpuovvT4b7p4.pgp Description: PGP signature
Bug#573187: transition: mpi-defaults
Am Samstag, den 20.03.2010, 14:01 +0100 schrieb Marc 'HE' Brockschmidt:
Manuel Prinz man...@debian.org writes:
This would enable to fix these in a timely fashion. I could start
doing that tomorrow night. How does that sound to you?
Sounds good. Sorry for my delayed answer :-/
No problem, same here!
I do not have everything sorted out yet but the issues seem to arise
from broken assumptions on where MPI stuff may reside. This is trivial
to fix in some cases and harder in others.
As an example, apbs expects {include,lib} in the same directory, which
is not the case for mpich2 currently. Lucas, do you think we should
fix this by providing this layout in the MPI -dev packages
under /usr/lib/{openmpi,mpich2}/? It seems that apbs is not the only
package assuming this but I did not generate numbers on that. It might
be worth to have some consistency here nevertheless, IMHO.
As for user-tagging, we can use the old-mpi-eol usertag.
Sounds good, though I don't know for which user - feel free to use
debian-rele...@lists.debian.org.
We have used debian-scie...@l.d.o, as mentioned in an earlier mail. The
BTS URL is [1].
Best regards,
Manuel
[1]
http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=old-mpi-eol;users=debian-scie...@lists.debian.org
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
On 23/03/10 at 19:11 +0100, Manuel Prinz wrote:
Am Samstag, den 20.03.2010, 14:01 +0100 schrieb Marc 'HE' Brockschmidt:
Manuel Prinz man...@debian.org writes:
This would enable to fix these in a timely fashion. I could start
doing that tomorrow night. How does that sound to you?
Sounds good. Sorry for my delayed answer :-/
No problem, same here!
I do not have everything sorted out yet but the issues seem to arise
from broken assumptions on where MPI stuff may reside. This is trivial
to fix in some cases and harder in others.
As an example, apbs expects {include,lib} in the same directory, which
is not the case for mpich2 currently. Lucas, do you think we should
fix this by providing this layout in the MPI -dev packages
under /usr/lib/{openmpi,mpich2}/? It seems that apbs is not the only
package assuming this but I did not generate numbers on that. It might
be worth to have some consistency here nevertheless, IMHO.
Given the low number of packages that FTBFS with mpich2 as mpi-default,
I would say that this could be fixed in the failing packages' build
system, no?
Fixing that my providing symlinks might cause subtle failures later,
when the package is actually used and thinks that the libs are in some
place while they aren't.
--
| Lucas Nussbaum
| lu...@lucas-nussbaum.net http://www.lucas-nussbaum.net/ |
| jabber: lu...@nussbaum.fr GPG: 1024D/023B3F4F |
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Am Dienstag, den 23.03.2010, 19:30 +0100 schrieb Lucas Nussbaum: Given the low number of packages that FTBFS with mpich2 as mpi-default, I would say that this could be fixed in the failing packages' build system, no? Sure. It might need some severe patching, though. Fixing that my providing symlinks might cause subtle failures later, when the package is actually used and thinks that the libs are in some place while they aren't. I do not see why it should. It's only relevant a compile time. The real libs are in /usr/lib, where the linker will find them later, and rpath is not involved. (The openmpi package has used that setup since forever.) Anyway, I'm OK with patching the rules file as well. Time spent on arguing can be better spent elsewhere. ;) Best regards, Manuel -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Manuel Prinz man...@debian.org writes: Am Mittwoch, den 17.03.2010, 13:24 +0100 schrieb Lucas Nussbaum: On 17/03/10 at 09:49 +0100, Marc 'HE' Brockschmidt wrote: I would propose that all (or at least) most of these bugs should be fixed before we actually do the switch in unstable. Manuel, does that seem OK to you? Erm. I'm very reluctant to filing bugs that can't easily be reproduced in unstable. At this point, you need a custom mpi-defaults package to reproduce them. Also, it is possible that some of them will not be fixable before the new mpi-defaults is in unstable (= you can't upload a fixed package that works if the new mpi-defaults isn't in unstable). I'm with Lucas here. If we file the bugs before the upload, we should be very explicit about the fact that it does not concern unstable yet. As a compromise, we could prepare patches, upload mpi-defaults and file the bugs with the respective patches. (Or most of them, did not check how complex the changes are.) That would be great. The point is that transitions suck most once they are half-started and preparing as much as possible helps to keep the sucking phase short. This would enable to fix these in a timely fashion. I could start doing that tomorrow night. How does that sound to you? Sounds good. Sorry for my delayed answer :-/ As for user-tagging, we can use the old-mpi-eol usertag. Sounds good, though I don't know for which user - feel free to use debian-rele...@lists.debian.org. Marc -- BOFH #293: You must've hit the wrong anykey. pgp7jP6VfHWNx.pgp Description: PGP signature
Bug#573187: transition: mpi-defaults
Hi, As asked during yesterday's release team meeting, I did a partial archive rebuild on amd64 with a modified mpi-defaults pointing to mpich2 instead of openmpi. The goal is to detect issues that will show up when the slower arches are switched to mpich2. (I know the plan is not to switch amd64 to mpich2) After searching for /mpi/ in build-depends and depends, I rebuilt the following 118 packages: advi apbs apron ara arpack avant-window-navigator blacs-mpi blitz++ boost-defaults boost1.40 boost1.42 bytecode camlp5 ccsm coccinelle code-saturne compiz-fusion-plugins-extra compiz-fusion-plugins-main compiz-fusion-plugins-unsupported compiz compizconfig-backend-gconf compizconfig-backend-kconfig compizconfig-python coq deal.ii dolfin ecs elmerfem eog epix exempi fftw fpc frama-c gdcm gearhead2 gearhead geneweb gerris gpiv gpivtools grace gromacs gst-plugins-bad0.10 haxe hdf5 hedgewars hpcc hypre icecc igstk illuminator imapcopy intercal ironpython ksplice kwwidgets laby lazarus libcompizconfig libfvm libgpiv liblicense libmesh libtool life log4cxx lwt m-tx meep-mpich meep-openmpi meep minc mpb mpi-defaults mpich2 mpich mpqc mseide-msegui mtasc mumble mumps music mypaint nautilus netpipe nordugrid-arc-nox octave3.0 octave3.2 openmpi paraview petsc pgapack piespy pmk protobuf-c python-scientific qemu-kvm qemu regina-normal rmpi root-system scala scalapack scotch slepc spooles ssreflect stlport5.2 tellico tracker tree-puzzle trilinos tuxguitar vtk why xmds xmpi The following 11 packages failed to build: apbs looks for mpi.h in the wrong place blacs-mpi missing target 'build-mpich2' in debian/rules gdcm ? igstk No rule to make target `/usr/lib/libmpi_cxx.so' (?) kwwidgets No rule to make target `/usr/lib/libmpi_cxx.so' life matrixpetsc.hpp:49: error: using typ edef-name 'MPI_Win' after 'struct' mumps ld: cannot find -lblacs-mpich2 petsc Nonexistent directory: /usr/lib/mpich2 for key with-mpi-dir (?) pgapack cd: 4: can't cd to lib/linux/ rmpi configure: error: Cannot find mpi.h header file scalapack missing target 'build-mpich2' in debian/rules So, it looks like it is mostly build system bugs, and it should not be too hard to fix. All the logs are available in http://people.debian.org/~lucas/logs/2010/03/16.mpi/ -- | Lucas Nussbaum | lu...@lucas-nussbaum.net http://www.lucas-nussbaum.net/ | | jabber: lu...@nussbaum.fr GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Lucas Nussbaum lu...@lucas-nussbaum.net writes: [test builds for mpi-default switch to mpich2] The following 11 packages failed to build: apbs looks for mpi.h in the wrong place blacs-mpi missing target 'build-mpich2' in debian/rules gdcm ? igstk No rule to make target `/usr/lib/libmpi_cxx.so' (?) kwwidgets No rule to make target `/usr/lib/libmpi_cxx.so' life matrixpetsc.hpp:49: error: using typ edef-name 'MPI_Win' after 'struct' mumps ld: cannot find -lblacs-mpich2 petsc Nonexistent directory: /usr/lib/mpich2 for key with-mpi-dir (?) pgapack cd: 4: can't cd to lib/linux/ rmpi configure: error: Cannot find mpi.h header file scalapack missing target 'build-mpich2' in debian/rules Could bugs about these issues be filed (with a usertag, so we can track them)? So, it looks like it is mostly build system bugs, and it should not be too hard to fix. Good to know, thanks for your work. I would propose that all (or at least) most of these bugs should be fixed before we actually do the switch in unstable. Manuel, does that seem OK to you? Marc -- Fachbegriffe der Informatik - Einfach erklärt 163: SMD Schwer Montierbare Dinger (Holger Köpke) pgpjJasdSI7LT.pgp Description: PGP signature
Bug#573187: transition: mpi-defaults
On 17/03/10 at 09:49 +0100, Marc 'HE' Brockschmidt wrote: Lucas Nussbaum lu...@lucas-nussbaum.net writes: [test builds for mpi-default switch to mpich2] The following 11 packages failed to build: apbs looks for mpi.h in the wrong place blacs-mpi missing target 'build-mpich2' in debian/rules gdcm ? igstk No rule to make target `/usr/lib/libmpi_cxx.so' (?) kwwidgets No rule to make target `/usr/lib/libmpi_cxx.so' life matrixpetsc.hpp:49: error: using typ edef-name 'MPI_Win' after 'struct' mumps ld: cannot find -lblacs-mpich2 petsc Nonexistent directory: /usr/lib/mpich2 for key with-mpi-dir (?) pgapack cd: 4: can't cd to lib/linux/ rmpi configure: error: Cannot find mpi.h header file scalapack missing target 'build-mpich2' in debian/rules Could bugs about these issues be filed (with a usertag, so we can track them)? So, it looks like it is mostly build system bugs, and it should not be too hard to fix. Good to know, thanks for your work. I would propose that all (or at least) most of these bugs should be fixed before we actually do the switch in unstable. Manuel, does that seem OK to you? Erm. I'm very reluctant to filing bugs that can't easily be reproduced in unstable. At this point, you need a custom mpi-defaults package to reproduce them. Also, it is possible that some of them will not be fixable before the new mpi-defaults is in unstable (= you can't upload a fixed package that works if the new mpi-defaults isn't in unstable). -- | Lucas Nussbaum | lu...@lucas-nussbaum.net http://www.lucas-nussbaum.net/ | | jabber: lu...@nussbaum.fr GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Am Mittwoch, den 17.03.2010, 13:24 +0100 schrieb Lucas Nussbaum: On 17/03/10 at 09:49 +0100, Marc 'HE' Brockschmidt wrote: Lucas Nussbaum lu...@lucas-nussbaum.net writes: So, it looks like it is mostly build system bugs, and it should not be too hard to fix. Good to know, thanks for your work. Thanks from me as well! I would propose that all (or at least) most of these bugs should be fixed before we actually do the switch in unstable. Manuel, does that seem OK to you? Erm. I'm very reluctant to filing bugs that can't easily be reproduced in unstable. At this point, you need a custom mpi-defaults package to reproduce them. Also, it is possible that some of them will not be fixable before the new mpi-defaults is in unstable (= you can't upload a fixed package that works if the new mpi-defaults isn't in unstable). I'm with Lucas here. If we file the bugs before the upload, we should be very explicit about the fact that it does not concern unstable yet. As a compromise, we could prepare patches, upload mpi-defaults and file the bugs with the respective patches. (Or most of them, did not check how complex the changes are.) This would enable to fix these in a timely fashion. I could start doing that tomorrow night. How does that sound to you? As for user-tagging, we can use the old-mpi-eol usertag. Best regards, Manuel -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#573187: transition: mpi-defaults
Package: release.debian.org Severity: normal User: release.debian@packages.debian.org Usertags: transition Dear release team, as requested by Marc Brockschmidt in 87ocj81kt4@solon.marcbrockschmidt.de, I file this bug for the mpi-defaults transition. As discussed on the Debian Science list, we'd like to get rid of two old MPI implementations, MPICH (mpich) and LAM/MPI (lam4). Both are not maintained upstream anymore and have successors already in the archive, MPICH2 (mpich2) and Open MPI (openmpi), respectively. The current plan is to EOL MPICH and LAM by removing all reverse dependencies on those packages. One big step in this direction would be via mpi-defaults. The current version depends on LAM/MPI for arches where Open MPI is not supported (yet). A package is prepared that will depend on MPICH2 on these arches. After uploading that, binNMUs are needed for the depending packages. This transition will interfere with the hdf5 transition, as hdf5 does build MPI-enabled versions. Bugs have been filed for the remaining packages that do not use mpi-defaults but depend on an MPI implementation directly. The can be tracked via [1]. They need to be fixed by sourceful uploads, I only mention them for reference. If you have any questions, feel free to ask on the Debian Science mailing list. And sorry for bringing it up so late in the release cycle! Best regards, Manuel [1] http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=old-mpi-eol;users=debian-scie...@lists.debian.org -- System Information: Debian Release: squeeze/sid APT prefers testing APT policy: (500, 'testing'), (200, 'unstable') Architecture: amd64 (x86_64) Kernel: Linux 2.6.32-trunk-amd64 (SMP w/2 CPU cores) Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
