On Thu, Jan 24, 2013 at 12:51 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> > > Begin forwarded message: > > *From: *Philip Papadopoulos <philip.papadopoulos at gmail.com> > *Subject: **Re: [petsc-maint #149715] Making a PetSc Roll and Rocks Module > * > *Date: *January 24, 2013 9:48:36 AM CST > *To: *"Schneider, Barry I." <bschneid at nsf.gov> > *Cc: *Barry Smith <bsmith at mcs.anl.gov> > > Dear Barry^2, > > The major observation about the build process is that building the > software so that it can livesw- > in a standard OS package is more work than it should be. Here is the > standard "meme". > Suppose you have three directories: > sw-src > tmpoot/sw-install > sw-install > > sw-src: the source directory tree for building the sw package (eg > petsc-3.3-p5) > sw-install: the directory where one wants sw installed on the final > system. We use something > like > /opt/petsc/compiler/mpi/interconnect/petsc-version/petsc-arch > > the last directory is an artifact of the way many people build software > for packaging. Instead of > installing into sw-install directly, you install into a > tmproot/sw-install. Then you direct the package > manager to grab all files in tmproot/sw-install to put in the package. > The package itself will > strip off the tmproot leading directory. In other words the package labels > all files as sw-install/. > > So the build issue that I ran into is that the build directory and/or the > tmproot directory path becomes embedded into a large number of files > (include files, mod files, etc). To get around this, > I did a bind mount of (tmproot/sw-install --> /sw-install) and told petsc > to install into /sw-install. I consider that to be a "hack", but petsc > isn't the only software that I've run into that requires this > mounting bit of legerdemain. > > Many software packages will support a dest= or DESTDIR variable for "make > install" that supports the "installation" into a tmproot directory but > leaves no trace of tmproot in any of its configuration files. ( > http://git.rocksclusters.org/cgi-bin/gitweb.cgi?p=core/python/.git;a=blob;f=src/python-2/Makefile;h=ced6cc1c6eb6a4f70dd0f3e1b0ccd1ac1e40f989;hb=HEAD) > shows a Makefile for python that supports this. > > Okay, we should be able to do that anyway I guess. I will do it (probably with Satish helping me). > Ultimately we create packages of software to simplify a number of things. > Including update of installed software on many machines. > > The other "issue" is not really an issue, but a request. When petsc is > built it creates a petscvariables (or similarly named) file with lots of > environment variables. It would be terrific if it could also create an > environment modules files with this same information. Users often want > different versions/different configs of the same software and environment > modules is now standard in Redhat/CentOS distributions. > > We already create a PETSc.mk file for pkgconfig, so doing another one is not hard. Here is us doing that: https://bitbucket.org/petsc/petsc-dev/src/b94b5d6f0ee5d4de5a64fc5f0d1a4e01c743cf04/config/PETSc/Configure.py?at=default#cl-108 and the Dump() function sets up the information in the petscvariables file by calling addMakeMacro(). Care to try it yourself? I do not know the format for a modules file. We are always available to help. Thanks, Matt > Hope this helps. > > > On Thu, Jan 24, 2013 at 2:44 AM, Schneider, Barry I. <bschneid at > nsf.gov>wrote: > >> Barry, >> I am sending this to Phil Papadopoulos and he can make much more precise >> recommendations. Glad you have thick skins. Being at NSF requires the >> same physiology. >> >> -----Original Message----- >> From: Barry Smith [mailto:bsmith at mcs.anl.gov] >> Sent: Wednesday, January 23, 2013 10:19 PM >> To: Schneider, Barry I. >> Subject: Re: [petsc-maint #149715] Making a PetSc Roll and Rocks Module >> >> >> Barry, >> >> By far the most common email we get is regarding installation, so we >> are always trying to understand how to improve that process. If we can >> eliminate just 20 percent of installation problems that would save us a >> great deal of work, so yes, any critique/suggestions are greatly >> appreciated, and after all these years we have thick skins so can survive >> even the harshest suggestions. >> >> Barry >> >> On Jan 23, 2013, at 6:13 PM, "Schneider, Barry I." <bschneid at nsf.gov> >> wrote: >> >> > Barry, >> > First, thanks for getting back to me so quickly. I view this as a way >> to make things easier for everyone for a library that is really invaluable >> to many of us. Phil P. is a pro at making rolls and modules for the Rocks >> distribution of CentOS that is widely used in the NSF cyberinfrastructure >> and by others. I am both a program manager for the XSEDE project and a >> large user as well so I appreciate the ability to use the library in its >> most transparent fashion. After Phil did his thing we tested it and it >> worked perfectly on our cluster. Basically, you load the module and any >> associated modules such as the Intel compiler and appropriate MPI module >> and then you just use it. No screwing around with building and the rest of >> it. That's the good news. Of course, you need to make a roll for every >> specific combination of compiler and MPI but I believe what Phil has >> learned makes that pretty straightforward to do. While I cannot speak for >> Phil, I would expect that anything he learned would be available to you >> folks if that's what you wanted. The next step for us will be to >> incorporate what we did in the Rocks distribution which will eventually >> propagate to lots of users. Eventually, there will be an XSEDE >> distribution which will be used by an even larger number of sites. We are >> talking about many thousands of users. So, I guess what I am really asking >> is whether you are interested enough in what Phil learned to perhaps modify >> the current PetSc so that it is easier for users to install or make >> available the technology for folks to make their own rolls. Of course, you >> could decide that is not on your agenda and that's fine. But if you >> capitalize on our experience and your great software that would be >> wonderful and perhaps offer users alternatives to the current way of doing >> things that would make life easier. >> > >> > ************************************************** >> > Dr Barry I. Schneider >> > Program Director for Cyberinfrastructure Past Chair, APS Division of >> > Computational Physics Physics Division National Science Foundation >> > 4201 Wilson Blvd. >> > Arlington, VA 22230 >> > Phone:(703) 292-7383 >> > FAX:(703) 292-9078 >> > Cell:(703) 589-2528 >> > ************************************************** >> > >> > >> > -----Original Message----- >> > From: Barry Smith [mailto:bsmith at mcs.anl.gov] >> > Sent: Wednesday, January 23, 2013 5:56 PM >> > To: petsc-maint at mcs.anl.gov; Schneider, Barry I. >> > Subject: Re: [petsc-maint #149715] Making a PetSc Roll and Rocks >> > Module >> > >> > >> > Barry, >> > >> > Thanks for the feedback. We actually desire to support "system" >> installs just as easily as "home directory" installs, though our >> documentation emphasizes "home directory" installs since we feel that is >> most practical for more users. We'd be interested in more specifics on the >> difficulties and any suggestions there may be to make the process >> (especially for multiple compilers/mpis) easier. That is what could we have >> done differently to make it easier for you? >> > >> > Thanks >> > >> > Barry >> > >> > On Jan 23, 2013, at 1:59 PM, "Schneider, Barry I." <bschneid at nsf.gov> >> wrote: >> > >> >> I thought I might pass on the following to the folks maintaining >> PetSc. Recently we had the occasion to make a Rocks roll and module to be >> used on a local cluster here at NSF. Phil Papadopoulos the developer of >> Rocks at SDSC took the time to help us to do that not simply because we >> wanted it but also because he wanted to be able to know how to do it and >> distribute it to a much larger set users of NSF resources and also on out >> NSF supported platforms. Here are his comments. Perhaps if you feel there >> are things that could be made easier that would be great. It also could be >> useful to you directly. >> >> >> >> BTW, Petsc is kind of a bear to package -- they really expect you to >> build it in your home directory :-). >> >> I took the time to make the roll pretty flexible in terms of >> compiler/mpi/network support to build various versions. >> >> This was modeled after other Triton rolls so that it is consistent >> with other software -- it likely becomes part of the standard SDSC software >> stack, so this was a good thing to do all the way around. >> >> >> >> I'm about ready to add this to our code repository, it will show up on >> git.rocksclusters.org<http://git.rocksclusters.org> tomorrow morning. >> >> >> >> >> >> ************************************************** >> >> Dr Barry I. Schneider >> >> Program Director for Cyberinfrastructure Past Chair, APS Division of >> >> Computational Physics Physics Division National Science Foundation >> >> 4201 Wilson Blvd. >> >> Arlington, VA 22230 >> >> Phone:(703) 292-7383 >> >> FAX:(703) 292-9078 >> >> Cell:(703) 589-2528 >> >> ************************************************** >> >> >> >> >> > >> >> > > > -- > Philip Papadopoulos, PhD > University of California, San Diego > 858-822-3628 (Ofc) > 619-331-2990 (Fax) > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130124/bb333c52/attachment-0001.html>