Begin forwarded message:
> From: Philip Papadopoulos <philip.papadopoulos at gmail.com> > Subject: Re: [petsc-maint #149715] Making a PetSc Roll and Rocks Module > Date: January 24, 2013 9:48:36 AM CST > To: "Schneider, Barry I." <bschneid at nsf.gov> > Cc: Barry Smith <bsmith at mcs.anl.gov> > > Dear Barry^2, > > The major observation about the build process is that building the software > so that it can livesw- > in a standard OS package is more work than it should be. Here is the standard > "meme". > Suppose you have three directories: > sw-src > tmpoot/sw-install > sw-install > > sw-src: the source directory tree for building the sw package (eg > petsc-3.3-p5) > sw-install: the directory where one wants sw installed on the final system. > We use something > like > /opt/petsc/compiler/mpi/interconnect/petsc-version/petsc-arch > > the last directory is an artifact of the way many people build software for > packaging. Instead of > installing into sw-install directly, you install into a tmproot/sw-install. > Then you direct the package > manager to grab all files in tmproot/sw-install to put in the package. The > package itself will > strip off the tmproot leading directory. In other words the package labels > all files as sw-install/. > > So the build issue that I ran into is that the build directory and/or the > tmproot directory path becomes embedded into a large number of files (include > files, mod files, etc). To get around this, > I did a bind mount of (tmproot/sw-install --> /sw-install) and told petsc to > install into /sw-install. I consider that to be a "hack", but petsc isn't > the only software that I've run into that requires this > mounting bit of legerdemain. > > Many software packages will support a dest= or DESTDIR variable for "make > install" that supports the "installation" into a tmproot directory but > leaves no trace of tmproot in any of its configuration files. > (http://git.rocksclusters.org/cgi-bin/gitweb.cgi?p=core/python/.git;a=blob;f=src/python-2/Makefile;h=ced6cc1c6eb6a4f70dd0f3e1b0ccd1ac1e40f989;hb=HEAD) > shows a Makefile for python that supports this. > > Ultimately we create packages of software to simplify a number of things. > Including update of installed software on many machines. > > The other "issue" is not really an issue, but a request. When petsc is built > it creates a petscvariables (or similarly named) file with lots of > environment variables. It would be terrific if it could also create an > environment modules files with this same information. Users often want > different versions/different configs of the same software and environment > modules is now standard in Redhat/CentOS distributions. > > Hope this helps. > > > On Thu, Jan 24, 2013 at 2:44 AM, Schneider, Barry I. <bschneid at nsf.gov> > wrote: > Barry, > I am sending this to Phil Papadopoulos and he can make much more precise > recommendations. Glad you have thick skins. Being at NSF requires the same > physiology. > > -----Original Message----- > From: Barry Smith [mailto:bsmith at mcs.anl.gov] > Sent: Wednesday, January 23, 2013 10:19 PM > To: Schneider, Barry I. > Subject: Re: [petsc-maint #149715] Making a PetSc Roll and Rocks Module > > > Barry, > > By far the most common email we get is regarding installation, so we are > always trying to understand how to improve that process. If we can eliminate > just 20 percent of installation problems that would save us a great deal of > work, so yes, any critique/suggestions are greatly appreciated, and after all > these years we have thick skins so can survive even the harshest suggestions. > > Barry > > On Jan 23, 2013, at 6:13 PM, "Schneider, Barry I." <bschneid at nsf.gov> > wrote: > > > Barry, > > First, thanks for getting back to me so quickly. I view this as a way to > > make things easier for everyone for a library that is really invaluable to > > many of us. Phil P. is a pro at making rolls and modules for the Rocks > > distribution of CentOS that is widely used in the NSF cyberinfrastructure > > and by others. I am both a program manager for the XSEDE project and a > > large user as well so I appreciate the ability to use the library in its > > most transparent fashion. After Phil did his thing we tested it and it > > worked perfectly on our cluster. Basically, you load the module and any > > associated modules such as the Intel compiler and appropriate MPI module > > and then you just use it. No screwing around with building and the rest of > > it. That's the good news. Of course, you need to make a roll for every > > specific combination of compiler and MPI but I believe what Phil has > > learned makes that pretty straightforward to do. While I cannot speak for > > Phil, I would expect that anything he learned would be available to you > > folks if that's what you wanted. The next step for us will be to > > incorporate what we did in the Rocks distribution which will eventually > > propagate to lots of users. Eventually, there will be an XSEDE > > distribution which will be used by an even larger number of sites. We are > > talking about many thousands of users. So, I guess what I am really asking > > is whether you are interested enough in what Phil learned to perhaps modify > > the current PetSc so that it is easier for users to install or make > > available the technology for folks to make their own rolls. Of course, you > > could decide that is not on your agenda and that's fine. But if you > > capitalize on our experience and your great software that would be > > wonderful and perhaps offer users alternatives to the current way of doing > > things that would make life easier. > > > > ************************************************** > > Dr Barry I. Schneider > > Program Director for Cyberinfrastructure Past Chair, APS Division of > > Computational Physics Physics Division National Science Foundation > > 4201 Wilson Blvd. > > Arlington, VA 22230 > > Phone:(703) 292-7383 > > FAX:(703) 292-9078 > > Cell:(703) 589-2528 > > ************************************************** > > > > > > -----Original Message----- > > From: Barry Smith [mailto:bsmith at mcs.anl.gov] > > Sent: Wednesday, January 23, 2013 5:56 PM > > To: petsc-maint at mcs.anl.gov; Schneider, Barry I. > > Subject: Re: [petsc-maint #149715] Making a PetSc Roll and Rocks > > Module > > > > > > Barry, > > > > Thanks for the feedback. We actually desire to support "system" > > installs just as easily as "home directory" installs, though our > > documentation emphasizes "home directory" installs since we feel that is > > most practical for more users. We'd be interested in more specifics on the > > difficulties and any suggestions there may be to make the process > > (especially for multiple compilers/mpis) easier. That is what could we have > > done differently to make it easier for you? > > > > Thanks > > > > Barry > > > > On Jan 23, 2013, at 1:59 PM, "Schneider, Barry I." <bschneid at nsf.gov> > > wrote: > > > >> I thought I might pass on the following to the folks maintaining PetSc. > >> Recently we had the occasion to make a Rocks roll and module to be used on > >> a local cluster here at NSF. Phil Papadopoulos the developer of Rocks at > >> SDSC took the time to help us to do that not simply because we wanted it > >> but also because he wanted to be able to know how to do it and distribute > >> it to a much larger set users of NSF resources and also on out NSF > >> supported platforms. Here are his comments. Perhaps if you feel there > >> are things that could be made easier that would be great. It also could > >> be useful to you directly. > >> > >> BTW, Petsc is kind of a bear to package -- they really expect you to build > >> it in your home directory :-). > >> I took the time to make the roll pretty flexible in terms of > >> compiler/mpi/network support to build various versions. > >> This was modeled after other Triton rolls so that it is consistent with > >> other software -- it likely becomes part of the standard SDSC software > >> stack, so this was a good thing to do all the way around. > >> > >> I'm about ready to add this to our code repository, it will show up on > >> git.rocksclusters.org<http://git.rocksclusters.org> tomorrow morning. > >> > >> > >> ************************************************** > >> Dr Barry I. Schneider > >> Program Director for Cyberinfrastructure Past Chair, APS Division of > >> Computational Physics Physics Division National Science Foundation > >> 4201 Wilson Blvd. > >> Arlington, VA 22230 > >> Phone:(703) 292-7383 > >> FAX:(703) 292-9078 > >> Cell:(703) 589-2528 > >> ************************************************** > >> > >> > > > > > > > -- > Philip Papadopoulos, PhD > University of California, San Diego > 858-822-3628 (Ofc) > 619-331-2990 (Fax) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130124/81d3a58c/attachment.html>
