Yes, I submitted jobs before I went to bed last night (in the UK) and it configure and built. So far so good. Thanks,
On Fri, Jun 30, 2017 at 11:32 PM, Richard Tran Mills <rtmi...@anl.gov> wrote: > > > On Fri, Jun 30, 2017 at 3:27 PM, Balay, Satish <ba...@mcs.anl.gov> wrote: > >> On Fri, 30 Jun 2017, Richard Tran Mills wrote: >> >> > Hmm. I get the same error when adding "--download-hypre=0" to an >> otherwise >> > functional configure script for Cori KNL. Hypre's configure doesn't like >> > something about cross-compiling for KNL. It might be possible to make >> this >> > work by building on the compute nodes, though this is, unfortunately, >> not >> > officially supported on Cori -- NERSC staff say that builds are not >> > guaranteed to work on the compute nodes, though I've managed to make it >> > work sometimes. >> >> Here is (a part of a prior email) regarding installing hypre on theta >> (using compute nodes). >> > > The approach Satish outlines below sounds like the most reasonable course > of action we ahve. I wonder if it might help a little bit to try to just > run the standalone Hypre configuration (using the incantation in the > configure.log) in the batch script, rather than invoking PETSc's > configure.py. > > --Richard > > >> Satish >> >> -------------------------------- >> >> The alternative is to run configure on the KNL nodes. But this is very >> slow. We could split up the build into 2 parts - only build some >> packages on KNL node and the rest on front-end. >> >> Ok - I tried "--download-hypre=1 >> --prefix=/home/balay/soft/hypre-xsdk-0.2.0-rc2" >> on the KNL node. >> [this way - I can --with-hypre-dir=/home/balay/soft/hypre-xsdk-0.2.0-rc2 >> for subsequent builds] >> >> This crashed halfway with 'Intel License Server' errors. The crash >> happened after hypre configure completed - so I could complete the >> build on the frontend. >> >> Anyhow - here is the process" >> >> caveat: PETSC_ARCH/externalpackages should be populated before doing >> this - as compute nodes don't have external network access. >> >> 1. schedule configure run in the batch queue: >> >> balay@thetalogin5:~/petsc.test> cat reconfigure-arch-theta-opt-cray.py >> #!/usr/bin/python >> if __name__ == '__main__': >> import sys >> import os >> sys.path.insert(0, os.path.abspath('config')) >> import configure >> configure_options = [ >> '--download-hypre=1', >> '--prefix=/home/balay/soft/hypre-xsdk-0.2.0-rc2', >> '--with-cc=cc', >> '--with-clib-autodetect=0', >> '--with-cxx=CC', >> '--with-cxxlib-autodetect=0', >> '--with-debugging=0', >> '--with-fc=ftn', >> '--with-fortranlib-autodetect=0', >> '--with-mpiexec=mpirun', >> '--with-memalign=64', >> 'COPTFLAGS=-g -O3 -fp-model fast -xMIC-AVX512', >> 'CXXOPTFLAGS=-g -O3 -fp-model fast -xMIC-AVX512', >> 'FOPTFLAGS=-g -O3 -fp-model fast -xMIC-AVX512', >> 'LIBS=-lstdc++', >> 'PETSC_ARCH=arch-theta-opt-cray', >> ] >> configure.petsc_configure(configure_options) >> >> balay@thetalogin5:~/petsc.test> cat x.sh >> #!/bin/sh >> aprun -n 1 python reconfigure-arch-theta-opt-cray.py >> >> balay@thetalogin5:~/petsc.test> qsub -t 45 -n 1 -A EarlyPerf_theta x.sh >> >> <<<<<<<<<<<<<<<<<< >> >> 2. wait for job to complete - [or atleast hypre configure to complete]. >> >> 3. if incomplete - complete the build on frontend - and install hypre >> libraries. >> >> 4. subsequently install PETSc using --with-hypre-dir=/home/balay/s >> oft/hypre-xsdk-0.2.0-rc2 >> [on the frontend] >> >> Satish >> >> >> >