Hm - for one - we rely on ~/.petscrc for our nightly builds.

petsc@petsc:~$ cat ~/.petscrc
-malloc_dump
-display 140.221.10.20:0.0
#-nox
#-nox_warning
petsc@petsc:~$ 

And secondly we have an option -skip_petscrc that would skip it.

using "-pc_type_hypre" in ~/.petscrc is a bad practice. [it makes
sense only for global options that are useful for multiple petsc
applications - like -malloc_dump]

We also read in $PWD/.petscrc and $PWD/petscrc

Satish

On Tue, 14 Apr 2015, Barry Smith wrote:

> 
>    Mark,
> 
>    Ok, you convinced me. 
> 
>   Satish,
> 
>     Can you please rip out all the checking for .petscrc in the home 
> directory.
> 
>    Thanks
> 
>     Barry
> 
>    Theoretically it is a great feature but no one uses it and those who do 
> use it forget that they used it.
> 
> > On Apr 14, 2015, at 7:41 AM, Mark Adams <mfad...@lbl.gov> wrote:
> > 
> > PETSc's design of looking for RC files in the users home directory really 
> > sucks.  I've complained about this before a few years ago and am going to 
> > again.
> > 
> > A perfectly reasonable apps person had a .petscrc file in his home 
> > directory with at "%-pc_type_hypre" in it.  This gave an error but he could 
> > not figure out where PETSc got this thing.  (The error message was garbled 
> > for some reason, which slowed things down. As soon as I saw 
> > "%-pc_type_hypre" I knew what the problem was.)  As you can see below he 
> > spent a day doing this.
> > 
> > I spent a day with another apps person on this same project a few years ago 
> > with this same problem.  This is an error prone construct and it does not 
> > show up until you have used PETSc for a few years and have forgotten that 
> > you have a .petscrc file in your home directory. Very bad.
> > 
> > Also, this code specified that RC file name as "petsc.rc".  It looks like 
> > PETSc is still picking up a .petscrc file anyway!!!  At the very least we 
> > should scrub ".petscrc" if the user supplies another name.
> > 
> > I think we should dump this design and stop looking at home directories and 
> > just tell users to change, as we always do when we improve the design.
> > 
> > Thanks,
> > Mark
> > 
> > 
> > ---------- Forwarded message ----------
> > From: Yoon, Eisung <yo...@rpi.edu>
> > Date: Mon, Apr 13, 2015 at 11:00 PM
> > Subject: RE: related to compiling your source code
> > To: Mark Adams <mfad...@lbl.gov>, Robert Hager <rha...@pppl.gov>
> > Cc: Choong-Seock Chang <csch...@pppl.gov>, Mark Shephard 
> > <sheph...@rpi.edu>, Seung-Hoe Ku <s...@pppl.gov>
> > 
> > 
> > --------
> > WARNING: At least one of the links in the message below goes to an IP 
> > address (e.g.10.1.1.1), which could be malicious. To learn how to protect 
> > yourself, please go here: https://commons.lbl.gov/x/_591B
> > --------
> > 
> > Dear Mark and Robert,
> > 
> > Wow! Thank you so much for all your comments and helps. After I spent a 
> > whole day, I am about to give up resolving this issue.
> > 
> > I confirmed that the file which has %-pc_type hypre is located at my home 
> > directory! But the file name caused the problem is not petsc.rc but 
> > .petscrc, which I guess I copied a long time ago. The source code directory 
> > where the XGC execution file is located has a petsc.rc which does not 
> > contain %-pc_type hypre and does not have .petscrc file in the directory. 
> > 
> > In summary, three directories were involved for running XGC; A source code 
> > directory where the XGC execution file is located, working directory where 
> > job is submitted, and just my home directory. And the problem was from my 
> > home directory.
> > 
> > In addition, it is quite interesting that petscinitialize subroutine passes 
> > "./petsc.rc" in the XGC source code, while petsc tried to find a DEFAULT 
> > file ".petscrc"  in my HOME directory!!!
> > 
> > I currently have removed .petscrc and submitted the job to see if XGC runs. 
> > 
> > Best,
> > Eisung Yoon
> > 
> > From: Mark Adams [mfad...@lbl.gov]
> > Sent: Monday, April 13, 2015 10:19 PM
> > To: Robert Hager
> > Cc: Yoon, Eisung; Choong-Seock Chang; Mark Shephard; Seung-Hoe Ku
> > 
> > Subject: Re: related to compiling your source code
> > 
> > Good try Robert :)
> > 
> > I'll bet Eisung has a petsc.rc file in his home directory.  Let me know.  I 
> > will use this as another data point to support my opinion that looking in 
> > your home directory is a bad idea.
> > 
> > BTW, Seung-Hoe (cc'ed) and I had this same problem a few years ago and it 
> > took us hours to figure it out,
> > 
> > Mark
> > 
> > On Mon, Apr 13, 2015 at 9:21 PM, Robert Hager <rha...@pppl.gov> wrote:
> >> It seems Petsc is looking at a certain directory, but cannot check where 
> >> it is.
> > 
> > This may be a clue. I always copy the executable to my run directory and 
> > call something like
> > 
> > aprun ...  ./xgca
> > 
> > In one of your earlier e-mails, I saw that you call
> > 
> > aprun ... {PATH_TO_XGCa_SOURCE}/xgca
> > 
> > If PETSc looks for petsc.rc in the directory of the executable, it will try 
> > to read a very old petsc.rc file that certainly does not work. Could you 
> > try with the executable to your run directory?
> > 
> > Best
> > 
> > Robert
> > 
> > On Apr 13, 2015, at 8:58 PM, Yoon, Eisung wrote:
> > 
> >> I attach the requested files.
> >> 
> >> I tried PETSc and petsc.rc file in the XGC1 example suggested by Mark as 
> >> well as the original input files in xgc_chang-hinton_test.tar. Also I 
> >> checked language options which are same with yours and tried sed command, 
> >> but all failed with almost same messages. 
> >> 
> >> There were rarely interesting error messages, which can be a clue to 
> >> resolve this issue. The error messages showed "Unknown statement in 
> >> options file: (%-pc_type hypre )" even though my petsc.rc doesn't have 
> >> that line. I checked petsc.rc files in XGC source file directory as well 
> >> as working(running) directory, but that line doesn't exist. Also the 
> >> default .petscrc doesn't exist in both directories. It seems Petsc is 
> >> looking at a certain directory, but cannot check where it is.
> >> 
> >> Best,
> >> Eisung Yoon
> >> 
> >> 
> >> 
> >> From: Robert Hager [rha...@pppl.gov]
> >> Sent: Monday, April 13, 2015 5:31 PM
> >> To: Yoon, Eisung
> >> Cc: Choong-Seock Chang; Mark Adams; Mark Shephard
> >> Subject: Re: related to compiling your source code
> >> 
> >> That looks ok.
> >> 
> >> I unpacked the tar-file I gave you and ran a diff with the petsc.rc that 
> >> is still working for me and found that they are identical.
> >> 
> >> Did you edit any of the files (possibly in a Microsoft environment)? Or 
> >> maybe your shell misinterprets characters. Did you specify any language in 
> >> your shell setup?
> >> 
> >> In case something added any control characters to the petsc.rc file, you 
> >> can run
> >> 
> >> sed -e 's/[^[:print:]]//g'
> >> 
> >> to remove them.
> >> 
> >> My language settings are
> >> 
> >> rhager@edison02:~/w/xgca_chang-hinton_test3> locale
> >> LANG=
> >> LC_CTYPE="POSIX"
> >> LC_NUMERIC="POSIX"
> >> LC_TIME="POSIX"
> >> LC_COLLATE="POSIX"
> >> LC_MONETARY="POSIX"
> >> LC_MESSAGES="POSIX"
> >> LC_PAPER="POSIX"
> >> LC_NAME="POSIX"
> >> LC_ADDRESS="POSIX"
> >> LC_TELEPHONE="POSIX"
> >> LC_MEASUREMENT="POSIX"
> >> LC_IDENTIFICATION="POSIX"
> >> LC_ALL=
> >> 
> >>  Could you send your makefile, defs.mk and rules.mk (possibly 
> >> rules_edison.mk) anyway, please?
> >> 
> >> Best regards
> >> 
> >> Robert
> >> 
> >> 
> >> 
> >> On Apr 13, 2015, at 5:01 PM, Yoon, Eisung wrote:
> >> 
> >>> Hi Robert,
> >>> 
> >>> I added below to .cshrc.ext  as you recommended
> >>> 
> >>>   module load cray-petsc
> >>>   module load cray-hdf5-parallel
> >>>   module load pspline
> >>>   module load adios/1.6.0
> >>> 
> >>> and got
> >>> 
> >>> Currently Loaded Modulefiles:
> >>>   1) modules/3.2.10.2                       7) intel/15.0.1.133           
> >>>            13) gni-headers/3.0-1.0502.9684.5.2.ari   19) 
> >>> PrgEnv-intel/5.2.40                   25) altd/2.0                        
> >>>       31) adios/1.6.0
> >>>   2) nsg/1.2.0                              8) cray-libsci/13.0.1         
> >>>            14) xpmem/0.1-2.0502.55507.3.2.ari        20) craype-ivybridge 
> >>>                      26) darshan/2.3.0
> >>>   3) eswrap/1.1.0-1.020200.1130.0           9) 
> >>> udreg/2.3.2-1.0502.9275.1.12.ari      15) 
> >>> dvs/2.5_0.9.0-1.0502.1873.1.145.ari   21) cray-shmem/7.1.1                
> >>>       27) usg-default-modules/1.1
> >>>   4) switch/1.0-1.0502.54233.2.96.ari      10) 
> >>> ugni/5.0-1.0502.9685.4.24.ari         16) alps/5.2.1-2.0502.9041.11.6.ari 
> >>>       22) cray-mpich/7.1.1                      28) cray-petsc/3.5.2.1
> >>>   5) craype-network-aries                  11) 
> >>> pmi/5.0.6-1.0000.10439.140.2.ari      17) 
> >>> rca/1.0.0-2.0502.53711.3.127.ari      23) torque/5.0.1                    
> >>>       29) cray-hdf5-parallel/1.8.13
> >>>   6) craype/2.2.1                          12) 
> >>> dmapp/7.0.1-1.0502.9501.5.219.ari     18) atp/1.7.5                       
> >>>       24) moab/8.0.1-2014110616-5c7a394-sles11  30) pspline/nersc1.0
> >>> 
> >>> I copied Makefile.edison to Makefile, and had no problem with compiling 
> >>> and linking. I will try to figure out with the petsc.rc file. Thank you!
> >>> 
> >>> Best,
> >>> Eisung Yoon
> >>> From: Robert Hager [rha...@pppl.gov]
> >>> Sent: Monday, April 13, 2015 4:55 PM
> >>> To: Choong-Seock Chang
> >>> Cc: Yoon, Eisung; Mark Adams; Mark Shephard
> >>> Subject: Re: related to compiling your source code
> >>> 
> >>> Hi Eisung,
> >>> 
> >>> I used this file with XGCa on Edison today. Which modules do you use and 
> >>> which set of makefiles?
> >>> 
> >>> Best
> >>> 
> >>> Robert
> >>> 
> >>> On Apr 13, 2015, at 4:44 PM, Choong-Seock Chang wrote:
> >>> 
> >>>> Please include Mark Adams in the PETSc related e-mails.
> >>>> He is in charge of PETSc in our project.  He needs to be aware of all 
> >>>> the conversations.
> >>>> Thanks,
> >>>> CS
> >>>> 
> >>>> On Apr 13, 2015, at 4:42 PM, Yoon, Eisung <yo...@rpi.edu> wrote:
> >>>> 
> >>>>> Hi Robert,
> >>>>> 
> >>>>> I tried to run XGC in Greene and Edison. Green still has a problem with 
> >>>>> PETSc. Even in edison, XGCa shows an error related to the petsc.rc file 
> >>>>> as below. Considering "invalid argument" in the message, I guess the 
> >>>>> petsc.rc included in the xgca_chang-hinton_test.tar doesn't work. 
> >>>>> Unfortunately, the characters for the unknown option shown in the 
> >>>>> message is broken. Do you have working petsc.rc?
> >>>>> 
> >>>>> Thank you!
> >>>>> ES
> >>>>> 
> >>>>> (t_initf) Read in prof_inparam namelist from: input
> >>>>>  PERF_SETOPTS: PAPI library not linked in. Request to enable PAPI 
> >>>>> ignored.
> >>>>>  (t_initf) Using profile_disable= F  profile_timer=           2
> >>>>>  (t_initf)  profile_depth_limit=       99999  profile_detail_limit=     
> >>>>>       1
> >>>>>  (t_initf)  profile_barrier= F  profile_outpe_num=           1
> >>>>>  (t_initf)  profile_outpe_stride=           1  profile_single_file= F
> >>>>>  (t_initf)  profile_global_stats= T  profile_papi_enable= F
> >>>>>  call petsc_init
> >>>>> [0]PETSC ERROR: --------------------- Error Message 
> >>>>> --------------------------------------------------------------
> >>>>> [0]PETSC ERROR: Invalid argument
> >>>>> [0]PETSC ERROR: Unknown statement in options file: (???~A'^D)
> >>>>> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> >>>>> for trouble shooting.
> >>>>> [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014
> >>>>> [0]PETSC ERROR: 
> >>>>> /global/u2/e/eyoon/branch/dev_rhager_esyoon/epsi/XGCa/xgca on a 
> >>>>> sandybridge named nid05677 by eyoon Mon Apr 13 13:31:32 2015
> >>>>> [0]PETSC ERROR: Configure options --known-mpi-int64_t=0 
> >>>>> --known-bits-per-byte=8 --known-level1-dcache-assoc=0 
> >>>>> --known-level1-dcache-linesize=32 --known-level1-dcache-size=32768 
> >>>>> --known-memcmp-ok=1 --known-mpi-c-double-complex=1 
> >>>>> --known-mpi-long-double=1 --known-mpi-shared-libraries=0 
> >>>>> --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 
> >>>>> --known-sizeof-char=1 --known-sizeof-double=8 --known-sizeof-float=4 
> >>>>> --known-sizeof-int=4 --known-sizeof-long-long=8 --known-sizeof-long=8 
> >>>>> --known-sizeof-short=2 --known-sizeof-size_t=8 --known-sizeof-void-p=8 
> >>>>> --with-ar=ar --with-batch=1 --with-cc=cc --with-clib-autodetect=0 
> >>>>> --with-cxx=CC --with-cxxlib-autodetect=0 --with-debugging=0 
> >>>>> --with-dependencies=0 --with-fc=ftn --with-fortran-datatypes=0 
> >>>>> --with-fortran-interfaces=0 --with-fortranlib-autodetect=0 
> >>>>> --with-ranlib=ranlib --with-scalar-type=real --with-shared-ld=ar 
> >>>>> --with-etags=0 --with-dependencies=0 --with-dependencies=0 
> >>>>> --with-mpi-dir=/opt/cray/mpt/7.0.0/gni/mpich2-intel/140 
> >>>>> --with-superlu=1 
> >>>>> --with-superlu-include=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/include
> >>>>>  
> >>>>> --with-superlu-lib=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/lib/libsuperlu.a
> >>>>>  --with-superlu_dist=1 
> >>>>> --with-superlu_dist-include=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/include
> >>>>>  
> >>>>> --with-superlu_dist-lib=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/lib/libsuperlu_dist.a
> >>>>>  --with-parmetis=1 
> >>>>> --with-parmetis-include=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/include
> >>>>>  
> >>>>> --with-parmetis-lib=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/lib/libparmetis.a
> >>>>>  --with-metis=1 
> >>>>> --with-metis-include=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/include 
> >>>>> --with-metis-lib=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/lib/libmetis.a
> >>>>>  --with-ptscotch=1 
> >>>>> --with-ptscotch-include=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/include
> >>>>>  --with-ptscotch-lib="-L/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/lib 
> >>>>> -lptscotch -lscotch -lptscotcherr -lscotcherr" --with-scalapack=1 
> >>>>> --with-scalapack-include=/opt/cray/libsci/13.0.0/INTEL/140/sandybridge/include
> >>>>>  
> >>>>> --with-scalapack-lib="-L/opt/cray/libsci/13.0.0/INTEL/140/sandybridge/lib
> >>>>>  -lsci_intel_mpi_mp -lsci_intel_mp" --with-mumps=1 
> >>>>> --with-mumps-include=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/include 
> >>>>> --with-mumps-lib="-L/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/lib 
> >>>>> -lcmumps -ldmumps -lesmumps -lsmumps -lzmumps -lmumps_common 
> >>>>> -lptesmumps -lpord" --CFLAGS="-xavx -openmp -O3 " --CXXFLAGS="-xavx 
> >>>>> -openmp -O3  " --FFLAGS="-xavx -openmp -O3  " --LIBS=-lstdc++ 
> >>>>> --CXX_LINKER_FLAGS= --PETSC_ARCH=sandybridge 
> >>>>> --prefix=/opt/cray/petsc/3.5.2.1/real/INTEL/140/sandybridge 
> >>>>> --with-hypre=1 
> >>>>> --with-hypre-include=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/include 
> >>>>> --with-hypre-lib=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/lib/libHYPRE.a
> >>>>>  --with-sundials=1 
> >>>>> --with-sundials-include=/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/include
> >>>>>  --with-sundials-lib="-L/opt/cray/tpsl/1.4.3/INTEL/140/sandybridge/lib 
> >>>>> -lsundials_cvode -lsundials_cvodes -lsundials_ida -lsundials_idas 
> >>>>> -lsundials_kinsol -lsundials_nvecparallel -lsundials_nvecserial"
> >>>>> [0]PETSC ERROR: #1 PetscOptionsInsertFile() line 534 in 
> >>>>> /b/cray-petsc/.cray-build/INTEL/140/sandybridge/cray-petsc-base-dynamic/petsc-3.5.2/src/sys/objects/options.c
> >>>>> [0]PETSC ERROR: #2 PetscOptionsInsert() line 716 in 
> >>>>> /b/cray-petsc/.cray-build/INTEL/140/sandybridge/cray-petsc-base-dynamic/petsc-3.5.2/src/sys/objects/options.c
> >>>>> [0]PETSC ERROR: PetscInitialize:Creating options database
> >>>>> PETSC ERROR: Logging has not been enabled.
> >>>>> You might have forgotten to call PetscInitialize().
> >>>>> Rank 0 [Mon Apr 13 13:31:32 2015] [c5-3c1s11n1] application called 
> >>>>> MPI_Abort(MPI_COMM_WORLD, 56) - process 0
> >>>>> forrtl: error (76): Abort trap signal
> >>>>> Image              PC                Routine            Line        
> >>>>> Source
> >>>>> xgca               0000000003363F21  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               0000000003362677  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               000000000331A2F4  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               000000000331A106  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               00000000032AE434  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               00000000032B53B1  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               0000000002F64B60  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               0000000002F64B1B  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               0000000003371B11  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               0000000003131922  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               0000000003100063  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               00000000008BD7F0  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               00000000008B2241  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               00000000008C1B41  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               000000000042554B  perf_monitor_mp_p        1875  
> >>>>> module.F90
> >>>>> xgca               000000000051F3BE  MAIN__                     95  
> >>>>> main.F90
> >>>>> xgca               0000000000405DEE  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               000000000336B6C1  Unknown               Unknown  
> >>>>> Unknown
> >>>>> xgca               0000000000405CD1  Unknown               Unknown  
> >>>>> Unknown
> >>>>> _pmiu_daemon(SIGCHLD): [NID 05677] [c5-3c1s11n1] [Mon Apr 13 13:31:32 
> >>>>> 2015] PE RANK 0 exit signal Aborted
> >>>>> [NID 05677] 2015-04-13 13:31:32 Apid 11750871: initiated application 
> >>>>> termination
> >>>>> Application 11750871 exit codes: 134
> >>>>> Application 11750871 exit signals: Killed
> >>>>> Application 11750871 resources: utime ~60s, stime ~12s, Rss ~29844, 
> >>>>> inblocks ~3174405, outblocks ~8270892
> >>>>> From: Robert Hager [rha...@pppl.gov]
> >>>>> Sent: Monday, April 13, 2015 2:16 PM
> >>>>> To: Yoon, Eisung
> >>>>> Cc: sheph...@rpi.edu; csch...@pppl.gov
> >>>>> Subject: Re: related to compiling your source code
> >>>>> 
> >>>>> Hi Eisung,
> >>>>> 
> >>>>> you can use the input in
> >>>>> 
> >>>>> /project/projectdirs/m499/rhager/xgca_chang-hinton_test.tar
> >>>>> 
> >>>>> Let me know if you have trouble reading the file.
> >>>>> 
> >>>>> Best regards
> >>>>> 
> >>>>> Robert
> >>>>> 
> >>>>> On Apr 13, 2015, at 1:46 PM, Yoon, Eisung wrote:
> >>>>> 
> >>>>>> Hi Robert,
> >>>>>> 
> >>>>>> Thank you for the information and explanation. I attach a text file 
> >>>>>> which contains issues of source code with TRIGRID and variable 
> >>>>>> collision time.
> >>>>>> 
> >>>>>> I'm sorry for not telling you previously that I was compiling the 
> >>>>>> source code in PPPL server. I've not ready to use XGC  in Edison yet 
> >>>>>> but I'm going to work it to be ready right now.
> >>>>>> 
> >>>>>> Could you send me an input file of XGCa for a collision test in Edison?
> >>>>>> 
> >>>>>> Thanks a lot!!!
> >>>>>> ES
> >>>>>> 
> >>>>>> 
> >>>>>> From: Robert Hager [rha...@pppl.gov]
> >>>>>> Sent: Monday, April 13, 2015 10:34 AM
> >>>>>> To: Yoon, Eisung
> >>>>>> Cc: sheph...@rpi.edu; csch...@pppl.gov
> >>>>>> Subject: Re: related to compiling your source code
> >>>>>> 
> >>>>>> Hi Eisung,
> >>>>>> 
> >>>>>> the TRIGRID directive should not cause any errors. Can I see the error 
> >>>>>> message?
> >>>>>> 
> >>>>>> I looked at Makefile.edison in your branch. It looks fine. You might 
> >>>>>> have to change defs.mk though. There is one include statement to 
> >>>>>> import some PETSc variable definitions. Depending on whether you use 
> >>>>>> PETSc 3.5 or 3.6, you have to use the first or the second line, 
> >>>>>> respectively.
> >>>>>> 
> >>>>>> On Edison, I load the following modules in addition to the default:
> >>>>>> 
> >>>>>>    module load cray-petsc
> >>>>>>    module load cray-hdf5-parallel
> >>>>>>    module load pspline
> >>>>>> 
> >>>>>> The output of module list is
> >>>>>> 
> >>>>>> Currently Loaded Modulefiles:
> >>>>>>   1) modules/3.2.10.2                      13) 
> >>>>>> gni-headers/3.0-1.0502.9684.5.2.ari   25) cray-petsc/3.5.2.1
> >>>>>>   2) nsg/1.2.0                             14) 
> >>>>>> xpmem/0.1-2.0502.55507.3.2.ari        26) cray-hdf5-parallel/1.8.13
> >>>>>>   3) eswrap/1.1.0-1.020200.1130.0          15) 
> >>>>>> dvs/2.5_0.9.0-1.0502.1873.1.145.ari   27) pspline/nersc1.0
> >>>>>>   4) switch/1.0-1.0502.54233.2.96.ari      16) 
> >>>>>> alps/5.2.1-2.0502.9041.11.6.ari       28) allineatools/5.0.1
> >>>>>>   5) craype-network-aries                  17) 
> >>>>>> rca/1.0.0-2.0502.53711.3.127.ari      29) idl/8.2
> >>>>>>   6) craype/2.2.1                          18) atp/1.7.5               
> >>>>>>               30) gv/3.7.3
> >>>>>>   7) intel/15.0.1.133                      19) PrgEnv-intel/5.2.40     
> >>>>>>               31) latex/2012
> >>>>>>   8) cray-libsci/13.0.1                    20) craype-ivybridge        
> >>>>>>               32) altd/2.0
> >>>>>>   9) udreg/2.3.2-1.0502.9275.1.12.ari      21) cray-shmem/7.1.1        
> >>>>>>               33) darshan/2.3.0
> >>>>>>  10) ugni/5.0-1.0502.9685.4.24.ari         22) cray-mpich/7.1.1        
> >>>>>>               34) usg-default-modules/1.1
> >>>>>>  11) pmi/5.0.6-1.0000.10439.140.2.ari      23) torque/5.0.1
> >>>>>>  12) dmapp/7.0.1-1.0502.9501.5.219.ari     24) 
> >>>>>> moab/8.0.1-2014110616-5c7a394-sles11
> >>>>>> 
> >>>>>> Last time I tried, the code compiled with these settings. It also ran 
> >>>>>> a couple of time steps. But there are still some bugs in the code. 
> >>>>>> Making the collision time step variable is a bit complicated because 
> >>>>>> the collision operation is usually run together with all other sources 
> >>>>>> like heating, etc. Therefore, the distribution function is evaluated 
> >>>>>> only every sml_f_source_period time steps. If a collision operation is 
> >>>>>> supposed to run at a different time step, f will not be available with 
> >>>>>> the current code. However, in order to test whether it is worth to 
> >>>>>> pursue this approach, I wanted to implement variable collision time 
> >>>>>> steps in the simplest possible way, i.e. sml_f_source_period=0 and all 
> >>>>>> sources except the collision operation deactivated. The collision 
> >>>>>> interval must have an upper limit which I set to 10 time steps in my 
> >>>>>> test. The interval for load-balancing should be a multiple of this 
> >>>>>> upper limit in order to be efficient. If this approach helps to 
> >>>>>> improve performance, we can think about how to implement variable 
> >>>>>> collision intervals in a cleaner way.
> >>>>>> 
> >>>>>> Let me know if you have any further problems.
> >>>>>> 
> >>>>>> Best
> >>>>>> 
> >>>>>> Robert
> >>>>>> 
> >>>>>> 
> >>>>>> On Apr 12, 2015, at 2:38 PM, Yoon, Eisung wrote:
> >>>>>> 
> >>>>>>> Hi Robert,
> >>>>>>> 
> >>>>>>> Thank you for the performance test data. I really appreciate your 
> >>>>>>> work.
> >>>>>>> 
> >>>>>>> As for variable collision time,  I've made a branch 
> >>>>>>> "dev_rhager_esyoon" as a copy of your source code, "dev_rhager". I've 
> >>>>>>> read your modification for variable collision time in the XGCa folder.
> >>>>>>> 
> >>>>>>> In order to run the code, I currently have trouble with compiling the 
> >>>>>>> source code. It appears preprocessing directives -DTRIGRID causes the 
> >>>>>>> error. Could you send me your Makefile to see working compile options?
> >>>>>>> 
> >>>>>>> Thank you.
> >>>>>>> 
> >>>>>>> Best,
> >>>>>>> ES
> >>>>>>> 
> >>>>>>> 
> >>>>>>> 
> >>>>>>> 
> >>>>>>> 
> >>>>>> 
> >>>>>> <code_reading.txt>
> >> 
> >> <defs.mk><Makefile><rules.mk>
> > 
> > 
> > 
> 
> 

Reply via email to