Jim By default, McLachlan is configured to be generic, i.e. many choices can be made at run time. For example, there are 4 additional gauge variables evolved (A and B^i). If you are willing to restrict your run-time choices, for example by fixing derivative order, disabling multi-block systems, choosing a particular gauge, choice of conformal factor, etc., then this will lead to a more efficient code. You can easily do so for McLachlan by generating a new version in Kranc. If you look in the beginning of McLachlan_BSSN.m in the section "Choose code to generate", then you'll see the parameters you can choose. We have made a particular set of choices for a configuration "BSSN_bench", but these are apparently different from LazEv. For example, we use 4th order finite differencing, and evolve B^i in time as well.
Can you specify all the details of your formulation? I notice that you didn't respond to my request to point me to the source code, and there are quite a few details that you also didn't describe yet (and are also not described in your paper), such as your choice of conformal factor, whether/how your code is integrated with ADMBase, whether the constraints are calculated from the BSSN or the ADM variables, how advection derivatives are handled, etc. For a true comparison, this all needs to be known. Unless we both know exactly what both the codes do, we'll never know where the physics differences are. -erik On Mon, Jul 27, 2015 at 5:39 PM, James Healy <[email protected]> wrote: > Hi Erik, > > We evolve alpha and beta for the gauge variables as given in equations 7a > and 7b of http://arxiv.org/pdf/1506.06153.pdf. The number I reported for > LazEv was using 5th order dissipation applied to all 21 evolved variables > (alp, beta^i, At_{ij}, gt_{ij}, Gammat^{i}, trK, and xi). > > LazEv is calculating the Hamiltonian and Momentum constraints as well as > the constraints on Gammat^i using 4th order stencils. > > I also forgot to mention that I did these tests on 16 nodes, and the > memory usage was relatively low ( ~2 GB/MPI process ). > > I will give the internal dissipation for McLachlan a try. > > Thanks, > Jim > > > On 07/27/2015 05:56 PM, Erik Schnetter wrote: > > Jim > > Thanks for posting the details. > > Can you give us more details about the LazEv scheme? In particular there > may be differences in the gauge. What (gauge) variables do you evolve? What > gauge conditions do you use? And what kind of dissipation do you apply? Can > you point us to the source code? > > For the new McLachlan, you would probably use the built-in dissipation > instead of thorn Dissipation, which should lead to a small speed-up. > > -erik > > On Mon, Jul 27, 2015 at 4:43 PM, James Healy <[email protected]> wrote: > >> Hello, >> >> I have been running some tests on Stampede comparing the run speed of >> McLachlan to RIT's evolution thorn LazEv. I started with the >> qc0-mclachlan.par parameter file included with the Einstein Toolkit, added >> a few refinement levels, increased the resolution and changed McLachlan to >> be 8th order (and increased the number of ghost zones to 5). I also >> increased the initial separation so the finest grids aren't already >> overlapping. To compare with LazEv, I removed the McLachlan and >> Dissipation thorns and replaced them with LazEv. Everything else in the >> parameter file is exactly the same. I tried using both the McLachlan >> master and rewrite branches. >> >> The grid setup is 10 levels of refinement, dx=4M on the coarsest with >> outer boundary at 400M, M/128 on the finest with r=0.6M, CFL is 0.25. Both >> use 8th order spatial differencing with ghost_size=5 and 5th order >> dissipation. >> >> Below is a summary of the results as reported at iteration 256 from >> Carpet::physical_time_per_hour: >> >> McLachlan - rewrite branch: 3.0596110 M/hr >> McLachlan - master branch: 3.8033607 M/hr >> LazEv - 4.1941544 M/hr >> >> I am using the stampede-impi.cfg configuration file in simfactory. >> "module list" returns: >> >> 1) TACC-paths 3) cluster-paths 5) xalt/0.4.6 7) TACC >> 2) Linux 4) intel/13.0.2.146 6) cluster 8) impi/4.1.0.030 >> >> Attached is my parameter file. I pasted the McLachlan parameters below. >> Are there any optimizations that I can use for McLachlan? Are the >> parameters I am using for it what would be used for production runs? >> >> ML_BSSN::harmonicN = 1 # 1+log >> ML_BSSN::harmonicF = 2.0 # 1+log >> ML_BSSN::ShiftGammaCoeff = 0.75 >> ML_BSSN::BetaDriver = 1.0 >> ML_BSSN::LapseAdvectionCoeff = 1.0 >> ML_BSSN::ShiftAdvectionCoeff = 1.0 >> >> ML_BSSN::MinimumLapse = 1.0e-8 >> >> ML_BSSN::my_initial_boundary_condition = "extrapolate-gammas" >> ML_BSSN::my_rhs_boundary_condition = "NewRad" >> Boundary::radpower = 2 >> >> ML_BSSN::ML_log_confac_bound = "none" >> ML_BSSN::ML_metric_bound = "none" >> ML_BSSN::ML_Gamma_bound = "none" >> ML_BSSN::ML_trace_curv_bound = "none" >> ML_BSSN::ML_curv_bound = "none" >> ML_BSSN::ML_lapse_bound = "none" >> ML_BSSN::ML_dtlapse_bound = "none" >> ML_BSSN::ML_shift_bound = "none" >> ML_BSSN::ML_dtshift_bound = "none" >> >> ML_BSSN::fdOrder = 8 >> >> ActiveThorns = "Dissipation" >> >> Dissipation::order = 5 >> Dissipation::vars = " >> ML_BSSN::ML_metric >> ML_BSSN::ML_trace_curv >> ML_BSSN::ML_curv >> ML_BSSN::ML_Gamma >> ML_BSSN::ML_lapse >> ML_BSSN::ML_shift >> ML_BSSN::ML_dtlapse >> ML_BSSN::ML_dtshift >> " >> >> ActiveThorns = "ML_ADMConstraints" >> >> >> Thanks, >> Jim Healy >> >> _______________________________________________ >> Users mailing list >> [email protected] >> http://lists.einsteintoolkit.org/mailman/listinfo/users >> >> > > > -- > Erik Schnetter <[email protected]> > http://www.perimeterinstitute.ca/personal/eschnetter/ > > > -- Erik Schnetter <[email protected]> http://www.perimeterinstitute.ca/personal/eschnetter/
_______________________________________________ Users mailing list [email protected] http://lists.einsteintoolkit.org/mailman/listinfo/users
