[Users] meeting minutes for 2024-09-05

[Cupp, Samuel D.]

Present: Sam, Bill, Steve, Maxwell, Peter, Leo, Roland

# ET Release
* release name "Sophie Germain"

# Mailing list issues
* ongoing struggles with managing the rejection list
* users list emails are rejected by yahoo, and emails from yahoo fail to send 
to the list

# Tickets
* Ticket 2822: on homebrew, there's a problem with the pkg-config; this works 
to provide
  a workaround; would also help to offer non-static 
compiling option
* Ticket 2821: update to petsc external library; newer compiler needs newer 
version of petsc,
  and newer petsc needs updates to the build script
* several tickets have proposed solutions and are ready for review

Next week
Chair: Zach
Minute taker: Roland
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2024-08-29

[Roland Haas]

attendees: Roland, Zach, Maxwell, Leo, Peter, Keith

ET release timeline
===

* no updates

questions
=

particleTracer_ET

* Leo asked about ways to make particleTracer_ET more efficient by
  reducing duplicate work on multiple ranks
* current patch seems to have caused test
  failures https://einsteintoolkit.github.io/tests/
* patched code should follow what multipole is doing and thus be fine

BNS simulation collapsing to BH

* Maxwell asked about experience running BNS simualtion that collapses
  to a black hole
* Zach provided some suggestions what to do
** add extra refinement levels once a AH forms (lapse drops)
** Leo may be able to provide parfile snippets
* GRHydro is lacking some checks on the conservatives that
  IllinoisGRMHD does and which can lead to instabilities
* Roland has fixes that are in the Faber et. al. paper

Wiki server

* apache needs to be updated
* Roland has updates to Debian 12
* needs some testing on real system (SQL access is lacking)

Fellowship program

* http://lists.einsteintoolkit.org/pipermail/users/2024-August/009436.html
  for promotion of open source software

tickets
===

* CarpetX implementing "Driver"
  https://bitbucket.org/einsteintoolkit/tickets/issues/2813 Steve updated
  the ticket to list only the active pull requests
* no updates on improved McLachlan
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2024-08-22

[Leo Rosa Werneck]

Chair: Sam
Minutes: Leo

Present: Sam Cupp, Steve Brandt, Maxwell Rizzo, Bill Gabella, Leo Werneck,
 Roland Haas, Peter Diener

ET Release:
  * Roland will contact folks regarding new contributions.
  * Leo proposes the GRHayL-based version of NRPyLeakage for inclusion and will
re-open ticket #2689.
  * SpEC inclusion? Steve haven't heard from anyone.
  * Release co-chair? Maxwell Rizzo volunteered. Maxwell is aware he is
expected to be manager of the next release.

Tickets ready for review:
  * TensorTypeAlias has been inconsistently used, and Lucas added the term to
the flesh's grammar so it is properly parsed. Needs review.
  * Peter and Zach are working on a paper related to improvements to McLachlan.

Next chair: Steve
Next minutes: Roland



--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2024-08-15

[Lucas Timotheo Sanches]

# Release timeline
* We are running a bit late. Roland says he is going to start shooting emails
  to gather contributions.
* NSF says our new funding is approved and *awarded*.
* Possible items for the contributions list:
  * Z4C. Steve expected to be able to produce something soonish (no promises).
  * Roland asks anyone who has photos of the ETK meeting at LSU and subsequent
LIGO visit to send pictures, if they have them, so these can be added to
our group pages.

# Unanswered questions on the mailing list
No questions

# Tickets
* 2813: Steve asks for a review. He says it is a trivial change and should not
  take too long to do so.
* 2814: Lucas asks for a re-review after updates on this.
* 963: Peter says that the improvements work for him and Zach on BBHs. Slow
  star collapse is not working. Peter said that this was somewhat expected,
  but the results are worse than expected, so this needs to be further
  investigated. Steve expressed interest in helping to figure this out. Zach
  and Peter offered to provide him with the materials necessary so that he
  could play around.
* 2772: Roland asked to poke the local Fuka maintainers so that this can get
  fixed. Zach asked Roland to send him a reminder for that and expressed that
  he would do so.

# Other topics
Zach asks if there will be a PI meeting to discuss the new grand. Roland says
yes. He also says he needs help distributing the grant money across
institutions.
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2024-08-08

[Roland Haas]

Present: Roland, Steve, Peter, Khalil Pierre, Yosef

ET release
==

* no updates, running late since in month 3 of the 6 month cycle

Working group lists
===

* no objection to retire no longer active lists
* keep archive around

unanswered questions


* Khalil is facing issues with stars in simulations on a new cluster
* Steve provided some advise on how to simplify setting up Einstein
  Toolkit on a new cluster
* Roland suggested to try different compilers

ad-hoc discussions
==

Lucas presented a prototype implementation of a transpiler to construct
Cactus CCL files from yaml files. These have standardized ways to
specify the structure that editors understand so can provide feedback
while writing the file.

open tickets


* Roland will look
  at 
https://bitbucket.org/einsteintoolkit/tickets/issues/2813/allow-carpetx-to-implement-driver
* Lucas will
  update 
https://bitbucket.org/einsteintoolkit/tickets/issues/2814/tensortypealias-support-in-interfaceccl
 based on review comments
* Peter will look
  at 
https://bitbucket.org/einsteintoolkit/tickets/issues/2812/make-extract-work-with-carpet
* Roland explained path parsing issue
  in 
https://bitbucket.org/einsteintoolkit/tickets/issues/2719/getcompnents-does-not-handle-symbolic
* Steve and Roland may look
  at 
https://bitbucket.org/einsteintoolkit/tickets/issues/2179/nanchecker-fix-output-of-nanmask
 next week
* 
https://bitbucket.org/einsteintoolkit/tickets/issues/2520/update-gauge-settings-in-tov-example-to-be
 Roland to check that David has proper write access

tickets open for review
===

* 
https://bitbucket.org/einsteintoolkit/tickets/issues/2787/einsteintoolkitth-no-longer-sorted
 needs reviewer
* 
https://bitbucket.org/einsteintoolkit/tickets/issues/2806/support-carpetx-grid-scalars-and-grid
 Roland says needs tests for checkpoint / recovery
* 
https://bitbucket.org/einsteintoolkit/tickets/issues/963/improve-mclachlan-accuracy
 Peter still working on this for BH, Zach has test runs with Baikal running for 
NS


Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgp4qz5YDYknH.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2024-08-01

[Steven Brandt]

Present: Steve, Roland, Lucas, Peter, Leo, David Boyer, Yosef, Li Wei, Zach

ET Timeline: Roland needs to make one.

Proposed for inclusion:
    Z4c
    NRPy Leakage

Full US ET 2024 workshop schedule is available:
    https://www.cct.lsu.edu/Einsteintoolkitworkshop2024

Mailing list:
Bruno has a Lorene image

Tickets:
CarpetX -> Driver is ready
Getcomponents doesn't handle symlinks correctly
Zach and Peter are still working on the slow-start collapse.
2520 - Update TOV settings for the gallery examples
    Put new settings and data files into the repo
ET Thornlist could be sorted - Roland should check

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2024-07-25

[Roland Haas]

Present: Steve, Peter, Roland, Bing-Ran HE, Lucas, Johnny, Nan Dong,
Jake, Johnny, Cheng-Hsin, Liwei, Maxwell, Yosef
 
ET release
==

* Maxwell will be release co-chair

Unanswered questions


* https://lists.einsteintoolkit.org/pipermail/users/2024-July/009393.html is 
caused by gcc being stricter about pointer types in C. Roland will respond.
* Steve will
  address http://lists.einsteintoolkit.org/pipermail/users/2024-July/009391.html

Open tickets


* https://bitbucket.org/einsteintoolkit/tickets/issues/2811 is ready to be 
applied
* Steve will look into #2786: support CarpetX in
  Multipole https://bitbucket.org/einsteintoolkit/tickets/issues/2786
* Steve is looking into #2179: NaNChecker: fix output of
  NaNMask https://bitbucket.org/einsteintoolkit/tickets/issues/2179
* #2520: update gauge settings in TOV example to be more typical seems
  ok to apply
  (Steve), https://bitbucket.org/einsteintoolkit/tickets/issues/2520
* #2808: prevent wvuthorns/IllinoisGRMHD from compiling in release past
  2024_05, https://bitbucket.org/einsteintoolkit/tickets/issues/2808 Leo will
  discuss this locally. Had a discussion on how the old and new IGM
  thorns agree / disagree
* Peter and Zach will work on #963: Improve McLachlan
  accuracy, 
https://bitbucket.org/einsteintoolkit/tickets/issues/963/improve-mclachlan-accuracy


Einstein Toolkit translation


Bing-Ran reported on efforts on creating Chinese language documentation
for the ET. Currently have MPI set up and working on local cluster. Had
a discussion on how to best incorporate this into the existing ET site.

Steve suggests to have each language on a site and have link from one
to the other. Steve and Bing-Ran will discuss this via email. 

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpDmU4wuv4ZH.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2024-07-18

[Leo Rosa Werneck]

Hi all,

Here are the minutes of today’s meeting.

Chair: Sam
Minutes: Leo

Present: A (student from China), Sam Cupp, Steve Brandt, David Boyer, Keith Dow,
 Lucas Sanches, Zach Etienne, Leo Werneck, Johnny Tsao, Yosef Zlochower,
 Roland Haas, Bing-Ran HE

* ET Release Timeline
  - Proposed new thorns and updates:
- David's new TOV initial data thorn.
- Probably SpacetimeX.
- Zach wants to upgrade NRPyPN to the latest version.
  - Still no vice-manager, no volunteers yet.
  - Nothing else to discuss since Roland is not present.
  - testsuites: needs to be updated to include Mac OS.

* Unanswered Questions
  - None.

* Open Tickets
  - #2809: Cactus Makefile needs to be updated in the ET_2024_05 release.
   Steve will check.
  - #2179: mask bug in NaNChecker has been fixed and is ready for review.
  - #2520: David updated the parameters in the TOV example to use more usual
   values to the ML_BSSN settings.

* Tickets Ready for Review:
  - #2179: See above.

* Zach reports on some faculty members in China who are interested in using the
  Einstein Toolkit, some of which are experts in numerical relativity and some
  which are enthusiastic about the field.

* Lucas reports on some good progress on trying to use Bear to setup clangd for
  the Einstein Toolkit. As clangd is a C/C++ LSP, this could provide a more
  friendly experience when using an IDE that supports it, such as VS Code and
  neovim.

Next chair: Steve
Next minutes: Roland

Best,
Leo


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2024-07-11

[Lucas Timotheo Sanches]

Present: Lucas (Minutes), Roland (chair), David Boyer, Leo, Sam, Yosef

# Current Etk release timeline
 * Things are looking OK for the release.
 * Test results OK (some failures on thorns known to be sensitive).
 * BNS gallery example is probably finished, the page is still to be updated.
 * BBH gallery example still missing. Deborah needs to be poked but may not
   have time. Roland will do the poking.

# Next ETK release
  * We need to form a release coordination. Roland will be the release chair.
  * We need a release co-chair. There have been no volunteers so far.
  * Decide on features we will include (needs to be done now). Start querying
people for possible features they would like to include.
  * Roland reports that Hayley J. Macpherson may have something to the
cosmology toolkit. She needs to be further questioned on this.
  * David Boyer, a member of Zach's group has an inclusion: David Made a TOV
solver that handles multiple ODEs, tabulated and piecewise EOS. He reports
that adaptive RK4 is enabled for the ETK version but more can be enabled
if the community desires. It is separated from the Nrpy EOS solver but
was tested against that.

# Mailing list
  * No unanswered questions on the mailing list.

# Tickets
 * 2748: Yosef will review SpacetimeX (Z4C). He says his schedule may not be
   compatible, so this may have to change.
 * 2520: Assigned to David Boyer.
 * 2780: Sam suggested that this ticket should be closed and the inclusion of
   Baikal X dropped. Sam reports that Zach's group (the developers of
   Baikal/BaikalX) are not actively working on CarpetX thorns and the RIT
   group originally interested in it may no longer be using it.
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2024-05-30

[Lucas Timotheo Sanches]

Present: Lucas (minutes), Leo (chair), Peter, Roland, Zach

# ET Release Timeline
 * We now have all the gallery runners.
 * No other updates since the latest release.

# ET summer school and workshop at LSU
 * Leo asked how to best allocate the 1-hour time slot for his talk.
   Roland suggests giving breaks during the talk and says a more
   detailed plan can be found at
   
https://docs.google.com/spreadsheets/d/1KYbKLKrle7kGITeWNWZvO_Rf0A2Rc6_AyaiqARy89R4/edit?pli=1#gid=0
 * Lucas asked about the `Error: machine slurmjupyter is missing a
required key: user` while trying to run simfactory in the workshop
tutorial server. Roland recommends trying simfactory's setup silent

# Mailing list
 * No questions.
 * Roland: Bruno Giacomazzo and Julia Conti had trouble compiling on
   MacOS (the known incompatible pointer). Roland fixed it.

# Open tickets
 * 2799: Simple, no readme file in the Thorn. Roland requested a
   change.
 * 2786: Put off until after the release.

# Tickets ready for review
 * 2772: Leo says that Sam reported the error fixed, but it hasn't
   really. Roland says there is a PR with a fix, but Sam said he
   wants to test more before merging. Roland encouraged Leo to bug
   him to fix this.
 * 963: Peter says that he and Zach have implemented sphynx bssn.
   Constraint damping is better than it was before, but there are
   still some issues. Peter says he is working on that.
 * 2776: Roland says he needs to test this *soon*. It is on his
   to-do list.

# Extras
 * There will be no meeting next week because of the workshop
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2024-05-23

[Roland Haas]

Present: Steve (chair), Peter, Roland (minute), Keith, Leo, Zach

ET summer school and workshop at LSU


* everything in order, have drivers for LIGO trip
* Steve will present on Cactus Reduction instead of NRPy+ work

ET release timeline
===

* running late, will release in June, will restart after workshop is
  over
* MultipoleX: hold off until next release to have more time to clean up
  Carpet + CarpetX compatible codes
* NewRadX is in
* GRHayL codes are all fine

gallery runners
===

* one more runner may be needed. Roland to ask Maxwell Rizzo if wants
  to run TOV example

release issues
==

* gcc-14 support needs more changes, mostly easy to handle pointer types
* will disable PAPI by default since it fails to compile with gcc-14 on
  macOS and is very hard to support there (does not do anything even
  when compiled)

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpCsOuk0aXB5.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2024-05-16

[Leo Rosa Werneck]

Hi all,

Here are the minutes for today’s meeting (apologies for the delay).

Cheers,
Leo

Chair: Zach Etienne
Minutes: Leo Werneck

Present: Sam Cupp, Leo Werneck, Steve Brandt, Peter Diener, Johnny Tsao,
 Lucas Sanches

* NA ET School/Workshop @ LSU
  - June 3-7 (https://www.cct.lsu.edu/Einsteintoolkitworkshop2024).
  - There will be an excursion to LIGO, scheduled for the afternoon of June 5.
  - Looks ready to go!

* ET Release
  - Will happen sometime in June
  - No updates on MultipoleX and NewradX.

* Mailing List
  - Misyura Maxim is interested in obtaining the data for GW150914. Message
will be forward to authors, as Zenodo doesn't contain all the data.
  - A student from Nanjing Normal University inquired about LORENE. Zach will
forward the message to Josh.

* Open Tickets
  - #2797: Leo addressed issue in NRPyElliptic.
  - #2796: Sam addressed issue in Baikal thorns with SIMD variables.

Next Chair: Leo
Next Minutes: Roland


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2024-05-09

[Lucas Timotheo Sanches]

Present: Steven R Brandt, Kai, Keith Dow, Samuel Cupp, Peter Diener,
Zachariah Etienne, Roland Hass, Leo Werneck

* We had a new member, Kai's, to join in the meeting. Kai is a
German software engineer creating a Game Engine. He wants to
model GR physics with his engine and is interested in learning
GR / NR.

* US ET Workshop June 3-7 [SB]
  Steve says that the workshop is "in pretty good shape, in
  terms of the number of speakers".

  Currently, there are too many funding requests to attend
  the workshop and too little money to fulfill these requests.
  There may be support from CCT for paying for the meals of
  participants. There is some spare money from the Spec
  benchmarks that may be used for the workshop.

  Steve expressed that van drivers for the Wednesday LIGO tour
  are needed.

  Philipp Moesta has announced the European ETK workshop in Amsterdam.

* ET release timeline [SB]
  The release timeline is def. going to slip.

  The new target is the second week in June.

  Roland urged that codes from personal git repositories be moved into
  the ETK organization repository.

  Seteve urged to contact the NewRadX devs to move their repo. The need
  to move CarpetX was pointed out as well. Roland will make sure that
  happens.

  Roland brought up that gallery example runners are needed. The gallery
  examples need to be done before the new release deadline.

  Steve has volunteered to take BNs example.

  Deborah Ferguson volunteered for BBHs example.

  Roland has a student coming in June and will ask him for a TOV example.

  Lucas volunteered to run the MultiPatch example.

* contributions (champion, reviewer)
  ** MultipoleX (Lucas, Roland): Steve wants to figure a way out to
 make that work with Roland.

  ** NewRadX (Alex, Liwei and Jay): Jay is reviewing the thorn.

  ** Z4c (Erik, Steve): Dead. Not present in this release. Not enough
 time for it to get ready. Has no tests.

  ** GRHayL-based IllinoisGRMHD (Sam, Lorenzo): On track.

  ** Baikal(Vacuum) using NRPy 2.0 (Sam, Terrence): On track.

  ** GRHayLHD(X) updated to include tabulated EOS (Sam, Terrence): On
 track.

* Unanswered question on the mailing list

  The issue with npernode value in MPI (Shamim Haque 1910511): Steve
  the problem is more or less solved.

* Open tickets sorted by update time
  SyntaxWarnings Raised During Build Process with Python 3.12: Roland
  requested changes to the simfactory ini file reader.

  TACC Stampede3-skx Compilation Failing
  [-Wincompatible-function-pointer-type]: Remove the restrict keyword
  from the thorn. Roland will do it. At some point, restrict needs to
  be restored, because in Fortran, a restrict is a "default".

  ET webserver no longer sends email: Closed.

  Add sorting capability to the manifest: Steve made a sorter to solve
  this problem, which could be used or not.

* Tickets ready for review

  SyntaxWarnings Raised During Build Process with Python 3.12: See
  above. Roland will investigate why the ticket has not yet been
  closed since he fixed the problem.

  Improve McLachlan accuracy: Peter and Zach have No updates yet,
  but they scheduled a meeting to work on this.

  zlib: also use pkg-config when searching for installed versions:
  Solved by Roland.

  Issues link against Kadath Thorns and Intel MKL due to link
  ordering: Sam wants/needs to do more stuff before it is closed.
  Maybe poke him.

* Talking to Kai about what he wants/ what he got from the meeting.

* Peter suggests Kai studying perturbations for particle motion
  solutions https://bhptoolkit.org/KerrGeodesics/
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes 2024-05-02

[Steven Brandt]

Present: Steve, Roland, Lucas, Keith Dow, Petr, Leo, Zach

Workshop is on track
    Lost one speaker - Nasir

Need to figure out what to do with MultipoleX and messed up inheritance
Need Status on NewRadX
Need Status GRHayL-based IllinoisGRMHD
Schedule is going to slip
    Family vacation for Steve

From a DB run
  Tests failed:

    test_o11 (from CarpetProlongateTest)
    Balsara1 (from IllinoisGRMHD)
    Balsara2 (from IllinoisGRMHD)
    Balsara3 (from IllinoisGRMHD)
    Balsara4 (from IllinoisGRMHD)
    Balsara5 (from IllinoisGRMHD)
    ML_BSSN_NewRad (from ML_BSSN_Test)
    ML_BSSN_O8_sgw3d (from ML_BSSN_Test)

Ticket 2794 - Reflection Symmetry probably needs restrict fixed.
Ticket 963 - Improve McL accuracy, Peter says his modification looks bad
Ticket 2772 - Link ordering someone has to poke Samuel
Ticket 2776 - Zlib

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2024-04-25

[Roland Haas]

Present: Steve, Roland, Lucas, Sam, Zach, Peter (others I forgot to
write down right away)

ET release
==

* all thorns need to be added to thorn list: Multipole updates, Z4c,
  NewRadX
* the upcoming 2024-05 will be "Landau",
  https://en.wikipedia.org/wiki/Lev_Landau
* release managers to contact champions about author lists, requested
  and suggested citations, check on formal inclusion criteria
* review criteria:
  https://docs.einsteintoolkit.org/et-docs/How_to_Review_a_new_component
* email templates:
  
https://docs.einsteintoolkit.org/et-docs/Release_Process#Emails_to_send_to_contributors_and_gallery_runners
* add this to release process wiki page

Unanswered questions


* no questions
* reminder for LSU summer school and workshop attendees to register to
  get reduced hotel rate and be provided lunch

Tickets
===

* https://bitbucket.org/einsteintoolkit/tickets/issues/2778
  grhaylhd-x-update-for-tabulated-eos will be closed
* https://bitbucket.org/einsteintoolkit/tickets/issues/2760
  syntaxwarnings-raised-during-build-process will be merged and closed
* https://bitbucket.org/einsteintoolkit/tickets/issues/963
  improve-mclachlan-accuracy Peter is still testing slow start lapse
* https://bitbucket.org/einsteintoolkit/tickets/issues/2789
  citation-suggestion-for-kranc will have to contact authors once more
  for decision on what to cite
* https://bitbucket.org/einsteintoolkit/tickets/issues/2790
  mol-documentation-out-of-sync no update, is documentation only
  change, nothing bad happns if one follows the current docs
* https://bitbucket.org/einsteintoolkit/tickets/issues/2788
  et-building-error not enough information, reporter has gone radio
  silent, will close.
* https://bitbucket.org/einsteintoolkit/tickets/issues/2787
  einsteintoolkitth-no-longer-sorted to be fixed, Steve suggests a
  script, Roland points out that there are comments that one needs to
  keep with their GetComponent stanza
* https://bitbucket.org/einsteintoolkit/tickets/issues/2772
  issues-link-against-kadath-thorns-and Waiting for input by Samuel
  Tootle
* https://bitbucket.org/einsteintoolkit/tickets/issues/2765
  nsimd-is-not-actually-build-with-simd-code Worked for Steve
  apparently, may be "invalid", will put "on hold"

Chapter for Springer book
=

* still in bad shape
* promised some text on GRHydro
* Steve urges Roland to add some more text to get this done soon

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgp7TaJRKLbI0.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2024-04-18

[Leo Rosa Werneck]

Hi all,

Here are the minutes for today’s ET call.

Chair: Zach Etienne
Minutes: Leo Werneck

Present: Roland Haas, Leo Werneck, Peter Diener, Sam Cupp, Zach Etienne
 Johnny Tsao, Lucas Sanches, Steve Brandt

* NA ET School/Workshop @ LSU
  - June 3-7 (https://www.cct.lsu.edu/Einsteintoolkitworkshop2024).
  - Checking participant support cost availability for both domestic and
international participants.
  - Preliminary schedule in progress, but still missing speakers.
  - There will be an excursion to LIGO, scheduled for the afternoon of June 5.
  - Leo requested a hands-on tutorial on how to set up and run containers in
HPC systems, for reproducibility and portability.

* ET Release
  - Review deadline: tomorrow!
  - Baikal/GRHayLHD: Terrence is actively reviewing.
  - IllinoisGRMHD: haven't heard from Lorenzo yet.

* Online ET Tutorial
  - Requested by a research group from Denmark who contributed to the toolkit.
  - The Amsterdam workshop can be useful, if it happens.
  - Roland looking for lecturers/tutorial givers.
  - Roland will know more details by next week.
  - For those in the West coast: keep in mind the time zone difference.

* Mailing List
  - Misyura Maxim is interested in obtaining the data for GW150914. Message
will be forward to authors, as Zenodo doesn't contain all the data.
  - A student from Nanjing Normal University inquired about LORENE. Zach will
forward the message to Josh.

* Open Tickets
  - #2760: simfactory had warnings with Python 3.12. Steve and Roland worked on
it, and asked Zach to review. Most of the fixes involved strings used for
regex into raw strings.
  - #2791: use apptainer rather than Docker for documentation.
  - #963: improve McLachlan accuracy; Zach and Peter will meet about it.

* Tickets ready for review
  - #2760: see above.
  - #963: see above.
  - #2786: Multipole support in CarpetX, stalled after discussion.

Next Chair: Lucas
Next Minutes: Roland

Best,
Leo


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2024-04-11

[Cupp, Samuel D.]

Present: Roland, Peter, Sam, Steve, Zach, Leo, Lucas, Yosef

NSF reporting time!
Those with ongoing or recently completed theses that use the Einstein Toolkit 
please reply to Roland's email so this information can be included in the 
report.

ET US workshop
==
LIGO tour is arranged

ET release timeline
===
Next deadline: reviews due April 19th

tickets
===
#2790
Minor changes to fix outdated documentation in MoL

#2696
Kuibit now has new version 1.4.1 (minimum python version 3.7)

tickets ready for review

#2786
Suggested changes so that Multipole will work for both Carpet and CarpetX led 
to conversation regarding grid implementation. Following this was a complicated 
discussion regarding inheritance, whether backward compatibility extends to old 
thornlists in general.

#963
Improvements to Baikal(Vacuum) planned for November release
Peter also plans to add these improvements to McLachlan

Other topics
==

Sam brought up CarpetX issue requesting the addition of a feature to control 
the rhs tag at runtime.

Steve suggested adding general build instructions to the website that 
recommends using the utils/Scripts/makethornlist to generate a smaller 
specialized thornlist based on a parfile.

Next week:
Chair - Zach
Minute taker - Leo
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2024-03-28

[Roland Haas]

Present: Roland, Peter, Sam, Steve, Zach

ET US workshop
==

* date set to June 3 - 7

ET release timeline
===

* April 5th deadlines moved to April 19th
* code is expected to be in master
** need to add Z4c
** need to add MultipoleX
** need to add NewRadX
* create tickets for each contribution

SpaceTimeX
==

* all in favor of trying to move SpaceTimeX to ET repos

CarpetX
===

* all in favor of trying to move CarpetX to ET repos

unanswered questions


*
  https://lists.einsteintoolkit.org/pipermail/users/2024-March/009256.html
  related to
https://bitbucket.org/einsteintoolkit/tickets/issues/2781 now merged to master

tickets
===

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2748/inclusion-of-spacetimex-in-einstein
  should be made more specific for release to be used to add spacetimex
  to ET repos

*
  https://bitbucket.org/einsteintoolkit/tickets/issues/2696/update-kuibit-to-140
  Max ran into issues with kuibit on db1 following the tutorial

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2786/support-carpetx-in-multipole
  this touches on the larger issue on CarpetX / Carpet non-compatibility

tickets ready for review


* no new tickets
*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/963/improve-mclachlan-accuracy
  Peter and Zach still working on this

No ET call April 4th due to APS meeting.

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpSu3YdRX6gt.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2024-02-29

[Roland Haas]

Present: Sam, Leo, Peter, Roland, Zach

* did no reach quota
* discussed book chapter with Springer, Roland to solicit extra authors
  for GRhydro section

* no new emails
* no new tickets

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpgSi8k1kfU0.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] meeting minutes for 2024-01-18

[Samuel Tootle]

Steve et al:

I'm very open to including a gallery example in the up-coming release.  
If you have some ideas, preferences, or constraints (i.e. has to be 
computed in a certain amount of time on a certain number of cores); let 
me know.  I have been testing the new development importers for FUKA ID 
with IGM and GRHayl, so it's possible we could combine some of these 
ideas for gallery examples.


Cheers,

Samuel


Postdoctoral Researcher
Department of Physics
University of Idaho
https://samueltootle.github.io

On 1/18/24 7:47 AM, Steven Brandt wrote:
Present: Bing-Ran HE, Steve, Roland, Lihaichang, Sam Cupp, shenfei, 
Zach, Leo

Chair: Roland
Minutes: Steve

Need to have a release schedule

List of updates for the next release:
    Reprimand - is there a use case?
    Gallery examples for ID readers FUKA, Sgrid, Canuda
    TOV solver based on Grhayl, tabulated EOS builtin
    New Grhayl-based MHD thorns - Gallery example?
    New tabulated eqn state
    Baikal updates

Bing asked about initial data
    he did not feel LORENE was friendly
    Roland pointed him to FUKA and Sgrid
http://einsteintoolkit.org/thornguide/ExternalLibraries/SGRID/documentation.html 

http://einsteintoolkit.org/thornguide/Fuka/KadathImporter/documentation.html 



Roland will try and replicate failure of BNS

Trying again to get the check from SPEC

No unanswered questions on the email list

Open tickets:
    2772 Kadath - linking issue on systems with mkl
    and 2755 POWER - Doesn't compute angular momentum correctly (wrong 
formula)


___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2024-03-21

[Leo Rosa Werneck]

Hi all,

Here are the minutes for today’s ET call.

Chair: Roland Haas
Minutes: Leo Werneck

Present: Roland Haas, Leo Werneck, Peter Diener, Sam Cupp, Zach Etienne
 Steve Brandt

* No meeting two weeks from now (April 4) due to APS April Meeting

* NA ET School/Workshop @ LSU
  - June 3-7 (https://www.cct.lsu.edu/Einsteintoolkitworkshop2024).
  - Steve is deciding on length of the talks.
  - Everyone in the call should be thinking about talk topics to email Steve.
  - There will be a field trip to LIGO.
  - Workshop is on track to happen.
  - Working workshop hackathon for CarpetX may or may not happen afterwards.

* ET Release
  - Moving as expected.
  - About to hit the 10 week mark ("all codes proposed for inclusion in master
branch").
  - Roland will try to create CarpetX versions of Grid and CartGrid3D.
  - Discuss these ideas in the regular CarpetX meeting.
  - Helvi suggested having a new repository for NewRadX so it doesn't have to
live in a private repo. Will have to be discussed later with her group and
the RIT group, which were not present in today's call.
  - Roland will start the process by creating a CarpetX repository.
  - MultipoleX: issue with grid, but very minor changes compared to Multipole.
  - NewRadX: documentation and no-harm review are in progress. Might need to
 poke people to figure out required citations.
  - Z4c: Steve will poke Erik, but already at "no-harm" stage.
  - GRHayL-based IllinoisGRMHD: in main now, tests updated, waiting to complete.
  - GRHayLHDX: Terrence is reviewing. Steve suggested having a TOV test.
  - Baikal with NRPy+2.0: Might need to merge PR, review underway.
  - BaikalX: Might not be included in this release, as more testing is needed.
  - The review deadlines are being extended by a week to help reviewers who are
attending the 2024 APS April Meeting.

* Mailing List
  - Someone had a problem when debug mode is enabled and PreSync is set to 
“off”.
A pointer isn't set, causing an assert can fail. Roland says this as a
feature, not a bug, so it shouldn't be fixed with the proposed way.
  - One more email Erik is already involved with, so nothing to discuss.

* Open Tickets
  - #2619: Alireza still interested in contributing Elliptica.
  - #2777: GRHayL-based IllinoisGRMHD --- Sam is working on fixing issues with
   backwards compatibility, as some of the tests are failing.
  - #2784: Cactus start up problems, Steve says he's working on it.
  - #2785: A friendly reminder that Zenodo is currently using the wrong URL.

* Tickets ready for review
  N/A

Next Chair: Sam
Next Minutes: Roland

Best,
Leo


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2024-03-14

[Cupp, Samuel D.]

Present: Roland, Samuel, Peter, Zach, Harsh Parashar, Leo

ET US summer school and workshop:
* ​draft of website now available: 
https://www.cct.lsu.edu/Einsteintoolkitworkshop2024

ET release:
* GRHayLHD update has a reviewer (Terrence)

ET testing system:
* Roland still seeking help resolving issues with this system: 
https://github.com/EinsteinToolkit/tests/issues

Tickets:
Ticket 
2785
* Zenodo uses old website url
* probably requires manual changes while making the next zenodo entry

Ticket 
2784
* CarpetX revealed new threading issues in Cactus
* also revealed an error with the read/write parsing for vector array variables
* both have PRs under review

Ticket 
2781
* issue with PreSync when CCTK_DEBUG is on and presync_only is the default "off"
* PR under review to fix this
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-12-14

[Steven R. Brandt]

Present: Zach, Steve, Vaishak Prasad, Sam, Leo
Chair: Sam
Minutes: Steve

Checked: http://einsteintoolkit.org/testsuite_results/index.php

Roland says we cannot compile on MacOS. Too many problems.

JupyterET - needs to rebuild and push to dockerhub
    Only Roland or Steve can do that.

Ready for Zenodo (Steve will do)

Gallery exmaples
    Deborah has a PR for the website repo for her changes
    The BNS has been run, but analysis is not yet done
    TOV is being done by Peter

Unanswered emails
    Question about atmosphere mask, need to ask original authors.

Simfactory2 website doc issue updated
NSIMD needs some testing and investigation

Next meeting: Jan 11

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-11-09

[Roland Haas]

Present: Roland, Peter, Steven, Leo, Sam, Zach

CarpetX failure due to non-threads safe flesh
=

* Cactus flesh is not thread save, but is called from multi-threaded
  context in CarpetX
* this causes a SEGFAULT, but only during PreSyncOnly mode
* ticket is https://bitbucket.org/einsteintoolkit/tickets/issues/2762
* this needs to be fixed before the release

ET release
==

* SGRID test has been updated and passes on CI and local machine
* Sam has drafted announcement email
* Collecting authors for contributing 
* Peter is running tests on sunrise, most ACCESS-CI machines not yet
  tested
* Debroah Ferguson will run BBH gallery example
* Roland will poke local students
* Peter ran into issues with ExternalLibraries/ADIOS and
  ExternalLibraries/MPI not picking up the mpicc wrapper. Roland says
  that ExternalLibraries/MPI is a measure or last resort, and he would
  avoid it unless no other option is possible

Springer book
=

* Sam and Leo will look into it
* Steve volunteers to contribute

Running ET on Arm64
===

* Roland reports he could successfully compile on his Arm64 box at home
  (TV Set-top box using Debian)
* two failing tests: MemSpeed (aborts due to not recognising memory
  layout it seems, maybe hwloc), ML_BSSN_NewRad (numerical comparison
  fails)

BitBucket account restrictions
==

* resolved for einsteintoolkit and cactuscode
* still in existence for Simfactory, needs action by Frank Loeffler

Simfactory restructuring


* Zach suggests that parts of simfactory may be split of and moved into
  the main build system of Cactus as a configure script or option list
* currently we are lacking good directions on the website on how to
  compile, should be close to download page
* would want to streamline download and build system for new users
* Zach suggests to have requirements shown from simfactory like tool

Open tickets

* https://bitbucket.org/einsteintoolkit/tickets/issues/2757 still needs
  to be fixed since it makes building fail unless Python's request is
  present
* https://bitbucket.org/einsteintoolkit/tickets/issues/2755 Roland will
  accepts this bug fix

Next chair: Zach
Next minutes: Sam

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpiLmfQdiN4T.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] meeting minutes for 2023-10-19

[Cupp, Samuel D.]

Hi Steve,
   These are the minutes for an older meeting, so they hadn't switched yet. You 
are currently assigned the BNS gallery example.

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho

From: Users  on behalf of Steven Brandt 

Sent: Monday, November 6, 2023 10:36 AM
To: users@einsteintoolkit.org 
Subject: Re: [Users] meeting minutes for 2023-10-19

I think the gallery examples differs from previous weeks? I think I was
trading BBH with some other test?

--Steve

On 10/19/2023 9:35 AM, Steven R. Brandt wrote:
> Present: Steve, Peter, Sam, Zach, Leo
>
> Release
> Gallery Examples
> - TOV is Peter
> - BBH is Steve
> - Roland might have students for the other three?
> Tests
> - Need to be run
> - Clusters:
> Stampede2 isn't listed, and Delta and Anvil are added.
> - Release name: Lise Meitner
> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FLise_Meitner&data=05%7C01%7Cscupp1%40my.apsu.edu%7C4b812167a1ed418733ee08dbdef75282%7Cfe42fdbd8df8483a9700ff2693323482%7C0%7C0%7C638348926065507796%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=xVPy8NfSjg9SgjnY2wu6LigGsJuASOR5uEtVrHD5sa0%3D&reserved=0<https://en.wikipedia.org/wiki/Lise_Meitner>
> - Autogenerated codes need to be regenerated, Leo will do it
>
> Mailing list moderation
> - Remind Roland to send Peter the password
>
> Unanswered question on the mailing list
> - Need answer for Enzo. Roland was going to answer?
> - Ticket 2749: Leo says he fixed it.
> - Ticket 2647: One of Zach's students is looking at that in Grail
> (sp)
> - Ticket 2609: Steve thinks he approved PR for Roland
>
> ___
> Users mailing list
> Users@einsteintoolkit.org
> https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.einsteintoolkit.org%2Fmailman%2Flistinfo%2Fusers&data=05%7C01%7Cscupp1%40my.apsu.edu%7C4b812167a1ed418733ee08dbdef75282%7Cfe42fdbd8df8483a9700ff2693323482%7C0%7C0%7C638348926065507796%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WsfyXWcVGjvXgQ8a4qsrdhDyGST0KKWPD5CUe3jxMak%3D&reserved=0<http://lists.einsteintoolkit.org/mailman/listinfo/users>
___
Users mailing list
Users@einsteintoolkit.org
https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.einsteintoolkit.org%2Fmailman%2Flistinfo%2Fusers&data=05%7C01%7Cscupp1%40my.apsu.edu%7C4b812167a1ed418733ee08dbdef75282%7Cfe42fdbd8df8483a9700ff2693323482%7C0%7C0%7C638348926065507796%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WsfyXWcVGjvXgQ8a4qsrdhDyGST0KKWPD5CUe3jxMak%3D&reserved=0<http://lists.einsteintoolkit.org/mailman/listinfo/users>
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] meeting minutes for 2023-10-19

[Steven Brandt]

I think the gallery examples differs from previous weeks? I think I was 
trading BBH with some other test?


--Steve

On 10/19/2023 9:35 AM, Steven R. Brandt wrote:

Present: Steve, Peter, Sam, Zach, Leo

Release
    Gallery Examples
    - TOV is Peter
    - BBH is Steve
    - Roland might have students for the other three?
    Tests
    - Need to be run
    - Clusters:
    Stampede2 isn't listed, and Delta and Anvil are added.
    - Release name: Lise Meitner 
https://en.wikipedia.org/wiki/Lise_Meitner

    - Autogenerated codes need to be regenerated, Leo will do it

Mailing list moderation
    - Remind Roland to send Peter the password

Unanswered question on the mailing list
    - Need answer for Enzo. Roland was going to answer?
    - Ticket 2749: Leo says he fixed it.
    - Ticket 2647: One of Zach's students is looking at that in Grail 
(sp)

    - Ticket 2609: Steve thinks he approved PR for Roland

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] meeting minutes for 2023-10-19

[Gabriele Bozzola]

Hi Sam,

I can confirm that if I set OMP_NUM_TRHEADS=1 the test passes.

Best,
Gabriele

On Thu, Nov 2, 2023 at 2:35 PM Cupp, Samuel D.  wrote:

> Hi Gabriele,
>I've been experimenting with this, and I've reproduced this issue on a
> different machine. In my case, it runs with 1 OMP thread but has the
> segfault with 2. Can you run that test parfile
> (repos/GRHayLET/GRHayLHDX/test/Balsara0.par) without OMP? If this is the
> source, it will help narrow down the problem. I can run the test with
> multiple threads on my local machine though, so I'm not sure why OMP only
> causes a segfault sometimes.
>
>Samuel Cupp
>Postdoctoral Researcher
>Department of Physics
>University of Idaho
> --
> *From:* Gabriele Bozzola 
> *Sent:* Sunday, October 29, 2023 6:16 PM
> *To:* Cupp, Samuel D. 
> *Cc:* Steven R. Brandt ; users@einsteintoolkit.org <
> users@einsteintoolkit.org>
> *Subject:* Re: [Users] meeting minutes for 2023-10-19
>
> Hi Sam,
>
> I verified that the testsuite passes if I remove the GRHayL thorns.
> Other than that, ET seems to work fine.
>
> Attached is the cfg file I used to compile ET on Anvil.
>
> Best,
> Gabriele
>
> On Thu, Oct 26, 2023 at 12:45 PM Cupp, Samuel D. 
> wrote:
>
> Hi Gabriele,
>We discussed this in the call this morning, and there's a few things we
> can try. First, it would be helpful if you created a ticket so we can track
> progress on the issue. This is especially true since the testsuite
> shouldn't hang if a test fails. The expected behavior would be for it to
> continue with testing, but for some reason it didn't. Also, it might help
> to remove GRHayLHDX from the thornlist, recompile, and see if that is the
> only test failing. Knowing if this is specific to the thorn or a broader
> issue would help diagnose the source of the problem.
>
>Samuel Cupp
>Postdoctoral Researcher
>Department of Physics
>University of Idaho
> --
> *From:* Gabriele Bozzola 
> *Sent:* Tuesday, October 24, 2023 1:27 PM
> *To:* Cupp, Samuel D. 
> *Cc:* Steven R. Brandt ; users@einsteintoolkit.org <
> users@einsteintoolkit.org>
> *Subject:* Re: [Users] meeting minutes for 2023-10-19
>
> Hi Sam,
>
> This is just CPU. The .out for the entire testsuite was attached to my
> first email.
>
> Best,
> Gabriele
>
>
>
> On Tue, Oct 24, 2023 at 12:14 PM Cupp, Samuel D. 
> wrote:
>
> Do you know if any other CarpetX tests fail? Also, is this using gpus or
> cpus?
>
>Samuel Cupp
>Postdoctoral Researcher
>Department of Physics
>    University of Idaho
> --
> *From:* Gabriele Bozzola 
> *Sent:* Tuesday, October 24, 2023 10:07 AM
> *To:* Cupp, Samuel D. 
> *Cc:* Steven R. Brandt ; users@einsteintoolkit.org <
> users@einsteintoolkit.org>
> *Subject:* Re: [Users] meeting minutes for 2023-10-19
>
> Hi Sam,
>
> I pulled the latest version. Tests are not failing anymore, but looking at
> the .out, it seems that the Balsara0 test fails and stalls execution of the
> entire testsuite.
>
> I am attaching the .out for the test.
>
> Best,
> Gabriele
>
> On Fri, Oct 20, 2023 at 2:32 PM Cupp, Samuel D. 
> wrote:
>
> Hi Gabriele,
>It looks like some of the failures are in GRHayLHD. I've been having
> trouble getting consistent output with nproc>1. The data doesn't change, it
> just gets rearranged in the datafile. I made changes to the tests this
> week, so could you rerun the GRHayLHD tests and tell me if they fail? If
> they do, I'll change the tests so that they only run for nproc=1.
>
> I don't know why it would take that long to run, however. Nothing in the
> ET CI suggests it should take that long, best I can tell. Do you have an
> idea of which test is taking so long?
>
>Samuel Cupp
>Postdoctoral Researcher
>Department of Physics
>University of Idaho
> --
> *From:* Users  on behalf of Gabriele
> Bozzola 
> *Sent:* Thursday, October 19, 2023 5:46 PM
> *To:* Steven R. Brandt 
> *Cc:* users@einsteintoolkit.org 
> *Subject:* Re: [Users] meeting minutes for 2023-10-19
>
> Hello,
>
> Is there a tested configuration for anvil?
>
> I compiled it one last weekend and ran the tests with the master branch. I
> found a few failures in the
> MPI runs, and the test suite does not complete even with a walltime of 6
> hours.
>
> I am attaching the output.
>
> Best,
> Gabriele
>
> On Thu, Oct 19, 2023 at 7:36 AM Steven R. Brandt 
> wrote:
>
> Present: Steve, Peter, Sam, Zach, L

Re: [Users] meeting minutes for 2023-10-19

[Cupp, Samuel D.]

Hi Gabriele,
   I've been experimenting with this, and I've reproduced this issue on a 
different machine. In my case, it runs with 1 OMP thread but has the segfault 
with 2. Can you run that test parfile 
(repos/GRHayLET/GRHayLHDX/test/Balsara0.par) without OMP? If this is the 
source, it will help narrow down the problem. I can run the test with multiple 
threads on my local machine though, so I'm not sure why OMP only causes a 
segfault sometimes.

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho

From: Gabriele Bozzola 
Sent: Sunday, October 29, 2023 6:16 PM
To: Cupp, Samuel D. 
Cc: Steven R. Brandt ; users@einsteintoolkit.org 

Subject: Re: [Users] meeting minutes for 2023-10-19

Hi Sam,

I verified that the testsuite passes if I remove the GRHayL thorns.
Other than that, ET seems to work fine.

Attached is the cfg file I used to compile ET on Anvil.

Best,
Gabriele

On Thu, Oct 26, 2023 at 12:45 PM Cupp, Samuel D. 
mailto:scu...@my.apsu.edu>> wrote:
Hi Gabriele,
   We discussed this in the call this morning, and there's a few things we can 
try. First, it would be helpful if you created a ticket so we can track 
progress on the issue. This is especially true since the testsuite shouldn't 
hang if a test fails. The expected behavior would be for it to continue with 
testing, but for some reason it didn't. Also, it might help to remove GRHayLHDX 
from the thornlist, recompile, and see if that is the only test failing. 
Knowing if this is specific to the thorn or a broader issue would help diagnose 
the source of the problem.

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho

From: Gabriele Bozzola 
mailto:bozzola.gabri...@gmail.com>>
Sent: Tuesday, October 24, 2023 1:27 PM
To: Cupp, Samuel D. mailto:scu...@my.apsu.edu>>
Cc: Steven R. Brandt mailto:sbra...@cct.lsu.edu>>; 
users@einsteintoolkit.org<mailto:users@einsteintoolkit.org> 
mailto:users@einsteintoolkit.org>>
Subject: Re: [Users] meeting minutes for 2023-10-19

Hi Sam,

This is just CPU. The .out for the entire testsuite was attached to my first 
email.

Best,
Gabriele



On Tue, Oct 24, 2023 at 12:14 PM Cupp, Samuel D. 
mailto:scu...@my.apsu.edu>> wrote:
Do you know if any other CarpetX tests fail? Also, is this using gpus or cpus?

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho

From: Gabriele Bozzola 
mailto:bozzola.gabri...@gmail.com>>
Sent: Tuesday, October 24, 2023 10:07 AM
To: Cupp, Samuel D. mailto:scu...@my.apsu.edu>>
Cc: Steven R. Brandt mailto:sbra...@cct.lsu.edu>>; 
users@einsteintoolkit.org<mailto:users@einsteintoolkit.org> 
mailto:users@einsteintoolkit.org>>
Subject: Re: [Users] meeting minutes for 2023-10-19

Hi Sam,

I pulled the latest version. Tests are not failing anymore, but looking at the 
.out, it seems that the Balsara0 test fails and stalls execution of the entire 
testsuite.

I am attaching the .out for the test.

Best,
Gabriele

On Fri, Oct 20, 2023 at 2:32 PM Cupp, Samuel D. 
mailto:scu...@my.apsu.edu>> wrote:
Hi Gabriele,
   It looks like some of the failures are in GRHayLHD. I've been having trouble 
getting consistent output with nproc>1. The data doesn't change, it just gets 
rearranged in the datafile. I made changes to the tests this week, so could you 
rerun the GRHayLHD tests and tell me if they fail? If they do, I'll change the 
tests so that they only run for nproc=1.

I don't know why it would take that long to run, however. Nothing in the ET CI 
suggests it should take that long, best I can tell. Do you have an idea of 
which test is taking so long?

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho

From: Users 
mailto:users-boun...@einsteintoolkit.org>> 
on behalf of Gabriele Bozzola 
mailto:bozzola.gabri...@gmail.com>>
Sent: Thursday, October 19, 2023 5:46 PM
To: Steven R. Brandt mailto:sbra...@cct.lsu.edu>>
Cc: users@einsteintoolkit.org<mailto:users@einsteintoolkit.org> 
mailto:users@einsteintoolkit.org>>
Subject: Re: [Users] meeting minutes for 2023-10-19

Hello,

Is there a tested configuration for anvil?

I compiled it one last weekend and ran the tests with the master branch. I 
found a few failures in the
MPI runs, and the test suite does not complete even with a walltime of 6 hours.

I am attaching the output.

Best,
Gabriele

On Thu, Oct 19, 2023 at 7:36 AM Steven R. Brandt 
mailto:sbra...@cct.lsu.edu>> wrote:
Present: Steve, Peter, Sam, Zach, Leo

Release
 Gallery Examples
 - TOV is Peter
 - BBH is Steve
 - Roland might have students for the other three?
 Tests
 - Need to be run
 - Clusters:
 Stampede2 isn&

[Users] Meeting minutes for 2023-11-02

[Leo Rosa Werneck]

Hi all,

Here are the minutes for today’s meeting:

Chair: Sam
Minutes: Leo
Present: Peter, Sam, Roland, Steve, Zach, Leo, Agnesh Pandey

* ET release
  - Release name already decided
  - Gallery example runners
o BNS: Steve
o BBH: Debrah? Roland will ask
o TOV: Peter
o Scalar wave: pending
  - Sam will try to compile on Deep Bayou to make sure everything is working
with GPUs (CarpetX)
o Some of the GRHayL thorns are failing even on CPUs; Sam will look into it
  - Further compilations test are running at the moment

* ET testing
  - Deep Bayou: Steve, SuperMIC
  - Sunrise: Peter
  - Frontera: Leo

* Contributing to Springer book
  - Roland shared more information with those involved; no other news

* Unanswered questions on the mailing list
  - Roland will respond to the user having issues on Macs and Expanse

* Zach asked: was the toolkit compiled on arm64 CPUs? Yes, you should be able to
  compile it on Raspberry Pis. Zach is trying to compile on his phone.

* A user had a question about the ET no longer working on his machine, but the
  issue could not be resolved. He will join the meeting again next week so we
  can further discuss the issues they are having.

* Open tickets
  ---

* Tickets ready for review
  - Only 4 remain!
  - Work has progressing on improving McLachlin
  - No more progress to report

Next chair: Leo
Next minutes: Roland

Cheers,
Leo


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] meeting minutes for 2023-10-19

[Gabriele Bozzola]

Hi Sam,

I verified that the testsuite passes if I remove the GRHayL thorns.
Other than that, ET seems to work fine.

Attached is the cfg file I used to compile ET on Anvil.

Best,
Gabriele

On Thu, Oct 26, 2023 at 12:45 PM Cupp, Samuel D.  wrote:

> Hi Gabriele,
>We discussed this in the call this morning, and there's a few things we
> can try. First, it would be helpful if you created a ticket so we can track
> progress on the issue. This is especially true since the testsuite
> shouldn't hang if a test fails. The expected behavior would be for it to
> continue with testing, but for some reason it didn't. Also, it might help
> to remove GRHayLHDX from the thornlist, recompile, and see if that is the
> only test failing. Knowing if this is specific to the thorn or a broader
> issue would help diagnose the source of the problem.
>
>Samuel Cupp
>Postdoctoral Researcher
>Department of Physics
>University of Idaho
> --
> *From:* Gabriele Bozzola 
> *Sent:* Tuesday, October 24, 2023 1:27 PM
> *To:* Cupp, Samuel D. 
> *Cc:* Steven R. Brandt ; users@einsteintoolkit.org <
> users@einsteintoolkit.org>
> *Subject:* Re: [Users] meeting minutes for 2023-10-19
>
> Hi Sam,
>
> This is just CPU. The .out for the entire testsuite was attached to my
> first email.
>
> Best,
> Gabriele
>
>
>
> On Tue, Oct 24, 2023 at 12:14 PM Cupp, Samuel D. 
> wrote:
>
> Do you know if any other CarpetX tests fail? Also, is this using gpus or
> cpus?
>
>Samuel Cupp
>Postdoctoral Researcher
>Department of Physics
>University of Idaho
> --
> *From:* Gabriele Bozzola 
> *Sent:* Tuesday, October 24, 2023 10:07 AM
> *To:* Cupp, Samuel D. 
> *Cc:* Steven R. Brandt ; users@einsteintoolkit.org <
> users@einsteintoolkit.org>
> *Subject:* Re: [Users] meeting minutes for 2023-10-19
>
> Hi Sam,
>
> I pulled the latest version. Tests are not failing anymore, but looking at
> the .out, it seems that the Balsara0 test fails and stalls execution of the
> entire testsuite.
>
> I am attaching the .out for the test.
>
> Best,
> Gabriele
>
> On Fri, Oct 20, 2023 at 2:32 PM Cupp, Samuel D. 
> wrote:
>
> Hi Gabriele,
>It looks like some of the failures are in GRHayLHD. I've been having
> trouble getting consistent output with nproc>1. The data doesn't change, it
> just gets rearranged in the datafile. I made changes to the tests this
> week, so could you rerun the GRHayLHD tests and tell me if they fail? If
> they do, I'll change the tests so that they only run for nproc=1.
>
> I don't know why it would take that long to run, however. Nothing in the
> ET CI suggests it should take that long, best I can tell. Do you have an
> idea of which test is taking so long?
>
>Samuel Cupp
>Postdoctoral Researcher
>Department of Physics
>University of Idaho
> --
> *From:* Users  on behalf of Gabriele
> Bozzola 
> *Sent:* Thursday, October 19, 2023 5:46 PM
> *To:* Steven R. Brandt 
> *Cc:* users@einsteintoolkit.org 
> *Subject:* Re: [Users] meeting minutes for 2023-10-19
>
> Hello,
>
> Is there a tested configuration for anvil?
>
> I compiled it one last weekend and ran the tests with the master branch. I
> found a few failures in the
> MPI runs, and the test suite does not complete even with a walltime of 6
> hours.
>
> I am attaching the output.
>
> Best,
> Gabriele
>
> On Thu, Oct 19, 2023 at 7:36 AM Steven R. Brandt 
> wrote:
>
> Present: Steve, Peter, Sam, Zach, Leo
>
> Release
>  Gallery Examples
>  - TOV is Peter
>  - BBH is Steve
>  - Roland might have students for the other three?
>  Tests
>  - Need to be run
>  - Clusters:
>  Stampede2 isn't listed, and Delta and Anvil are added.
>  - Release name: Lise Meitner
> https://en.wikipedia.org/wiki/Lise_Meitner
>  - Autogenerated codes need to be regenerated, Leo will do it
>
> Mailing list moderation
>  - Remind Roland to send Peter the password
>
> Unanswered question on the mailing list
>  - Need answer for Enzo. Roland was going to answer?
>  - Ticket 2749: Leo says he fixed it.
>  - Ticket 2647: One of Zach's students is looking at that in Grail (sp)
>  - Ticket 2609: Steve thinks he approved PR for Roland
>
> ___
> Users mailing list
> Users@einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
>
>


anvil.cfg
Description: Binary data
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] meeting minutes for 2023-10-19

[Cupp, Samuel D.]

Hi Gabriele,
   We discussed this in the call this morning, and there's a few things we can 
try. First, it would be helpful if you created a ticket so we can track 
progress on the issue. This is especially true since the testsuite shouldn't 
hang if a test fails. The expected behavior would be for it to continue with 
testing, but for some reason it didn't. Also, it might help to remove GRHayLHDX 
from the thornlist, recompile, and see if that is the only test failing. 
Knowing if this is specific to the thorn or a broader issue would help diagnose 
the source of the problem.

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho

From: Gabriele Bozzola 
Sent: Tuesday, October 24, 2023 1:27 PM
To: Cupp, Samuel D. 
Cc: Steven R. Brandt ; users@einsteintoolkit.org 

Subject: Re: [Users] meeting minutes for 2023-10-19

Hi Sam,

This is just CPU. The .out for the entire testsuite was attached to my first 
email.

Best,
Gabriele



On Tue, Oct 24, 2023 at 12:14 PM Cupp, Samuel D. 
mailto:scu...@my.apsu.edu>> wrote:
Do you know if any other CarpetX tests fail? Also, is this using gpus or cpus?

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho

From: Gabriele Bozzola 
mailto:bozzola.gabri...@gmail.com>>
Sent: Tuesday, October 24, 2023 10:07 AM
To: Cupp, Samuel D. mailto:scu...@my.apsu.edu>>
Cc: Steven R. Brandt mailto:sbra...@cct.lsu.edu>>; 
users@einsteintoolkit.org<mailto:users@einsteintoolkit.org> 
mailto:users@einsteintoolkit.org>>
Subject: Re: [Users] meeting minutes for 2023-10-19

Hi Sam,

I pulled the latest version. Tests are not failing anymore, but looking at the 
.out, it seems that the Balsara0 test fails and stalls execution of the entire 
testsuite.

I am attaching the .out for the test.

Best,
Gabriele

On Fri, Oct 20, 2023 at 2:32 PM Cupp, Samuel D. 
mailto:scu...@my.apsu.edu>> wrote:
Hi Gabriele,
   It looks like some of the failures are in GRHayLHD. I've been having trouble 
getting consistent output with nproc>1. The data doesn't change, it just gets 
rearranged in the datafile. I made changes to the tests this week, so could you 
rerun the GRHayLHD tests and tell me if they fail? If they do, I'll change the 
tests so that they only run for nproc=1.

I don't know why it would take that long to run, however. Nothing in the ET CI 
suggests it should take that long, best I can tell. Do you have an idea of 
which test is taking so long?

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho

From: Users 
mailto:users-boun...@einsteintoolkit.org>> 
on behalf of Gabriele Bozzola 
mailto:bozzola.gabri...@gmail.com>>
Sent: Thursday, October 19, 2023 5:46 PM
To: Steven R. Brandt mailto:sbra...@cct.lsu.edu>>
Cc: users@einsteintoolkit.org<mailto:users@einsteintoolkit.org> 
mailto:users@einsteintoolkit.org>>
Subject: Re: [Users] meeting minutes for 2023-10-19

Hello,

Is there a tested configuration for anvil?

I compiled it one last weekend and ran the tests with the master branch. I 
found a few failures in the
MPI runs, and the test suite does not complete even with a walltime of 6 hours.

I am attaching the output.

Best,
Gabriele

On Thu, Oct 19, 2023 at 7:36 AM Steven R. Brandt 
mailto:sbra...@cct.lsu.edu>> wrote:
Present: Steve, Peter, Sam, Zach, Leo

Release
 Gallery Examples
 - TOV is Peter
 - BBH is Steve
 - Roland might have students for the other three?
 Tests
 - Need to be run
 - Clusters:
 Stampede2 isn't listed, and Delta and Anvil are added.
 - Release name: Lise Meitner https://en.wikipedia.org/wiki/Lise_Meitner
 - Autogenerated codes need to be regenerated, Leo will do it

Mailing list moderation
 - Remind Roland to send Peter the password

Unanswered question on the mailing list
 - Need answer for Enzo. Roland was going to answer?
 - Ticket 2749: Leo says he fixed it.
 - Ticket 2647: One of Zach's students is looking at that in Grail (sp)
 - Ticket 2609: Steve thinks he approved PR for Roland

___
Users mailing list
Users@einsteintoolkit.org<mailto:Users@einsteintoolkit.org>
http://lists.einsteintoolkit.org/mailman/listinfo/users
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting Minutes for 2023-10-26

[Cupp, Samuel D.]

Present: Sam, Peter, Roland, Steve, Zach, Leo

GRHayLHDX test failing on Anvil due to segfault
running on cpu
a ticket would be helpful
Testsuite shouldn't hang once a test fails
Try removing GRHayLHDX and see if other tests fail

Gallery runners:
Deborah might do BBH, Steve will switch to BNS

Request came in July for contribution to Springer book on GRMHD codes in ET, 
but it was never discussed
People representing several codes (GR-Athena, SPHINCS BSSN, KHARMA) have agreed 
to contribute other sections to this book
GRHydro (Steve?) and IllinoisGRMHD (Sam, Leo) would be in an ET section; should 
also discuss the spacetime codes (ML_BSSN, Baikal)

Unanswered emails:
Help with GW150914 Black Hole simulation
Problem: vtk files are missing in simulation output
Gallery example sets vtk output to behave the same as 3D output, but 3D output 
is disabled.

Tickets:
kuibit currently has an issue with the most recent version of tikzplotlib. It 
appears to be a bug with that package and not kuibit itself; Gabriele is 
working on a workaround

#963
Peter implemented Zach's ML_BSSN improvements into SPHINCS BSSN, and it appears 
to have significant reduction of constraint violations (still running)
May release should include this fix for both ML_BSSN and Baikal

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] meeting minutes for 2023-10-19

[Gabriele Bozzola]

Hi Sam,

This is just CPU. The .out for the entire testsuite was attached to my
first email.

Best,
Gabriele



On Tue, Oct 24, 2023 at 12:14 PM Cupp, Samuel D.  wrote:

> Do you know if any other CarpetX tests fail? Also, is this using gpus or
> cpus?
>
>Samuel Cupp
>Postdoctoral Researcher
>Department of Physics
>University of Idaho
> --
> *From:* Gabriele Bozzola 
> *Sent:* Tuesday, October 24, 2023 10:07 AM
> *To:* Cupp, Samuel D. 
> *Cc:* Steven R. Brandt ; users@einsteintoolkit.org <
> users@einsteintoolkit.org>
> *Subject:* Re: [Users] meeting minutes for 2023-10-19
>
> Hi Sam,
>
> I pulled the latest version. Tests are not failing anymore, but looking at
> the .out, it seems that the Balsara0 test fails and stalls execution of the
> entire testsuite.
>
> I am attaching the .out for the test.
>
> Best,
> Gabriele
>
> On Fri, Oct 20, 2023 at 2:32 PM Cupp, Samuel D. 
> wrote:
>
> Hi Gabriele,
>It looks like some of the failures are in GRHayLHD. I've been having
> trouble getting consistent output with nproc>1. The data doesn't change, it
> just gets rearranged in the datafile. I made changes to the tests this
> week, so could you rerun the GRHayLHD tests and tell me if they fail? If
> they do, I'll change the tests so that they only run for nproc=1.
>
> I don't know why it would take that long to run, however. Nothing in the
> ET CI suggests it should take that long, best I can tell. Do you have an
> idea of which test is taking so long?
>
>Samuel Cupp
>Postdoctoral Researcher
>Department of Physics
>University of Idaho
> --
> *From:* Users  on behalf of Gabriele
> Bozzola 
> *Sent:* Thursday, October 19, 2023 5:46 PM
> *To:* Steven R. Brandt 
> *Cc:* users@einsteintoolkit.org 
> *Subject:* Re: [Users] meeting minutes for 2023-10-19
>
> Hello,
>
> Is there a tested configuration for anvil?
>
> I compiled it one last weekend and ran the tests with the master branch. I
> found a few failures in the
> MPI runs, and the test suite does not complete even with a walltime of 6
> hours.
>
> I am attaching the output.
>
> Best,
> Gabriele
>
> On Thu, Oct 19, 2023 at 7:36 AM Steven R. Brandt 
> wrote:
>
> Present: Steve, Peter, Sam, Zach, Leo
>
> Release
>  Gallery Examples
>  - TOV is Peter
>  - BBH is Steve
>  - Roland might have students for the other three?
>  Tests
>  - Need to be run
>  - Clusters:
>  Stampede2 isn't listed, and Delta and Anvil are added.
>  - Release name: Lise Meitner
> https://en.wikipedia.org/wiki/Lise_Meitner
>  - Autogenerated codes need to be regenerated, Leo will do it
>
> Mailing list moderation
>  - Remind Roland to send Peter the password
>
> Unanswered question on the mailing list
>  - Need answer for Enzo. Roland was going to answer?
>  - Ticket 2749: Leo says he fixed it.
>  - Ticket 2647: One of Zach's students is looking at that in Grail (sp)
>  - Ticket 2609: Steve thinks he approved PR for Roland
>
> ___
> Users mailing list
> Users@einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
>
>
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] meeting minutes for 2023-10-19

[Cupp, Samuel D.]

Do you know if any other CarpetX tests fail? Also, is this using gpus or cpus?

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho

From: Gabriele Bozzola 
Sent: Tuesday, October 24, 2023 10:07 AM
To: Cupp, Samuel D. 
Cc: Steven R. Brandt ; users@einsteintoolkit.org 

Subject: Re: [Users] meeting minutes for 2023-10-19

Hi Sam,

I pulled the latest version. Tests are not failing anymore, but looking at the 
.out, it seems that the Balsara0 test fails and stalls execution of the entire 
testsuite.

I am attaching the .out for the test.

Best,
Gabriele

On Fri, Oct 20, 2023 at 2:32 PM Cupp, Samuel D. 
mailto:scu...@my.apsu.edu>> wrote:
Hi Gabriele,
   It looks like some of the failures are in GRHayLHD. I've been having trouble 
getting consistent output with nproc>1. The data doesn't change, it just gets 
rearranged in the datafile. I made changes to the tests this week, so could you 
rerun the GRHayLHD tests and tell me if they fail? If they do, I'll change the 
tests so that they only run for nproc=1.

I don't know why it would take that long to run, however. Nothing in the ET CI 
suggests it should take that long, best I can tell. Do you have an idea of 
which test is taking so long?

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho

From: Users 
mailto:users-boun...@einsteintoolkit.org>> 
on behalf of Gabriele Bozzola 
mailto:bozzola.gabri...@gmail.com>>
Sent: Thursday, October 19, 2023 5:46 PM
To: Steven R. Brandt mailto:sbra...@cct.lsu.edu>>
Cc: users@einsteintoolkit.org<mailto:users@einsteintoolkit.org> 
mailto:users@einsteintoolkit.org>>
Subject: Re: [Users] meeting minutes for 2023-10-19

Hello,

Is there a tested configuration for anvil?

I compiled it one last weekend and ran the tests with the master branch. I 
found a few failures in the
MPI runs, and the test suite does not complete even with a walltime of 6 hours.

I am attaching the output.

Best,
Gabriele

On Thu, Oct 19, 2023 at 7:36 AM Steven R. Brandt 
mailto:sbra...@cct.lsu.edu>> wrote:
Present: Steve, Peter, Sam, Zach, Leo

Release
 Gallery Examples
 - TOV is Peter
 - BBH is Steve
 - Roland might have students for the other three?
 Tests
 - Need to be run
 - Clusters:
 Stampede2 isn't listed, and Delta and Anvil are added.
 - Release name: Lise Meitner https://en.wikipedia.org/wiki/Lise_Meitner
 - Autogenerated codes need to be regenerated, Leo will do it

Mailing list moderation
 - Remind Roland to send Peter the password

Unanswered question on the mailing list
 - Need answer for Enzo. Roland was going to answer?
 - Ticket 2749: Leo says he fixed it.
 - Ticket 2647: One of Zach's students is looking at that in Grail (sp)
 - Ticket 2609: Steve thinks he approved PR for Roland

___
Users mailing list
Users@einsteintoolkit.org<mailto:Users@einsteintoolkit.org>
http://lists.einsteintoolkit.org/mailman/listinfo/users
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] meeting minutes for 2023-10-19

[Gabriele Bozzola]

Hi Sam,

I pulled the latest version. Tests are not failing anymore, but looking at
the .out, it seems that the Balsara0 test fails and stalls execution of the
entire testsuite.

I am attaching the .out for the test.

Best,
Gabriele

On Fri, Oct 20, 2023 at 2:32 PM Cupp, Samuel D.  wrote:

> Hi Gabriele,
>It looks like some of the failures are in GRHayLHD. I've been having
> trouble getting consistent output with nproc>1. The data doesn't change, it
> just gets rearranged in the datafile. I made changes to the tests this
> week, so could you rerun the GRHayLHD tests and tell me if they fail? If
> they do, I'll change the tests so that they only run for nproc=1.
>
> I don't know why it would take that long to run, however. Nothing in the
> ET CI suggests it should take that long, best I can tell. Do you have an
> idea of which test is taking so long?
>
>Samuel Cupp
>Postdoctoral Researcher
>Department of Physics
>University of Idaho
> --
> *From:* Users  on behalf of Gabriele
> Bozzola 
> *Sent:* Thursday, October 19, 2023 5:46 PM
> *To:* Steven R. Brandt 
> *Cc:* users@einsteintoolkit.org 
> *Subject:* Re: [Users] meeting minutes for 2023-10-19
>
> Hello,
>
> Is there a tested configuration for anvil?
>
> I compiled it one last weekend and ran the tests with the master branch. I
> found a few failures in the
> MPI runs, and the test suite does not complete even with a walltime of 6
> hours.
>
> I am attaching the output.
>
> Best,
> Gabriele
>
> On Thu, Oct 19, 2023 at 7:36 AM Steven R. Brandt 
> wrote:
>
> Present: Steve, Peter, Sam, Zach, Leo
>
> Release
>  Gallery Examples
>  - TOV is Peter
>  - BBH is Steve
>  - Roland might have students for the other three?
>  Tests
>  - Need to be run
>  - Clusters:
>  Stampede2 isn't listed, and Delta and Anvil are added.
>  - Release name: Lise Meitner
> https://en.wikipedia.org/wiki/Lise_Meitner
>  - Autogenerated codes need to be regenerated, Leo will do it
>
> Mailing list moderation
>  - Remind Roland to send Peter the password
>
> Unanswered question on the mailing list
>  - Need answer for Enzo. Roland was going to answer?
>  - Ticket 2749: Leo says he fixed it.
>  - Ticket 2647: One of Zach's students is looking at that in Grail (sp)
>  - Ticket 2609: Steve thinks he approved PR for Roland
>
> ___
> Users mailing list
> Users@einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
>
>
/anvil/scratch/x-bozzola/simulations/ettests_2proc_again/output-/TEST/sim/GRHayLHDX
Preparing:
+ set -e
+ cd output--active
+ module list
++ /opt/lmod/lmod/libexec/lmod bash list

Currently Loaded Modules:
  1) xalt/2.10.45 (S)   7) gcc/11.2.0  13) boost/1.74.0
  2) modtree/cpu8) numactl/2.0.14  14) papi/6.0.0.1
  3) libfabric/1.12.0   9) libszip/2.1.1   15) fftw/3.3.8
  4) gmp/6.2.1 10) openmpi/4.0.6   16) hdf5/1.10.7
  5) mpfr/4.0.211) gsl/2.4 17) openblas/0.3.17
  6) mpc/1.1.0 12) zlib/1.2.11 18) python/3.9.5

  Where:
   S:  Module is Sticky, requires --force to unload or purge

 

+ eval 'MODULEPATH=/opt/spack/cpu-20211007/openmpi/4.0.6-3navcwb/gcc/11.2.0:/opt/spack/cpu-20211007/gcc/11.2.0:/opt/spack/cpu/Core;' export 'MODULEPATH;' '_ModuleTable001_=X01vZHVsZVRhYmxlXyA9IHsKTVR2ZXJzaW9uID0gMywKY19yZWJ1aWxkVGltZSA9IGZhbHNlLApjX3Nob3J0VGltZSA9IGZhbHNlLApkZXB0aFQgPSB7fSwKZmFtaWx5ID0gewpjb21waWxlciA9ICJnY2MiLAptcGkgPSAib3Blbm1waSIsCnB5dGhvbiA9ICJweXRob24iLAp9LAptVCA9IHsKYm9vc3QgPSB7CmZuID0gIi9vcHQvc3BhY2svY3B1LTIwMjExMDA3L29wZW5tcGkvNC4wLjYtM25hdmN3Yi9nY2MvMTEuMi4wL2Jvb3N0LzEuNzQuMC5sdWEiLApmdWxsTmFtZSA9ICJib29zdC8xLjc0LjAiLApsb2FkT3JkZXIgPSAxMywKcHJvcFQgPSB7fSwKc3RhY2tEZXB0aCA9IDAsCnN0YXR1cyA9ICJhY3RpdmUiLAp1c2VyTmFtZSA9ICJib29zdC8xLjc0LjAiLAp3ViA9ICJeMDAw;' export '_ModuleTable001_;' '_ModuleTable002_=MDAwMDEuMDAwMDAwMDc0Lip6ZmluYWwiLAp9LApmZnR3ID0gewpmbiA9ICIvb3B0L3NwYWNrL2NwdS0yMDIxMTAwNy9vcGVubXBpLzQuMC42LTNuYXZjd2IvZ2NjLzExLjIuMC9mZnR3LzMuMy44Lmx1YSIsCmZ1bGxOYW1lID0gImZmdHcvMy4zLjgiLApsb2FkT3JkZXIgPSAxNSwKcHJvcFQgPSB7fSwKc3RhY2tEZXB0aCA9IDAsCnN0YXR1cyA9ICJhY3RpdmUiLAp1c2VyTmFtZSA9ICJmZnR3LzMuMy44IiwKd1YgPSAiXjAwMDAwMDAzLjAwMDAwMDAwMy4wMDAwMDAwMDguKnpmaW5hbCIsCn0sCmdjYyA9IHsKZm4gPSAiL29wdC9zcGFjay9jcHUvQ29yZS9nY2MvMTEuMi4wLmx1YSIsCmZ1bGxOYW1lID0gImdjYy8xMS4yLjAiLApsb2FkT3JkZXIgPSA3LApwcm9wVCA9IHt9LApz;' export 

Re: [Users] meeting minutes for 2023-10-19

[Cupp, Samuel D.]

Hi Gabriele,
   It looks like some of the failures are in GRHayLHD. I've been having trouble 
getting consistent output with nproc>1. The data doesn't change, it just gets 
rearranged in the datafile. I made changes to the tests this week, so could you 
rerun the GRHayLHD tests and tell me if they fail? If they do, I'll change the 
tests so that they only run for nproc=1.

I don't know why it would take that long to run, however. Nothing in the ET CI 
suggests it should take that long, best I can tell. Do you have an idea of 
which test is taking so long?

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho

From: Users  on behalf of Gabriele Bozzola 

Sent: Thursday, October 19, 2023 5:46 PM
To: Steven R. Brandt 
Cc: users@einsteintoolkit.org 
Subject: Re: [Users] meeting minutes for 2023-10-19

Hello,

Is there a tested configuration for anvil?

I compiled it one last weekend and ran the tests with the master branch. I 
found a few failures in the
MPI runs, and the test suite does not complete even with a walltime of 6 hours.

I am attaching the output.

Best,
Gabriele

On Thu, Oct 19, 2023 at 7:36 AM Steven R. Brandt 
mailto:sbra...@cct.lsu.edu>> wrote:
Present: Steve, Peter, Sam, Zach, Leo

Release
 Gallery Examples
 - TOV is Peter
 - BBH is Steve
 - Roland might have students for the other three?
 Tests
 - Need to be run
 - Clusters:
 Stampede2 isn't listed, and Delta and Anvil are added.
 - Release name: Lise Meitner https://en.wikipedia.org/wiki/Lise_Meitner
 - Autogenerated codes need to be regenerated, Leo will do it

Mailing list moderation
 - Remind Roland to send Peter the password

Unanswered question on the mailing list
 - Need answer for Enzo. Roland was going to answer?
 - Ticket 2749: Leo says he fixed it.
 - Ticket 2647: One of Zach's students is looking at that in Grail (sp)
 - Ticket 2609: Steve thinks he approved PR for Roland

___
Users mailing list
Users@einsteintoolkit.org<mailto:Users@einsteintoolkit.org>
http://lists.einsteintoolkit.org/mailman/listinfo/users
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-10-19

[Steven R. Brandt]

Present: Steve, Peter, Sam, Zach, Leo

Release
    Gallery Examples
    - TOV is Peter
    - BBH is Steve
    - Roland might have students for the other three?
    Tests
    - Need to be run
    - Clusters:
    Stampede2 isn't listed, and Delta and Anvil are added.
    - Release name: Lise Meitner https://en.wikipedia.org/wiki/Lise_Meitner
    - Autogenerated codes need to be regenerated, Leo will do it

Mailing list moderation
    - Remind Roland to send Peter the password

Unanswered question on the mailing list
    - Need answer for Enzo. Roland was going to answer?
    - Ticket 2749: Leo says he fixed it.
    - Ticket 2647: One of Zach's students is looking at that in Grail (sp)
    - Ticket 2609: Steve thinks he approved PR for Roland

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-10-12

[Roland Haas]

Present: Roland, Steve, Peter, Zach

Testsuites
==

* Steve compiled code with nvcc and finds that some code tries to use
  GPU outside of CarpetX and fails
* need to verify on different machines
* this can be a blocker for the release, needs to be sorted out

ET release
==

* not updates from GRHayL
* SeedMagneticField is in master, no further updates
* Lucas is working on CarpetX documentation in branch and
  documentation.tex
** CarpetX still contains what looks like dead code, hard to tell what
is what though.
** would also need tests on AMD GPU system. Only Forntier system uses
AMD GPUs right now. Maybe solicit help from RIT who is using Frontier

CarpetX release
===

* worry of not yet fully at quality for ET due to incomplete review
* would favor inclusion as "preview" in release with caveat that API
  may change
** would want to include to keep up momentum of using CarpetX
* current only "science" doable with stock CarpetX included in toolkit
  is the WaveToy (SpaceTimeX and AsterX are not part of the release)

ET size
===

Steve brought up large size and time used to compile the full Einstein
Toolkit. Suggests using MakeThornList script to compile only what is
needed for parameter file instead of compiling the full toolkit all the
time.

* Zach agrees that default workflow should migrate that way
* this would not compile many of the more experimental code (eg
  CarpetX) for most users and reduce compilation issues
* Steve suggests to, by default, run MakeThornList on some typical
  parfiles to produce a "typical" thornlist to compile
* discussed what the exemplar parfiles should be: vacuum, hydro, Canuda
* want "desktop sized" and "cluster sized" parfiles
* also update tutorial notebooks

Unanswered email


*
  https://lists.einsteintoolkit.org/pipermail/users/2023-October/009083.html
  Zach will respond. Looks like out of memory on a personal laptop.

*
  https://lists.einsteintoolkit.org/pipermail/users/2023-October/009086.html
  Roland will respond. The vtk files are not included the small data
  available on Zenodo (says so in the website as well) nor produced by
  default by the GW150914.rpar file. To produce them one has to update
  the rpar file

Open tickets


*
  https://bitbucket.org/einsteintoolkit/tickets/issues/2759
  twopunctures-swap_xz-results-in-left Zach provided some more details
  on the issue described in the ticket. Should have actually rotated
  coordinates instead of swapping. Steve requests a pull request with
  documentation warning and call to CCTK_VINFO in the code. Roland
  argues against using CCTK_VWARN (one output per MPI rank).

Galley runners
==

claimed: BBH, TOV: Steve
unclaimed: BNS, Scalar Wave, Poisson Equation

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpJJkOvjFBFT.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2023-10-05

[Leo Rosa Werneck]

Dear all,

Here are today’s minutes.

Chair   : Peter
Minutes : Leo
Present : Zach, Steve, Sam, Roland, Peter, Leo


Release:

- Sam: every time a new thorn is added to the thornlist for checkout, the CI
  testing does not get that thorn, and so Roland or Steve has to manually update
  the CI. Steve: there are instructions on how to do it somewhere.

- SGRID: Liwei is reviewing.

- Sam pushed an update to the CarpetX thorns in GRHayL. GRHayL functions still
  lack the proper declaration of device functions in them, but Sam and Leo will
  work on that in the near future.

- Steve suggests that maybe the CarpetX thorns in GRHayL should be included in
  the next release instead, as they have not been shown to run on GPUs. Roland
  said that as long as the code compiles on a GPU build, it should be fine;
  having GPU support is not a requirement for inclusion. Steve says that even if
  we get everything compiled and properly declared, it likely that benchmark
  tests won't have been run, and that is something that we might want done
  before inclusion.

- Seed_Magnetic_Fields: Zach sent a review to Sam.

- CarpetX: Steve found some issues, but now has a working cfg file for Deep
  Bayou, which may be useful for other systems.

Unanswered questions on mailing list:

- Extract thorn issue: likely needs to activate additional time levels, change
  the prolongation type, or simply request the output to be generated only when
  all time levels are aligned. Leo responded.

Next chair   : Steve
Next Minutes : Roland

Chair/minutes rotation list, taking effect next week.
.-..-.
|  Date   | Chair  | Minutes |
.-..-.
| 10-12-23 | Steve  | Roland |
| 10-19-23 | Peter  | Steve  |
| 10-26-23 | Roland | Sam|
| 11-02-23 | Sam| Leo|
| 11-09-23 | Leo| Roland |
| 11-16-23 | Zach   | Steve  |
| ...  | ...| ...|
.-..-.

Cheers,
Leo


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-09-28

[Roland Haas]

Present: Roland, Steve, Peter, Leo, Sam, Zach

New thorns in release:

* deadline for review is Oct 11th
* Sam and Roland added mising GRHayL thorns to autoamted tests

SGRID importer:
* Sam eventually heard back from authors
* tried out provided thorn, fails since it is missing SGRID itself
* Roland mentioned new ExternalLibraries-SGRID thorn

GRHayL MHD thorns:
* Zach is looking will need some spare seconds

CarpetX review:
* Steve looked at some of the code, not much progress yet

ET release name:

* possible options are listed on
  https://docs.einsteintoolkit.org/et-docs/Release_Details#Release_names
* suggested name: Lise Meitner

Unanswered questions:
* no new emails

Open tickets:

* no new non-release related tickets
* icpc 19 cannot compile AMReX
  https://bitbucket.org/einsteintoolkit/tickets/issues/2758 Roland
  reports that this is still happening and a workaround may be possible
  but large. Liwei has tried a modern OneAPI based Intel compiler
  (clang based) on Fronterra and seems to have had success
* Peter and Zach will interface about
  https://bitbucket.org/einsteintoolkit/tickets/issues/963 improved ML
  accuracy
* no progress on
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2647/incorrect-weno-coefficient-in-grhydro-weno


chair next week: Peter
minutes next week: Leo


Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.


pgpuxo2fa8jhN.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2023-09-21

[Steven Brandt]

Present: Peter, Steve, Roland, Erik, Leo, Sam, Zach

Minutes: Steve
Chair: Peter

The release date is pushed back to Dec 6.

Update on the review of Grail (Peter looked at the test suite). There is 
an issue with initial data. Shock is exactly at a grid point which gives 
a rounding issue with intel vs. gnu.


Lucas will be doing the documentation for CarpetX in October.

The tests are not running in the test suite yet. They do get run once in 
a while (pull request).


Sameer Sende (friend of Steve's) offers a hardware lab with lots of 
different versions. We could use this for testing.


Intel 19.1.3.304 Sep 2020 on Deep Bayou fails to compile AMReX.

sgrid... is it going in?

Steve asks if we should use the MakeThornList script on the download 
page? Roland thinks we should add it to the new user's tutorial.


Update on Seed Magnetic Fields? Zach asks to be reminded in a week.

Test suite server has unreliable results? Still the case. Something is 
glitchy in detecting which tests fail. Balsara is still failing. Email 
is fixed. Should we email maintainers of thorns that are failing?


Unanswered questions: Peter forgot to reply to Haleigh McPherson.

Zach has been working on constraint damping and improving BSSN. Peter 
wants to use it to close out the ticket he is working on for McLachlan. 
He also wants to use it for Sphynx BSSN.


Leo fixed several WVU thorns following Roland's suggestion. Ticket 
should be closed.


NSIMD is not required by CarpetX, but is nice to have. NSIMD's install 
downloads something using requests.


2754 can probably be closed.

Do we still need HTTPD? Does it work at all?

Can have tiers of support, tier 1 is actively used, tier 3 moves to 
unsupported. How do people show active support, etc.


McLachlin has incorrect reads declaration. Ian doesn't have time to look 
at it. Roland has a patch for Kranc.


Zach says NRPy 2.0 is pip installable that works with Python 3.6 and up.

Next Chair: Leo
Next Minute Taker: Roland

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting Minutes for 2023-9-14

[Cupp, Samuel D.]

Present: Steve, Sam, Roland, Leo, Jose, Peter, Yosef

ET_2023_11:
CarpetX review
Still has a severe lack of documentation, and it is much harder to build 
successfully than Carpet

Roland: compilation failed on Expanse because the NSIMD external library thorn 
tries to download sleef (Issue 
#2757).
 This should be resolvable by including sleef in the thorn.

Steve: I think another thing that should be done before inclusion is cleaning 
up the dead code, which it has a lot of. There is some room to push back 
release into December, and we should consider extending the release window to 
get CarpetX into the release.

CarpetX thorns should be added to master thornlist to see what breaks on the 
different clusters.
--
LORENE fails to build with spack, and spack doesn't reliably work across 
multiple clusters with the same setup
--
GRHayL
needs more documentation on the wiki; add some more discussion to the thorn doc
GRHayLHD test failed on intel; Peter is looking into it
test also failed in automated testing due to misconfigured test.ccl
--
sgrid
Roland: Wolfgang and Michal are still working on getting the code working as a 
standalone thorn that can be directly checked out; not sure how far along they 
are or what dependencies their code has
--

Unanswered emails:
Peter will follow up with unanswered email from Hayley.

Next meeting (9/21):
Chair - Peter
Minute taker - Steve

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-09-07

[Roland Haas]

Present: Roland, Steve, Peter, Yosef, Sam, Johnny, Leo, Zach

ET release:

* deadline to include thorns in thornlist was yesterday, none have been
  added
** Sam asks for a reviewer to look at GRHayl thornlist, Steve will look
   at pull request
** CarpetX still has issues, Roland will fix LoopControl problem
** no updates from SGRID

Tickets:
* merging SeedMagneticField in
https://bitbucket.org/einsteintoolkit/tickets/issues/2746/make-seed_magnetic_fields-independent-of
 Sam has made some changes to push things forward

* https://bitbucket.org/einsteintoolkit/tickets/issues/2745 Zach and
  Sam on fence on whether to push for GRHayL in this release or the
  next to replace IllinoisGRMHD

*
  https://bitbucket.org/einsteintoolkit/tickets/issues/2756 Roland
  described issue being caused by E and P cores on Intel cpus and
  unexpected number of logical CPUs reported by the OS. Steve asked if
  there is an easy way to to identify them. Roland is not aware. Yosef
  asks if it is possible to limit to only one sort of cores. The ET
  currently tries to avoid HT but the issue here is really the uneven
  number of logical cpus compared to cores. Currenlty lscpu --extended
  can be used to identify the cores based on their maximum frequency,
  but it is not nice.

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2752/parameter-file-parser-generates-illegal-c
  causes issues since we copy the string context verbatim to C, then
  LaTeX then HTML code. Had a discussion about allowing everything and
  escaping, or listing forbidden characters, or listing allowed
  characters. Discussion favored escaping for C, LaTeX and HTML output.

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2749/wvuthorns-sprintf-use-violates-standard
  Zach will fix this

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2751/ehfinder-array-reference-out-of-bounds
  Peter will fix

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2750/httpd-array-access-out-of-bounds
  Roland will fix

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2739/change-admmacros-spatial_order-to-grhydro
  Steve will review

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2754/inconsistent-definition-of-b-in-evolution
  Sam will add a parameter

Email:

*
https://lists.einsteintoolkit.org/pipermail/users/2023-September/009059.html
"Loss of precision after postintial?". Yosef suggested that init_3_tl
could also cause changes in POSTPOSTINITIAL. Peter is not sure if
anything more can be said without having seen the parameter file. Peter
will try and contact Hayley.


Chair next week: Steve
Minutes next week: Sam

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.


pgpA8l0WvApNG.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2023-08-31

[Leo Rosa Werneck]

Hi all,

Einstein Toolkit Meeting - August 31, 2023

Chair: Peter
Minute taker : Leo
Present  : Sam Cupp, Yosef Zlochower, Peter Diener, Leo Werneck,
   Zach Etienne, Roland Haas

* Next ET Release

  - Should CarpetX be enabled or disabled by default? Sam believes it would be
weird to include all of the new thorns that depend on CarpetX and then have
people do extra work to have access to them. This will force everyone to
have a compiler that supports C++17, but this is not a particularly high
bar. Erik also supports enabling it by default.

  - Roland said some modification might be needed for simulation factory to
account for this. There is also some consideration about the default
compiler, which would mostly affect the "generic" configuration. The new
configuration would need to check the system until it finds a
C++17-compatible compiler, something that is currently not done. Further,
some older versions of GCC require you to pass additional options to link
against the correct, "experimental" standard library.

  - Zach shared a useful list of supported C++ features by compiler:
https://en.cppreference.com/w/cpp/compiler_support/17

  - Roland suggested enabling CarpetX now so that problems can be fixed as they
appear.

  - Loop control, as it exists, prevents Carpet and CarpetX from existing in the
same executable. The provided macros are drastically different for the two
drivers because of the way looping is done in AMReX. One option is to have
thorns require "Loop control" OR "Loop control X" so that the header file
can pick the appropriate macros at compile time. This is not yet
implemented, but Roland will work on it.

  - Peter should review GRHayLHDX, but skip GRHayLMHDX.

* Unanswered Questions on the Mailing List

  - There are bugs in POWER. Roland will look into it.

* Open Tickets:

  - #2754: there are different definitions of magnetic fields in the toolkit,
   differing by a factor of sqrt(4pi). Note that the evolution of GRMHD
   quantities is still consistent, as these factors are accounted for
   when computing the magnetic pressure that is used to compute the
   energy-momentum tensor. As Zach points out, this only affects very
   specific scenarios where direct comparisons between e.g., GRHydro and
   IllinoisGRMHD are performed.

  - In the future, it would be nice do modify the Seed_Magnetic_Fields thorns
from WVUThorns* to provide a more user friendly way of setting up the
magnetic fields (for example, the user can specify the ratio of magnetic to
fluid pressure).

Next week's chair: Zach
Next week's minute taker : Roland

Cheers,
Leo


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting Minutes, August 24, 2023

[Cupp, Samuel D.]

Chair - Leo
Minutes - Sam
Attendees - Roland Peter Yosef

ET_2023_11 release
SpaceTimeX likely will not be included
Yosef - this means there's no spacetime evolution thorn, greatly reducing what 
can be done with CarpetX

While true, these thorns are also on Erik's plate, for the most part, and he is 
busy with CarpetX release. They also haven't been used for a publication or an 
equivalent quality control. Even if CarpetX is included without a spacetime 
evolution thorn, having CarpetX in the toolkit will simplify future releases 
with CarpetX-based codes. Also, if people want to use the Z4c thorn in 
SpacetimeX, they can still check it out at their own peril.


AsterX also doesn't have a publication, so the reviewer will need to check 
whether AsterX is actually ready for inclusion. Is the paper just limited by 
writing, or is something still not working correctly?

Roland - I think there was an issue with TOV star evolution, but I don't know 
if that was resolved. Bruno hasn't responded in regard to being a reviewer for 
the AsterX arrangement.

Sam - Samuel Tootle volunteered to review it, though I don't know if one person 
is enough for the whole arrangement. I'll make sure the reviewer(s) clarify 
whether AsterX is completely working and the paper is just a matter of writing, 
or if there's still something being debugged.


Unanswered emails:
"Loss of precision after postintial?"
Con2Prim runs after initial data is set, even with checkpoint/restart. This 
means that the data can potentially change depending on which evolution thorn 
is being used. Since finite differences were mentioned, one could also try 
increasing resolution and see how this affects the precision.


Tickets:
Ticket 
2749
 - Leo will address this issue.


Next week's meeting:
Chair - Peter
Minute taker - Leo

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting Minutes, August 17, 2023

[Cupp, Samuel D.]

Chair - Peter
Minute taker - Sam
Attendees - Leo, Steve

ET_2023_11 release
AsterX still needs reviewer
SpacetimeX might need a second reviewer

As a reminder, reviews should be completed by 9/20/23. Please add any 
questions/comments to the respective tickets on the ticket 
tracker.

Unanswered emails:
"Problems running BBH example" - Steve replied and it was resolved; may not 
have been reply all

"Issues at installation" - Sam, Leo, and/or Zach will check if anyone replied 
without a reply all; otherwise, they will reply to this

"Error Estimation for RK45CK" - Peter will work on replying to this

Next week (8/24):
Chair - Leo
Minute Taker - Steve

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-08-10

[Roland Haas]

Present: Roland, Leo, Zach, Peter, Sam, Yosef

ET release
==

* reviewers needed for AsterX. Roland to poke Bruno and EU people.
* Yosef is reviewer for SpaceTimeX, still not specific list of thorns
  to include. Sam will ask for final list on mailing list. Thorns not
  volunteered and champions stepping forward will be dropped from the
  release.
* Reviewer found for SGRID
* need testsuite runners and gallery runners
** Steve would do BBH gallery example again
** please suggest possible helpers
* Sam created / reactivated tickets for release

ET summer school


* recordings are still being processed (Yosef is the sole editor right
  now)

Unanswered emails
=

*
  http://lists.einsteintoolkit.org/pipermail/users/2023-August/009021.html
  Roland will try and follow up, but is not sure what is going on.
  Might be simfactory failure or might be in simfactory or some
  homebrew / machports / user editing issue

*
  http://lists.einsteintoolkit.org/pipermail/users/2023-August/009020.html
  Roland had been in contact. Not sure what is going on, might be an
  issue with simfactory setting up new segments instead of creating a
  new simulation.


Tickets
===

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2706/update-default-twopunctures-parameters-or
  Zach will coordinate with Steve about TP paramters

* pull request
  https://bitbucket.org/cactuscode/cactusnumerical/pull-requests/13
  Peter will take a look to see what / if needs to be done

Tickets for review
==

* no updates for most of them
*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/963/improve-mclachlan-accuracy
  Peter will look into the after Zach's paper with improved methods is
  out
*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2653/carpet-presync-triggering-syncs-for-all
  issue is triggered if there are no b/c to apply, then PreSync will
  try and SYNC + bc every time. There probably is some better fix for
  this. Some fine tuning of warnings / errors may be required.

chair next week: Peter
minutes taker next week: Sam 

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.


pgprhI5Qd4QNJ.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes. August 3, 2023

[Yosef Zlochower]

Attendees:  Samuel Cupp, Roland Haas, Peter Diener, Steven Brandt, Zach 
Etienne, Keith Dow, Yosef Zlochower

Summer School recap:
    We need more disk space
    Encourage each tutorial speaker to test notebooks

    Need to report number of both in person and online participants.

ET 2023_11 release:

    Need volunteer reviewers (see list at 
https://docs.einsteintoolkit.org/et-docs/Meeting_agenda 2023-08-03 minutes)

   Need more CarpetX documentation
   Need more CarpetX testsuites

   May need multiple reviewers for spacetimeX (Yosef volunteers). Who 
are the champions ?

   Potential issues of duplicated thorns/code when Carpet and CarpetX 
are both in the toolkit.

   Reviewers needed for GRHayL thorns (Peter volunteers)

   Reviewers needed for AsterX


Chair Next week: Samuel Cupp
Minute taker:

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] meeting minutes for 2023-07-13

[Roland Haas]

Hello Erik, all,

> > * Roland suggests to use that as  
> 
> Roland, what do you suggest? The sentence ends here.

Ah, I was going to suggest this as a topic for the working workshop /
hackathon at RIT end of July. Since there are are least 2 groups
interested (RIT and UIUC). So forming a group of say 3-4 persons, might
work well.

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpeQRO5XfRPr.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] meeting minutes for 2023-07-13

[Erik Schnetter]

On Thu, Jul 13, 2023 at 11:02 AM Roland Haas  wrote:

[...]

> Subcyling in CarpetX
> 
>
> * Yosef and Allen find very slow speeds for BBH simulations
> * Alexandru Dima and Helvi are finding that comparing Lean_public and
>   Lean_public in CarpetX that subcycling is responsible for the
>   majority of slowdown (factor of 6 or so)
> * Roland suggests to use that as

Roland, what do you suggest? The sentence ends here.

-erik

-- 
Erik Schnetter 
http://www.perimeterinstitute.ca/personal/eschnetter/
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-07-13

[Roland Haas]

Present: Roland, Steve, Yosef, Leo, Sam, Zach

thorns proposed for inclusion
=

* no updates

Subcyling in CarpetX


* Yosef and Allen find very slow speeds for BBH simulations 
* Alexandru Dima and Helvi are finding that comparing Lean_public and
  Lean_public in CarpetX that subcycling is responsible for the
  majority of slowdown (factor of 6 or so)
* Roland suggests to use that as 

Subscribe to NR community call
==

* no objections

Users mailing list moderation
=

* currently Peter is the only person moderating
  users@einsteintoolkit.org
* also handles subscription requests
* Yosef suggested adding more persons as moderators and add it to the
  ET call agenda
* had some discussion on how to try and improve this process

Unanswered emails
=

*
  https://lists.einsteintoolkit.org/pipermail/users/2023-July/008978.html
  Roland will respond

Tickets
===

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2669/math-is-not-rendered-correctly-in-online
  Zach suggests to also provide PDF as fallback on the website

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2647/incorrect-weno-coefficient-in-grhydro-wenohttps://bitbucket.org/einsteintoolkit/tickets/issues/2647/incorrect-weno-coefficient-in-grhydro-weno
  Roland to check what existing reconstruction methods in GRhydro are
  actually doing (interpol or reconstruct)

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2739/change-admmacros-spatial_order-to-grhydro
  needs review, code from 2022 and review request from April 2023

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2526/mclachlan-has-incorrect-read-declarations
  needs some internal Kranc knowledge, volunteer needed

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2653/carpet-presync-triggering-syncs-for-all
  had some discussion on where checks for "already valid" happen in
  Carpet & Presync. Sam did observe that the changes proposed sped
  things up. Should be re-investigated to see if this still applies. 

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2637/update-cactus-copyright-file-to-lgpl
  needs review, mostly finishes up a prior change to README files

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/963/improve-mclachlan-accuracy
  discussion is going on between Peter and Zach where Zach proposes a
  better and more effective solution to the issue. Hope to revisit this
  after summer.

Chair 2 weeks from now: Steve
Minute take 2 weeks from now: Yosef

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpZPpaRmWQxS.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-07-06

[Steven R. Brandt]

Present: Steven, Roland, Sam, Yosef

No updates on SPEC benchmark

ET release planning
    Sam's updates:
    - Illinois GRMHD updated to use the thorn that provides Grhayllib
    - Grhayllib proposed for includion
    - GrhaylHD is a pure hydro version that also depends on Grhayllib
    - GrhaylHDX is the CarpetX version
  and other Grhayl thorns.
    - 3 new thorns + 2 or 3 if CarpetX makes it.
    - Updates to 2 thorns
    - Also Sam has a list of deprecations
    CarpetX is attempting to be on track
    - no docs as yet
    - no clear idea which thorns come with it
    - Erik is renaming to avoid conflicts

The RIT workshop:
    - Mostly set
    - One change in speaker but no change in topics
    - Visualization video instead of live
    - Request notebooks get sent to me ahead of time
Working Workshop:
    - Roland will set things up

Tickets:
    - rdwr.pl is too noisy

Next week:
Chair: Yosef
Minutes: Roland

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting Minutes for 23-06-29

[Cupp, Samuel D.]

Chair: Leo
Minutes: Sam
Present: Leo Sam Roland Yosef

ET_2023_11 release
People can begin considering possible inclusions for the ET_2023_11 release.
There is still ~1 month before finalizing this list, but it would be good to 
start
proposing these sooner rather than later.

Roland: Will the Elliptica reader be proposed again?
Leo sent email during previous release detailing goals for inclusion. We don't
know if they plan to propose it again for this release.

Ticket 2497
Leo and Sam made some suggestions, but it is still unclear what the desired 
behavior
is. Because of how TmunuBase schedules things, it appears that setting 
stress_energy_at_RHS
to "no" will never set the Tmunu variables, meaning they will be nan.
SetTmunu should probably also be scheduled in CCTK_INITIAL to allow for fixed 
Tmunu.
The scheduling would likely be similar to HydroBase_Prim2ConInitial.
Roland will propose a possible solution in the ticket.

Ticket 2738
Canuda tests are now working

ET US Meeting:
RIT Inn answered ambiguously about whether they provide a shuttle to/from 
airport
or just to/from university. Yosef will clarify this with the hotel.
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-06-15

[Roland Haas]

Present: Peter, Roland, Leo, Sam, Steve, Yosef

open tickets:

Ticket 2735
===
* proposed simplification of schedule.ccl for ADMBase, HydroBase, and TmunuBase
* Sam simplified the schedule making use of more modern
  STORAGE foo[timelevels] constructs
* there is also checks if intial_FOO is "none" in which case no storage
  is allocated for FOO
* Roland says that these should be retained

Ticket 2742
===
* proposed restructuring of Seed_Magnetic_Fields
* Sam described the two Seed_Magnetic_Fields thorns in WVUThorns and
  WVUThorns_Diagnostics and desire to merge them into a single thorn
* added schedule statements to make thorns use with just HydroBase
  rather than IllinoisGRMHD
* PR is about to be created
* plan is to deprecated Seed_Magnetic_Fields_BNS in the next release and
  remove the release after next

ticket 2627
===
* announce removal of TmunuBase inherting from ADMBase and
  StaticConformal
* would remove more uses of StaticConformal and simplify code
* will proceed with deprecation warning

ticket 2497
===
* Sam and Leo are in conversation with Vikram Manikantan about issues
  they face running IllinoisGRMHD code
* Leo suggests to use magnetized TOV test as parfile to study this

ticket 2741
===
* some thorns seem to incorrectly use CCTK_GFINDEX4D
* Sam replace some more instances of CCTK_GFINDEX4D with
  CCTK_VECTGFINDEX3D
* Sam will close ticket

ticket 2526
===
* McLachlan has incorrect READ declarations for presync.
* Ian is busy right now and is asking for others that could volunteer
  to review change to Kranc

ET release:

gallery examples

* BBH gallery example updated by Steve
* Peter will update gallery example 

GRHayL library inclusion proposal in ET_2023_11
===
* Ticket 2497, as scheduling doesn't mix con2prim and updating Tmunu
* IllinoisGRMHD would no longer see feature updates, be marked as
  deprecated and eventually archived.
* Peter suggests that a "transition guide" from IllinoisGRMHD to GRHayL
  be provided when IllinoisGRMHD is no longer updated. Sam has a draft
  of such a document written already.
* Roland mentioned to carry IllinoisGRMHD along in the ET even after it
  no longer sees updates until maintaining it becomes prohibitive. At
  which point it would move to a archivedthorns-IllinoisGRMHD repository
* this will not affect GiRaFFE

process to deprecate features
=
* Sam had asked about procedure
* Release announcement will be official notification
* process
https://docs.einsteintoolkit.org/et-docs/Policies_to_retire_functionality

release process updates ideas
=
* Sam reported on issues he encountered some issues when creating the
  new release
* Sam will update "unshallow" commands to download instructions
* notes added to
  https://docs.einsteintoolkit.org/et-docs/Release_Process_Updates

timing for ET_2023_11 meeting
=
* schedule: https://docs.einsteintoolkit.org/et-docs/Release_Details
* proposal deadline for new features would be weeks of RIT meeting
* Sam (release manager) will shift choose features date back by one
  week, keeping other dates the same

ET meetings:

EU ET meeting
=
* Steve has been working on making the tutorial server a multi-node
  setup, will test after meeting
* will skip ET call next week

US ET meeting
=
* please check that you have received registration confirmation email
  if you registered. If no email was received then the registration is
  not complete.
* RIT Inn reservation code sent at registration, if you already
  reserved the room without the reservation code, contact the
  organizers to have the code applied (and be able to check out on
  Saturday if so desired)
* the reserved block will go away soon, please reserve your room soon
* Yosef will check if hotel shuttle runs to the airport
* Shuttle from RIT Inn to RIT is provided during extended business
  hours, shuttle runs by request

chair week after next (June 29): Leo
minutes week after next (June 29): Sam

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpLAcSfmV6wv.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2023-06-08

[Leonardo Rosa Werneck]

Hi all,

Here are the minutes for today’s meeting:

Chair  : Steve
Minutes: Leo

Present: Steve Brandt, Peter Diener, Zach Etienne,
 Roland Haas, Leo Werneck, Yosef Zlochower

Nothing specific added to the agenda.

* Next ET release manager search:
  - Sam Cupp is currently expected to be the lead release manager of the
ET_11_2023 release, but he was not in today's telecon. Leo will ask him in
person.

* ET Summer School and Workshop:
  - Yosef said a discount code that allows folks to book their room at the RIT
Inn has been sent to everyone who already registered. Participants are
encouraged to book their rooms as soon as possible. As a reminder, the RIT
Inn is the only hotel that offers free ground transportation to/from the
RIT.

  - If someone booked their hotel earlier without using their code, they should
reach out to Yosef so that they can see what can be done.

  - There are people who began their registration but never
finished. Unfortunately the organizers do not have access to any information
that identifies these people. They are encouraging everyone who wants to
register to do so by completing the registration forms all the way through.

* NRPy+ modernization:
  - Zach wants to announce that NRPy+ is undergoing an extensive modernization
that will allow it to be pip installed.

  - Steve suggests all core NRPy+ modules should be moved inside a nrpy/
directory as soon as possible to avoid future issues.

  - Steve suggests a 'project class' that would hold all global variables (e.g.,
gridfunction list), but Zach does not think it is necessary since modules
containing global variables contain very few of them.

  - This new version of NRPy+ (still under development) requires Python >=3.6,
and thus supports f-strings and type hinting/mypy. Support for Python 2 will
be dropped once this new version becomes the official version.

* Unanswered question on the mailing list:
  - Bernard Kelly asked a question regarding whether or not ADMMacros should be
removed from parameter files that use GRHydro for the MHD evolution. Steve
is writing a reply to that question, which likely involves removing the
ADMMacros thorn from the parameter file but updating the value of a new
GRHydro parameter that replaces the one from ADMMacros.

Next week chair  : Peter
Next week minutes: Roland

Cheers,
Leo


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-06-01

[Roland Haas]

Present: Zach, Keith, Roland, Steve,  Leo, Peter

SPECv8 contribution
===

* no specific news, progressing 

ET release coordinator search
=

* had planned to include CarpetX and AsterX
* no documentation exists for it yet
* Roland estimates that CarpetX is not closer to inclusion ready than 6
  months ago. CarpetX reviewers 
* paired coordinator approach was useful for Leo in the 2023_05 release
* will pick this up again next week due to small number of participants

Unanswered question
===

*
  https://lists.einsteintoolkit.org/pipermail/users/2023-May/008930.html
  Roland will forward similar answer about grid structure consistency

*
  https://lists.einsteintoolkit.org/pipermail/users/2023-May/008935.html 
  Roland will respond suggesting that a full C++11 stack is required

Error message shown about missing
http://lists.einsteintoolkit.org/pipermail/users/2023-June/date.html is likely 
due there not yet being a posting to the mailing list for June and the script 
cannot distinguish "no posting yet" from "server is down or cannot access the 
file".

Galley examples
===

* Steve will update the BBH example
  (https://www.einsteintoolkit.org/gallery/bbh/index.html)
** Zach mentioned ticket
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2706/increase-twopunctures-resolution-in-bbh
* Peter had added convergence plot to TOV example
  (https://www.einsteintoolkit.org/gallery/ns/index.html). Convergence
  order is between 1 and 2.

Zach brought up the issue that gallery examples are typically very low
resolution and one often has to increase resolution significantly
observed for example good convergence. This typically clashes with the
desire to keep the gallery examples small enough so that they can be
easily run.

Zach noted that the TOV example is not featured in the cycler on the
main page. Steve and Zach had a discussion about how one could create a
more visually interesting image using VisIt.

Steve brought up work on openPMD reader in Python. Roland will work
with summer student in summer to add mesh refined openPMD support to
VisIt. Will meet with Steve and (NCSA) local visualization expert.

Updated tickets
===

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2734/consider-adding-if-statement-functionality
** Steve mentioned current situation in CarpetX where storage is always
   one.
** Steve considers that he may be able to add auto-allocation of
   storage to the WRITE statements, READ would result in invalid data to
   read from

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2733/twopunctures-contains-globally-visible
** we should fix this


Open topics
===

* Clang and Flang support. Zach mentioned benchmarks that have clang
  outperform gcc, asks about support for clang and flang in simfactory
  and the Einstein Toolkit.
** Roland says no clang support in the past  due to lack of Fortran
   compiler. Steve and Zach point out that gfortran can be used and the
   there is now also flang.
** Roland mentions that there are "old" flang based on PGI and
   "new" flang based on llvm. Also AMD compiler and Cray compiler on
   Crusher / Frontier are llvm based, so are modern version of the Intel
   compiler suite.
** currently have not knowledge about clang / flang in the Linux
   known-architecture files, only in Darwin files for (older) clang

Next Chair: Steve
Next Minutes Taker: Leo

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgp3tOIuaaYwS.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2023-05-25

[Cupp, Samuel D.]

Present: Sam, Keith, Yosef, Peter, Roland, Zach
Chair: Peter
Notetaker: Sam

* ET_2023_05 Release is out!

Peter: Gallery example now has multiple resolutions. Should we include all the 
data? Will increase the amount of data being stored.

Roland: We could just add the data specifically for the new plot and parameter 
files.

Peter: I've already added the parfiles and can add the new data.

* SPECv8
Roland: They've asked us for the exact version number of the code they are 
using. As far as we know, the ball is still in their court for moving forward 
with this.

* Tickets
Sam: I still need to close the tickets associated with the release, which I'll 
do today.

Roland: Since we have just released a new version, now is the time to start 
considering more serious changes to code. One change is removing the compile 
flags from the linker stage. This is because for compiling with CUDA the linker 
stage shouldn't be assuming that it is the c++ compiler. This is sufficient to 
compile CarpetX, but it remains to be seen whether this change will be 
sufficient for external libraries.

Zach: What is the plan for including CarpetX? Is it still coming soon?

Roland: The idea is to have it added in the November release, but this will 
require quite a bit of work to get CarpetX ready for a release, both with code 
cleanup and testing compilation on different systems.

Sam: I know some of the thorns need to be renamed to allow for smooth 
integration with the rest of the toolkit. Hopefully this will happen sooner 
rather than later.

Peter: Had a conversation with Zach about Ticket #963 but hasn't had a chance 
to look at McLachlan again since that call.

Zach: I found a different version of the 'third' change which has even better 
results than the original. I'll share that soon.

Roland: We can merge Ticket #2731 (checking for incomplete perl install). A fix 
is also proposed for Ticket #2526 (incorrect read/writes generated by Kranc). 
Ian doesn't have time to review this, so Roland will try to track down the 
person who added the READ/WRITE declarations to Kranc.

Next week the chair is Zach and the minute taker is Roland.
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2023-05-18

[Steven Brandt]

Present: Steve, Sam, Yosef, Keith, Roland, Zach, Peter, Jose
Chair: Sam
Notes: Steve

Release: Most of the branches and tags have been done.
Gallery: Not all done (Steve forgot). Peter did 4 different resolutions.

Roland is moving the tutorial server to github
Only Steve, Erik, and Roland can push to the dockerhub

Steve needs to run tests on LSU machines.

Issue with ticket 2497. IGMHD also has a parameter that replictaes one 
in TmunuBase. IGMHD will be deprecated to be replaced by GrhaylET.


GrhaylET has 90% code coverage.

US ET summer school:
Registration is open
Tutorials have been arranged

EU ET summer school.

Spec should still be happening.

Peter has a convergence plot for NSs. Possible mug image.

Steve needs to reply to email

Next week: Chair: Peter, Minute Taker: Sam

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-05-11

[Roland Haas]

Present: Roland, Peter, Zach, Jose, Keith, Sam, Steve, Leo, Johnny,
Yosef

ET release
==
* release managers will send out draft of release statement later today
* 2 weeks to release, target date May 24th
* no issues with remaining inclusions
* Steve is asking for images to put on the release mug, Zach suggested
  looking for kuibit based images
* Leo is not able to assign tickets on
  https://trac.einsteintoolkit.org, Roland will check if there is
  something off with his account compared to Sam, who can

ET testsuite status
===
* Peter is compiling on db1 to run TOV star
* Steve will run BBH
* Roland has run BNS
* Johnny has run Poisson equation, will update website
* Multipatch scalar wave assigned to Jake (UTAusin), runs finished,
  working on uploading to website
* Leo reduced runtime of NRPyEllipticET test and will push change

Unanswered questions

* Steve will contact Shamin Haque about issues on clusters
* Zach suggested creating a "unanswered tickets"  lists similar to
  unanswered questions. Roland says this should be possible with some
  Python code and REST API calls to bitbucket's API. Estimated work
  load is about 2hrs if one knows the API.
* Leo will look into Tmunu ticket in Baikal
https://bitbucket.org/einsteintoolkit/tickets/issues/2497/illinoisgrmhd-is-incompatible-with-setting
* WENO coefficient ticket still up for review, will be done after
  release
https://bitbucket.org/einsteintoolkit/tickets/issues/2647/incorrect-weno-coefficient-in-grhydro-weno


ET summer school

* Registration is open, email is out 
* Travel support can be requested at registration time

CarpetX
===
* Sam is asking about existing interpolation / deaverage code in
  CarpetX to from cell centered averages to samples or averages at
  cells, faces, vertices. Roland suggests that the prolongation
  operators could be used for some of this, but that most code so far
  seems to roll their own

AEILocalInterp
==
* Roland and Zach looked into C auto-generating Maple code
* Roland could re-run the scripts and re-generate the code
* results looks identical up to renaming of variables and pass the
  current ET testsuites
** Roland noticed that 3d generation log files are missing, will check
   what is going on
* Peter has 5th order Hermite codes for field to particle interpolation
  in GR-SPH code, will discuss with Zach if that can be made available
* coeffs derived in MMA and code exported to Fortran from there

Next chair: Sam
Next minute taker: Steve

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgp9FssiyXeMI.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-05-04

[Roland Haas]

Present: Roland, Peter, Zach, Gabriele, Leo, Sam

ET release timeline
===

* release date will be May 24th
* Leo and Sam are sending emails to the contributors and drafting the
  release announcement
* inclusions are reviewed ok

kuibit
==

* Gabriele finds that installing kuibit using pip is getting harder and
  coming up with generic instructions is difficult
* Ubuntu 23.04 (and other new versions) prohibit use of `pip install
  --user` and strongly favor virtualenv
* Also link required package instructions as described in
https://bitbucket.org/einsteintoolkit/tickets/issues/2724/update-download-page

Testsuite status


* no updates
* test need to be re-run after fix to Fortran AliasedFunctions code
* Peter will re-run on sunrise once release branches are in place
* Roland has updated tests for Frontera and will re-run other tests
* Anuj will re-run tests on thornyflat

Gallery tests
=

BBH: no update

BNS: Roland updated example

TOV: Peter will run

Multipatch: no update

Poisson: no volunteer, Leo and Sam to look for people

Gabriele suggested to have more detailed change list listing bugs more
explicitly
* Zach suggests using the script that shows changes for review by the
  release managers
* Gabriele suggests providing a link to this information, also to
  resolved tickets 
* link to revolved and closed tickets sorted by date:
https://bitbucket.org/einsteintoolkit/tickets/issues?status=resolved&status=invalid&status=closed&status=wontfix

questions
=

* Gabriele mentioned large number of warning messages from Cactus when
  trying to run lean_public with presync_warn flags
https://bitbucket.org/einsteintoolkit/tickets/issues/2723/no-boundary-conditions-registered-for
** there was some more generic discussion about how presync handles it
   various options

* Leo brought up about HydroBaseX for CarpetX where currently there is
  no direct way to handle staggered variables 
** this is more generic problem with variables that may sensibly be
   defined or used with different sorts of staggering. Current HydroBase
   assumes a single global choice is good enough for all codes that
   interface and that translation between representations is almost
   error free.

emails
==

* existing emails by Shamim Haque, in conversation with Steve, ask
  Steve if is continuing the converstation

tickets
===

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2722/getcomponents-places-kadaththorn-inside
  is needed for release, Roland will ask Steve

chair next week: Zach
minutes next week: Steve

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpopW4G0V5Hd.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-04-27

[Roland Haas]

Present: Roland, Peter, Leo, Steve, Zach, Sam, Yosef

ET release:

* Elliptica Reader will be postponed to next release
** needs a library of ID
** comparison between ET and published result
* FUKA: on track
* partialTracer_ET: on track
* release managers will start drafting release announcement
* release date is: 2023-05-24

Testsuite status:
* newly failing test in GRHydro_InitData/magtov_poloidal
** Peter debugged and found that issue is with Fortran if condition.
Fortran does not short-circuit logic or guarantee order of evaluation.
Peter will file pull request.

Unanswered question:
*
http://lists.einsteintoolkit.org/pipermail/users/2023-April/008906.html Cactus 
hangs on startup. Steve had been in contact with Shamin Haque about this. 
Problem still persists even in most recent try. Steve will respond.

Tickets on mailing list:
* https://bitbucket.org/einsteintoolkit/tickets/issues/2719 issue in
  GetComponents when dealing with symbolic links. Workaround exists.
  Currently affects KadathThorn
* https://bitbucket.org/einsteintoolkit/tickets/issues/2623 compile
  failure when paths contain ":". Had some discussion of updating or
  replacing autoconf in Cactus with something else.

Tickets ready for review:
* in code freeze state, only important bug fixes

SPECv8 review
* waiting to hear back from SPEC organizers
* contribution still on track to be included

CarpetX:
* Zach asked about any horizon finder for CarpetX based runs
** Roland says that currently no working version exists
** AHFinderDirect may be portable using Sam's and Roland's work to
interface CarpetX's interpolator interface with Cactus' interpolator
interface. Native CarpetX may already include support for derivatives.
** Sam worked on interpolator but also no longer quite recalls the
current state

next chair: Peter
next minute taker: Steve

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpypKQIbiKHQ.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2023-04-20

[Leonardo Rosa Werneck]

Hi all,

Apologies for the long delay. Here are the minutes from last week’s call.

Chair   : Johnny
Minutes : Leo
Present : Peter, Alireza, Johnny, Roland, Zach, Leo, Yosef

-> Next ET Release
  * Feature freeze coming up April 26, 2023
  * Elliptica reader
- Alireza is taking a look at the latest comments made to the
  ticket. Trajectories differences are due to issues in the parameter file,
  which are being fixed.
- Will there be initial data files available? Yes. The solver will be made
  open source soon. Roland says a reasonable number of files is a
  requirement if the code is not open source.

  * Fuka reader
- There is a compilation issue related to the fact that GetComponents does
  not download the code's submodules. Samuel should take a look at it as
  soon as possible.

  * particle_tracerET
- All tests passing.

  * TwoPunctures_BBHSF
- Leo and will contact Cheng-Hsin regarding additional tests that may have
  been requested.

  * kuibit
- Gabriele was able to restore compatibility with Python 3.7.

  * CarpetX pre-release
- Steve will be the coordinator.

  * The next release will be named Schwarzschild.

  * The preliminary release data will be May 24, 2023.

  * Peter will run the TOV gallery example. Johnny will run the Scalar Wave
gallery example.

  * SPEC: Roland reports we are still on track for the inclusion.

-> Unanswered questions on the mailing list
  * The mailing list server was updated last week and should be back online.

-> Open tickets
  * Building PAPI on Linux broke after a recent attempt to fix compilation
issues on macOS. Roland already has a pull request that fixes the issue, but
it needs to be reviewed.

  * NRPyPN may be movied from arrangements/EinsteinInitialData to
utils/InitialData, since it is not a thorn. Zach is the assignee and will
review it.

  * Zach suggested the default values of grid resolution for TwoPunctures are
too low and should be adjusted. Roland says: changing parameters' default
values is tricky, and thus not recommended. That being said, we could update
the values in readily available parameter files that use TwoPunctures.

  * KadathThorn assume git repository and use non-POSIX functionality.

*Had to leave*


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2023-04-06

[Bing-Jyun Tsao]

Participants: Leo, Johnny, Roland, Peter, Keith, Zach, Samuel, Steve

   - ET_2023_05
  - Elliptica reader (and solver timeline) Ticket #2619
  

--
  gave (final) final deadline for tests: 4/12; thorns are already
in manifest
  to stay on track, but may be removed depending on final status
 - [LW] Extend deadline by 1 week, generate thorn again with
 Kranc/NRPy by Friday
 - test will require same random generator of particles on
 different machines
  - Need testers for gallery examples: BNS: Roland, BBH: Steve
  - [SB] SuperMike-III (LSU) add to list of machines that works,
  Deep-Bayou tested
  - Name for the release: [SB] suggest “Schwarzschild”
  - [RH] Now changing current CI to add Elliptica
  - CarpetX
   code
  review -- Erik is champion for CarpetX; Steve would be the coordinator
  (keep track of who reviewed what) for the review
   - ET uptime monitor, original uptime free service shutting down, set up
   in github: [1]  [RH]
   - mailing list:
  - [2]
  
  “Thorn setup taking too much time in cluster”, Steve suggested that the
  simulation wasn’t purged, but will need to go over parameter file to find
  why this happens
   - open tickets
  - [LW] close two tickets #2300
  

  #2690
  

  (include piecewise polytrope and tabulated EOS), IllinoisGRMHD
will not be
  touched for next release, will just add implementation to GRAIL instead,
  [ZE] new thorn GRAIL_ET
 - [RH] [ZE] manage old thorns that people use, especially now
 CarpetX will be included [SB] instead of download the whole
toolkit, record
 which thorns users download and warn that old thorns are no longer
 maintained [RH] popularity contest
 

 - [SB] suggestion: give a name to Cactus in ETK, “CactusET”
  - #1775
  

  [RH] several thorns use Boost, huge to compile and require many
  dependencies [SB] only generate headers for boost and not compile the
  entire package, link to an external thorn
   - No meeting next week for APS, next meeting in two weeks (04/20):
  - Chair: Johnny, minute: Steve
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-03-30

[Roland Haas]

Present: Roland, Peter, Alireza, Beyhan, Johnny, Keith, Leo, Sam, Zach

Elliptica inclusion:

* discussed progress of review, including code quality review
* asking for a comparison of new ET reader module generated simulation
  with published work using BAM, "by eye trajectory agreement" would be
  acceptable to all

FUKA reader:

* review completed 
* currently depends on Boost, can only be included as a DISABLED thorn
** not easy to extract needed parts of Boost and include according to
   author
* had a larger scale discussion on what including Boost would involve
** Zach would like to avoid including Boost since it is quite large and
   slow to compile

particleTracer_ET:

* included in master branch
* needs updates to test tolerances to avoid breaking on different
  systems

TwoPuncturesBBH_EF:

* review concluded
* look for pull request

kuibit:

* Gabriele managed to remove the requirement for Python 3.8
* Sam will contact Gabriele about any changes required to ThornList

ET Release:

* Leo asked about steps to regenerate MMA codes, Roland referred to the
  release process wiki page which should list the steps, essentially
  runing "make"
* Peter offered help generating them in case a MMA license is not
  available at UIdaho

Unanswered questions:

* LORENE build errors
  http://lists.einsteintoolkit.org/pipermail/users/2023-March/008890.html
  had some discussion. Leo mentioned that even with modules loaded one
  still needs to set up correct localsettings files. Leo will respond.

Open tickets:

* https://bitbucket.org/einsteintoolkit/tickets/issues/2700
  Peter will review, faced git issues
* no other new tickets

chair next week: Leo
minute taker next week: Johnny 

Yours,
Roland


-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.


pgpqCD7FmBrJ7.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] Meeting minutes for 2023-03-16

[Gabriele Bozzola]

Hello,

  - kuibit: The new version of the code requires Python 3.8+. Steve hoped
> that
> kuibit would support Python 3.7, since that's what installed in some
> of the
> LSU clusters. That being said, both Steve and Peter have successfully
> installed kuibit with Python 3.7 by using Gabriele's suggestion:
>   $ pip install --upgrade 'kuibit<=1.4.0'


I managed to restore compatibility with Python 3.7 for kuibit 1.4.0 without
any
drawback for users.  All the tests pass for Python 3.7 - 3.11, which should
cover
the entire range of machines supported for the upcoming release.

(Incidentally, Stampede has Python 3.8. It is in the GNU modules).

Best,
Gabriele


On Thu, Mar 16, 2023 at 3:17 PM Leonardo Rosa Werneck 
wrote:

> Hi all,
>
> Here are the minutes of today’s meeting:
>
> Chair: Steve
> Minute taker: Leo
> Present: Steve, Leo, Sam, Cheng-Hsin, Zach, Roland, Peter, Yosef
>
> * ET_release
>
>   - Elliptica reader: Alireza said testing should be completed in the next
> two
> weeks. Beyhan expressed some concern about the fact that there are
> lots of
> files for the Elliptica library, but Steve and Roland assured this is
> fine
> as long as the library is reviewed properly. One important thing to
> keep in
> mind is that if the library provide any globals outside of a
> namespace, the
> names of these globals must be prefixed with the libraries' name.
>
>   - Fuka reader: Review finished a while back and should be good to go.
>
>   - CarpetX pre-release: Review is progressing slowly.
>
>   - particle_tracerET: Review completed last week and should be merged into
> master today.
>
>   - Baikal reads/writes: Code modifications were very minor and should be
> merged
> into master today.
>
>   - TwoPunctures_BBHSF: Review is complete and pull request is being
> created.
>
>   - NPScalars_SF: Review hasn't started yet, but Peter will perform the
> "do no
> harm" review asap. Pull request should be created soon.
>
>   - kuibit: The new version of the code requires Python 3.8+. Steve hoped
> that
> kuibit would support Python 3.7, since that's what installed in some
> of the
> LSU clusters. That being said, both Steve and Peter have successfully
> installed kuibit with Python 3.7 by using Gabriele's suggestion:
>   $ pip install --upgrade 'kuibit<=1.4.0'
>
>   - Overall comments on the release: everything is progressing timely. We
> will
> begin discussing the release name soon (maybe next week).
>
> * ET school: Scientific committee information will be released today;
>   registration will open soon and a modest fee will be charged to cover
> basic
>   expenses; hotel near the RIT closed, but there will be free shuttles
> to/from
>   the hotels.
>
> * CI tests: NRPyEllipticET test is broken (Leo will look into it).
>
> * List of clusters and testers have been added to the release details web
> page
>   (https://docs.einsteintoolkit.org/et-docs/Release_Details).
>
> Next chair: Peter
> Next minute taker: ChatGPT/Zach
>
>
> --
> Leonardo R. Werneck, Ph.D.
> Postdoctoral researcher
> Office EP 314 | Department of Physics | University of Idaho
> 875 Perimeter Dr. MS 0903
> Moscow, ID 83844-0903, USA
> leona...@uidaho.edu
> https://leowerneck.github.io
>
> ___
> Users mailing list
> Users@einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users
>
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2023-03-16

[Leonardo Rosa Werneck]

Hi all,

Here are the minutes of today’s meeting:

Chair: Steve
Minute taker: Leo
Present: Steve, Leo, Sam, Cheng-Hsin, Zach, Roland, Peter, Yosef

* ET_release

  - Elliptica reader: Alireza said testing should be completed in the next two
weeks. Beyhan expressed some concern about the fact that there are lots of
files for the Elliptica library, but Steve and Roland assured this is fine
as long as the library is reviewed properly. One important thing to keep in
mind is that if the library provide any globals outside of a namespace, the
names of these globals must be prefixed with the libraries' name.

  - Fuka reader: Review finished a while back and should be good to go.

  - CarpetX pre-release: Review is progressing slowly.

  - particle_tracerET: Review completed last week and should be merged into
master today.

  - Baikal reads/writes: Code modifications were very minor and should be merged
into master today.

  - TwoPunctures_BBHSF: Review is complete and pull request is being created.

  - NPScalars_SF: Review hasn't started yet, but Peter will perform the "do no
harm" review asap. Pull request should be created soon.

  - kuibit: The new version of the code requires Python 3.8+. Steve hoped that
kuibit would support Python 3.7, since that's what installed in some of the
LSU clusters. That being said, both Steve and Peter have successfully
installed kuibit with Python 3.7 by using Gabriele's suggestion:
  $ pip install --upgrade 'kuibit<=1.4.0'

  - Overall comments on the release: everything is progressing timely. We will
begin discussing the release name soon (maybe next week).

* ET school: Scientific committee information will be released today;
  registration will open soon and a modest fee will be charged to cover basic
  expenses; hotel near the RIT closed, but there will be free shuttles to/from
  the hotels.

* CI tests: NRPyEllipticET test is broken (Leo will look into it).

* List of clusters and testers have been added to the release details web page
  (https://docs.einsteintoolkit.org/et-docs/Release_Details).

Next chair: Peter
Next minute taker: ChatGPT/Zach


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-03-02

[Roland Haas]

Present: Roland, Keith, Alex, Leo, Steve, Zach, Yosef

Status of compiling ET on ARM cpus

* Roland reports that ET can be compiled on M1 macOS if one comments
  out PAPI or with the changes in
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2682/updates-to-compile-et-on-macos-m1-systems

Upcoming ET release

* Leo and Sam have been in touch with release champions
* updated release targets due to removal of IllinoisGRMHD
* most reviewers have finished or are about to finish the 
* Roland mentioned apparently still development for Elliptica reader.
  Leo reported that right now the code is included in a thorn and not
  as an externallibrary

CarpetX review

* Roland to be added to reviewers
* will try and reach out to Erik to see if he would like to champion it

Unanswered questions:

*
  http://lists.einsteintoolkit.org/pipermail/users/2023-February/008857.html
  Roland suggests to try and make sure that, since the code is compiled
  using -xHost or -march=native that the CPU on the login node is the
  same as on the compute nodes

*
  http://lists.einsteintoolkit.org/pipermail/users/2023-February/008856.html
  no one present with Llama

*
  http://lists.einsteintoolkit.org/pipermail/users/2023-February/008847.html
  no one is using Anvil right now

ET testing system

* currently broken and does not send emails
* Zach suggested to explore if GitHub has ARM based runners

Chair 2 weeks from now: Steve
Minutes 2 weeks from now: Leo

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.


pgpYCV_qmYWvo.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2023-02-23

[Bing-Jyun Tsao]

   - Present: Steve, Helvi, Peter, Leo, Cheng-Hsin, Keith, Roland, Yosef,
   Zach, Johnny
   - seminar proposal: Adam J. Peterson, "Code generation for AMReX with
   applications to numerical relativity", 2301.08354
   , which uses AMReX for Z4c evolution
  - March 09th, recording available on seminar
   page and youtube
   - Unanswered questions on mailing list
  - email
  

  from Roland: nobody used Anvil before
  - [SB] Another email regarding running on Rome powered supercomputer:
  [RH] Delta is powered by Rome, Steve or Roland will ask the user
to provide
  the files for compiling (optionlists, etc.) and send this email to the
  mailing list
  - New GRMHD library: take IllinoisGRMHD and NRPyEOS and make it into
  a library, [LW] No need to review NRPyEOS and NRPyLeakage, will need
  reviewers for other parts. Terrence will review particle code, Peter can
  review NPSolverSX (?), [HW] have reviewers and champions in contact
  - [RH] People who agree to test the cluster, Steven Brandt - deep
  bayou, Peter - sunrise, Roland - stampede2, expanse, Summit,
Frontera, [LW]
  add detail to the release
   - Open tickets
  - SummationByParts
  
:
  [RH] prefix needs to be fix to not pollute namespace, fix name conflict
  when Fortran code add underscore at the end, warning: might
break something
  in Llama
  - Mac os Ventura
  
:
  [RH] duplicate from the mailing list, could be listed as solved
  - Compile on the M1 system
  

  [RH] PAPI needs significant changes, HomeBrew changes the links
  - Slab fails
  

  [RH] Slab claim to optionally use MPI and Kranc tutorial did without it,
  but it actually does use MPI so a pull request is made to fix this
  - Include NRPyLeakage
  

  [LW]: will be included for the next release
  - EHFinder parfile
  
:
  [PD] will look into it
  - [YZ] Large ET jobs on Frontera, startup overwhelming the system,
  every MPI proc starts reading files. Proposed solution: at a checkpoint,
  only a group of MPI procs to open files, workaround by
staggering when the
  job starts, issue might also occur with CarpetX. [RH] Assign
token barrier
  to MPI. Roland can send the token to Yosef. Frontera should fix
their file
  system.
   - Next week:
   Chair: Zach
   Minute: Roland
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-02-09

[Steven R. Brandt]

Present: Peter, Steve, Roland, Keith, Zach, Leo, Sam, Gabriele Bozzola, 
Yosef

UNAM workshop
     - Untarring problem at scale
     - Better messages from Simfactory
     - Adding Kuibit version to update
     - New page: https://docs.einsteintoolkit.org/et-docs/Tutorial_Planning

May Release
     - Testing:
   Need to have a list of critical systems, add Delta. Maybe add Anvil?
   Send a message out to the user's list to help with testsuites?
   http://einsteintoolkit.org/testsuite_results/index.php
   Gabriele: Expanse
   Peter: Sunrise
   Roland: Summit, Corey, Frontera, Crusher
   Steve: Docker containers, LSU systems
   Leo & Sam: National lab systems
   Maria: Thornyflat? (last time run by Anush)
     - Additions that don't have reviewers
     - https://lists.einsteintoolkit.org has no good cert
   Steve pinged IT support

Next meeting:
     Chair: Leo
     Minutes: Yosef if makes it, Sam otherwise

Yosef suggests that maybe put names in a ring?

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-02-02

[Roland Haas]

Present: Roland, Peter, Sam, Alireza, Aviral, CHeng-Hsin, Gabriele,
Helvi, Jay, Keith, Leo, Liwei, Steve, Yosef, Zach

ET release contributions:

* FB Moncrief disk
** Roland reported on status, still rough around the edges
** Issues reported with underlying thorn that need to be fixed first
** Zach will look into thorn issues
** UIdaho is starting up a group of students that are working on
creating cvisualizations for this

* READ / WRITE for Baikal
** updates to Baikal, mostly regular development
** removal of checkpoint / recovery for RHS variables which do not need
to be checkpointed
** removes an no-op ApplyBCs
** Leo will review

* Elliptica reader
** spectral XCTS solver, arbitrary spin for fluid body, mixed systems
possible
** Alireza has reader public for BAM, ID already exists, solver will be
public as well
** would want to change so that it no longer uses a temporary
** please keep release managers (Leo, Sam) in the loop

* FUKA reader
** Samuel could not make it
** ticket has some description of what is available
** used in multiple publications
** code is used in publications

* CarpetX inclusion
** too large an inclusion and not yet in state ready to include
** will start review now, aiming for inclusion in November
** 3 reviewers would be great
** Steve, Zach
** UIUC strong gravity group might be interested to review (porting
physics code)

* AsterX inclusion
** depends on CarpetX so postponed

* IllinoisGRMHD, NRPyEOS, NRPyLeakage
** effectively one large update to IllinoisGRMHD that changes con2prim
(Palenzuela, Newman&Hamlin, methods from Siegel et al. paper)
** support tabulated EOS, interpolators ported from O'Conner et al.
cleaned up slightly
** specific EOS generated depends on compile time code generation
** NRPyLeakage is simple leakage code based on Ruffert prescription
** both NRPyEOS and NRPyLeakage are independent of IllinoisGRMHD
** used in publication, documented in jupyter notebooks
** Gabriele asked about ID thorn for TOV stars, IllinoisGRMHD updates
will have a new TOV ID generator to produce EOS for this
** Sam will review IllinoisGRMHD

* particle_tracerET
** major bugfix earlier in 2022 that affected interaction with mesh
refinement
** this new change adds more output formats and more data that can be
output
** currently this custom format is the default
** may announce change of default output behaviour

* TwoPunctures_BBHSF and NPScalars_SF 
** produces initial data for black holes and a massive scalar field
** part of Canuda
** NPScalar_SF computes Newman-Penrose scalars including contributions
from these massive scalar fields
** Zach is asking if it would be possible to instead extend the
existing TP thorn. Helvi explained that some changes go beyond adding a
term to Tmunu since scalar field scale differently.
** Gabriele mentioned that TP was at one point extended to support
arbitray scaled matter field, but that this functionality was never
really well documented and seems tightly coupled to TOVSolver
** Gabriele brought up issue that default values of parameters in
TwoPunctures are no longer always the best, and that using Taylor
expansion for data interpolation is no longer required (for speed
reasons). In the new thorn the defaults could be modernized.

* kuibit update to 1.4
** mostly library update
** impossible to list pip dependencies that allow use with Python 3.7
** can use kuibit 3.6 on older clusters, "pip install kuibit" will
install the newest version available

chair next week: Sam
minutes next week: Steve

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpzBOtjN5mZO.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-01-26

[Steven R. Brandt]

Present Roland, Steve, Peter, Sam, Cheng-Hsin Cheng, Leo, Zach, 
Alexandru Dima, Helvi, Keith Dow

Contribution propoals for the next release were discusssed. Next week, 
decisions will be made about what will be included. Roland suggests that 
each contribution have a champion and a link that people can look at. 
CarpetX AsterX in the next release? An email will be sent out to reach 
out to the contributors.

SSL Cert for the mailing list archives. The URL for the testing system 
will probably work by next.

The first version of the ET swag shop is up: 
https://www.etsy.com/shop/EinsteinToolkitShop
Please send comments / thoughts / suggestions to sbra...@cct.lsu.edu

ET Meeting in Europe June 19-23 at the University Aveiro in Portugal.

The testsuite system needs a reviewer.

Majority of CCTK_REAL is used in the ET, but a float would break many of 
the thorns. We should start testing this feature. There is a CCTK_REAL8. 
CCTK_PRECISION can be set in the OptionList.

Various tickets are ready for review. Didn't look at all of these.

https://bitbucket.org/cactuscode/cactus/pull-requests/145/implement-scalars-default-to-everywhere

Next week:
     Peter Diener, famed in song and story, shall be chair!
     The Grand and Glorious Roland Haas shall be the minute taker!

Black Holes on Star Trek: 
https://www.tvfanatic.com/quotes/one-problem-at-a-time/

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-01-19

[Roland Haas]

Present: Roland, Leo, Cheng-Hsin, Helvi, Keith, Sam, Steve, Yosef, Zach

new ET modules proposed
* went over the prior existing suggestions
* TwoPunctures extended to add extra fields
** not yet fully sure if will have publication published by the time
of the inclusion
** Sam and Leo pointed out that this is not a strict requirement but
might be good to do to avoid others using the code for the planned
publication
* will vote on new features 2 weeks from now (Feb 2nd)

NR school at UNAM
* Helvi Witek is in contact with Miguel Alcubierre to set up a summer
  school on NR at UNAM
* will try and use ET tutorial server
* will contact Steve

Conference announcement
* Steve is co-organizing an adaptive many-tasks conference
  https://amte2023.stellar-group.org in Europe (Cyprus) in summer as
  part of EuroPar (https://2023.euro-par.org/)
* adaptive many tasks systems would be similar to HPX, Charm++,
  UIntah, etc. that have multiple small tasks that are scheduled in
  parallel
* audience is users and developers of these systems

SSL certificate for lists.einsteintoolkit.org
* Steve will contact LSU IT to ask to have the certificate update

Updates to agenda page
* agenda page for old agendas (pre 2017) has been separated from main
  agenda page

Steve is collection pictures related to past ET release
* will be available on mugs in Etsy store

Job openings:
* the AEI in Potsdam is advertising for a scientific programmer
  position
https://hyperspace.uni-frankfurt.de/2023/01/18/scientific-programmer-position-at-the-max-planck-institute-for-gravitational-physics-potsdam-germany/

No unanswered emails on the mailing list

No new interesting updated tickets.

No new tickets to review

Next week chair: Sam
Next week minutes: Steve

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpxxpvCpJ1J1.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting Minutes: 2023-01-12

[Yosef Zlochower]

Present: Zach, Roland, Leo, Samuel, Yosef.



Additional New features for next release:

    * Feature update for IGM (neutrino physics and tabulated EOS support)
    * https://docs.einsteintoolkit.org/et-docs/Release_Details


Announcements:
   * GRHayL Library version of GRMHD code under development at U Idaho
    (to generalize across infrastructures). To be presented at a telecon 
later in the year.

   * Pull requests for update so that ETK works on Mac M1 out of the box.

   *   Next US ET summer school will be at RIT. Details to be released 
soon.

Discussion: Improving build system and external library support.
  Building on a new cluster can be quite complicated and often 
requires sysadmin support.
  Perhaps improving the build system should be part of the next 
proposal.

Chair for next week: Leo
Minute Taker: Roland


Yosef

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] meeting minutes for 2023-01-05

[Roland Haas]

Present: Leo, Roland, Steve, Zach

nrpytutorial development:
* Sam and Steve were active over winter break modernizing code 
** Steve observes breakage in Giraffe code
** Zach will chat with Ken about this

ET release contribution
* Leo and Sam are release chair
* Fishbone-Moncrief gallery example: Roland will collect files and
  created website
* Elliptica reader: release managers will contact
* FUKA: release managers will contact 

Unaswered questions:
*
http://lists.einsteintoolkit.org/pipermail/users/2022-December/008802.html
is about ET summer school. Will answer, next US meeting is RIT, next EU
meeting is Miguel Zilhao (Portugal, Porto?)

FB-Moncrief ID and NRPy+ modernization:
* Zach's group will start on this
(https://bitbucket.org/einsteintoolkit/tickets/issues/2503/various-small-problems-with)
* Baikal will get read / writes support (by Sam)

tickets ready for review:
* Roland and Terrence will discuss WENO coefficient ticket 
https://bitbucket.org/einsteintoolkit/tickets/issues/2647
* Discussed McLachlan improvement ticket
  https://bitbucket.org/einsteintoolkit/tickets/issues/963 (Zach and
  Peter were working on this)

Yours,
Roland


pgp7HI5Q6JkOX.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2023-01-05

[Roland Haas]

Present: Leo, Roland, Steve, Zach

nrpytutorial development:
* Sam and Steve were active over winter break modernizing code 
** Steve observes breakage in Giraffe code
** Zach will chat with Ken about this

ET release contribution
* Leo and Sam are release chair
* Fishbone-Moncrief gallery example: Roland will collect files and
  created website
* Elliptica reader: release managers will contact
* FUKA: release managers will contact 

Unaswered questions:
*
http://lists.einsteintoolkit.org/pipermail/users/2022-December/008802.html
is about ET summer school. Will answer, next US meeting is RIT, next EU
meeting is Miguel Zilhao (Portugal, Porto?)

FB-Moncrief ID and NRPy+ modernization:
* Zach's group will start on this
(https://bitbucket.org/einsteintoolkit/tickets/issues/2503/various-small-problems-with)
* Baikal will get read / writes support (by Sam)

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpoYSozLAbQh.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] Meeting minutes for 2022-12-15

[Erik Schnetter]

Bruno

Yes, this was one of the main design goals of the Simulation Factory.
Many things can go wrong on supercomputers, and Simfactory automates
what one should do to be safe and efficient.

Of course, people can circumvent this safety feature by escaping
upwards with the output directory or by using hard-coded paths. Please
don't do this. Set the output directory to "$parfile".

-erik

On Fri, Dec 16, 2022 at 4:01 AM Bruno Giacomazzo
 wrote:
>>
>> - Safety feature to avoid HDF5 files from being corrupted
>>   * Leo requests a feature that would allow the user to e.g., generate one
>> output file per restart. With kuibit, there was interested in switching 
>> from
>> the ASCII data files to the HDF5 in our research group. However, in a 
>> recent
>> simulation it turned out that a node failure caused a crash as one of the
>> HDF5 was being written to and we lost all data for an important
>> gridfunction. If one HDF5 file was written per restart (or another safety
>> feature was in place), then this would have not been an issue, as only 
>> one
>> of the chunks of data would have been corrupted. Leo will open a ticket
>> about this.
>
>
> Isn't this done automatically when using simfactory? I have my hdf5 data 
> written in the separate output-00?? directories (the ones generated by 
> symfactory at each restart) so that if one run has problems I do not lose all 
> the data.
>
> Cheers,
> Bruno
>
>
> --
>
> Prof. Bruno Giacomazzo
> Department of Physics
> University of Milano-Bicocca
> Piazza della Scienza 3
> 20126 Milano
> Italy
>
> email: bruno.giacoma...@unimib.it
> phone: (+39) 02 6448 2321
> web: http://www.brunogiacomazzo.org
>
> -
> There are only 10 types of people in the world:
> Those who understand binary, and those who don't
> --
>
> ___
> Users mailing list
> Users@einsteintoolkit.org
> http://lists.einsteintoolkit.org/mailman/listinfo/users



-- 
Erik Schnetter 
http://www.perimeterinstitute.ca/personal/eschnetter/
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] Meeting minutes for 2022-12-15

[Bruno Giacomazzo]

When using simfactory, if I write in my parameter file that the output
directory for the hdf5 files is "./hdf5" (e.g., CarpetIOHDF5::out2D_dir =
"./hdf5_2D") then this directory is created in the output-00?? directory.
Therefore each run creates a separate hdf5 directory in the corresponding
output-00?? one. This is how I avoid losing all data if one run has
problems.

Cheers,
Bruno


Il giorno ven 16 dic 2022 alle ore 10:10 Samuel Tootle <
too...@itp.uni-frankfurt.de> ha scritto:

> I don't think this is by default per say.  I use batchtools (instead of
> simfactory) exactly for this reason and discourage new users from
> restarting in the same directory as the parent checkpoints to avoid this
> exact outcome.  An additional issue that can arise is if a job is
> terminated before the walltime such that the data stored in ASCII/HDF5 goes
> beyond the last checkpoint are potential ingestion issues due to data
> mismatching from the restart.  I think overall Kuibit handles this well,
> but it is has been an issue in the past for some users before learning to
> separate restarts into individual directories.
>
> Cheers,
>
> Samuel
> On 12/16/22 10:01 AM, Bruno Giacomazzo wrote:
>
> - Safety feature to avoid HDF5 files from being corrupted
>>   * Leo requests a feature that would allow the user to e.g., generate one
>> output file per restart. With kuibit, there was interested in
>> switching from
>> the ASCII data files to the HDF5 in our research group. However, in a
>> recent
>> simulation it turned out that a node failure caused a crash as one of
>> the
>> HDF5 was being written to and we lost all data for an important
>> gridfunction. If one HDF5 file was written per restart (or another
>> safety
>> feature was in place), then this would have not been an issue, as
>> only one
>> of the chunks of data would have been corrupted. Leo will open a
>> ticket
>> about this.
>>
>
> Isn't this done automatically when using simfactory? I have my hdf5 data
> written in the separate output-00?? directories (the ones generated by
> symfactory at each restart) so that if one run has problems I do not lose
> all the data.
>
> Cheers,
> Bruno
>
>
> --
>
> Prof. Bruno Giacomazzo
> Department of Physics
> University of Milano-Bicocca
> Piazza della Scienza 3
> 20126 Milano
> Italy
>
> email: bruno.giacoma...@unimib.it
> phone: (+39) 02 6448 2321
> web: http://www.brunogiacomazzo.org
>
> -
> There are only 10 types of people in the world:
> Those who understand binary, and those who don't
> --
>
> ___
> Users mailing 
> listUsers@einsteintoolkit.orghttp://lists.einsteintoolkit.org/mailman/listinfo/users
>
>

-- 

Prof. Bruno Giacomazzo
Department of Physics
University of Milano-Bicocca
Piazza della Scienza 3
20126 Milano
Italy

email: bruno.giacoma...@unimib.it
phone: (+39) 02 6448 2321
web: http://www.brunogiacomazzo.org

-
There are only 10 types of people in the world:
Those who understand binary, and those who don't
--
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] Meeting minutes for 2022-12-15

[Samuel Tootle]

I don't think this is by default per say.  I use batchtools (instead of 
simfactory) exactly for this reason and discourage new users from 
restarting in the same directory as the parent checkpoints to avoid this 
exact outcome.  An additional issue that can arise is if a job is 
terminated before the walltime such that the data stored in ASCII/HDF5 
goes beyond the last checkpoint are potential ingestion issues due to 
data mismatching from the restart.  I think overall Kuibit handles this 
well, but it is has been an issue in the past for some users before 
learning to separate restarts into individual directories.


Cheers,

Samuel

On 12/16/22 10:01 AM, Bruno Giacomazzo wrote:


- Safety feature to avoid HDF5 files from being corrupted
  * Leo requests a feature that would allow the user to e.g.,
generate one
    output file per restart. With kuibit, there was interested in
switching from
    the ASCII data files to the HDF5 in our research group.
However, in a recent
    simulation it turned out that a node failure caused a crash as
one of the
    HDF5 was being written to and we lost all data for an important
    gridfunction. If one HDF5 file was written per restart (or
another safety
    feature was in place), then this would have not been an issue,
as only one
    of the chunks of data would have been corrupted. Leo will open
a ticket
    about this.


Isn't this done automatically when using simfactory? I have my hdf5 
data written in the separate output-00?? directories (the ones 
generated by symfactory at each restart) so that if one run has 
problems I do not lose all the data.


Cheers,
Bruno


--

Prof. Bruno Giacomazzo
Department of Physics
University of Milano-Bicocca
Piazza della Scienza 3
20126 Milano
Italy

email: bruno.giacoma...@unimib.it
phone: (+39) 02 6448 2321
web: http://www.brunogiacomazzo.org

-
There are only 10 types of people in the world:
Those who understand binary, and those who don't
--


___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] Meeting minutes for 2022-12-15

[Bruno Giacomazzo]

>
> - Safety feature to avoid HDF5 files from being corrupted
>   * Leo requests a feature that would allow the user to e.g., generate one
> output file per restart. With kuibit, there was interested in
> switching from
> the ASCII data files to the HDF5 in our research group. However, in a
> recent
> simulation it turned out that a node failure caused a crash as one of
> the
> HDF5 was being written to and we lost all data for an important
> gridfunction. If one HDF5 file was written per restart (or another
> safety
> feature was in place), then this would have not been an issue, as only
> one
> of the chunks of data would have been corrupted. Leo will open a ticket
> about this.
>

Isn't this done automatically when using simfactory? I have my hdf5 data
written in the separate output-00?? directories (the ones generated by
symfactory at each restart) so that if one run has problems I do not lose
all the data.

Cheers,
Bruno


-- 

Prof. Bruno Giacomazzo
Department of Physics
University of Milano-Bicocca
Piazza della Scienza 3
20126 Milano
Italy

email: bruno.giacoma...@unimib.it
phone: (+39) 02 6448 2321
web: http://www.brunogiacomazzo.org

-
There are only 10 types of people in the world:
Those who understand binary, and those who don't
--
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2022-12-15

[Leonardo Rosa Werneck]

Hi all,

Here are the minutes for today’s meeting.

Cheers,
Leo



Chair  : Sam Cupp
Notes  : Leo Werneck
Present: Leo Werneck, Peter Diener, Sam Cupp, Steve Brandt

- Any updates on github & Ian Hindler's account?
  * Something was compromised with his account, but github canceled the token 
and
looks like nothing bad happened. Ian is working on it.

- ET mailing list archives not using good SSL certificates
  * Steve will contact Roland, since this was not discussed last week but was in
the agenda.

- Safety feature to avoid HDF5 files from being corrupted
  * Leo requests a feature that would allow the user to e.g., generate one
output file per restart. With kuibit, there was interested in switching from
the ASCII data files to the HDF5 in our research group. However, in a recent
simulation it turned out that a node failure caused a crash as one of the
HDF5 was being written to and we lost all data for an important
gridfunction. If one HDF5 file was written per restart (or another safety
feature was in place), then this would have not been an issue, as only one
of the chunks of data would have been corrupted. Leo will open a ticket
about this.

- Unanswered questions on the mailing list
  * Spandan Sarma had an issue where it seemed the code didn't scale as
expected, but Peter pointed out that the scaling is more or less what's
expected.

- Open tickets
  * All website related tickets are likely more interesting to Roland, so we
won't be discussing them today.

Next meeting : possibly Jan 5, 2023
Next chair   : pending
Next minute taker: pending


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] Meeting minutes for 2022-12-08

[Roland Haas]

Hello all,

> Chair: Sam
> Minute Taker: Roland

Some other commitments came up and I won't be able to make the call
(and thus cannot take minutes). If someone else could volunteer in
before the call that would be great.

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpQvKj3mapqP.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2022-12-08

[Steven R. Brandt]

Present: Steve, Roland, Keith, Yosef, Sam, Zach, Gabriele, Leo

Next ET release chair search: Leo, deputy chair: Sam

Call for new contribution proposals: Fishbone Moncrief Gallery Example 
developed at the ICERM Brown workshop

CarpetX prerelease
     These are the things we said we wanted: 
https://docs.einsteintoolkit.org/et-docs/CarpetX_Transition
     Have we finalized how CarpetX works? E.g. boundary conditions
     Maybe more people need to be involved in discussing the BC's?
     Need to schedule a BC call to make sure we have all our 
requirements?
     Will Hyperslab-based BC's work?
     Reach out to AMReX developers?
     Some classes might need to be renamed, maybe the repo, some thorns 
should be rearranged

Gabriele Bozzala
     Does not test that the code is converging
     Want the ability to run convergence tests for Thorn Developers
     Bring up on the CarpetX call?

Emails about Ian Hinder's account being compromised. Not sure whether 
this is a concern.

Zach has said that there is confusion over name of the toolkit. And it's 
true.
     Can we make it clearer that ET is not Cactus. (Einstein Toolkit != 
Cactus)
     Don't want people to feel non-cactus codes are being washed out.
     The debate should continue.

Ticket 963 Peter ran the gallery example and constraint damping. It made 
a big difference in the number of orbits. Reduced orbits from 5 to 2. 
Not sure what's wrong. Restarting with constraints output. Will discuss 
with Zach.

Chair: Sam
Minute Taker: Roland

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for December 1st 2022

[Bruno Giacomazzo]

Dear All,
   here are the minutes from today's meeting. Sorry if I missed someone or
something.

Present: Bruno Giacomazzo, Steven Brandt, Roland Haas, Peter Diener,
Beyhan, Artectek, Keith Dow, Zachariah Etienne, Gabriele Bozzola, Leo Rosa
Werneck, Allen Wen, Yosef Zlochower

Chair: Leo
Minutes: Bruno

*ET release recap
Roland provided a recap. The delay was due to the addition of an Intel
Compiler 19 workaround for GRHydro (it seems that there is a compiler bug
that cannot recognize properly that arrays were initialized). In the future
we also need a release co-chair because it is becoming too much work for
just one person. It was a very large release with 3 new contributions and a
large list of contributors with many students contributing to the testing.

Steve thinks we need to better recognize the work done by the release chair
since it is a lot of work. Moreover there is always some missing point in
the documentation regarding how to manage a new release.
Zach agrees with what Steve said.
We all thank Roland for his work for this release.


**missing tests in the new release
Roland pointed out that Canuda does not have a testsuite. We did not notice
it during the release.


*Next ET release chair search
The search has begun. Any volunteers?
Leo is thinking about it, but he is looking for a co-chair


*Open tickets:
#2670

(opened
by Artectek): it seems to be a failure in the Carpet grid division
algorithm happening after regridding, but Roland does not have an idea of
what is causing this. Nobody has seen this error before. Roland pointed out
that the sum of dens reported in INFO is ~1e-4 while the maximum of dens in
~1e-3, but this may be OK since the sum is the sum of the values weighted
by the grid spacing (to get the integral one would need to multiply the sum
by the coarsest grid spacing). Roland suggests having a look at the output
for dens. Artectek says that he did have a look at the output using the
script in the gallery example and the NSs are still there. Artectek's
simulation is based on the BNS gallery example.
Gabriele suggests starting from the gallery example and changing only one
parameter at a time. Artectek said that he only changed the EOS and the
grid setup for the initial separation of the NSs.  Roland suggests to run
with the same EOS of the gallery example (gamma law) and just change the
grid setup. If this works then Gabriele suggests trying with a piecewise
polytropic EOS.
Roland suggests producing a checkpoint at the iteration before the error
appears so that by looking at the checkpoint maybe we can understand what
is happening. Gabriele also noted that the memory requirement for the run
is 11 GB, which seems to be very low for a BNS. It could be that there are
too few points maybe shared through too many MPI processes (and therefore
each MPI may have too small grids). Gabriele and Roland therefore suggest
running it on only one node as a first test. Leo also suggests checking
that the NS is fully inside the finest grid (but this would probably give
errors in C2P and not in regridding, since regridding is done also if the
NS was fully contained).

#2671
:
Roland says that it would be useful to make it more clear to users which
information to share when encountering problems with the Einstein Toolkit.

#2665
:
it was opened by Gabriele. There seems to be a couple of typos in the
documentation of the TOV Solver and there was a pull request to fix it.


*Call for new contribution proposals
Fishbone-Moncrief gallery example
Elliptica reader (and solver timeline) ticket
FUKA reader announcement

Roland said that Elliptica and FUKA should be included in the next release
and we need to ping their authors. The  Fishbone-Moncrief gallery example
was done recently and it should be included.


*simfactory website repo server migration
Steve said it is going to be moved from svn to git.


*ET Jenkins replacement status update
Roland said that we used to run tests on Jenkins, but there was the idea to
move to git actions. Some students worked on this and it seems to be
working fine. See einsteintoolkit.github.io/tests/ (it runs on github
resources that we can access for free since the einstein toolkit has an
academic license and therefore we do not need to manage virtual machines by
ourselves as we do now). Github action also builds documentation every day.


*ET seminar and hackathon search
Roland says that we should start these again. Any volunteers for seminars?
What about a tutorial on NRPY Elliptic?


*NRPy+ plans
Zach said that Steve and Zach's group would like to get together for future
plans since Steve has a very nice fork of NRPY that improves users'
interaction. Zach also has his own fork with som

[Users] Meeting minutes for 2022-11-17

[Beyhan Karakaş]

Dear All,

Here are the minutes from today's meeting.

Present: Keith, Peter, Allen, Bruno, Zach, Yosef, Leo.

Chair: Roland
Minutes: Beyhan

ET release:


   - BBH gallery example: Allen has some issues with his cluster, still
   needs 24 hours to figure it out.
   - Canuda Scalar thorns: documentation and tests are missing. Roland
   added some documentations for the arrangement. It is too late for test
   cases, Roland will ask Miguel Zilhao to have some test cases.
   - Failures in GRHydro persists in many clusters (Sunrise, SuperMuc,
   Supermic, QueenBee, db1 and Cori):

Might be related to the intel compiler but stampede that also uses intel
compiler seems to be fine (Roland). Might depend on the version of the
compiler (Peter). Probably needs to be older than intel 19. Roland will try
and see whether it will be overcomed by using an older intel version on
QueenBee. It might also be a compiler bug. Yosef thinks he has reported
this. They were getting nans in the matter fields after the first iteration
with GRHydro (in the gallery example).

   - Release will probably not be made today due to these failures
   (Roland). Roland will  try to do it before December.


Unanswered questions


   - Issue with compiling ET on cluster [Shamim Haque]:
   http://lists.einsteintoolkit.org/pipermail/users/2022-November/008761.html

 Shamim was in contact with Erik and Roland who provided some suggestions.
It would be great to have a volunteer to answer Shahim's further questions
due to the release management of Roland.

Open tickets:

   - #2664
   

   : Publication webpage is outdated, sphinxcontrib-citations may help keeping
   it updated automatically:

Suggestion and comments by Roland, Zach, Bruno, Yosef and Leo on a better
way to find which papers are using ET.

Roland could give this a try. We are not going to get all of them because
Zenodo doi's are not very often cited, people sometimes still cite original
ET papers and sometimes do not cite anything just say we used ET.

Zach: script that looks at all thorns in the ET, finding any paper that
corresponds to them would really help with this.

Bruno: It would be a good idea to use the script and manually evaluate what
the script produced.

Roland: It requires familiarity with the people in the field.

Zach suggested that it would be effective to add this to a regular item in
each call. It is similar to the unanswered question in the mailing list.

Roland: We can put it on the agenda. We could also in principle have a web
form with just a text box that arxiv links could be dropped in.

Bruno: We can just search for the paper that cites the ET paper.

Zach  has some concerns because people who use the ET do not always cite
the ET paper, they cite the thorn they use.

Yosef: Most people that use ET just mean it.


   - #2663
   
:
   add a notice about minimum Intel / GCC version required to configure
   output:

Most of the questions about not compiling ET with intel compiler ends up
being wrong standard C++ template library.

For C++11, the error message given is not really helpful. Error message
should tell the user "you need to use at least intel 2017 and gcc 6".
(Roland)


   - #2662
   
:
   Try to get email from ~/.gitconfig:

Something to make simfactory look for a couple places to guess a good email
address when setting up a new machine (Roland).


   - #963:
   

   Improve McLachlan accuracy:


Peter gets things running on dbu. Running the standard version and trying
to rerun it with a modified version to see whether it makes a difference.
Zach would like to be informed about the difference if it exists.

Suggestion:

Zach:Would it be better for us to have a wiki page that just has tasks
anyone in the community could help (voluntarily) with (updating wiki pages
including list of publications and many other tasks which do not require a
lot of physics knowledge)?

Roland: Updated Summer student project wiki page (*
) and
will add this suggestion to the next agenda.

No call for next week due to thanksgiving.

Next call (12.01):

Chair: Leo

Minutes: Bruno

Best regards,

Beyhan.
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2022-11-10

[Beyhan Karakaş]

Present: Roland, Steve, Peter, Erik, Keith, Lorenzo, Samuel, Liwei, Maria,
Allen, Jay

Chair: Zach,
Minutes: Beyhan


ET Release update (Roland)

   - Release date has been pushed by a week.
   - Everything is up and ready.
   - Two gallery examples (BBH
   

   and Multipatch
   

   need to be updated).
   - Tests are running well on clusters.
   - Lots of failures in GRHydro which also exist in the last release in
   many clusters (Sunrise, Supermuc, Supermic, QueenBee, db1 and Cori).
   - There will be a new gallery example (Fishbone-Moncrief disk) that will
   probably be put after the release.
   - Canuda Scalar thorns do not have any documentation files, which are
   required for inclusion (See ticket
   

   )

Samuel's presentation on modularizing IllinoisGRMHD (Modular GRMHD:
Con2prim Routines)

   - Recording of the presentation is available at here
   
   - Working version of the code currently available at here
   

Unanswered questions:

   - Nothing was discussed.

Open tickets:

   - #2662: Try to get email from ~/.gitconfig (link
   
):
   It is not for this release and has a low priority (Steve).


Next call:


   - Next call chair: Roland
   - Next call minute taker: Beyhan


Best regards,

Beyhan.
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting Minutes for 2022-11-03

[Steven R. Brandt]

Nov 3, 2022
Present: Steve, Peter, Keith Dow, Beyhan, Johnny Taso, Roland, Leo, Zach

Release:
Behind on the testing
Nearly all contributers have responded
Larger than usual release team
Code freeze in 2-3 days

Debated bug fix for synchronizing scalars.

One person is not enough to do the release

The schedule was a bit weird and didn't quite work out. Roland will make 
notes to improve it.

Discussed https://docs.einsteintoolkit.org/et-docs/CarpetX_Transition
What thorns need to be ported over?

Ticket Roland wants in: incorrect WENO coefficients.
Terrance has been traveling. Zach and Leo will remind him.

Zach wants to break components of GRHydro into pieces,
e.g. Con2Prim, allowing them to mix and match. Zach is
trying to make an infrastructure agnostic version, usable
in other codes. Wants to show it off next week and get
comments.

Ticket 963 improve McLaughlin accuracy, cannot compile because
of hwloc and cactus fails with an assertion error.

Ticket 2364 avoid opening and closing hdf5 files during output. Next 
release.

Chair for next week: Zach
Minute taker for next week: Beyhan

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2022-10-27

[Bing-Jyun Tsao]

Present: Leo, Johnny, Roland, Steve, Peter, Keith, Beyhan, Samuel, Yosef

   - ET release
  - testing has started:
  
  http://einsteintoolkit.org/testsuite_results/index.php
 - M1 and generic macOS
 

issues
 [RH], clang doesn’t support openMP
 - homebrew moved [RH]
 - test run on sunrise [PD]
  - release name variation "Sophie Kowalevski" which is (say wikipedia)
  how she transcribed her name in her papers [RH], agreed in meeting
  - release teams definition update / [
  
  https://docs.einsteintoolkit.org/et-docs/Meeting_agenda#2020-05-21recap]:

 - release coordinator(s)
 - test runner(s)
 - gallery example runner(s)
 - anyone putting in work specifically for the release
 - reviewer(s): be members to ET, test runners and gallery example
 runners who will look at the contribution for the new release add as
 co-authors
  - Gallery runs:
  - BNS example
  

--
  Johnny: ran on expanse
  - BBH example
  

--
  Allen Wen (need email address): already ran without simfactory
  - Multipatch
  

--
  Hrishikesh Kalyanaraman, will be followed up by RH
  - Poisson
  

--
  Nadine Kuo
  - Single NS
  

--
  Lisa Leung
   - Tickets:
  - #2644 :
  issue with later version of ET
   - Other:
  - Switching to CarpetX [ES], need gallery example, maybe GW150914
  [SB], no sub-cycling in time with CarpetX might not be
sufficient for BBH,
  will need to check walltime for an example run [YZ], need documentation
  [LW], symmetry boundary condition and periodic boundary exist, but no
  radiated boundary condition yet [RH]
  - Citation of ET [YZ]
   - Next week
  - Chair: Peter
  - Minute: Steve
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2022-10-20

[Leonardo Rosa Werneck]

Hi all,

Here are the minutes from today’s meeting.

Chair: Sam
Minutes: Leo

Attending: Sam, Leo, Peter, Cheng-Hsin, Johnny, Keith, Roland, Zach, Yosef

ET Release
  - Progress is being made, fixing failures in clusters that
are not frequently used.
  - Odd compilation error in Summit cluster that hasn't
been fixed yet.
  - Roland will contact contributors.
  - All galery examples are running.
  - Peter will re-run tests on Sunrise.

Extension of PreSync's Capabilities with Carpet
  - PreSync is used to fill in ghostzones and apply BCs
automatically whenever needed.
  - Extend to fill in "auxiliary" quantities automatically
when access is requested. For example, if a thorn evolves
the BSSN variables, then it provides a function to compute
the ADMBase variables from the BSSN variables and PreSync
would schedule this function appropriately whenever another
function needs to access the ADMase variables.
  - CarpetX has some support for gridfunction dependencies, which
could potentially guide this process.
  - There are subtleties that could lead to circular logic, so
care must be taken when doing this.
  - First idea is to not deal with more complicated cases where
e.g., a function adds something to a gridfunction (like KO
dissipation). Instead, the new feature would fill in gridfunctions
for which a "READ" was requested but the gridfunctions are
not valid anywhere.

Open Tickets
  - No warning printed by the Perl scripts for ambiguous
READ/WRITES since it is unlikely users will attempt to use
the old PreSync standard. Zach suggests: would be nice to have
the option of being more verbose instead of completely removing
the warning, if possible. Roland says: maybe we can give the
Perl script another option, e.g. "Picky" (Verbose is already
taken) so that it can print the warnings if requested.
  - There is a ticket ready for review which prevents PreSync from
syncing coarser refinement levels if those have already been
synced previously.
  - There's an interface error in some of the ADMBase functions
where read-only gridfunctions are maked as "OUT". Sam says:
changing "OUT" to "IN" fixes the interface. Fix is working
on local machines but needs more testing.
  - cctk_time and cctk_deltas are wrong sometimes and a fix has
been implemented that needs further review.
  - With PreSync, CarpetRegrid does not change the validity of
the gridfunctions after a regrid, which leads to problems.
The logic should be: after a regrid all gridfunctions are
valid nowhere. Then the gridfunctions that support prolongation
can be flagged as valid in the interior after prolongation.
Everything else should remain invalid everywhere until something
else sets it.

Unanswered Questions in the Mailing List
  None.

Next Chair: Leo
Next Minutes: Johnny

Cheers,
Leo


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2022-10-13

[Roland Haas]

Present: Roland, Peter, Steve, Beyhan, Keith, Leo, Lorenzo, Sam, Zach

ET release:
* feature freeze today
* no progress testing yet
* Roland will send email with release date
* Roland will send email to contributors about authorship etc.
* Steve would like to get pull request to change scalar READS/WRITES t
  be "everywhere" by default

Tickets:
*
https://bitbucket.org/einsteintoolkit/tickets/issues/2654/pr-for-admanalysis-trace-function-with
ADMAnalysis has incorrect PROVIDES FUNCTION prototype (OUT when IN is
correct)

*
https://bitbucket.org/einsteintoolkit/tickets/issues/2653/carpet-presync-triggering-syncs-for-all
has a fix provided but needs to be reviewed

*
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2648/empty-applybcs-call-in-leanbssnmol
  has superfluous ApplyBCs schedule. Zach says this had been copied
  from Lean_public and can be removed. Zach suggests reaching out to
  Lean developers.

Unanswered questions:
*
http://lists.einsteintoolkit.org/pipermail/users/2022-October/008719.html
about reducing number of checkpoints. Roland suggests that reducing
checkpoint frequency to one per segment may not be worse than 8hrs
since output HDF5 files tend to get corrupted if open when the code
crashes

* Leo will create pull request to handle time refinement in tracers for
  IGM correctly

* Discussed how re-generating Baikal is handled about how to best
  handle inclusion of readme and documentation files. 

next week chair: Sam
next week minute taker: Leo

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.


pgpr_3iFgnPYO.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2022-10-06

[Bing-Jyun Tsao]

Present: Peter, Keith, Steven, Samuel, Roland, Beyhan, Johnny

   - ET release
  - Name was decided last time [RH]
  - testing has started:
  
  http://einsteintoolkit.org/testsuite_results/index.php BNS example
  (Johnny), BBH (Yosef’s student), Beyhan can do TOV or Multipatch
scalar but
  maybe for someone else who is new to ETK, will look for students [RH]
   - Mailing list questions
  - work station: writing an email [RH]
   - Issues
  - ticket #2653: performance issue, is currently fixed by commenting
  out the recursive loop, someone else should try to look and check [SC],
  will need a pull request [SB]
  - ticket #2639: th::regrid and CarpetLib has different time, send the
  parfile that shows the issue [SC] find where time is set for syncing the
  time levels, will need to check with Erik [RH]
  - ticket #2650: scalars have no interior so presync does not do
  anything [SC], partial fix is changing WRITE in presync [RH],
complete fix
  is to modify the parsing method [SB]
  - ticket #2643: add macros to Meudon ID thorns [SC]
  - ticket #2642: error when running on deep bayou [PD] default option
  takes g++ compiler that might be too old, will need to set the
g++ name gxx-name
  path-to-modern-g++ [RH]
  - major bug fix, will create a new ticket [LW]
  - set sourcebasedir in def.local.ini to current directory [SB]
  setup-silent as it is now should work, need to account for all users for
  people who use simfactory for other purposes than ETK [RH]
   - 2022-10-13 (-1mo): Feature freeze [RH]

Next call:

Chair: Steve
Minutes: Roland
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2022-09-29

[Roland Haas]

Present: Roland, Steve, Peter, Beyhan, Johnny, Keith, Leo, Sam, Yosef

ET testing:
* testing has begun
* keep test "life" for 1month rather than 1week
* would want some volunteers to run on some clusters
** Peter will run at Stockholm
* name for next release:  Kovalevskaya
(https://en.wikipedia.org/wiki/Sofya_Kovalevskaya)

* open questions:
** Roland to respond to
  http://lists.einsteintoolkit.org/pipermail/users/2022-August/008667.html
  suggest to write a basic SLURM Cactus batch

* open tickets:
** Peter will run tests runs on
  
https://bitbucket.org/einsteintoolkit/tickets/issues/963/improve-mclachlan-accuracy
** urgently need reviewer for
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2647/incorrect-weno-coefficient-in-grhydro-weno
** Leo has fix for
  
https://bitbucket.org/einsteintoolkit/tickets/issues/2642/toolkit-fails-to-compile-on-sawtooth-idaho
  and will apply it and close the ticket

chair next week: Peter
note taker: Johnny

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.


pgpSBmOpmStLa.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting Minutes 2022-09-22

[Bruno Giacomazzo]

Present:  Leo Werneck (chair), Bruno Giacomazzo (minute taker), Steven
Brandt, Beyhan, Peter Diener, Gabriele Bozzola, Keith, Roland Haas, Samuel
Cupp, Zachariah Etienne

*1) vote on modules suggested for inclusion for the next release*

*1a) Canuda scalar and complex fields* -- proposed by Miguel Zilhao
Zach asks if it was reviewed since he didn't hear anything recently. Roland
said that the review finished. No one opposed the inclusion.

*1b) FLRW solver* -- proposed by Hayley Macpherson
No one opposed the inclusion.

*1c) SelfForce-1D* -- proposed by Samuel Cupp
Peter mentioned that there are new functionalities that have been added.
Roland asked if there are new examples, but Peter said no. Steve asked for
examples with movies.
No one opposed the inclusion.

*1d) NRPyEllipticET* -- proposed by Leo Werneck, Thiago Assumpcao, Zach
Etienne
Roland mentioned that there are no test cases included with it.
No one opposed the inclusion with the condition that at least a test case
is added.

Leo mentioned that some documentation of WVUThorns does not show correctly
online and Roland said that html latex is stricter than pdflatex when
checking latex syntax.

*2) ET release*
Roland said that is coming up in November, all thorns compile, and all
tests pass.
Test results are available on
http://einsteintoolkit.org/testsuite_results/index.php

Steve reminded people that the subversion servers are going to be switched
off. There seems to be only one still used and maybe it is time to move to
another versioning system any repository still on svn.

Gabriele said that kuibit will soon require python 3. Zach mentioned that
NRPy will continue to support python 2 since python 2 is still used on many
HPC systems. Steve suggested checking which is the default version of
python available in the clusters we use.

*4) unanswered question on mailing list*
Regarding configuration of workstation (Indrani Banerjee): no volunteer
found to answer this question yet. Not clear whether it would be possible
to compute waveforms on a workstation.

*5) open tickets sorted by update time*
The following tickets were discussed:
#2644: NaNs in ADMBASE metric components with RNSID -> no volunteer found
#2642: Toolkit Fails to Compile on Sawtooth (Idaho National Lab) -> it
seems that C++ 14 standard is needed and this should solve the issue.
#2643: Adding read/write declarations to Meudon ID thorns -> it is a
pull request (please have a look)
#2641: ML_BSSN macro usage bypasses personalized read/write macros

*6) tickets ready for review*
#2630: WaveMoL example computes energy from unsynchronized grid variables
#181: AEILocalInterp should not off-centre the interpolation stencil by
default
#2364: avoid opening and closing HDF5 files multiple times during output
#963: Improve McLachlan accuracy

No comment on any of these tickets.

*7) Other comments*
Zach said that the update to Baikal will be a regular update (as suggested
by Roland) and he will open a pull request.


Next Chair: Steve
Next Minute Tacker: Roland
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] Meeting Minutes 2022-9-15

[Roland Haas]

Hello all,

> In that case, I probably just misunderstood the context, then.

if the (draft_) policy's meaning is not clear, then we should rewrite
it.

Ie essentially boils down to:

* announce in one release
* remove in the next (or the devel branches just after the announcement
  release)
* must not break tests by removing things

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://keys.gnupg.net.


pgpPlkjSUnwBa.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] Meeting Minutes 2022-9-15

[Cupp, Samuel D.]

>>* There is no established policy on deprecating or renaming
>>  thorns/parameters. Zach proposed that we should set a clear policy
>>  for the timescale on which these things can be done, and the
>>  procedure for doing so.

>Such a policy exists:

> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.einsteintoolkit.org%2Fet-docs%2FPolicies_to_retire_functionality&data=05%7C01%7Cscupp1%40my.apsu.edu%7Cabc0c1210dfb465155fd08da9740177c%7Cfe42fdbd8df8483a9700ff2693323482%7C0%7C0%7C637988598768257547%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=VwOw%2Fh%2Fh8JASmYgW9DfrlXZDsR0Dpsbo3yplmsZ7Dj0%3D&reserved=0

In that case, I probably just misunderstood the context, then.

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

Re: [Users] Meeting Minutes 2022-9-15

[Roland Haas]

Hello all,

Sorry for not making it, was chatting with locals.

>New ET modules:
>* no news on Canuda review

I will poke Taishi about his verdict. Asduming the expected positive response 
we can vote on inclusion in the call next week (and announce on the vote soon).

>* Haley has addressed reviewers' comments for FLRW solver

I will poke Zach about his verdict. Asduming the expected positive response we 
can vote on inclusion in the call next week (and announce on the vote soon).


>* Sam has addressed reviewers' comments for selfforce-1d;

I will poke Peter about his verdict. Asduming the expected positive response we 
can vote on inclusion in the call next week (and announce on the vote soon).


>  only remaining task is to regenerate the FORD documentation

Must be done for the release anyway (so not a blocker right now).


>* Leo has addressed reviewers' comments for NRPyEllipticET;
>  notebook which generates the code is not yet ready, but this
>  is currently being worked on

I will poke Cheng-Hsin and Guiseppe about their verdict. Asduming the expected 
positive response we can vote on inclusion in the call next week (and announce 
on the vote soon). The notebook is mot usually reviewed (since NRPy+ is not in 
the ET), only the generated code.


>* Baikal update: Zach hasn't heard back from Helvi. Since the
>  new Baikal is waiting for inclusion, the old Baikal's notebook
>  isn't being maintained in favor of the new Baikal


I suggest to treat this as a regular Baikal update with regular pull request 
and ticket which makes review open to all who want to give it a try.

>
>
>Other questions:
>* For the particle tracer Leo is working on, he needs to access the
>  cctk_timefac for the finest level, but cctk_timefac only gives the
>  value for the current time level. He does this by accessing the
>  Carpet parameter time_refinement_factors using CCTK_ParameterGet()
>  but wanted to know if there was a better/more 'proper' way of
>  accessing the parameters. The function is necessary because the
>  parameter is private and cannot be accessed via the interface.
>
>  Steve said that this is how he would do it, but Leo should send the
>  question to the mailing list to see if Roland or Erik have other
>  suggestions.

I woukd suggest to REQUIRE: Carpet tge include carpet.hh then use 
Carpet::timerefsfacs (or so) which is a C++ vector of tbe factors.

>
>* There is no established policy on deprecating or renaming
>  thorns/parameters. Zach proposed that we should set a clear policy
>  for the timescale on which these things can be done, and the
>  procedure for doing so.

Such a policy exists: 


https://docs.einsteintoolkit.org/et-docs/Policies_to_retire_functionality

>
>Zach said that one solution is to just leave it up to the user to
>decide if it is sane to update these quantities every n timesteps.

It is not easy to e en.do the "every n steps".



>Since several thorns need this feature, discussion arose on whether
>this should be provided as an aliased function by Carpet to easily
>provide access to the needed information to validate the given values
>of these types of output_every parameters. Carpet could set some
>out_every_safe that provides the minimal safest out_every.
>

yup.

Yours,
Roland
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting Minutes 2022-9-15

[Cupp, Samuel D.]

Present: Sam, Steve, Keith, Gabriele, Peter, Leo, Zach, Bruno

New ET modules:
* no news on Canuda review
* Haley has addressed reviewers' comments for FLRW solver
* Sam has addressed reviewers' comments for selfforce-1d;
  only remaining task is to regenerate the FORD documentation
* Leo has addressed reviewers' comments for NRPyEllipticET;
  notebook which generates the code is not yet ready, but this
  is currently being worked on
* Baikal update: Zach hasn't heard back from Helvi. Since the
  new Baikal is waiting for inclusion, the old Baikal's notebook
  isn't being maintained in favor of the new Baikal


Other questions:
* For the particle tracer Leo is working on, he needs to access the
  cctk_timefac for the finest level, but cctk_timefac only gives the
  value for the current time level. He does this by accessing the
  Carpet parameter time_refinement_factors using CCTK_ParameterGet()
  but wanted to know if there was a better/more 'proper' way of
  accessing the parameters. The function is necessary because the
  parameter is private and cannot be accessed via the interface.

  Steve said that this is how he would do it, but Leo should send the
  question to the mailing list to see if Roland or Erik have other
  suggestions.


* We would like to shut down the svn servers which aren't being used
  anymore and have been maintained for preservation of old thornlists
  for a long time. By shutting them down, we can see if anyone
  complains about it and also track down anywhere else on the website
  that might reference them.

* There is no established policy on deprecating or renaming
  thorns/parameters. Zach proposed that we should set a clear policy
  for the timescale on which these things can be done, and the
  procedure for doing so.

* Someone asked about whether there will be recordings of the European
  ET meeting, but Peter said there were not. However, they do intend
  to upload the slides at some point.

* Next European meeting was set and will be in Aveiro, Portugal.

Questions on the mailing list:
No one answered 'Running with SLURM' from last week; Steve will respond to it.
http://lists.einsteintoolkit.org/pipermail/users/2022-August/008667.html

Open tickets:
Gabriele brought up ticket issue #2629
https://bitbucket.org/einsteintoolkit/tickets/issues/2629/mol_pseudoevolution-vs-analysis
about when one should schedule in CCTK_ANALYSIS vs MoL_PseudoEvolution.
The particular concern is how this will interact with parameters such
as the calculate_constraints_every in LeanBSSNMoL, as some choices
of output_every type parameters will be invalid depending on the
number of active refinement levels. However, since these calculations
are expensive, it is important to be able to control the calculation
to only occur when it will be output.

Zach said that one solution is to just leave it up to the user to
decide if it is sane to update these quantities every n timesteps.

This is similar to the issue Leo has been facing with the particle
tracer, as it must compute and output only so many iterations.
However, Cactus counts iterations based on the smallest possible
refinement level, even if it isn't active. This leads to the need
to determine the dtfac for the finest possible level, which is the
topic of the question raised earlier. He then errors out if the
parameters will produce incorrect output.

Since several thorns need this feature, discussion arose on whether
this should be provided as an aliased function by Carpet to easily
provide access to the needed information to validate the given values
of these types of output_every parameters. Carpet could set some
out_every_safe that provides the minimal safest out_every.

The chair for next week (9/22) is Leo, and the minutes taker is Bruno.

   Samuel Cupp
   Postdoctoral Researcher
   Department of Physics
   University of Idaho
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2022-09-08

[Leonardo Rosa Werneck]

Hi all,

Here are the minutes from today’s meeting.

Chair: Peter Minutes: Leo Present: Peter Diener, Leo Werneck, Sam Cupp,
   Zach Etienne, Keith Dow, Steve Brandt, Beyhan, Gabriele Bozzola,
   Roland Haas, Yosef Zlochower

* Updates on new thorns proposed for inclusion and revisions:

  - NRPyEllipticET: got nice feedback from reviewers. Leo will be
working on updates over the next couple of weeks.

  - SelfForce1D: no progress during the past week.

  - Baikal: Updates are mostly going to be applied to the code
generation scripts.

  - FLRW Solver: thorn has been thoroughly reviewed for best
programming practices and is looking good.

  - Canuda Solver: No update since no reviewer or champion present.

* Zach announces: CarpetX-compatible versions of Baikal and IllinoisGRMHD
  will be developed. "BaikalX" may already exist, will look into it.

* Steve asks: can NRPyElliptic handle the surfaces of NSs when generating
  ID? Zach says: BNS initial data will be the subject of a future project.
  There is no doubt that NRPyElliptic can solve the elliptic problem, but
  the techniques requires for handling the NS surface are yet not
  completely clear.

* Zach suggests: some of ADMBase's functionalities (e.g., setting ID) should
  be moved elsewhere, as currently the thorn does (too) much more than what
  its name suggests.

Cheers,
Leo


--
Leonardo R. Werneck, Ph.D.
Postdoctoral researcher
Office EP 314 | Department of Physics | University of Idaho
875 Perimeter Dr. MS 0903
Moscow, ID 83844-0903, USA
leona...@uidaho.edu 
https://leowerneck.github.io 
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2022-08-25

[Roland Haas]

Present: Roland, Peter, Keith, Zach, Beyhan, Leo, Gabriele, Yosef

new ET modules:
* Zach reviewed FLRW solver, thumbs up for does no harm
** Roland to look at file not found error during compile
* no news on Canuda review
* Peter is reviewing SelfForce1D, thumbs up for harm
* NRPyEllipticET, Leo provided reviewers with parfile
** Leo to contact reviewers


Functionality to retire:
* keep ongoing list in wiki
* Roland will push pull request that remove use of "REQUIRES THORNS"
  from current ET thorns

Questions on mailing list:
* nothing new

Open tickets:
* Zach reported
  https://bitbucket.org/einsteintoolkit/tickets/issues/2635 that
  contains a compile time warning about working only with Cartesian
  grid, suggests removal

* New ticket https://bitbucket.org/einsteintoolkit/tickets/issues/2634/
  mentioning lack of documentation on CCTK_BUILTIN_EXPECT

* Two tickets on SummationByParts assigned to Peter,
  https://bitbucket.org/einsteintoolkit/tickets/issues/2633 and
  https://bitbucket.org/einsteintoolkit/tickets/issues/2632

* Roland filed a ticket about non-conforming handling of initial data
  selection parameters
  https://bitbucket.org/einsteintoolkit/tickets/issues/2631 . Zach
  provided some recap that setting shift to 0 rather than using
  LORENE's shift gives better evolution, in particular much less
  eccentricity. Gabriele suggests adding a comment about this to the
  gallery example BNS parfile, instead of using LORENE's shift. 

* Zach and Gabriele suggest adding warning about this to code, also
  suggest adding similar runtime warning for other settings that we
  expect to produce failures or strange results

* Gabriele brought
  up https://bitbucket.org/einsteintoolkit/tickets/issues/2629 . Had a
  discussion on how to handle scheduling to compute a quantity every N
  steps. Roland to send email about what he recollects about doing this
  in EVOL vs ANALYSIS to the mailing list.


No ET call next week due to overlap with EU ET summer school.

chair next time: Peter
minutes next time: Leo

Yours,
Roland
-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgpnAaRSMD2Ov.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] meeting minutes for 2022-08-18

[Roland Haas]

Present: Roland, Peter, Zach, Steve, Atul. Yosef, Beyhan, Cheng-Hsin

New contributions:

* Zach, Cheng-Hsin: NRPyElliptic, can compile with ET
* Hayley, Zach: FLRW started review, interacting, Python dependency
  removed
* Alireza, Beyhan: time constraints, will likely skip this review and
  release
* Sam, Peter: SelfForce1D checked out and compiling

Moving back to 9:00am
* accepted, no objections

gallery examples:
* BBH - Steve is working on it (simulation finished)

functionality to retire:
* Tmunu inheritance from ADMBase, StaticConformal
* REQUIRES THHORN is deprecated, Roland is replacing its use by REQUIRES
* also add CoordGauge

Messages on list:
*
http://lists.einsteintoolkit.org/pipermail/users/2022-August/008654.html is an 
internal Carpet issue, Roland thinks that it means that Carpet tries to copy 
between refinement levels with different resolution, wold need stack backtrace 
to see exactly where it is a called from 

*
  http://lists.einsteintoolkit.org/pipermail/users/2022-August/008649.html
  is an MPI failure on a new cluster. Roland was in contact with her,
  error message indicates a failure in MPI in reduction call. Strange
  thing is that error happens relatively late, since there should have
  been reductions before. Peter says he has seen delayed SEGFAULTs if
  one does not ask or enough memory in the submit script (#SBATCH --mem
  XXX).Beyhan notes that the output indicates that 1GB is required for
  the simulations

Open tickets:

* Zach worked on FishboneMoncrief ID thorn
  (https://bitbucket.org/einsteintoolkit/tickets/issues/2503) with
  significant updates in sympy / NRPy+. Will include improved
  integration with ADMBase's methods.
* Zach found that the proposed change can lead to slight differences
  wrt to centering
  (https://bitbucket.org/einsteintoolkit/tickets/issues/181)
* Improve McLachlan accuracy
  (https://bitbucket.org/einsteintoolkit/tickets/issues/963), Zach
  added a comment, found big improvement, Peter had looked in during
  hackathon at UIdaho, but could not find obvious change

ET conventions
* Zach brought up old code in ADMBase and non-intuitive required usage
* Yosef and Steve brought up suggested code style. Cactus has one in
  MaintGuide.pdf chapter A2 "Coding Style" (not online but in
  Cactus/docs/)

Zach suggested adding discussions on CarpetX transition to the agenda
* full transition and retirement for Carpet may be years in the future
* many thorns currently require re-implementation, adapting
* Steve mentions that for a while both stacks may need to be maintained
* Yosef suggests that if these are many small changes then these would
  be good hackathon projects
* Sam Cupp is working on porting IllinoisGRMHD to CarpetX
* Roland mentioned weekly CarpetX call Wed 9am Central Time (join
http://lists.einsteintoolkit.org/mailman/listinfo/carpetx-developers to receive 
Zoom invite )
* add as recurring item agenda

ET workshop in Dublin:
* no call during workshop (2 weeks from now)
* Steve, Peter, Roland in person
* Steve will present on NRPy+, will interact with Zach

Chair next week: Zach
Minutes next week: Roland (or Leo)

Yours,
Roland

-- 
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from http://pgp.mit.edu .


pgp_grxsN09Sj.pgp
Description: OpenPGP digital signature
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users

[Users] Meeting minutes for 2002-08-11

[Steven R. Brandt]

Present: Steve (minutes), Roland, Bruno, Peter, Beyhan (chair), Yosef, 
Zach, Leo

Done with the 3rd Spec Benchmark: CPU8
     We have some licensing issues that Roland discovered.

Canuda progress: The review has started and sent some comments to the 
authors so Roland is optimistic.

FLRW Solver progress: Zach isn't a cosmologist, but he is reviewing 
based on coding.

Selfforce-1D: Peter has started, so it's progressing

NRPyEllipticET: Leo says the review will start early next week, tests 
were done by hand. Some cleaning up of the code has been in progress. A 
script for the empirical formula for computing the damping was added 
because the referee asked for it.

Roland Haas is trying to make sure the reviews are done in a timely way 
and the results of the published paper can be reproduced.

Steve suggested adding coverage testing for new thorns.

ID Reader for Elliptica. Beyhan has compiled it, but that's all so far. 
This may not get reviewed by the end of the month and so we might move 
it to the next release. Roland is not sure if the library of data files 
are ready. Not in a publication. Roland would like the reader in the 
toolkit, but would prefer not to have something that's rushed.

BBH Gallery example is running.

New thorns should probably add a gallery example. This would also create 
more visibility. Zach thinks the self-force code should have a gallery 
example.

Functionality to remove ADMBase and StaticConformal inheritance from 
TmunuBase.

Zach wants StaticConformal to go altogether. We could deprecate it. 
Could replace it with a stub of a thorn that just errors out. Steve 
suggests we add compile/runtime warnings during the deprecation. Spec 
code has such a mechanism. We could do something similar. Yosef wants to 
not build deprecated unless the user chooses an option. Maybe disable it 
in the thorn list. So announce removal, then disabled, then removed.

Open tickets: Does number 2626 have an update? Roland is not sure the 
changes actually make things go faster.

Ticket 2598? Mostly about clusters at West Virginia. Started by Maria. 
The maintainer is Anush. Roland will check on the pull request.

Ticket 2625? Gabrielle Bozolla made a pull request.

Yosef tried to use GetComponents to checkout thorns. Multiple branches 
cannot doesn't work. Needs --no-shallow.

Yosef askes if we want to move the meeting back to 9am? We will put the 
question on the mailing list.

Steve will try and update the intro to cactus presentation to use the 
wave equtaion instead of the TOV star.

Volunteers for the next call: Roland will take minutes. Yosef will be chair.

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users