[slurm-dev] Dry run upgrade procedure for the slurmdbd database

[Ole Holm Nielsen]

We're planning to upgrade Slurm 16.05 to 17.02 soon.  The most critical 
step seems to me to be the upgrade of the slurmdbd database, which may 
also take tens of minutes.

I thought it's a good idea to test the slurmdbd database upgrade locally 
on a drained compute node in order to verify both correctness and the 
time required.

I've developed the dry run upgrade procedure documented in the Wiki page 
https://wiki.fysik.dtu.dk/niflheim/Slurm_installation#upgrading-slurm

Question 1: Would people who have real-world Slurm upgrade experience 
kindly offer comments on this procedure?

My testing was actually successful, and the database conversion took 
less than 5 minutes in our case.

A crucial step is starting the slurmdbd manually after the upgrade.  But 
how can we be sure that the database conversion has been 100% completed?

Question 2: Can anyone confirm that the output "slurmdbd: debug2: 
Everything rolled up" indeed signifies that conversion is complete?

Thanks,
Ole