Hi Niels Carl,

On 16-04-2021 14:41, Niels Carl Hansen wrote:
For each account do
      sacctmgr modify account name=<accountname> set GrpJobs=0

After sync'ing, resume with
      sacctmgr modify account name=<accountname> set GrpJobs=-1

Yes, but this would block all jobs from <accountname> immediately. If this account had a week-long job running, it couldn't run any shorter jobs until after the migration.

That's why I'm thinking that a system reservation excluding all jobs from <accountname> could be created several weeks in advance, so that <accountname> jobs could keep starting and running until they would be blocked by the reservation.

I'm thinking of a reservation something like this:

scontrol create reservation starttime=... duration=12:00:00 ReservationName=migrate_physics nodes=ALL Accounts=-physics

Would this work as expected?

Best regards,
Ole

On 16/04/2021 14.23, Ole Holm Nielsen wrote:
I need to migrate several sets of user home directories from an old NFS file server to a new NFS file server.  Each group of users belong to specific Slurm accounts organized in a hierarchical tree.

I want to make the migration while the cluster is in full production mode for all the other accounts (the terms "service window" or "downtime" don't exist for me :-)

My idea is to make a Slurm reservation so that the accounts in question will have zero jobs running during the reservation, and I also need to kick users off the login nodes.  During the reservation I can rsync the home directories from the old NFS server to the new NFS server and update the NFS automounter links.

Question:  Does anyone have experiences with this type of scenario? Any good ideas or suggestions for other methods for data migration?

Reply via email to