[slurm-users] Re: Slurmctld Problems
On 25.06.2024 17:54, stth via slurm-users wrote: Hi Timo, Thanks, The old data wasn’t important so I did that. I changed the line as follows in the /usr/lib/systemd/system/slurmctld.service : ExecStart=/usr/sbin/slurmctld --systemd -i $SLURMCTLD_OPTIONS You should be able to immediately remove it again. I'd have probably just launched slurmctld maually via cli with -i once. -- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-le...@lists.schedmd.com
[slurm-users] Re: Slurmctld Problems
Hi Timo, Thanks, The old data wasn’t important so I did that. I changed the line as follows in the /usr/lib/systemd/system/slurmctld.service : ExecStart=/usr/sbin/slurmctld --systemd -i $SLURMCTLD_OPTIONS Slurmctld is now active Timo Rothenpieler via slurm-users schrieb am Di. 25. Juni 2024 um 17:26: > On 25/06/2024 12:20, stth via slurm-users wrote: > > Jun 25 10:06:39 server slurmctld[63738]: slurmctld: fatal: Can not > > recover last_conf_lite, incompatible version, (9472 not between 9728 and > > 10240), start with '-i' to ignore this. Warning: using -i will lose the > > data that can't be recovered. > > Seems like it's not the first time, but the first time in a long while. > If there is no important data in that old db, just do what the error > says as a one-off. > > -- > slurm-users mailing list -- slurm-users@lists.schedmd.com > To unsubscribe send an email to slurm-users-le...@lists.schedmd.com > -- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-le...@lists.schedmd.com
[slurm-users] Re: Slurmctld Problems
On 25/06/2024 12:20, stth via slurm-users wrote: Jun 25 10:06:39 server slurmctld[63738]: slurmctld: fatal: Can not recover last_conf_lite, incompatible version, (9472 not between 9728 and 10240), start with '-i' to ignore this. Warning: using -i will lose the data that can't be recovered. Seems like it's not the first time, but the first time in a long while. If there is no important data in that old db, just do what the error says as a one-off. -- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-le...@lists.schedmd.com
[slurm-users] Re: Slurmctld Problems
Hello Lorenzo, Thank you for your reply. Yes I got the 23.11.4 version. Lorenzo Bosio schrieb am Di. 25. Juni 2024 um 16:50: > Hello, > > I suppose the actual error is: > > slurmctld: fatal: Can not recover last_conf_lite, incompatible version, > (9472 not between 9728 and 10240), start with '-i' to ignore this. > Warning: using -i will lose the data that can't be recovered. > > did you upgrade from Slurm 21.08 (9472) to your actual version 23.11 > (10240) ? See here for numbers reference: > https://github.com/SchedMD/slurm/blob/40058e4df5fa243f4c340db9622ed559ce771778/src/common/slurm_protocol_common.h#L63 > > You have to stay in a 2 releases window for the upgrades to work. > > Best regards, > Lorenzo > On 25/06/24 16:30, stth via slurm-users wrote: > > Hello, > slurmctld.log and journalctl -u slurmctld --no-pager give the same info as > I have already provided. > “ Referenced but unset environment variable evaluates to an empty string: > SLURMCTLD_OPTIONS* " has to do with the files on /etc/default > (slurmdbd/slurmctld/slurmd), where there is a line: SLURMDBD_OPTIONS="". > > But it does not have anything to do with the fact that the deamon is not > active > > On Tue, Jun 25, 2024 at 3:49 PM daijiangkuicgo--- via slurm-users < > slurm-users@lists.schedmd.com> wrote: > >> What's your “ Referenced but unset environment variable evaluates to an >> empty string: >> SLURMCTLD_OPTIONS* ”？ Meanwhile, you can check slurmctld.log and >> journalctl -u slurmctld --no-pager. >> >> -- >> slurm-users mailing list -- slurm-users@lists.schedmd.com >> To unsubscribe send an email to slurm-users-le...@lists.schedmd.com >> > > > -- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-le...@lists.schedmd.com
[slurm-users] Re: Slurmctld Problems
Hello, I suppose the actual error is: slurmctld: fatal: Can not recover last_conf_lite, incompatible version, (9472 not between 9728 and 10240), start with '-i' to ignore this. Warning: using -i will lose the data that can't be recovered. did you upgrade from Slurm 21.08 (9472) to your actual version 23.11 (10240) ? See here for numbers reference: https://github.com/SchedMD/slurm/blob/40058e4df5fa243f4c340db9622ed559ce771778/src/common/slurm_protocol_common.h#L63 You have to stay in a 2 releases window for the upgrades to work. Best regards, Lorenzo On 25/06/24 16:30, stth via slurm-users wrote: Hello, slurmctld.log and journalctl -u slurmctld --no-pager give the same info as I have already provided. “ Referenced but unset environment variable evaluates to an empty string: SLURMCTLD_OPTIONS* " has to do with the files on /etc/default (slurmdbd/slurmctld/slurmd), where there is a line: SLURMDBD_OPTIONS="". But it does not have anything to do with the fact that the deamon is not active On Tue, Jun 25, 2024 at 3:49 PM daijiangkuicgo--- via slurm-users wrote: What's your “ Referenced but unset environment variable evaluates to an empty string: SLURMCTLD_OPTIONS* ”？ Meanwhile, you can check slurmctld.log and journalctl -u slurmctld --no-pager. -- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-le...@lists.schedmd.com -- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-le...@lists.schedmd.com
[slurm-users] Re: Slurmctld Problems
Hello, slurmctld.log and journalctl -u slurmctld --no-pager give the same info as I have already provided. “ Referenced but unset environment variable evaluates to an empty string: SLURMCTLD_OPTIONS* " has to do with the files on /etc/default (slurmdbd/slurmctld/slurmd), where there is a line: SLURMDBD_OPTIONS="". But it does not have anything to do with the fact that the deamon is not active On Tue, Jun 25, 2024 at 3:49 PM daijiangkuicgo--- via slurm-users < slurm-users@lists.schedmd.com> wrote: > What's your “ Referenced but unset environment variable evaluates to an > empty string: > SLURMCTLD_OPTIONS* ”？ Meanwhile, you can check slurmctld.log and > journalctl -u slurmctld --no-pager. > > -- > slurm-users mailing list -- slurm-users@lists.schedmd.com > To unsubscribe send an email to slurm-users-le...@lists.schedmd.com > -- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-le...@lists.schedmd.com
[slurm-users] Re: Slurmctld Problems
What's your “ Referenced but unset environment variable evaluates to an empty string: SLURMCTLD_OPTIONS* ”？ Meanwhile, you can check slurmctld.log and journalctl -u slurmctld --no-pager. -- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-le...@lists.schedmd.com
