Hello, yesterday, we tried to add some accounting rules (new partitions for one of our accounts), but the new rules didn't work. After a restart of the slurmdbd, "sacctmgr list assoc" hangs and returns after a timeout with the error message "No error".
sacctmgr -vvv list assoc sacctmgr: debug: Reading slurm.conf file: /etc/slurm-llnl/slurm.conf sacctmgr: Accounting storage SLURMDBD plugin loaded with AuthInfo=(null) sacctmgr: auth plugin for Munge (http://code.google.com/p/munge/) loaded sacctmgr: debug: slurmdbd: Sent DbdInit msg sacctmgr: error: slurmdbd: Getting response to message type 1410 sacctmgr: error: slurmdbd: DBD_GET_ASSOCS failure: No error Error with request: No error sacctmgr: debug: slurmdbd: Sent fini msg while other request as "sacctmgr list user" still work perfectly. Even a reboot of the slurmctld/slurmdbd host didn't help. The mysql database seems to be ok, manual login works and requests on all tables seem to be possible. The slurm cluster is implemented on a Ubuntu 14.04.03 LTS system with slurm-llnl version 2.6.5, the slurmdbd config is appended below. Please let me know if you need further information. Any idea or hint whats going wrong? Best regards, Andreas Löbel sacctmgr show config Configuration data as of 2015-09-09T07:53:11 AccountingStorageBackupHost = (null) AccountingStorageHost = opt151 AccountingStorageLoc = N/A AccountingStoragePass = (null) AccountingStoragePort = 7031 AccountingStorageType = accounting_storage/slurmdbd AccountingStorageUser = N/A AuthType = auth/munge MessageTimeout = 10 sec PluginDir = /usr/lib/slurm PrivateData = none SlurmUserId = slurm(64030) SLURM_CONF = /etc/slurm-llnl/slurm.conf SLURM_VERSION = 2.6.5 TrackWCKey = 0 SlurmDBD configuration: ArchiveDir = /local/tmp ArchiveEvents = 1 ArchiveJobs = 1 ArchiveScript = (null) ArchiveSteps = 1 ArchiveSupend = 1 AuthInfo = (null) AuthType = auth/munge BOOT_TIME = 2015-09-09T00:54:50 DbdAddr = localhost DbdBackupHost = (null) DbdHost = opt151 DbdPort = 7031 DebugLevel = info DefaultQOS = (null) LogFile = /var/log/slurm-llnl/slurmdbd.log MessageTimeout = 300 secs PidFile = /var/run/slurm-llnl/slurmdbd.pid PluginDir = /usr/lib/slurm PrivateData = jobs,users PurgeEventAfter = NONE PurgeJobAfter = NONE PurgeResvAfter = 0 months PurgeStepAfter = NONE PurgeSuspendAfter = NONE SLURMDBD_CONF = /etc/slurm-llnl/slurmdbd.conf SLURMDBD_VERSION = 2.6.5 SlurmUser = slurm(64030) StorageBackupHost = (null) StorageHost = mysql.zib.de StorageLoc = opti_slurm_acct_db StoragePort = 3306 StorageType = accounting_storage/mysql StorageUser = opti_slurm TrackWCKey = 0 TrackSlurmctldDown = 0
