Hi all,
since our 1.1 migration we've had some misterious behaviours with
bibsched that, apparently, nobody else has. It may be related to having
more than one Invenio in a single system, but I haven't been able to
prove it.
One very curious behaviour is that the first task after midnight
switches to SCHEDULED state but doesn't run. On the test server it
happens to Traces, and on the production it happens to DDD. No matter
how many hours I spend (and I have spent many!) finding why and how, the
mystery continues.
I get the friendly daily mail as such:
Emergency from http://ddd.uab.cat: BibSched halted: Process bibsort
(task_id: 140997) was launched but seems not to be able to reach
RUNNING status.
Anyhow, I needed a mechanism to automate my daily manual task to put
bibsched into manual mode, know which is the task in SCHEDULED state,
run it and put bibsched back to automatic mode.
I have been patching bibsched to allow, at least, those basic scripting
capabilities. I don't know how many more tasks do I need (acKnowledge
errors, maybe?). I'm unsure on the names I have choosen.
As I feel that it may be useful to somebody else, so I submit it for
your consideration.
Comments welcome,
Ferran
BibSched: enable scripting commands [DRAFT]
BibSched is a curses commands with only a few command-line options. This
first patch adds a new command (appropiately called command) with two
options:
--mode=[automatic, manual]
--key=k:task_id
The first one allows to swith from manual to automatic modes, and the second
allows to apply commads to tasks. Currently only one is implemented: R for
run.
---
modules/bibsched/lib/bibsched.py | 155 +
1 files changed, 138 insertions(+), 17 deletions(-)
diff --git a/modules/bibsched/lib/bibsched.py b/modules/bibsched/lib/bibsched.py
index 01314ae..dfba9fb 100644
--- a/modules/bibsched/lib/bibsched.py
+++ b/modules/bibsched/lib/bibsched.py
@@ -31,6 +31,7 @@ import getopt
from itertools import chain
from socket import gethostname
from subprocess import Popen
+from cStringIO import StringIO
import signal
from invenio.bibtask_config import \
@@ -281,9 +282,78 @@ def bibsched_send_signal(proc, task_id, sig):
return False
return False
+def parse_report_queue_status():
+'''Get queue status parting the output of report_queue_status.
+
+Returns: a dictionary with the numeric task_id as key and a
+dictionary for each value
+'''
+# print "calling report_queue_status..."
+out = StringIO()
+report_queue_status(verbose=True, status=('WAITING', 'RUNNING', 'SCHEDULED'), stream=out)
+report = out.getvalue()
+tasks = {}
+for line in report.split('\n'):
+fields = {}
+if '"' in line:
+words = line.split('"')
+while words:
+word = words.pop(0)
+if word.endswith('='):
+key = word[:-1].split()[-1]
+value = words.pop(0)
+fields[key] = value
+key = int(fields['ID'])
+tasks[key] = fields
+return tasks
+
+
+def command(opt="", arg=""):
+'''Check command parameters and call Manager with the appropiate values'''
+
+print "opt = [%s] arg = [%s]" % (opt, arg)
+if opt in ('-m', '--mode'):
+arg = arg.upper()
+if 'AUTOMATIC'.startswith(arg):
+mode = 'A'
+elif 'MANUAL'.startswith(arg):
+mode = 'M'
+else:
+mode = None
+print >>sys.stderr,'Unknown mode: %s' % (arg)
+sys.exit(1)
+if mode:
+print 'Manager, mode = %s' % (mode)
+print 'redirect...'
+# old_stdout, old_stderr = redirect_stdout_and_stderr()
+old_stdout = sys.stdout
+Manager(old_stdout, mode)
+elif opt in ('-k', '--key'):
+if arg.count(':') != 1:
+print >>sys.stderr, "Error: syntax: K:task_id"
+sys.exit(1)
+else:
+(cmd, task_id) = arg.split(':')
+if len(cmd) == 1:
+cmd = cmd.upper()
+else:
+print >>sys.stderr, "Error: Key must be single character"
+sys.exit(1)
+if not task_id.isdigit():
+prit >>sys.stderr, "Error: task id not numeric"
+sys.exit(1)
+print 'Manager, command = %s. [%s] [%s]' % (arg, cmd, task_id)
+print 'redirect...'
+# old_stdout, old_stderr = redirect_stdout_and_stderr()
+old_stdout = sys.stdout
+task_id = int(task_id)
+if cmd == "R":
+Manager(old_stdout, cmd, task_id)
+
+
class Manager(object):
-def __init__(self, old_stdout):
+def __init__(self, old_stdout, key='', task_id=0):
import curses
import curses.panel
from curses.wrapper import wrapper
@@ -316,8 +386,40 @@ class Manager(object):
self.header_lines = 3
