Package: mailman3-web Version: 0+20180916-8 Severity: normal My server ran out of power today and totally crashed. This seems to have corrupted the Whoosh search database, because I would receive a traceback a minute by email after rebooting:
Subject: Cron <www-data@marcos> [ -f /usr/bin/django-admin ] && flock -n /var/run/mailman3-web/cron.minutely /usr/share/mailman3-web/manage.py runjobs minutely To: www-d...@anarc.at Date: Thu, 17 Oct 2019 14:43:34 -0400 [ERROR/MainProcess] Failed indexing 1 - 11 (retry 5/5): ord() expected a character, but string of length 0 found (pid 32248): ord() expected a character, but string of length 0 found Traceback (most recent call last): File "/usr/lib/python3/dist-packages/haystack/management/commands/update_index.py", line 97, in do_update backend.update(index, current_qs, commit=commit) File "/usr/lib/python3/dist-packages/haystack/backends/whoosh_backend.py", line 178, in update self.setup() File "/usr/lib/python3/dist-packages/haystack/backends/whoosh_backend.py", line 126, in setup self.index = self.storage.open_index(schema=self.schema) File "/usr/lib/python3/dist-packages/whoosh/filedb/filestore.py", line 176, in open_index return indexclass(self, schema=schema, indexname=indexname) File "/usr/lib/python3/dist-packages/whoosh/index.py", line 421, in __init__ TOC.read(self.storage, self.indexname, schema=self._schema) File "/usr/lib/python3/dist-packages/whoosh/index.py", line 632, in read check_size("int", _INT_SIZE) File "/usr/lib/python3/dist-packages/whoosh/index.py", line 626, in check_size sz = stream.read_varint() File "/usr/lib/python3/dist-packages/whoosh/filedb/structfile.py", line 191, in read_varint return read_varint(self.read) File "/usr/lib/python3/dist-packages/whoosh/util/varints.py", line 102, in read_varint b = ord(readfn(1)) TypeError: ord() expected a character, but string of length 0 found There are a few problems here: 1. it's very hard to act upon that error message: it doesn't say which file is corrupted or where 2. it sends this *every minute* which is a lot 3. it shouldn't happen in the first place: if the database is corrupted, it can regenerate it from scratch (maybe?) This, for the record, is the "corrupt" database: anarcat@marcos:~(master)$ sudo ls -al /var/lib/mailman3/web/fulltext_index.orig total 8 drwxr-xr-x 2 www-data www-data 4096 oct 17 10:46 . drwxr-x--- 5 www-data www-data 4096 oct 17 15:19 .. -rw-r--r-- 1 www-data www-data 0 oct 17 10:46 _MAIN_367516.toc -rw-r--r-- 1 www-data www-data 0 oct 17 10:46 MAIN_rv0glpgztufuptbi.seg -rwxr-xr-x 1 www-data www-data 0 fév 3 2019 MAIN_WRITELOCK ie. a bunch of empty files. That specific corruption could be easy to recover from... -- System Information: Debian Release: 10.1 APT prefers stable APT policy: (500, 'stable') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.19.0-6-amd64 (SMP w/2 CPU cores) Locale: LANG=fr_CA.UTF-8, LC_CTYPE=fr_CA.UTF-8 (charmap=UTF-8), LANGUAGE=fr_CA.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages mailman3-web depends on: ii dbconfig-sqlite3 2.0.11+deb10u1 ii debconf [debconf-2.0] 1.5.71 ii lsb-base 10.2019051400 ii node-less 1.6.3~dfsg-3 ii python3 3.7.3-1 ii python3-django-hyperkitty 1.2.2-1 ii python3-django-postorius 1.2.4-1 ii python3-mysqldb 1.3.10-2 ii python3-psycopg2 2.7.7-1 ii python3-whoosh 2.7.4+git6-g9134ad92-4 ii sassc 3.5.0-1 ii ucf 3.0038+nmu1 ii uwsgi 2.0.18-1 ii uwsgi-plugin-python3 2.0.18-1 Versions of packages mailman3-web recommends: ii nginx 1.14.2-2+deb10u1 ii nginx-full [nginx] 1.14.2-2+deb10u1 Versions of packages mailman3-web suggests: ii mariadb-server-10.3 [virtual-mysql-server] 1:10.3.17-0+deb10u1 -- debconf information: * mailman3-web/db/basepath: /var/lib/dbconfig-common/sqlite3/mailman3-web mailman3-web/remote/port: 3306 * mailman3-web/superuser-mail: root@localhost mailman3-web/pgsql/authmethod-user: password mailman3-web/mysql/method: Unix socket mailman3-web/internal/skip-preseed: false * mailman3-web/database-type: sqlite3 mailman3-web/passwords-do-not-match: mailman3-web/pgsql/manualconf: mailman3-web/pgsql/authmethod-admin: ident * mailman3-web/db/dbname: mailman3web.db mailman3-web/dbconfig-upgrade: true * mailman3-web/dbconfig-reinstall: false mailman3-web/purge: false * mailman3-web/mysql/admin-user: debian-sys-maint mailman3-web/upgrade-error: abort mailman3-web/install-error: abort * mailman3-web/django-site: lists.anarc.at mailman3-web/remote/newhost: mailman3-web/remove-error: abort mailman3-web/pgsql/admin-user: postgres * mailman3-web/dbconfig-install: true * mailman3-web/emailname: anarc.at mailman3-web/missing-db-package-error: abort * mailman3-web/configure-webserver: none mailman3-web/upgrade-backup: true mailman3-web/pgsql/method: TCP/IP mailman3-web/remote/host: localhost mailman3-web/nginx-choice: mailman3-web/pgsql/changeconf: false * mailman3-web/restart-webserver: false mailman3-web/internal/reconfiguring: false mailman3-web/dbconfig-remove: true mailman3-web/pgsql/no-empty-passwords: mailman3-web/db/app-user: mailman3-web@localhost * mailman3-web/superuser-name: admin