jenkins-bot has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/390142 )
Change subject: Stop categorization job during database update
......................................................................
Stop categorization job during database update
This commits changes which exist on labs today.
Change-Id: I26784cd24b5ccd52202ca45f02f15ce4bbbfe990
---
M bin/categorize_images.sh
M bin/defaults.sh
M bin/update_monuments.sh
3 files changed, 10 insertions(+), 0 deletions(-)
Approvals:
Jean-Frédéric: Looks good to me, approved
jenkins-bot: Verified
diff --git a/bin/categorize_images.sh b/bin/categorize_images.sh
index 342b437..884f3d1 100755
--- a/bin/categorize_images.sh
+++ b/bin/categorize_images.sh
@@ -5,6 +5,9 @@
CURRENT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
. $CURRENT_DIR/defaults.sh
+# Use a virtual environment with our requirements
+source $VIRTUAL_ENV_PATH/bin/activate
+
# Make sure we are in our homedir
cd $HOME_DIR || exit
diff --git a/bin/defaults.sh b/bin/defaults.sh
index 29fd598..4b185a0 100644
--- a/bin/defaults.sh
+++ b/bin/defaults.sh
@@ -7,6 +7,7 @@
: ${SOURCE_PATH:=$HOME_DIR/heritage}
: ${ERFGOED_PATH:=$SOURCE_PATH/erfgoedbot}
: ${VIRTUAL_ENV_PATH:=$HOME_DIR/.venv}
+: ${LOGS_PATH:=$SOURCE_PATH/logs}
# Database config
: ${DATABASE:=s51138__heritage_p}
diff --git a/bin/update_monuments.sh b/bin/update_monuments.sh
index 2d96a9c..22ef27a 100755
--- a/bin/update_monuments.sh
+++ b/bin/update_monuments.sh
@@ -28,10 +28,16 @@
echo_time "Full source database update..."
$PYWIKIBOT_BIN $ERFGOED_PATH/update_database.py -fullupdate -log -skip_wd
+# stop categorization job as next stage locks the database
+jstop categorize_images
+
# Update the all monuments table
echo_time "Update monuments_all table..."
$MYSQL_BIN -h $DB_SERVER $DATABASE <
$ERFGOED_PATH/sql/fill_table_monuments_all.sql
+# restart the categorization job
+jsub -l release=trusty -mem 1000m -once -j y -o
$LOGS_PATH/categorize_images.log -N categorize_images
$SOURCE_PATH/bin/categorize_images.sh >> $LOGS_PATH/crontab.log
+
## Update the image table. Is now another job
# echo_time "Update image table..."
# PYWIKIBOT_BIN $ERFGOED_PATH/populate_image_table.py
--
To view, visit https://gerrit.wikimedia.org/r/390142
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I26784cd24b5ccd52202ca45f02f15ce4bbbfe990
Gerrit-PatchSet: 3
Gerrit-Project: labs/tools/heritage
Gerrit-Branch: master
Gerrit-Owner: Lokal Profil <lokal.pro...@gmail.com>
Gerrit-Reviewer: Jean-Frédéric <jeanfrederic.w...@gmail.com>
Gerrit-Reviewer: Lokal Profil <lokal.pro...@gmail.com>
Gerrit-Reviewer: Multichill <maar...@mdammers.nl>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
