Erik Zachte has submitted this change and it was merged.
Change subject: Make dammit aggregation try to grab its semaphore only once
......................................................................
Make dammit aggregation try to grab its semaphore only once
Monthly dammit aggregation is started through dammit_compact_daily.sh,
which grabbed the “dammit_compact_monthly.semaphore” semaphore before
invocing the monthly aggregation bash script. This bash script then
again tried to grab the same semaphore before spawning the perl
aggregation script.
So monthly aggregation always failed because the second try to grab
the semaphore failed.
We now drop grabbing the semaphore from the daily aggregation bash
script, as the monthly aggregation bash script grabs the semaphore
anyways.
Bug: 65627
Bug: 66004
Change-Id: I3b9e78920f9285c9de2212886c4a7c5b44f52305
---
M dammit.lt/bash/dammit_compact_daily.sh
1 file changed, 1 insertion(+), 3 deletions(-)
Approvals:
Erik Zachte: Verified; Looks good to me, approved
jenkins-bot: Verified
diff --git a/dammit.lt/bash/dammit_compact_daily.sh
b/dammit.lt/bash/dammit_compact_daily.sh
index 58d0a01..05e1e1c 100755
--- a/dammit.lt/bash/dammit_compact_daily.sh
+++ b/dammit.lt/bash/dammit_compact_daily.sh
@@ -39,9 +39,7 @@
cmd="nice perl DammitCompactHourlyOrDailyPageCountFiles.pl $mode -a $maxage -i
$input -o $output -t $temp | tee -a $logfile | cat"
flock -n -e $bash/dammit_compact_daily.semaphore -c "$cmd" || { echo "Script
is already running: lock on ../bash/dammit_compact_daily.semaphore" ; exit 1 ;
} >&2
-echo_ "Consolidate pagecount files for whole month into one monthly file -
only finds work to do on first day of new month\n"
-# c1 $bash/dammit_compact_monthly.sh
-flock -n -e $bash/dammit_compact_monthly.semaphore -c
"$bash/dammit_compact_monthly.sh" || { echo "Script dammit_compact_monthly.sh
is already running: lock on ../bash/dammit_compact_monthly.semaphore" ; exit 1
; } >&2
+"$bash"/dammit_compact_monthly.sh || exit 1
echo_ "Publish new files\n"
rsync -arv --include=*.bz2 $output/* $dataset1001 | tee -a $logfile | cat
--
To view, visit https://gerrit.wikimedia.org/r/138405
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I3b9e78920f9285c9de2212886c4a7c5b44f52305
Gerrit-PatchSet: 1
Gerrit-Project: analytics/wikistats
Gerrit-Branch: master
Gerrit-Owner: QChris <[email protected]>
Gerrit-Reviewer: Erik Zachte <[email protected]>
Gerrit-Reviewer: QChris <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
