EBernhardson has uploaded a new change for review. (
https://gerrit.wikimedia.org/r/394108 )
Change subject: Better error message if wiki missing in data_pipeline
......................................................................
Better error message if wiki missing in data_pipeline
Not sure how this happens, but on one run through viwiki was
in the input data but didn't make it to the check that we
have expected sampling. This change will still error, but will
give better messages about what went wrong.
Change-Id: If7af29be5022c0b374c9b8836322ccf074467575
---
M mjolnir/utilities/data_pipeline.py
1 file changed, 6 insertions(+), 2 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/search/MjoLniR
refs/changes/08/394108/1
diff --git a/mjolnir/utilities/data_pipeline.py
b/mjolnir/utilities/data_pipeline.py
index 20b06f0..ee479c8 100644
--- a/mjolnir/utilities/data_pipeline.py
+++ b/mjolnir/utilities/data_pipeline.py
@@ -145,8 +145,12 @@
for wiki in wikis:
# We cant have more samples than we started with
expected = min(samples_per_wiki, hit_page_id_counts[wiki])
- actual = actual_samples_per_wiki[wiki]
- if expected / float(actual) < samples_size_tolerance:
+ try:
+ actual = actual_samples_per_wiki[wiki]
+ except KeyError:
+ # This will probably still error, but give better messages.
+ actual = 0
+ if actual == 0 or expected / float(actual) < samples_size_tolerance:
not_enough_samples.append(
'Collected %d samples from %s which is less than %d%% of the
requested sample size %d'
% (actual, wiki, samples_size_tolerance*100, expected))
--
To view, visit https://gerrit.wikimedia.org/r/394108
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: If7af29be5022c0b374c9b8836322ccf074467575
Gerrit-PatchSet: 1
Gerrit-Project: search/MjoLniR
Gerrit-Branch: master
Gerrit-Owner: EBernhardson <ebernhard...@wikimedia.org>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
