Mforns has uploaded a new change for review.
https://gerrit.wikimedia.org/r/169223
Change subject: Normalize windows line endings in cohort csvs
......................................................................
Normalize windows line endings in cohort csvs
When uploaded cohort csv files come from windows machines
they have the \r\n line endings.
These were normalized in a way that a blank line was inserted.
This fix prevents the blank line insertion.
Bug: 72581
Change-Id: I1f87b27bf667da76440929664da5b3c3ffa3d13e
---
M tests/test_controllers/test_cohort_upload.py
M wikimetrics/forms/cohort_upload.py
2 files changed, 7 insertions(+), 4 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/analytics/wikimetrics
refs/changes/23/169223/1
diff --git a/tests/test_controllers/test_cohort_upload.py
b/tests/test_controllers/test_cohort_upload.py
index 9caf521..f37cd98 100644
--- a/tests/test_controllers/test_cohort_upload.py
+++ b/tests/test_controllers/test_cohort_upload.py
@@ -35,14 +35,15 @@
def test_normalize_newlines(self):
stream = [
'blahblah\r',
- 'blahblahblahnor',
+ 'blahblahblah\r\n',
+ 'blahblahblahnormal',
'blahblah1\rblahblah2',
]
lines = list(normalize_newlines(stream))
assert_equal(len(lines), 5)
assert_equal(lines[0], 'blahblah')
- assert_equal(lines[1], '')
- assert_equal(lines[2], 'blahblahblahnor')
+ assert_equal(lines[1], 'blahblahblah')
+ assert_equal(lines[2], 'blahblahblahnormal')
assert_equal(lines[3], 'blahblah1')
assert_equal(lines[4], 'blahblah2')
diff --git a/wikimetrics/forms/cohort_upload.py
b/wikimetrics/forms/cohort_upload.py
index 63ef404..36c9687 100644
--- a/wikimetrics/forms/cohort_upload.py
+++ b/wikimetrics/forms/cohort_upload.py
@@ -105,7 +105,9 @@
for line in lines:
if '\r' in line:
for tok in line.split('\r'):
- yield tok
+ tok = tok.strip()
+ if tok != '':
+ yield tok
else:
yield line
--
To view, visit https://gerrit.wikimedia.org/r/169223
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1f87b27bf667da76440929664da5b3c3ffa3d13e
Gerrit-PatchSet: 1
Gerrit-Project: analytics/wikimetrics
Gerrit-Branch: master
Gerrit-Owner: Mforns <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
