jenkins-bot has submitted this change and it was merged. Change subject: Add tests for XML export v0.10 ......................................................................
Add tests for XML export v0.10 The XML unit tests use XML data files using export v0.3. MediaWiki export format is currently version v0.10. Moved XML export v0.3 data files into tests/data/xml subdirectory. Add allowed_failure test for edit summary not being decoded. Bug: T85334 Change-Id: I82fccdafcf302d77dd3c6c5ce7713e72a4412e5c --- R tests/data/xml/article-pear.xml R tests/data/xml/article-pyrus-utf16.xml R tests/data/xml/article-pyrus-utf16.xml.bz2 R tests/data/xml/article-pyrus.xml R tests/data/xml/article-pyrus.xml.bz2 A tests/data/xml/pair-0.10.xml M tests/xmlreader_tests.py 7 files changed, 172 insertions(+), 12 deletions(-) Approvals: John Vandenberg: Looks good to me, but someone else must approve XZise: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/data/article-pear.xml b/tests/data/xml/article-pear.xml similarity index 100% rename from tests/data/article-pear.xml rename to tests/data/xml/article-pear.xml diff --git a/tests/data/article-pyrus-utf16.xml b/tests/data/xml/article-pyrus-utf16.xml similarity index 100% rename from tests/data/article-pyrus-utf16.xml rename to tests/data/xml/article-pyrus-utf16.xml Binary files differ diff --git a/tests/data/article-pyrus-utf16.xml.bz2 b/tests/data/xml/article-pyrus-utf16.xml.bz2 similarity index 100% rename from tests/data/article-pyrus-utf16.xml.bz2 rename to tests/data/xml/article-pyrus-utf16.xml.bz2 Binary files differ diff --git a/tests/data/article-pyrus.xml b/tests/data/xml/article-pyrus.xml similarity index 100% rename from tests/data/article-pyrus.xml rename to tests/data/xml/article-pyrus.xml diff --git a/tests/data/article-pyrus.xml.bz2 b/tests/data/xml/article-pyrus.xml.bz2 similarity index 100% rename from tests/data/article-pyrus.xml.bz2 rename to tests/data/xml/article-pyrus.xml.bz2 Binary files differ diff --git a/tests/data/xml/pair-0.10.xml b/tests/data/xml/pair-0.10.xml new file mode 100644 index 0000000..500f4dd --- /dev/null +++ b/tests/data/xml/pair-0.10.xml @@ -0,0 +1,112 @@ +<mediawiki xmlns="http://www.mediawiki.org/xml/export-0.10/"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://www.mediawiki.org/xml/export-0.10/ http://www.mediawiki.org/xml/export-0.10.xsd"; version="0.10" xml:lang="en"> + <siteinfo> + <sitename>Wikipedia</sitename> + <dbname>enwiki</dbname> + <base>http://en.wikipedia.org/wiki/Main_Page</base> + <generator>MediaWiki 1.25wmf12</generator> + <case>first-letter</case> + <namespaces> + <namespace key="-2" case="first-letter">Media</namespace> + <namespace key="-1" case="first-letter">Special</namespace> + <namespace key="0" case="first-letter" /> + <namespace key="1" case="first-letter">Talk</namespace> + <namespace key="2" case="first-letter">User</namespace> + <namespace key="3" case="first-letter">User talk</namespace> + <namespace key="4" case="first-letter">Wikipedia</namespace> + <namespace key="5" case="first-letter">Wikipedia talk</namespace> + <namespace key="6" case="first-letter">File</namespace> + <namespace key="7" case="first-letter">File talk</namespace> + <namespace key="8" case="first-letter">MediaWiki</namespace> + <namespace key="9" case="first-letter">MediaWiki talk</namespace> + <namespace key="10" case="first-letter">Template</namespace> + <namespace key="11" case="first-letter">Template talk</namespace> + <namespace key="12" case="first-letter">Help</namespace> + <namespace key="13" case="first-letter">Help talk</namespace> + <namespace key="14" case="first-letter">Category</namespace> + <namespace key="15" case="first-letter">Category talk</namespace> + <namespace key="100" case="first-letter">Portal</namespace> + <namespace key="101" case="first-letter">Portal talk</namespace> + <namespace key="108" case="first-letter">Book</namespace> + <namespace key="109" case="first-letter">Book talk</namespace> + <namespace key="118" case="first-letter">Draft</namespace> + <namespace key="119" case="first-letter">Draft talk</namespace> + <namespace key="446" case="first-letter">Education Program</namespace> + <namespace key="447" case="first-letter">Education Program talk</namespace> + <namespace key="710" case="first-letter">TimedText</namespace> + <namespace key="711" case="first-letter">TimedText talk</namespace> + <namespace key="828" case="first-letter">Module</namespace> + <namespace key="829" case="first-letter">Module talk</namespace> + <namespace key="2600" case="first-letter">Topic</namespace> + </namespaces> + </siteinfo> + <page> + <title>Çullu, Agdam</title> + <ns>0</ns> + <id>19252820</id> + <revision> + <id>237382899</id> + <timestamp>2008-09-09T22:40:15Z</timestamp> + <contributor> + <username>Carlossuarez46</username> + <id>23407</id> + </contributor> + <comment>moved [[Çullu, Agdam]] to [[Çullu, Quzanlı]]:&#32;dab</comment> + <model>wikitext</model> + <format>text/x-wiki</format> + <text xml:space="preserve" bytes="30">#REDIRECT [[Çullu, Quzanlı]]</text> + <sha1>9onarlg8ywgp11wnrddqebdry0jyz56</sha1> + </revision> + <revision> + <id>237383099</id> + <parentid>237382899</parentid> + <timestamp>2008-09-09T22:41:28Z</timestamp> + <contributor> + <username>Carlossuarez46</username> + <id>23407</id> + </contributor> + <comment>create dab</comment> + <model>wikitext</model> + <format>text/x-wiki</format> + <text xml:space="preserve" bytes="305">'''Çullu, Agdam''' may refer to: +*[[Çullu, Quzanlı]] +*[[Çullu (Chullu Vtoroye), Agdam]] + +{{geodis}} + +<!-- Long comment to avoid being listed on short pages --></text> + <sha1>senkho7ycefskq5d8mllalt33oxiy7v</sha1> + </revision> + </page> + <page> + <title>Talk:Çullu, Agdam</title> + <ns>1</ns> + <id>19252824</id> + <revision> + <id>237382916</id> + <timestamp>2008-09-09T22:40:18Z</timestamp> + <contributor> + <username>Carlossuarez46</username> + <id>23407</id> + </contributor> + <comment>moved [[Talk:Çullu, Agdam]] to [[Talk:Çullu, Quzanlı]]:&#32;dab</comment> + <model>wikitext</model> + <format>text/x-wiki</format> + <text xml:space="preserve" bytes="35">#REDIRECT [[Talk:Çullu, Quzanlı]]</text> + <sha1>co53382onr803k6tzfhpi5g83b2egxf</sha1> + </revision> + <revision> + <id>237383127</id> + <parentid>237382916</parentid> + <timestamp>2008-09-09T22:41:38Z</timestamp> + <contributor> + <username>Carlossuarez46</username> + <id>23407</id> + </contributor> + <comment>proj</comment> + <model>wikitext</model> + <format>text/x-wiki</format> + <text xml:space="preserve" bytes="19">{{DisambigProject}}</text> + <sha1>efe5s8flu02zejuj539rfokh6bcwya6</sha1> + </revision> + </page> +</mediawiki> diff --git a/tests/xmlreader_tests.py b/tests/xmlreader_tests.py index 8399d0a..3681e19 100644 --- a/tests/xmlreader_tests.py +++ b/tests/xmlreader_tests.py @@ -7,11 +7,15 @@ # __version__ = '$Id$' - import os.path + from pywikibot import xmlreader + from tests import _data_dir from tests.aspects import unittest, TestCase +from tests.utils import allowed_failure + +_xml_data_dir = os.path.join(_data_dir, 'xml') class XmlReaderTestCase(TestCase): @@ -20,11 +24,19 @@ net = False + def _get_entries(self, filename, **kwargs): + entries = [r for r in + xmlreader.XmlDump(os.path.join(_xml_data_dir, filename), + **kwargs).parse()] + return entries + + +class ExportDotThreeTestCase(XmlReaderTestCase): + + """XML export version 0.3 tests.""" + def test_XmlDumpAllRevs(self): - pages = [r for r in - xmlreader.XmlDump(os.path.join(_data_dir, - "article-pear.xml"), - allrevisions=True).parse()] + pages = self._get_entries('article-pear.xml', allrevisions=True) self.assertEqual(4, len(pages)) self.assertEqual(u"Automated conversion", pages[0].comment) self.assertEqual(u"Pear", pages[0].title) @@ -34,9 +46,7 @@ self.assertEqual(u"Pear", pages[0].title) def test_XmlDumpFirstRev(self): - pages = [r for r in - xmlreader.XmlDump(os.path.join(_data_dir, - "article-pear.xml")).parse()] + pages = self._get_entries("article-pear.xml", allrevisions=False) self.assertEqual(1, len(pages)) self.assertEqual(u"Automated conversion", pages[0].comment) self.assertEqual(u"Pear", pages[0].title) @@ -45,15 +55,16 @@ self.assertTrue(not pages[0].isredirect) def test_XmlDumpRedirect(self): + pages = self._get_entries('article-pyrus.xml', allrevisions=True) pages = [r for r in - xmlreader.XmlDump(os.path.join(_data_dir, + xmlreader.XmlDump(os.path.join(_xml_data_dir, "article-pyrus.xml")).parse()] self.assertTrue(pages[0].isredirect) def _compare(self, previous, variant, all_revisions): - result = [entry.__dict__ for entry in xmlreader.XmlDump( - os.path.join(_data_dir, 'article-pyrus' + variant), - all_revisions).parse()] + entries = self._get_entries('article-pyrus' + variant, + allrevisions=all_revisions) + result = [entry.__dict__ for entry in entries] if previous: self.assertEqual(previous, result) return result @@ -72,6 +83,43 @@ self._compare_variants(False) +class ExportDotTenTestCase(XmlReaderTestCase): + + """XML export version 0.10 tests.""" + + def test_pair(self): + entries = self._get_entries('pair-0.10.xml', allrevisions=True) + self.assertEqual(4, len(entries)) + self.assertTrue(all(entry.title.endswith(u"Çullu, Agdam") + for entry in entries)) + self.assertTrue(all(entry.username == 'Carlossuarez46' + for entry in entries)) + self.assertTrue(all(entry.isredirect is False for entry in entries)) + + articles = [entry for entry in entries if entry.ns == "0"] + talks = [entry for entry in entries if entry.ns == "1"] + + self.assertEqual(2, len(articles)) + self.assertTrue(all(entry.id == "19252820" for entry in articles)) + self.assertTrue(all(u'Çullu, Quzanlı' in entry.text + for entry in articles)) + self.assertEqual(articles[0].text, u'#REDIRECT [[Çullu, Quzanlı]]') + + self.assertEqual(2, len(talks)) + self.assertTrue(all(entry.id == "19252824" for entry in talks)) + self.assertEqual(talks[1].text, '{{DisambigProject}}') + self.assertEqual(talks[1].comment, 'proj') + + @allowed_failure + def test_edit_summary_decoding(self): + """Test edit summaries are decoded.""" + entries = self._get_entries('pair-0.10.xml', allrevisions=True) + articles = [entry for entry in entries if entry.ns == "0"] + + self.assertEqual(articles[0].comment, + 'moved [[Çullu, Agdam]] to [[Çullu, Quzanlı]]: dab') + + if __name__ == '__main__': try: unittest.main() -- To view, visit https://gerrit.wikimedia.org/r/181713 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I82fccdafcf302d77dd3c6c5ce7713e72a4412e5c Gerrit-PatchSet: 3 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: John Vandenberg <[email protected]> Gerrit-Reviewer: John Vandenberg <[email protected]> Gerrit-Reviewer: Ladsgroup <[email protected]> Gerrit-Reviewer: Merlijn van Deen <[email protected]> Gerrit-Reviewer: XZise <[email protected]> Gerrit-Reviewer: jenkins-bot <> _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
