| Dalba added a comment. |
The problem can be minimized to:
(Make sure wsbeta-family.py is already created using the python -m generate_family_file 'http://en.wikisource.beta.wmflabs.org/' 'wsbeta' 'y' command)
>>> site = Site('en', 'wikipedia') >>> site.interwiki('oldwikisource') # takes some time Traceback (most recent call last): File "<input>", line 1, in <module> File "...ibot-core\pywikibot\site.py", line 944, in interwiki return self._interwikimap[prefix].site File "...ibot-core\pywikibot\site.py", line 699, in __getitem__ if isinstance(self._iw_sites[prefix].site, BaseSite): File "...ibot-core\pywikibot\site.py", line 667, in site self._site = pywikibot.Site(url=self.url) File "...ibot-core\pywikibot\__init__.py", line 854, in Site code = family.from_url(url) File "...ibot-core\pywikibot\family.py", line 1228, in from_url .format(url, ', '.join(str(s) for s in matched_sites))) RuntimeError: Found multiple matches for URL "https://wikisource.org/wiki/$1": wsbeta:vep, wsbeta:kbd, wsbeta:mt, wsbeta:srn, wsbeta:pap, wsbeta:sg, wsbeta:pcd, wsbeta:su, wsbeta:war, wsbeta:bug, wsbeta:ksh, wsbeta:ki, wsbeta:olo, wsbeta:kw, wsbeta:stq, wsbeta:nds, wsbeta:ky, wsbeta:gsw, wsbeta:pam, wsbeta:bat-smg, wsbeta:pa, wsbeta:tg, wsbeta:dv, wsbeta:egl, wsbeta:hak, wsbeta:ng, wsbeta:bpy, wsbeta:sn, wsbeta:rmy, wsbeta:lmo, wsbeta:ln, wsbeta:lb, wsbeta:lbe, wsbeta:bm, wsbeta:mo, wsbeta:arz, wsbeta:pms, wsbeta:arc, wsbeta:ext, wsbeta:lad, wsbeta:nap, wsbeta:ab, wsbeta:nv, wsbeta:wuu, wsbeta:an, wsbeta:nah, wsbeta:ie, wsbeta:ny, wsbeta:ve, wsbeta:frp, wsbeta:ka, wsbeta:sq, wsbeta:rm, wsbeta:ts, wsbeta:lij, wsbeta:yo, wsbeta:ik, wsbeta:min, wsbeta:oc, wsbeta:si, wsbeta:bjn, wsbeta:cr, wsbeta:roa-rup, wsbeta:ch, wsbeta:ps, wsbeta:ku, wsbeta:mi, wsbeta:hif, wsbeta:pdc, wsbeta:cu, wsbeta:tk, wsbeta:chy, wsbeta:nrm, wsbeta:yue, wsbeta:ne, wsbeta:wo, wsbeta:sc, wsbeta:ee, wsbeta:nov, wsbeta:dsb, wsbeta:ba, wsbeta:sgs, wsbeta:glk, wsbeta:bi, wsbeta:eu, wsbeta:nds-nl, wsbeta:ce, wsbeta:om, wsbeta:av, wsbeta:mai, wsbeta:szl, wsbeta:chr, wsbeta:got, wsbeta:gag, wsbeta:jam, wsbeta:mg, wsbeta:xmf, wsbeta:hi, wsbeta:fur, wsbeta:zu, wsbeta:ss, wsbeta:cbk-zam, wsbeta:ha, wsbeta:rn, wsbeta:tt, wsbeta:jbo, wsbeta:gan, wsbeta:pnt, wsbeta:zea, wsbeta:eml, wsbeta:af, wsbeta:ga, wsbeta:bar, wsbeta:rup, wsbeta:vro, wsbeta:io, wsbeta:tn, wsbeta:tcy, wsbeta:als, wsbeta:am, wsbeta:ak, wsbeta:ast, wsbeta:simple, wsbeta:gv, wsbeta:ilo, wsbeta:bh, wsbeta:pih, wsbeta:aa, wsbeta:myv, wsbeta:mus, wsbeta:wa, wsbeta:lrc, wsbeta:xal, wsbeta:ay, wsbeta:cv, wsbeta:lg, wsbeta:ug, wsbeta:ady, wsbeta:vls, wsbeta:pfl, wsbeta:tl, wsbeta:fj, wsbeta:se, wsbeta:pag, wsbeta:so, wsbeta:ur, wsbeta:lv, wsbeta:ace, wsbeta:ff, wsbeta:mrj, wsbeta:ii, wsbeta:diq, wsbeta:ltg, wsbeta:sw, wsbeta:my, wsbeta:gn, wsbeta:ia, wsbeta:xh, wsbeta:kg, wsbeta:lez, wsbeta:tet, wsbeta:to, wsbeta:sh, wsbeta:new, wsbeta:pi, wsbeta:rw, wsbeta:scn, wsbeta:gom, wsbeta:tw, wsbeta:roa-tara, wsbeta:cho, wsbeta:fiu-vro, wsbeta:km, wsbeta:krc, wsbeta:kv, wsbeta:mh, wsbeta:ty, wsbeta:uz, wsbeta:os, wsbeta:csb, wsbeta:kab, wsbeta:sco, wsbeta:zh-classical, wsbeta:koi, wsbeta:iu, wsbeta:ho, wsbeta:ti, wsbeta:cdo, wsbeta:sd, wsbeta:mzn, wsbeta:kr, wsbeta:azb, wsbeta:kl, wsbeta:st, wsbeta:vo, wsbeta:haw, wsbeta:hsb, wsbeta:map-bms, wsbeta:tyv, wsbeta:dz, wsbeta:mn, wsbeta:kk, wsbeta:sm, wsbeta:ms, wsbeta:fy, wsbeta:mwl, wsbeta:co, wsbeta:crh, wsbeta:udm, wsbeta:bo, wsbeta:tpi, wsbeta:lo, wsbeta:nso, wsbeta:qu, wsbeta:zh-yue, wsbeta:ig, wsbeta:nn, wsbeta:jv, wsbeta:kaa, wsbeta:lzh, wsbeta:mhr, wsbeta:mdf, wsbeta:kj, wsbeta:ks, wsbeta:frr, wsbeta:hz, wsbeta:bxr, wsbeta:pnb, wsbeta:rue, wsbeta:ceb, wsbeta:tum, wsbeta:na, wsbeta:za, wsbeta:ckb, wsbeta:gd, wsbeta:bcl
Some observations:
oldwikisource is mapped to https://wikisource.org/wiki/$1 via siteinfo:
>>> next(filter(lambda i: i['prefix'] == 'oldwikisource', site.siteinfo['interwikimap']))['url'] 'https://wikisource.org/wiki/$1' >>> next(filter(lambda i: i['prefix'] == 'wikisource', site.siteinfo['interwikimap']))['url'] 'https://en.wikisource.org/wiki/$1'At least part of the problem relies in the generated family file:
self.langs = { 'en': 'en.wikisource.beta.wmflabs.org', 'aa': 'wikisource.org', 'ab': 'wikisource.org', 'ace': 'wikisource.org', 'ady': 'wikisource.org', 'af': 'wikisource.org', 'ak': 'wikisource.org', 'als': 'wikisource.org', 'am': 'wikisource.org', 'an': 'wikisource.org', 'ang': 'ang.wikisource.org', 'ar': 'ar.wikisource.org', 'arc': 'wikisource.org',Many language codes have duplicate values in this dictionary.
When a beta cluster for language does not exist, for example https://aa.wikisource.org/wiki/, it will be redirected to https://wikisource.org/wiki/Main_Page. The result is those duplicate values in self.lang.
Here is what's happening:
- site.interwiki('oldwikisource') is called
- After a few internal calls, site._iw_sites entries are looked up to find any site that matches the requested prefix.
- _iw_sites first looks into siteinfo['interwikimap'] to find the URL for the requested prefix. It will see that the url for oldwikisource is https://wikisource.org/wiki/$1.
- Then it tries to create a site using the found URL using the _pywikibot.Site(url=""> command.
- Site tries to use Family.from_url(url) to find the code for the given URL.
- from_url uses the codes in Family.langs (provided by the generated family file) to find a matching code.
- As stated earlier, generated family file contains multiple duplicate values. Therefore multiple codes are found.
- from_url is sensitive to multiple matched codes and raises the RuntimeError.
Here are some possible solutions:
- When generating the family file, somehow eliminate duplicate values in Family.langs.
- Don't raise RuntimeError inside from_url, instead just warn the user and use the first matching code.
- Don't run this test on wsbeta. Wait for T120427 and other related issues on beta cluster to be resolved.
I'm going with the third option.
TASK DETAIL
EMAIL PREFERENCES
To: Dalba
Cc: Aklapper, pywikibot-bugs-list, Dalba, Magul, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, Sn1per, jayvdb, Masti, Alchimista, Rxy
Cc: Aklapper, pywikibot-bugs-list, Dalba, Magul, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, Sn1per, jayvdb, Masti, Alchimista, Rxy
_______________________________________________ pywikibot-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
