[Wikidata-bugs] [Maniphest] [Changed Subscribers] T91981: Show an auto-generated article description when a user-contributed one is unavailable.
Popcorndude added a subscriber: Popcorndude. TASK DETAIL https://phabricator.wikimedia.org/T91981 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Popcorndude Cc: Popcorndude, Agabi10, StudiesWorld, Filceolaire, Nemo_bis, Jdlrobson, Magnus, thiemowmde, Lydia_Pintscher, Tgr, Deskana, bearND, Dbrant, Aklapper, Cxyshine, Wikidata-bugs, aude, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T105126: Evaluate pattern constraints (safely)
Popcorndude added a comment. What if backtracking, grouping, and alternatives were all disabled, and each property could have multiple format constraints, and a value would pass if it matched any of them? Most paterns would need rewriting, but only for about 18 of them would it actually be difficult. TASK DETAIL https://phabricator.wikimedia.org/T105126 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Popcorndude Cc: thiemowmde, Nikki, Popcorndude, Aklapper, daniel, Wikidata-bugs, aude, GWicke, csteipp ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T105126: Evaluate pattern constraints (safely)
Popcorndude added a comment.
I messed with the constraints a bit, and it would be pretty easy to get up to
~50% with the constraints you outlined (the numbers I gave before may have
forgotten to skip newlines, lowering the count). Adding + and * covers 3/4, and
most of the rest could be rewritten without to much trouble (other than
https://phabricator.wikimedia.org/P1793 and possibly a few others that are
really basically impossible).
(\\.|[^(){}\[\]\\]|\{\d+(,\d+)?\}|\[(?!^)(\\\[|\\\]|[^\[\]])*\])*
should work (though it does let character classes through).
After a bit more fiddling:
https://phabricator.wikimedia.org/P234, https://phabricator.wikimedia.org/P274,
https://phabricator.wikimedia.org/P305, P353,
https://phabricator.wikimedia.org/P395, https://phabricator.wikimedia.org/P428,
https://phabricator.wikimedia.org/P529, https://phabricator.wikimedia.org/P866,
https://phabricator.wikimedia.org/P998,
https://phabricator.wikimedia.org/P1190,
https://phabricator.wikimedia.org/P1256,
https://phabricator.wikimedia.org/P1257,
https://phabricator.wikimedia.org/P1472,
https://phabricator.wikimedia.org/P1612,
https://phabricator.wikimedia.org/P1662,
https://phabricator.wikimedia.org/P1793,
https://phabricator.wikimedia.org/P1986,
https://phabricator.wikimedia.org/P2015 are the only properties where I don't
see a way to rewrite the format with your constraints plus infinite repetition
(i.e. no groups), and with a lot of these there probably is a upper limit, it's
just that no one bothered to write it into the format.
With that in mind, the list shortens to https://phabricator.wikimedia.org/P234,
https://phabricator.wikimedia.org/P274, https://phabricator.wikimedia.org/P428,
https://phabricator.wikimedia.org/P1472,
https://phabricator.wikimedia.org/P1612,
https://phabricator.wikimedia.org/P1793 (most of which are probably things you
didn't intend to support anyway).
TASK DETAIL
https://phabricator.wikimedia.org/T105126
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Popcorndude
Cc: Nikki, Popcorndude, Aklapper, daniel, Wikidata-bugs, aude, GWicke, csteipp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T105126: Evaluate pattern constraints (safely)
Popcorndude added a comment.
this matches the constraints I suggested:
^(?!.*?\.(\+|\*|\{\d+,\})\()(\\.|[^()\\\[\]]|\[([^\\\[\]]|\\.)*\]|\((?!\?)(\\.|[^()\\]|\[([^\\\[\]]|\\.)*\])*\))+$
these convert infinite repetition (other than ##.*()##, ##.+()##, and
##.{n,}()##) to atomic groups:
| ##(\[([^\[\]]|\\.)*\])(\+|\*|\{\d+,\})## | ##(?>\1\2)##|
| ##((?\1\2)##|
| ##((?\1\2)##|
| ##(\(([^()]|\\.)*\))(\+|\*|\{\d+,\})## | ##(?>\1\2)##|
| ##\.(\+|\*|\{\d+,\})(\\.|[^()\[\]\\])## | ##(?>[^\2]\1)\2## |
| ##\.(\+|\*|\{\d+,\})\[(([^\\\[\]]|\\.)*)\]## | ##(?>[^\2]\1)[\2]## |
(incidentally, this accepts 669/715 (93%) of the current constraints)
TASK DETAIL
https://phabricator.wikimedia.org/T105126
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Popcorndude
Cc: Nikki, Popcorndude, Aklapper, daniel, Wikidata-bugs, aude, GWicke, csteipp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T105126: Evaluate pattern constraints (safely)
Popcorndude added a comment. Those criteria accept 62 (8%) of the current constraints. Adding character classes (\d is everywhere) brings it up to 166 (23%) I would suggest allowing infinite repetition if the thing being repeated cannot overlap with the next thing. Although to prevent that from requiring the checking regex to execute the format on itself maybe infinite repetition should be allowed on character classes if they don't overlap with the following thing. (e.g. \d*[a-z] but not \d*0), and then also treat them as atomic (automatically convert \d+ to (?>\d+)) It would probably make things a lot easier for the people making the formats if each property was allowed to have multiple format strings (match this or this). As for grouping, maybe allow it but only non-nested atomic, since the alternative is to rewrite this (admittedly slightly extreme example) as a flat list of alternatives https?://((academia|android|anime|apple|arduino|astronomy|aviation|beer|bicycles|biology|bitcoin|blender|boardgames|bricks|buddhism|chemistry|chess|chinese|christianity|codegolf|codereview|cogsci|cooking|craftcms|crypto|cs|cstheory|datascience|dba|diy|drupal|dsp|earthscience|ebooks|electronics|ell|emacs|english|expatriates|expressionengine|fitness|freelancing|french|gamedev|gaming|gardening|genealogy|german|gis|graphicdesign|ham|hermeneutics|hinduism|history|homebrew|islam|italian|japanese|joomla|judaism|linguistics|magento|martialarts|math|matheducators|mathematica|mechanics|meta|moderators|money|movies|music|networkengineering|opendata|outdoors|parenting|patents|pets|philosophy|photo|physics|pm|poker|politics|productivity|programmers|puzzling|quant|raspberrypi|reverseengineering|robotics|rpg|russian|salesforce|scicomp|scifi|security|sharepoint|skeptics|softwarerecs|sound|space|spanish|sports|sqa|startups|stats|sustainability|tex|tor|travel|tridion|unix|ux|video|webapps|webmasters| windowsphone|wordpress|workplace|writers)\.stackexchange\.com|askubuntu\.com|mathoverflow\.net|pt\.stackoverflow\.com|serverfault\.com|stackapps\.com|stackoverflow\.com|superuser\.com)/(tags|questions/tagged)/.* (https://phabricator.wikimedia.org/P1482) I'll see if I can make some regexs for this in a bit. TASK DETAIL https://phabricator.wikimedia.org/T105126 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Popcorndude Cc: Popcorndude, Aklapper, daniel, Wikidata-bugs, aude, GWicke, csteipp ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T105126: Evaluate pattern constraints (safely)
Popcorndude added a comment.
My apologies. I eliminated those in my initial analysis and forgot to mention
it. The full list of things with backslashes in front of them:
bdDpsSwx2()[]{}|^\/$?+*,-.
TASK DETAIL
https://phabricator.wikimedia.org/T105126
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Popcorndude
Cc: Nikki, Popcorndude, Aklapper, daniel, Wikidata-bugs, aude, GWicke, csteipp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T105126: Evaluate pattern constraints (safely)
Popcorndude added a comment.
I did some analysis of what regex features are actually used:
https://www.wikidata.org/wiki/User:Popcorndude/formats
Only 6 properties use backrefs, lookahead, or lookbehind:
https://phabricator.wikimedia.org/P1256,
https://phabricator.wikimedia.org/P1257,
https://phabricator.wikimedia.org/P1297,
https://phabricator.wikimedia.org/P1472,
https://phabricator.wikimedia.org/P1612,
https://phabricator.wikimedia.org/P1793. Everything else uses only "?*+ [] () |
{} {,} \d\D\s\S\w ."
Is this analysis helpful? If so, is there anything I else you'd like me to
check?
TASK DETAIL
https://phabricator.wikimedia.org/T105126
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Popcorndude
Cc: Popcorndude, Aklapper, daniel, Wikidata-bugs, aude, GWicke, csteipp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Subscribers] T105126: Evaluate pattern constraints (safely)
Popcorndude added a subscriber: Popcorndude. TASK DETAIL https://phabricator.wikimedia.org/T105126 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Popcorndude Cc: Popcorndude, Aklapper, daniel, Wikidata-bugs, aude, GWicke, csteipp ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T105126: Evaluate pattern constraints (safely)
Popcorndude added a comment. Of those 6 properties, 2 have "optionally the same character twice", 2 have "does not start with", and the other 2 are actually non-capturing groups that I misidentified as lookarounds. TASK DETAIL https://phabricator.wikimedia.org/T105126 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Popcorndude Cc: Popcorndude, Aklapper, daniel, Wikidata-bugs, aude, GWicke, csteipp ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T105126: Evaluate pattern constraints (safely)
Popcorndude added a comment.
\(?\|?([^*+(){}\[\]]|\[(\\\]|[^\]])*\]|\(([^()*+]|[+*]\|)*\)|(?!<\\)\\[+*]|(\\d|\[0-9\])([*+]|\{\d+(,\d+|)\})(?!\\d|\d|\[0-9)|\{\d+\})*([*+]|\{\d+(,\d+|)\})?\|?\)?
matches 624 of the ok ones, and should only match ok ones, though some will
fail.
TASK DETAIL
https://phabricator.wikimedia.org/T105126
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Popcorndude
Cc: Popcorndude, Aklapper, daniel, Wikidata-bugs, aude, GWicke, csteipp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T105126: Evaluate pattern constraints (safely)
Popcorndude added a comment.
I letting things like 0?\d{8} through the filter, and most of what's left is
checking file extensions. I can make them not backtrack at all if commons
filenames don't contain periods (I don't know what characters are allowed).
They are generally of the form .*\.()
Besides files and the 6 I mentioned before, the only ones I see are
https://phabricator.wikimedia.org/P1949 oai:.*:.*
https://phabricator.wikimedia.org/P274
([αβγδφωλμπ]-)?([([]*[A-Z☐][ub]?[a-z]?[₁₂₃₄₅₆₇₈₉₀]*(\)?[¹²³⁴⁵⁶⁷⁸⁹⁰]*[⁺⁻]?)?[])|,₁₂₃₄₅₆₇₈₉₀]*(·\(?[-0-9.]*n?\)?)?)+
https://phabricator.wikimedia.org/P367 .+ (S|s)ymbol.*\.svg
https://phabricator.wikimedia.org/P395 ([A-Z0-9\p{L}]+(,\w)?)+
https://phabricator.wikimedia.org/P866
(([a-z]+|39)\-)*([1-9][0-9]*|[a-z]+|[1-9][a-z0-9][a-z0-9])
everything else either doesn't have this problem (as far as I can tell) or are
guaranteed to be looking for less than 10 characters.
TASK DETAIL
https://phabricator.wikimedia.org/T105126
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Popcorndude
Cc: Popcorndude, Aklapper, daniel, Wikidata-bugs, aude, GWicke, csteipp
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Subscribers] T90871: write additional tours
Popcorndude added a subscriber: Popcorndude. TASK DETAIL https://phabricator.wikimedia.org/T90871 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Popcorndude Cc: Popcorndude, Aklapper, Lucie, Bene, Lydia_Pintscher, Wikidata-bugs, aude ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Subscribers] T98172: [Story] Implement Unit Selector widget
Popcorndude added a subscriber: Popcorndude. TASK DETAIL https://phabricator.wikimedia.org/T98172 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Jonas, Popcorndude Cc: Popcorndude, Conny, Micru, Ricordisamoa, Jc3s5h, Jonas, daniel, Aklapper, Snaterlicious, adrianheine, Tobi_WMDE_SW, thiemowmde, Snipre, Lydia_Pintscher, Bene, Wikidata-bugs, aude, Malyacko ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T106748: Display What statements link here on item pages and add a Wikibase-Client function to access these statements from the 'object' item
Popcorndude added a comment. Maybe just load the first ~20 statements with a Load More button? Incidentally, this would less than double Q2 (71 statements, linked to by 68 items) TASK DETAIL https://phabricator.wikimedia.org/T106748 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Popcorndude Cc: Daniel_Mietchen, Lydia_Pintscher, Bene, Multichill, Pasleim, Mbch331, Popcorndude, Filceolaire, Aklapper, Wikidata-bugs, aude, Malyacko, P.Copp ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Subscribers] T106748: Display What statements link here on item pages and add a Wikibase-Client function to access these statements from the 'object' item
Popcorndude added a subscriber: Popcorndude. TASK DETAIL https://phabricator.wikimedia.org/T106748 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Popcorndude Cc: Popcorndude, Filceolaire, Aklapper, Wikidata-bugs, aude, Malyacko, P.Copp ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
