[Wikidata-bugs] [Maniphest] [Commented On] T124054: [Task] Convert wikibase and data values libraries to use extension registration
Reedy added a comment. I propose we actually close this invalid, and just move these to being included properly by composer into vendorTASK DETAILhttps://phabricator.wikimedia.org/T124054EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ReedyCc: Reedy, Ricordisamoa, Liuxinyu970226, aude, Aklapper, QZanden, Salgo60, D3r1ck01, Izno, Wikidata-bugs, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Closed] T146740: Create an operational reconciliation service for Wikidata on OpenRefine
Pintoch closed this task as "Resolved".Pintoch claimed this task.Pintoch added a comment. This issue is resolved, the feature was released in OpenRefine 2.7 rc1.TASK DETAILhttps://phabricator.wikimedia.org/T146740EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: PintochCc: Pintoch, Micru, Esc3300, Pauljmackay, -jem-, Spinster, Magnus, Aklapper, QZanden, Salgo60, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Edited] T161527: Canonical data URLs for machine readable page content
daniel edited the task description. (Show Details) EDIT DETAILS...* The ```/data/ path is rewritten to a special page, Special:PageData * Special Special:PageData will redirect (with status 303) to an appropriate (and typically cacheable) URL for retrieving the page data. For now, this will use the action="" style="padding: 0 2px; color: #33; background: rgba(251, 175, 175, .7);">``` interface * The REST API offers fairly clean URLs, but they still expose details about the web application and API version. Even the fact that they expose the fact that this is an API is too specific in a context where URLs are used as identifiers. * Apply conten* "URLs don't negotiation to the established page URLs using the ```/wiki/``` path ed to be pretty" * The ``/wiki/`` path really points to a UI for interacting with the pageWhile URLs do not have to be pretty, they should be stable, especially when they are to be used as stable unique identifiers. Remocing all application specific information from the URL provides more stability by adding a layer of abstraction. == Open Questions and Concerns == * We could apply content negotiation to the established page URLs using the `/wiki/` path. Such URLs are already in use for referring to Wikipedia pages in RDF (e.g. Using it to refer to the page content can be confusingby DBpedia and also by Wikidata). On the other hand, the `/wiki/` path is really a UI entry point, such URLs are already in use for referring to Wikipedia pages in RDFand it seems like a good idea to keep the UI separate from the data identifiers. * "URLs don't need to be pretty"The proposed URL scheme does not have room for slot names. We will not be able to refer to slots other than the main slot. Possible solution: https://commons.wikimedia.org/data/main/Avignon_City_Wall.map. This is looking more and more thike the REST API URLs. * While URLs do* The porposed schemes are not have to be pretty,stable against page renames. they shWe could be stable,use page IDs instead of the title. especially when they are to be used as stable unique identifiers.That makes the URLs a lot less intuitive, Remocing all application specific information from the URL provides more stability by adding a layer of abstraction. and requires database access in order to construct them. == Resources ==...TASK DETAILhttps://phabricator.wikimedia.org/T161527EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: MZMcBride, Rybesh, Dzahn, GWicke, tstarling, Aklapper, Jonas, Smalyshev, mkroetzsch, Lydia_Pintscher, daniel, QZanden, Salgo60, D3r1ck01, Izno, suriyaa, Eevans, mobrovac, Hardikj, Wikidata-bugs, aude, jayvdb, Southparkfan, fbstj, RobLa-WMF, santhosh, Mbch331, Jay8g, Ltrlg, Glaisher, bd808, Krenair, Legoktm___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T161527: Canonical data URLs for machine readable page content
daniel added a comment. @MZMcBride good point about MCR slots. When designing a scheme like this, we should account for this. I of all people should know. I suppose I need to amend my proposal. With respect to /w/index.php, /w/api.php: we are not treating them as stable identifier prefixes. They are entry points. Tim asked me to not use the term URI, since they are the same as URLs in this context. But perhaps it's a useful distinction after all: not everything that is a decent URL is an acceptable URI. URI should be really stable, not for 10 years but for 100. It should be easy and straight forward to make them work with a completely different technology. Having ".php" anywhere in there is a no-go.TASK DETAILhttps://phabricator.wikimedia.org/T161527EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: MZMcBride, Rybesh, Dzahn, GWicke, tstarling, Aklapper, Jonas, Smalyshev, mkroetzsch, Lydia_Pintscher, daniel, QZanden, Salgo60, D3r1ck01, Izno, suriyaa, Eevans, mobrovac, Hardikj, Wikidata-bugs, aude, jayvdb, Southparkfan, fbstj, RobLa-WMF, santhosh, Mbch331, Jay8g, Ltrlg, Glaisher, bd808, Krenair, Legoktm___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Edited] T161527: Canonical data URLs for machine readable page content
daniel edited the task description. (Show Details) EDIT DETAILS...* Do not include the namespace after /data/, e.g. https://commons.wikimedia.org/data/Avignon_City_Wall.map TBD * That would mean this URL pattern cannot be used as a general mechanism to refer to page content. It would be specific to the Data namespace on Commons. * Use "raw" instead of "data", e.g. https://commons.wikimedia.org/raw/Data:Avignon_City_Wall.map TBD * "raw" is less descriptive, and may not be correct if content negotiation is applied. * Use REST API URLS TBD * The REST API offers fairly clean URLs, but they still expose details about the web application and API version. Even the fact that they expose the fact that this is an API is too specific in a context where URLs are used as identifiers. * Apply content negotiation to the established page URLs using the ```/wiki/``` path TBD * The ``/wiki/`` path really points to a UI for interacting with the page. Using it to refer to the page content can be confusing. On the other hand, such URLs are already in use for referring to Wikipedia pages in RDF. * "URLs don't need to be pretty" TBD * While URLs do not have to be pretty, they should be stable, especially when they are to be used as stable unique identifiers. Remocing all application specific information from the URL provides more stability by adding a layer of abstraction. == Resources ==...TASK DETAILhttps://phabricator.wikimedia.org/T161527EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: MZMcBride, Rybesh, Dzahn, GWicke, tstarling, Aklapper, Jonas, Smalyshev, mkroetzsch, Lydia_Pintscher, daniel, QZanden, Salgo60, D3r1ck01, Izno, suriyaa, Eevans, mobrovac, Hardikj, Wikidata-bugs, aude, jayvdb, Southparkfan, fbstj, RobLa-WMF, santhosh, Mbch331, Jay8g, Ltrlg, Glaisher, bd808, Krenair, Legoktm___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T150950: Additions to WQS autocompletion
Esc3300 added a comment. Works. Thanks! Reminds me that I mainly know SPARQL thanks to (and with) Autocomplete !TASK DETAILhttps://phabricator.wikimedia.org/T150950EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, Esc3300Cc: gerritbot, TerraCodes, Jonas, Aklapper, Esc3300, Adik2382, Soteriaspace, Th3d3v1ls, JakeTheDeveloper, Ramalepe, Liugev6, QZanden, EBjune, merbst, Salgo60, Avner, Lewizho99, Maathavan, DatGuy, debt, Gehel, Abbe98, D3r1ck01, FloNight, Xmlizer, MuhammadShuaib, Izno, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, aude, Deskana, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T161527: Canonical data URLs for machine readable page content
MZMcBride added a comment. "E.g. https://www.wikidata.org/w/index.php?title=Q23=""> does not work." seems completely orthogonal to this task. We already have /w/index.php, /w/api.php, and /wiki/ entry points. These are stable paths and we have more than ten years of evidence of this. Using /raw/ would essentially be implementing action paths (cf. T19981: Short URLs for history pages (set up wgActionPaths for Wikimedia sites))? Part of the beauty of using /wiki/ is that it's vaguely more internationalized. During discussions of proposals to change /wiki/ to use action paths such as /view/ or /history/, using only English-language verbs in the URL has been criticized for its lack of localization. Maybe I'm still just missing the obvious, but with a move toward "multi-content revisions," I would think the concept of wanting to get "all the data from the English Wikipedia's Barack Obama article" (i.e., /data/Barack_Obama) would become even murkier and ill-defined. I still don't understand who the audience is and what their use-cases are for this proposed new entry point. I guess I need to focus on T159517: [RFC] RDF mapping for geo-shape / URIs for commons data pages.TASK DETAILhttps://phabricator.wikimedia.org/T161527EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MZMcBrideCc: MZMcBride, Rybesh, Dzahn, GWicke, tstarling, Aklapper, Jonas, Smalyshev, mkroetzsch, Lydia_Pintscher, daniel, QZanden, Salgo60, D3r1ck01, Izno, suriyaa, Eevans, mobrovac, Hardikj, Wikidata-bugs, aude, jayvdb, Southparkfan, fbstj, RobLa-WMF, santhosh, Mbch331, Jay8g, Ltrlg, Glaisher, bd808, Krenair, Legoktm___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T119976: Track number of stubs on top 20 wikipedias
Esc3300 added a comment. Doesn't seem to be related to Wikidata ..TASK DETAILhttps://phabricator.wikimedia.org/T119976EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Esc3300Cc: Esc3300, Liuxinyu970226, ChrisPins, Mbch331, Izno, aude, Aklapper, Lydia_Pintscher, Addshore, StudiesWorld, QZanden, vjudge404, Salgo60, D3r1ck01, Cwek, Wikidata-bugs, zhuyifei1999, Shizhao___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T161529: Create Wikipedia Doteli
Esc3300 added a comment. https://www.wikidata.org/wiki/Wikidata:Bot_requests#Import_interwikis_from_Doteli_WikipediaTASK DETAILhttps://phabricator.wikimedia.org/T161529EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Esc3300Cc: Esc3300, Dzahn, Stashbot, Rschen7754, Liuxinyu970226, gerritbot, Meno25, Aklapper, MF-Warburg, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, Salgo60, Lewizho99, Zppix, Maathavan, Urbanecm, D3r1ck01, JEumerus, Tulsi_Bhagat, Izno, Luke081515, biplabanand, Wikidata-bugs, Snowolf, aude, faidon, Matanya, Mbch331, Rxy, Jay8g, Krenair, fgiunchedi___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T161529: Create Wikipedia Doteli
Esc3300 added a project: Wikidata. TASK DETAILhttps://phabricator.wikimedia.org/T161529EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Esc3300Cc: Dzahn, Stashbot, Rschen7754, Liuxinyu970226, gerritbot, Meno25, Aklapper, MF-Warburg, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, Salgo60, Lewizho99, Zppix, Maathavan, Urbanecm, D3r1ck01, JEumerus, Tulsi_Bhagat, Izno, Luke081515, biplabanand, Wikidata-bugs, Snowolf, aude, faidon, Matanya, Mbch331, Rxy, Jay8g, Krenair, fgiunchedi___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Edited] T161527: Canonical data URLs for machine readable page content
daniel edited the task description. (Show Details) EDIT DETAILS...== Proposed Solution ==: * Use URLs of the form https://commons.wikimedia.org/data/Data:Avignon_City_Wall.map to identify and retrieve machine readable page content* Do not include the namespace after /data/, e.g. https://commons.wikimedia.org/data/Avignon_City_Wall.map : TBD * Use "raw" instead of "data", e.g. https://commons.wikimedia.org/raw/Data:Avignon_City_Wall.map : TBD * Use REST API URLS : TBD * Apply content negotiation to the established page URLs using the /wiki/ path : TBD * "URLs don't need to be pretty" : TBD...TASK DETAILhttps://phabricator.wikimedia.org/T161527EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: MZMcBride, Rybesh, Dzahn, GWicke, tstarling, Aklapper, Jonas, Smalyshev, mkroetzsch, Lydia_Pintscher, daniel, QZanden, Salgo60, D3r1ck01, Izno, suriyaa, Eevans, mobrovac, Hardikj, Wikidata-bugs, aude, jayvdb, Southparkfan, fbstj, RobLa-WMF, santhosh, Mbch331, Jay8g, Ltrlg, Glaisher, bd808, Krenair, Legoktm___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Retitled] T161527: Canonical data URLs for machine readable page content
daniel changed the title from "Canonical data URIs and URLs for machine readable page content" to "Canonical data URLs for machine readable page content".daniel edited the task description. (Show Details)
EDIT DETAILS**Revised after public discussion, April 1 2017**
== Problem ==
Wikimedia is managing a growing amount of machine readable data as wiki page content. The latest addition is the Data namespace on commons, which hosts tabular data like [[https://commons.wikimedia.org/wiki/Data:Dolmens_of_the_Preseli_Hills.tab|Data:Dolmens_of_the_Preseli_Hills.tab]] and geographic data like [[https://commons.wikimedia.org/wiki/Data:Avignon_City_Wall.map|Data:Avignon_City_Wall.map]].
There is currently no canonical URL for referring to and retrieving these data sets. Canonical URLs are needed as stable identifiers (URIs) in linked data.
**Concrete need:** Wikidata can reference geo-shape data from the Data namespace on Commons. To represent such references in RDF, the data set needs a canonical URI. See {T159517}
Problem:== Proposed Solution ==:
* Use URLs of the form https://commons.wikimedia.org/data/Data:Avignon_City_Wall.map to identify and retrieve machine readable page content.
* The ```/data/``` path is rewritten to a special page, Special:PageData
There is currently no canonical URI/URL for referring to and retrieving these data sets.
Concrete need:* Special Special:PageData will redirect (with status 303) to an appropriate (and typically cacheable) URL for retrieving the page data. For now, this will use the ```action="" interface.
* Special:PageData may apply content negotiation based on the Accept header sent by the client. In the first iteration, it will only check if any accept header sent by the client is compatible with the content model of the requested page.
Wikidata can reference geo-shape data from the Data namespace on Commons. To represent such references in RDF* The 303 redirects are not cecheable for now, the data set needs a canonical URI.because they depend on the Accept header; See {T159517}complex normalization would be needed to allow the cache to vary on the Accept header without causing massive cache fragementation.
Current solutions:Note that in contrast to Wikidata entity URIs, the above URIs identify //descriptions// (data), not the thing described by the data.
== Status Quo ==
* A* There is a way to get raw page data for most data types, using action="" with the "ugly" URL form:
* Wikidata uses as the canonical URI of concepts, and as the canonical URI of the description. Both apply content negotiation and trigger a 303 redirect. The canonical URL for a specific serialization has the form .
== Concerns an Alternatives Considered ==
Proposed URIs for * Do not include the namespace after /data:/, e.g. https://commons.wikimedia.org/data/Avignon_City_Wall.map
* Special case for the data namespace: https://commons.wikimedia.org/data/Avignon_City_Wall.map
* ...or with the namespace,: TBD
* Use "raw" instead of "data", e.g. so other kinds of data can be added: https://commons.wikimedia.org/dataraw/Data:Avignon_City_Wall.map
* ...or bind it to action="" explicity: https://commons.wikimedia.org/raw/Data:Avignon_City_Wall.map: TBD
Note that in contrast to Wikidata concept URIs, the above URIs identify //descriptions// (data), not the thing described by the data.* Use REST API URLS
: TBD
Also note that these would return the "internal" serialization of the data (with the appropriate MIME type in the response header). They do not support custom serialization or apply content negotiation.
Question:
Do we need to plan for supporting custom serialization and content negotiation? Is it sufficient to later add a query parameter to specify an alternative serialization?* Apply content negotiation to the established page URLs using the /wiki/ path
: TBD
Example: to get .tab data as CSV instead of JSON, one would use a URL like .
Note that specifying the format makes no sense for a "pure" URI, this is only relevant when resolving the URI as a URL and fetching the associated data.
Useful reading for URI design:* "URLs don't need to be pretty"
: TBD
== Resources ==
* https://www.w3.org/TR/cooluris/...TASK DETAILhttps://phabricator.wikimedia.org/T161527EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: MZMcBride, Rybesh, Dzahn, GWicke, tstarling, Aklapper, Jonas, Smalyshev, mkroetzsch, Lydia_Pintscher, daniel, QZanden, Salgo60, D3r1ck01, Izno, suriyaa, Eevans, mobrovac, Hardikj, Wikidata-bugs, aude, jayvdb, Southparkfan, fbstj, RobLa-WMF, santhosh, Mbch331, Jay8g, Ltrlg, Glaisher, bd808, Krenair, Legoktm___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T161527: Canonical data URIs and URLs for machine readable page content
daniel added a comment. This RFC was discussed in a public meeting on IRC on March 28th. Full log: https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.2017-03-29-21.01.log.html The outcome was that @daniel will revise the RFC based on the discussion, and then put it on last call. If no new concerns are raised, the revised RFC is due for final review and possible approval on April 12th.TASK DETAILhttps://phabricator.wikimedia.org/T161527EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: MZMcBride, Rybesh, Dzahn, GWicke, tstarling, Aklapper, Jonas, Smalyshev, mkroetzsch, Lydia_Pintscher, daniel, QZanden, Salgo60, D3r1ck01, Izno, suriyaa, Eevans, mobrovac, Hardikj, Wikidata-bugs, aude, jayvdb, Southparkfan, fbstj, RobLa-WMF, santhosh, Mbch331, Jay8g, Ltrlg, Glaisher, bd808, Krenair, Legoktm___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T161527: Canonical data URIs and URLs for machine readable page content
daniel added a comment. @MZMcBride It's not about the URLs bein "pretty" URLs as much as it's about the URLs being stable. That is much easier to achieve if the URL is "clean": it should not expose technical details about the underlying web application or access mechanism. Clean URLs provide a layer of abstraction. This improves stability of data that uses the URLs as identifiers. Software that uses URLs can be updated. Data that uses URLs cannot. Ideally, URLs used as URIs should stay valid forever. So it's impogenertant to think about what they should look like, so we can still happily serve them when we have replaced MediaWiki with a hive of genetically engineered cyborg termites in 100 years. The basic idea is that stable URIs should expose a minimum of information. Have a look at https://www.w3.org/TR/cooluris/ and the other resources I linked in the description.TASK DETAILhttps://phabricator.wikimedia.org/T161527EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: MZMcBride, Rybesh, Dzahn, GWicke, tstarling, Aklapper, Jonas, Smalyshev, mkroetzsch, Lydia_Pintscher, daniel, QZanden, Salgo60, D3r1ck01, Izno, suriyaa, Eevans, mobrovac, Hardikj, Wikidata-bugs, aude, jayvdb, Southparkfan, fbstj, RobLa-WMF, santhosh, Mbch331, Jay8g, Ltrlg, Glaisher, bd808, Krenair, Legoktm___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
