[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[Toni_001]

Toni_001 added a comment.


  Hello. I stumbled over this ticket out of curiosity. Here my contribution:
  
  1. GeoSPARQL [1], which defines the `geo:wktLiteral` datatype, does not 
mention the concept of precision. Therefore, the number of digits does not 
imply any precision (at least not when following the standard).
  2. This is similar to the datatype `xsd:decimal` [2], which does not carry 
any information about precision. The standard states that explicitly and gives 
an example where 2.0 equals 2.00 (in that space of `xsd:decimal`).
  3. Wikibase already handles precision [3] with a `"precision"` field for 
coordinates and time, and `"upperBound"` and `"lowerBound"` for 
quantities/numbers.
  
  Therefore I'd conclude that the precision-argument alone is not sufficient to 
justify truncating digits. (There might be other, practical considerations, 
though.)
  
  [1] https://www.ogc.org/standards/geosparql/
  [2] https://www.w3.org/TR/xmlschema-2/#decimal
  [3] https://www.mediawiki.org/wiki/Wikibase/DataModel/JSON

TASK DETAIL
  https://phabricator.wikimedia.org/T174504

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Toni_001
Cc: Toni_001, Rishikumaralwal, Lucas_Werkmeister_WMDE, Lydia_Pintscher, 
WMDE-leszek, seav, thiemowmde, daniel, Aklapper, Smalyshev, CBogen, 
darthmon_wmde, ET4Eva, Nandana, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, 
Jayprakash12345, QZanden, EBjune, merbst, LawExplorer, Avner, Gehel, _jensen, 
rosalieper, Scott_WUaS, Jonas, FloNight, Xmlizer, Wong128hk, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE added a comment.


  Revert merged and backported, hopefully in time for this week’s RDF dumps (if 
I read the cron config correctly, they start at 23:00, presumably UTC).

TASK DETAIL
  https://phabricator.wikimedia.org/T174504

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Rishikumaralwal, Lucas_Werkmeister_WMDE, Lydia_Pintscher, WMDE-leszek, 
seav, thiemowmde, daniel, Aklapper, Smalyshev, darthmon_wmde, ET4Eva, 
DannyS712, Nandana, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, 
Jayprakash12345, QZanden, EBjune, merbst, LawExplorer, Avner, Gehel, _jensen, 
rosalieper, Jonas, FloNight, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, 
Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[Stashbot]

Stashbot added a comment.


  Mentioned in SAL (#wikimedia-operations) [2019-10-07T19:09:33Z] 
 Synchronized 
php-1.34.0-wmf.25/extensions/Wikibase: SWAT: [[gerrit:540419|Revert "Format 
coordinates with limited precision" (T174504 
)]] (duration: 00m 57s)

TASK DETAIL
  https://phabricator.wikimedia.org/T174504

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Stashbot
Cc: Rishikumaralwal, Lucas_Werkmeister_WMDE, Lydia_Pintscher, WMDE-leszek, 
seav, thiemowmde, daniel, Aklapper, Smalyshev, darthmon_wmde, ET4Eva, 
DannyS712, Nandana, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, 
Jayprakash12345, QZanden, EBjune, merbst, LawExplorer, Avner, Gehel, _jensen, 
rosalieper, Jonas, FloNight, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, 
Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[Lydia_Pintscher]

Lydia_Pintscher added a comment.


  Thanks! :)

TASK DETAIL
  https://phabricator.wikimedia.org/T174504

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lydia_Pintscher
Cc: Rishikumaralwal, Lucas_Werkmeister_WMDE, Lydia_Pintscher, WMDE-leszek, 
seav, thiemowmde, daniel, Aklapper, Smalyshev, darthmon_wmde, ET4Eva, 
DannyS712, Nandana, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, 
Jayprakash12345, QZanden, EBjune, merbst, LawExplorer, Avner, Gehel, _jensen, 
rosalieper, Jonas, FloNight, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, 
Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[WMDE-leszek]

WMDE-leszek added a comment.


  https://gerrit.wikimedia.org/r/521984 has been reverted, see 
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/540402
  Reverting does not mean dismissing points brought up by @Smalyshev here and 
in T232984 . Revert is only meant as 
a mitigation to issues reported by @seav in T232984 
. Issues raised by @Smalyshev hold, 
they would need a bit more appropriate solution than what we have tried in 
https://gerrit.wikimedia.org/r/521984.

TASK DETAIL
  https://phabricator.wikimedia.org/T174504

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: WMDE-leszek
Cc: Rishikumaralwal, Lucas_Werkmeister_WMDE, Lydia_Pintscher, WMDE-leszek, 
seav, thiemowmde, daniel, Aklapper, Smalyshev, darthmon_wmde, ET4Eva, 
DannyS712, Nandana, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, 
Jayprakash12345, QZanden, EBjune, merbst, LawExplorer, Avner, Gehel, _jensen, 
rosalieper, Jonas, FloNight, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, 
Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[thiemowmde]

thiemowmde added a comment.


  Thanks for the additional analysis. Another thing I realized later is that 
this might cause //actual// data loss in the Wikidata database. This can happen 
when a tool uses data it got from the query service to edit the original 
Wikidata entity. I believe this scenario should be rare, but wanted to mention 
it.

TASK DETAIL
  https://phabricator.wikimedia.org/T174504

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: thiemowmde
Cc: Rishikumaralwal, Lucas_Werkmeister_WMDE, Lydia_Pintscher, WMDE-leszek, 
seav, thiemowmde, daniel, Aklapper, Smalyshev, darthmon_wmde, ET4Eva, 
DannyS712, Nandana, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, 
Jayprakash12345, QZanden, EBjune, merbst, LawExplorer, Avner, Gehel, _jensen, 
rosalieper, Jonas, FloNight, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, 
Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE added a comment.


  Personally, I think we should revert this change. I also don’t know why it 
was suddenly merged, but //if// we decide to do it, the implementation should 
take the specified precision of the coordinate value into account instead of 
hard-coding a certain number of digits, the change should be announced in 
advance, and we should take care that it applies to all coordinate values, not 
just those of whichever items are edited after the change is deployed. (In 
practice, that last bullet point would probably require a full reload – I doubt 
that updating //all// coordinate value nodes 

 is feasible.)

TASK DETAIL
  https://phabricator.wikimedia.org/T174504

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Rishikumaralwal, Lucas_Werkmeister_WMDE, Lydia_Pintscher, WMDE-leszek, 
seav, thiemowmde, daniel, Aklapper, Smalyshev, darthmon_wmde, ET4Eva, 
DannyS712, Nandana, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, 
Jayprakash12345, QZanden, EBjune, merbst, LawExplorer, Avner, Gehel, _jensen, 
rosalieper, Jonas, FloNight, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, 
Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE added a comment.


  Note that changing the exported precision in RDF does not invalidate the hash 
of the data values, so even if we revert the code change and all affected items 
are edited again, reduced-precision coordinates will remain in the query 
service until the value nodes are updated 

 or a full reload is done 
.

TASK DETAIL
  https://phabricator.wikimedia.org/T174504

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Rishikumaralwal, Lucas_Werkmeister_WMDE, Lydia_Pintscher, WMDE-leszek, 
seav, thiemowmde, daniel, Aklapper, Smalyshev, darthmon_wmde, ET4Eva, 
DannyS712, Nandana, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, 
Jayprakash12345, QZanden, EBjune, merbst, LawExplorer, Avner, Gehel, _jensen, 
rosalieper, Jonas, FloNight, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, 
Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[gerritbot]

gerritbot added a comment.


  Change 521984 **merged** by jenkins-bot:
  [mediawiki/extensions/Wikibase@master] Format coordinates with limited 
precision
  
  https://gerrit.wikimedia.org/r/521984

TASK DETAIL
  https://phabricator.wikimedia.org/T174504

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: thiemowmde, daniel, Aklapper, Smalyshev, Hook696, Daryl-TTMG, RomaAmorRoma, 
0010318400, E.S.A-Sheild, darthmon_wmde, joker88john, ET4Eva, DannyS712, 
CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, 
Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Af420, Darkminds3113, Bsandipan, 
Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, 
Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, WSH1906, Avner, 
Lewizho99, Maathavan, Gehel, _jensen, rosalieper, Jonas, FloNight, Xmlizer, 
jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[gerritbot]

gerritbot added a comment.


  Change 521984 had a related patch set uploaded (by Smalyshev; owner: 
Smalyshev):
  [mediawiki/extensions/Wikibase@master] Format coordinates with limited 
precision
  
  https://gerrit.wikimedia.org/r/521984

TASK DETAIL
  https://phabricator.wikimedia.org/T174504

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: thiemowmde, daniel, Aklapper, Smalyshev, darthmon_wmde, ET4Eva, Nandana, 
Lahi, Gq86, Darkminds3113, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, 
EBjune, merbst, LawExplorer, Avner, Gehel, _jensen, rosalieper, Jonas, 
FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[Smalyshev]

Smalyshev added a comment.


  Now this bug has its own xkcd: https://xkcd.com/2170/

TASK DETAIL
  https://phabricator.wikimedia.org/T174504

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: thiemowmde, daniel, Aklapper, Smalyshev, darthmon_wmde, ET4Eva, Nandana, 
Lahi, Gq86, Darkminds3113, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, 
EBjune, merbst, LawExplorer, Avner, Gehel, _jensen, rosalieper, Jonas, 
FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[thiemowmde]

thiemowmde added a comment.
I can follow all your arguments. It's just that I think the effect of this (actually well defined) behavior on users is really, really negligible. Most users are never going to see coordinates as numbers anyway, but as dots or shapes on maps.

And even if, which user will think of sub-millimeters when they see a representation like 13.3667? Especially when the object is a city, or any larger shape. Most users don't even know what 1 degree is in meters or miles.

That said, I agree this could be improved, and even have an actual suggestion I want to implement some day, either in the RDF export or somewhere deeper in the Wikibase code base: Basically, cut off decimal places that do not have any effect on any of the output formats we support. This algorithm should consider all output formats, because when such an algorithm is applied we don't know which output format will be used.

But this idea requires coordinates to be stored as strings, which they are not. Basically, this requires a new datatype.TASK DETAILhttps://phabricator.wikimedia.org/T174504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: thiemowmdeCc: thiemowmde, daniel, Aklapper, Smalyshev, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, Avner, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T174504: Coordinates are exported into RDF with excessive precision

[Smalyshev]

Smalyshev added a comment.
Mostly because most of these digits are not representing any real data, it's just junk produced by decimal representation with overly big precision and produced by various conversions and calculations. We're just dragging around those meaningless characters that do not have any use and do not represent any data (nobody really measured that coordinate with micron precision and got 13.3667, what happened most probably that it was measured in another system, then calculation involved 40.1/3 (probably when converting degrees and minutes to decimal) and the result came out as 13.3667. And then we convert back, we'd get 40.1001 - again, junk data in 11 last decimal places.TASK DETAILhttps://phabricator.wikimedia.org/T174504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: thiemowmde, daniel, Aklapper, Smalyshev, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, Avner, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs