[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[colibri_vola]

colibri_vola added a subscriber: colibri_vola.
colibri_vola added a comment.

I would really appreciate if there could be something like MQL for accessing 
Wikidata. 
Since Google will get rid of Freebase, and Wikidata is the official successor 
of the data, many people would appreciate to also have the same, or at least a 
similar query-language.

Also: the best part about MQL was the online query editor. This made it really 
easy to use it.

https://www.freebase.com/query


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: colibri_vola
Cc: colibri_vola, MBlissett, Spencerk, waldyrious, Addshore, thiemowmde, 
Daniel_Mietchen, JeroenDeDauw, JanZerebecki, aude, Ricordisamoa, mobrovac, 
GWicke, Tpt, daniel, Hardikj, Smalyshev, Manybubbles, hoo, Eevans, jkroll, 
Wikidata-bugs, Jdouglas



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[GWicke]

GWicke added a comment.

In https://phabricator.wikimedia.org/T85181#971763, @JanZerebecki wrote:

> No, just a list of things that were brought up, when people talked about 
> querying Wikidata. I don't like Ask. Only ever used Ask and Cypher. I think 
> the way Cypher goes about the problem is neat to work with (query by example 
> for graphs). I find MQL a bit unintuitive.


Cypher looks pretty imperative, which IMHO is not ideal if we'd like to 
independently optimize / rewrite queries.

> Is MQL a superset of wqd.wmflabs.org ?


Functionality-wise I believe that it covers pretty much everything wdq does, 
except for the AROUND predicate. In MQL as exposed by freebase, the best you 
can currently do is a bounding box with range queries on lat/lon. Adding an 
AROUND-like operator should not be too hard though. The support around 
optional: forbidden vs. != might be a bit more powerful in MQL.

> How would a recursive query (WITH RECURSIVE in SQL) / query including 
> trasitive properties look like in MQL? Example: everything that is directly 
> or indirectly/recursively subclass of organization.


I'm pretty sure that we'll want to flatten the common use cases for recursive 
queries into indexes:

- instance of / type hierarchy ('type' in freebase)
- geolocation containment

With this done, you can directly ask for the parent class & automatically match 
all items that are directly in a sub-class. Similarly, for geolocation you can 
transitively match anything that's within some region, even if it is defined to 
be within a sub-region.

MQL does support following edges, but encourages a limited depth of this 
traversal by requiring the query to spell out the exact structure. It does not 
allow queries with unspecified traversal depth.


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: GWicke
Cc: Spencerk, waldyrious, Addshore, thiemowmde, Daniel_Mietchen, JeroenDeDauw, 
JanZerebecki, aude, Ricordisamoa, mobrovac, GWicke, Tpt, daniel, Hardikj, 
Smalyshev, Manybubbles, hoo, jkroll, Wikidata-bugs, Jdouglas



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[JanZerebecki]

JanZerebecki added a comment.

No, just a list of things that were brought up, when people talked about 
querying Wikidata. I don't like Ask. Only ever used Ask and Cypher. I think the 
way Cypher goes about the problem is neat to work with (query by example for 
graphs). I find MQL a bit unintuitive.

Is MQL a superset of wqd.wmflabs.org ?
How would a recursive query (WITH RECURSIVE in SQL) / query including trasitive 
properties look like in MQL? Example: everything that is directly or 
indirectly/recursively subclass of organization. Which means if there is a 
subclass of connection like charitable org. -> nonprofit org. -> organization 
and charitable org. is not directly subclass of organization i would still get 
charitable org. as a result for this query.
How would this look like for instance of and subclass of?


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: GWicke, JanZerebecki
Cc: GWicke, Smalyshev, JanZerebecki, mobrovac, JeroenDeDauw, daniel, 
thiemowmde, aude, Hardikj, waldyrious, Ricordisamoa, Spencerk, Manybubbles, 
Daniel_Mietchen, Tpt, Addshore, hoo, jkroll, Wikidata-bugs, Jdouglas



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[GWicke]

GWicke added a comment.

@janzerebecki: Are there any in that list that you would prefer over MQL? If 
so, for which reasons?


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: GWicke
Cc: GWicke, Smalyshev, JanZerebecki, mobrovac, JeroenDeDauw, daniel, 
thiemowmde, aude, Hardikj, waldyrious, Ricordisamoa, Spencerk, Manybubbles, 
Daniel_Mietchen, Tpt, Addshore, hoo, jkroll, Wikidata-bugs, Jdouglas



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[JanZerebecki]

JanZerebecki added a comment.

Some other examples of or query languages that were mentioned: Ask (from SMW), 
Facebook (Graph API or Query Language),  Neo4j (cypher; tinkerpop 3 has a 
similar functionality).


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: GWicke, JanZerebecki
Cc: GWicke, Smalyshev, JanZerebecki, mobrovac, JeroenDeDauw, daniel, 
thiemowmde, aude, Hardikj, waldyrious, Ricordisamoa, Spencerk, Manybubbles, 
Daniel_Mietchen, Tpt, Addshore, hoo, jkroll, Wikidata-bugs, Jdouglas



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[GWicke]

GWicke added a comment.

In https://phabricator.wikimedia.org/T85181#958820, @JeroenDeDauw wrote:

> As far as I can tell, decisions are currently made based on the use case the 
> WMF has, rather than also holding the Wkidata plans into account. I'm getting 
> this impression because I'm not seeing much questions going from your teams 
> side to the Wikidata one. Perhaps I'm missing something?


This ticket is meant to facilitate a discussion, so I'm hoping for broad 
participation of all interested parties, especially the 
https://phabricator.wikimedia.org/tag/wikidata/ team.

It would be very helpful if you could point out specific use cases that we 
should consider & aren't covered yet.


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: GWicke
Cc: GWicke, Smalyshev, JanZerebecki, mobrovac, JeroenDeDauw, daniel, 
thiemowmde, aude, Hardikj, waldyrious, Ricordisamoa, Spencerk, jkroll, 
Wikidata-bugs, Jdouglas, Manybubbles



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[JeroenDeDauw]

JeroenDeDauw added a comment.

@GWicke: Indeed. I would however not extend this to "every solution that is a 
good fit for the WMF use case will also be a good fit for those that are on the 
Wikidata teams roadmap". Keep in mind that the Wikidata team also wants to have 
complex queries. As far as I can tell, decisions are currently made based on 
the use case the WMF has, rather than also holding the Wkidata plans into 
account. I'm getting this impression because I'm not seeing much questions 
going from your teams side to the Wikidata one. Perhaps I'm missing something?


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: GWicke, JeroenDeDauw
Cc: GWicke, Smalyshev, JanZerebecki, mobrovac, JeroenDeDauw, daniel, 
thiemowmde, aude, Hardikj, jkroll, Wikidata-bugs, Jdouglas, Manybubbles



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[GWicke]

GWicke added a comment.

@jeroendedauw: I think it's possible to implement a simple query interface 
(such as the current wikibase API, as far as I understand it) on top of a more 
powerful one such as the one we are discussing here. The reverse is not 
necessarily true.

What is your view on this?


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: GWicke
Cc: GWicke, Smalyshev, JanZerebecki, mobrovac, JeroenDeDauw, daniel, 
thiemowmde, aude, jkroll, Wikidata-bugs, Jdouglas, Manybubbles



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[JeroenDeDauw]

JeroenDeDauw added a comment.

Has any thought been put into how to combine this with the plans the Wikidata 
team has for queries?


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: JeroenDeDauw
Cc: GWicke, Smalyshev, JanZerebecki, mobrovac, JeroenDeDauw, jkroll, 
Wikidata-bugs, Jdouglas, aude, Manybubbles, daniel



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[mobrovac]

mobrovac added a comment.

How about using the Sirent JSON  format? 
It aims at giving a complete description of a resource, complete with URIs. 
Granted, it's more verbose than other solutions, but it could be used also to 
create different pre- and post-parsers for other formats (MQL, WDQ, etc.) due 
to its completeness.


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: mobrovac
Cc: GWicke, Smalyshev, JanZerebecki, mobrovac, jkroll, Wikidata-bugs, Jdouglas, 
aude, Manybubbles, daniel



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[Smalyshev]

Smalyshev added a comment.

Agreed, format like JSON would be much better since everybody knows how to 
handle it.


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: GWicke, Smalyshev, JanZerebecki, mobrovac, jkroll, Wikidata-bugs, Jdouglas, 
aude, Manybubbles, daniel



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[GWicke]

GWicke added a comment.

In https://phabricator.wikimedia.org/T85181#940893, @Smalyshev wrote:

> Are we considering supporting WDQ API mini-language as the option for the 
> queries or it's not a viable option?


The problem I see with the WDQ language is the need to perform error-prone 
custom quoting and serialization in clients. For numeric identifiers this might 
not be so bad, but strings, dates etc will need escaping. JSON basically avoids 
these issues by letting the JSON serializer / parser deal with it.

We could still compile a more human-friendly DSL like the WDQ language to JSON, 
but forcing every client to deal with custom escaping and serialization issues 
does not sound like a great idea for a public API.


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: GWicke
Cc: GWicke, Smalyshev, JanZerebecki, mobrovac, jkroll, Wikidata-bugs, Jdouglas, 
aude, Manybubbles, daniel



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[Smalyshev]

Smalyshev added a comment.

@janZerebecki Gremlin is basically shell access, since it can run arbitrary 
Java code. So we can have it for internal purposes, but we need frontend API 
since we probably won't be comfortable with giving everybody shell access, and 
sanitizing Gremlin probably would be more complex than handling any simpler 
language.


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: GWicke, Smalyshev, JanZerebecki, jkroll, Wikidata-bugs, Jdouglas, aude, 
Manybubbles, daniel



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate & design public API, possibly using MQL

[Smalyshev]

Smalyshev added a comment.

Are we considering supporting WDQ API mini-language as the option for the 
queries or it's not a viable option?


TASK DETAIL
  https://phabricator.wikimedia.org/T85181

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: GWicke, Smalyshev, jkroll, Wikidata-bugs, Jdouglas, aude, Manybubbles, 
daniel



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs