[Wikidata-bugs] [Maniphest] [Created] T76509: Figure out why Titan gets OOM when importing data

2014-12-02 Thread Smalyshev
Smalyshev created this task. Smalyshev claimed this task. Smalyshev added subscribers: Smalyshev, Manybubbles, GWicke, JanZerebecki. Smalyshev added a project: wikidata-query-service. Smalyshev changed Security from none to none. TASK DESCRIPTION On import, Titan gets GC loops. Checking

[Wikidata-bugs] [Maniphest] [Commented On] T76509: Figure out why Titan gets OOM when importing data

2014-12-02 Thread Smalyshev
Smalyshev added a comment. @GWicke, it looks like it, the loader script reports processing 51K rows, which means 51K vertices inserted/updated, so 48K is pretty close to that number (not sure why they are not equal though). The script now commits every 1000 rows, but I'm not sure how Titan

[Wikidata-bugs] [Maniphest] [Commented On] T76509: Figure out why Titan gets OOM when importing data

2014-12-02 Thread Smalyshev
Smalyshev added a comment. @manybubbles I can give you the heap dumps (or attach it here if huge files are ok here) if it helps. Also, the code I'm using is here: https://github.com/smalyshev/wikidata-gremlin/tree/titan TASK DETAIL https://phabricator.wikimedia.org/T76509 REPLY HANDLER

[Wikidata-bugs] [Maniphest] [Commented On] T76509: Figure out why Titan gets OOM when importing data

2014-12-02 Thread Smalyshev
Smalyshev added a comment. Oh, another missing piece - the data is loaded now by somethine like: dataLoader.gzipFile(/Users/smalyshev/Downloads/xaax.gz).load(100) So DataLoader is the starting point. Sorry, it's work in progress, a bit messy now :) @aude will definitely check it out

[Wikidata-bugs] [Maniphest] [Commented On] T76509: Figure out why Titan gets OOM when importing data

2014-12-02 Thread Smalyshev
Smalyshev added a comment. Yes, the docs say: https://thinkaurelius.github.io/titan/javadoc/current/com/thinkaurelius/titan/core/TitanGraph.html#getManagementSystem() TitanManagement getManagementSystem() Returns the management system for this graph instance. The management system provides

[Wikidata-bugs] [Maniphest] [Unblock] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-02 Thread Smalyshev
Smalyshev closed blocking task T76509: Figure out why Titan gets OOM when importing data as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T76373 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. To: Smalyshev Cc

[Wikidata-bugs] [Maniphest] [Closed] T76509: Figure out why Titan gets OOM when importing data

2014-12-02 Thread Smalyshev
Smalyshev closed this task as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T76509 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. To: Smalyshev Cc: Aklapper, Smalyshev, Manybubbles, GWicke, JanZerebecki, aude, jkroll

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-02 Thread Smalyshev
Smalyshev added a comment. OK, setting `thrift_framed_transport_size_in_mb` in `cassandra.yaml` in both Cassandra and Titan (they both have the yaml file) to 256 seems to eliminate the Frame size error, now `g.wd('Q5').in('P31').labelEn[0]` words and produces 'Douglas Adams' as it should

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-03 Thread Smalyshev
Smalyshev added a comment. @janzerebecki The issue here is that many items do not have preferred state. I.e. take https://www.wikidata.org/wiki/Q30. What is the population of the USA? We don't have any number marked as preferred. We either have to report we have no idea about US population

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-03 Thread Smalyshev
Smalyshev added a comment. So we have chicken-and-egg problem here. Should we code for data that is ranked properly (but does not exist yet) and hope the data will catch up, and the querying will be complicated until then, or should we code for current data to make querying the current data

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-03 Thread Smalyshev
Smalyshev added a comment. OK, this can be done but the issue here is we can't evaluate a solution (e.g. for performance, fitness to data, etc.) such as Titan/Gremlin if we have no data to test it on. Meaning, assume we coded up all the queries under assumption the data is ranked properly

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-04 Thread Smalyshev
Smalyshev added a comment. Here is my initial proposal about the data model: https://www.mediawiki.org/wiki/Wikibase/Indexing/Data_Model Please comment. TASK DETAIL https://phabricator.wikimedia.org/T76373 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-05 Thread Smalyshev
Smalyshev added a comment. I also put some more comments to the discussion page https://www.mediawiki.org/wiki/Talk:Wikibase/Indexing/Data_Model - I think it makes sense to discuss/clarify things there, but if anybody thinks there's a better place please tell. TASK DETAIL https

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-08 Thread Smalyshev
Smalyshev added a comment. https://www.mediawiki.org/wiki/Wikibase/Indexing has been updated according to the comments. Main change - claims are now vertices. TASK DETAIL https://phabricator.wikimedia.org/T76373 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-09 Thread Smalyshev
Smalyshev added a comment. Doing the import with the new model I see that the import is significantly slower when claims have their own vertices. Not sure if it's a big deal or not. If it's an issue we may want to reconsider going back to claims as edges model. TASK DETAIL https

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-10 Thread Smalyshev
Smalyshev added a comment. Maybe worth checking out this: http://www.tinkerpop.com/docs/3.0.0.M6/#vertex-properties Titan 0.9 has TinkerPop 3, which has significantly expanded property model - in particular, the property can have multiple other properties attached to it, and itself can have

[Wikidata-bugs] [Maniphest] [Changed CC] T1095: Investigate Titan for WDQ

2014-12-10 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. TASK DETAIL https://phabricator.wikimedia.org/T1095 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Created] T85045: Figure out how to detect that claim is updated

2014-12-19 Thread Smalyshev
Smalyshev created this task. Smalyshev claimed this task. Smalyshev added subscribers: Smalyshev, Manybubbles, GWicke, JanZerebecki, aude, Lydia_Pintscher, Eloquence. Smalyshev added projects: Wikidata, wikidata-query-service. Smalyshev changed Security from none to none. TASK DESCRIPTION I

[Wikidata-bugs] [Maniphest] [Commented On] T85045: Figure out how to detect that claim is updated

2014-12-20 Thread Smalyshev
Smalyshev added a comment. Probably will have to switch to use content hashes as identifier for change. TASK DETAIL https://phabricator.wikimedia.org/T85045 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Closed] T85045: Figure out how to detect that claim is updated

2014-12-22 Thread Smalyshev
Smalyshev closed this task as Resolved. Smalyshev added a comment. We'll use content hash instead of claim ID to detect changes. We'll also use lastrevid on the item to track revisions. TASK DETAIL https://phabricator.wikimedia.org/T85045 REPLY HANDLER ACTIONS Reply to comment or attach

[Wikidata-bugs] [Maniphest] [Unblock] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-22 Thread Smalyshev
Smalyshev closed blocking task T85045: Figure out how to detect that claim is updated as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T76373 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Changed CC] T85181: Investigate design public API, possibly using MQL

2014-12-22 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate design public API, possibly using MQL

2014-12-22 Thread Smalyshev
Smalyshev added a comment. Are we considering supporting WDQ API mini-language as the option for the queries or it's not a viable option? TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-22 Thread Smalyshev
Smalyshev added a comment. For SET it won't be more complex to maintain, probably, but I'm not sure if the lookups would be fast enough. I could create an additional field for that and see how it behaves, and then we could drop the field that is not needed. For Date, I wonder if support can't

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate design public API, possibly using MQL

2014-12-23 Thread Smalyshev
Smalyshev added a comment. @janZerebecki Gremlin is basically shell access, since it can run arbitrary Java code. So we can have it for internal purposes, but we need frontend API since we probably won't be comfortable with giving everybody shell access, and sanitizing Gremlin probably would

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-23 Thread Smalyshev
Smalyshev added a comment. Elasticsearch totally supports sets. Right, but Titan unfortunately doesn't support mixed indexes on SET properties. I would assume it's not a hard limitation but rather them not getting to implementing it yet. The mixed index type support is very limited now http

[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate design public API, possibly using MQL

2014-12-23 Thread Smalyshev
Smalyshev added a comment. Agreed, format like JSON would be much better since everybody knows how to handle it. TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-24 Thread Smalyshev
Smalyshev added a comment. Proposed storage format for dates: 1. Dates are stored as long signed integers, representing number of seconds since 1970-01-01 00:00:00 UTC. 2. This gives us range of 292 bln years http://www.wolframalpha.com/input/?i=9223372036854775807+seconds+in+years. 3. When

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-24 Thread Smalyshev
Smalyshev added a comment. @GWicke Do we really need per-second precision transitioning between year 292M and 292M+1? I'm not sure it is ever required. We should ensure, of course, that seconds(292M-12-31T23:59:59) seconds(292M+1) and also the same for low dates, but beyond that I'm not sure

[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-24 Thread Smalyshev
Smalyshev added a comment. For dates beyond real Gregorian calendar, the values more precise than years have little meaning anyway, so I don't think it matters too much as long as comparisons and lookups (i.e. which Greek philosopher was born in 427 BCE) work. TASK DETAIL https

[Wikidata-bugs] [Maniphest] [Created] T85296: Bad dates (like September 31) in some entries

2014-12-24 Thread Smalyshev
Smalyshev created this task. Smalyshev added a subscriber: Smalyshev. Smalyshev added a project: MediaWiki-extensions-WikibaseRepository. TASK DESCRIPTION I've noticed there are bad dates for some items in Wikidata - e.g. https://www.wikidata.org/wiki/Q5906 and https://www.wikidata.org/wiki

[Wikidata-bugs] [Maniphest] [Created] T85298: Figure out quantity representation

2014-12-24 Thread Smalyshev
Smalyshev created this task. Smalyshev claimed this task. Smalyshev added subscribers: Smalyshev, Manybubbles, GWicke, JanZerebecki, aude, Lydia_Pintscher, Eloquence, aaron. Smalyshev added projects: MediaWiki-Core-Team, Wikidata, wikidata-query-service. TASK DESCRIPTION Right now we represent

[Wikidata-bugs] [Maniphest] [Commented On] T85298: Figure out quantity representation

2014-12-24 Thread Smalyshev
Smalyshev added a comment. From WDQ source https://bitbucket.org/magnusmanske/wikidataquery/src/f1667a81c3c693606c519587d6588c7bde8af455/wd_inmem.cpp?at=master#cl-103, WDQ treats it as double. So maybe Double precision is enough? TASK DETAIL https://phabricator.wikimedia.org/T85298 REPLY

[Wikidata-bugs] [Maniphest] [Closed] T85298: Figure out quantity representation

2014-12-29 Thread Smalyshev
Smalyshev closed this task as Resolved. Smalyshev added a comment. I think I'll decide to represent them as Double for now. If anybody objects, we can reopen and change it. TASK DETAIL https://phabricator.wikimedia.org/T85298 REPLY HANDLER ACTIONS Reply to comment or attach files

[Wikidata-bugs] [Maniphest] [Reopened] T85298: Figure out quantity representation

2014-12-29 Thread Smalyshev
Smalyshev reopened this task as Open. Smalyshev added a comment. Turns out there is a complication - Titan can not use floats (including double) in Vertex indices: http://s3.thinkaurelius.com/docs/titan/0.5.2/common-questions.html#_floating_point_numbers_in_vertex_centric_indices Need

[Wikidata-bugs] [Maniphest] [Block] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-29 Thread Smalyshev
Smalyshev reopened blocking task T85298: Figure out quantity representation as Open. TASK DETAIL https://phabricator.wikimedia.org/T76373 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Created] T85513: Restoring Titan functionality after Cassandra disconnect

2014-12-29 Thread Smalyshev
Smalyshev created this task. Smalyshev claimed this task. Smalyshev added subscribers: Smalyshev, Manybubbles, GWicke, JanZerebecki, aude, Lydia_Pintscher, Eloquence, aaron. Smalyshev added projects: MediaWiki-Core-Team, Wikidata, wikidata-query-service. TASK DESCRIPTION Right now

[Wikidata-bugs] [Maniphest] [Triaged] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service

2014-12-29 Thread Smalyshev
Smalyshev triaged this task as Normal priority. TASK DETAIL https://phabricator.wikimedia.org/T76373 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T85513: Restoring Titan functionality after Cassandra disconnect

2014-12-30 Thread Smalyshev
Smalyshev added a comment. See also description in: https://groups.google.com/forum/#!topic/aureliusgraphs/Ktq1B1AGEb0 TASK DETAIL https://phabricator.wikimedia.org/T85513 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T85298: Figure out quantity representation

2014-12-30 Thread Smalyshev
Smalyshev added a comment. Titan allows indexing the floats, but not in vertex-centric indexes. Elasticsearch indexes support floats, for example. I'm not sure what is the actual impact of this limitation yet - it probably depends on the kind of lookups we would do, as for some vertix-centered

[Wikidata-bugs] [Maniphest] [Commented On] T88550: Mourn Titan

2015-02-04 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. Smalyshev added a comment. https://www.youtube.com/watch?v=vodd6C5ryUU TASK DETAIL https://phabricator.wikimedia.org/T88550 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T88571: Figure out if Neo4j is a possible alternative to Titan

2015-02-04 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. Smalyshev added a comment. @JanZerebecki is master/slave not enough for us? The only write scenario is updates, and these can be done on master, all queries are read-only and as such can be run on slaves. TASK DETAIL https://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Commented On] T88549: Investigate ArangoDB for Wikidata Query

2015-02-04 Thread Smalyshev
Smalyshev added a comment. Preliminary notes: - Graph is a set of vertex collections and edge collections, both document collection with some special attributes. - ArangoDB keeps indexes only in memory and rebuilds them on collection load. Which takes about 1 hour for the dump of 3M objects

[Wikidata-bugs] [Maniphest] [Commented On] T88551: Find another TinkerPop 3 implementation to add to the Wikidata Query candidate list

2015-02-04 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. Smalyshev added a comment. Here is the list of all TinkerPop implementations (not TP3, but may be helpful as a starting point): https://github.com/tinkerpop/blueprints/wiki On OrientDB there are a number of questions about TP3 but no answers so far

[Wikidata-bugs] [Maniphest] [Commented On] T85298: Figure out quantity representation

2015-02-02 Thread Smalyshev
Smalyshev added a comment. Also we probably need to get to implementing value ranges/precision eventually. TASK DETAIL https://phabricator.wikimedia.org/T85298 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T88299: Must use an order preserving data type

2015-02-02 Thread Smalyshev
Smalyshev added a comment. @manybubbles Do you have a backtrace per chance? That'd help. TASK DETAIL https://phabricator.wikimedia.org/T88299 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Commented On] T88571: Figure out if Neo4j is a possible alternative to Titan

2015-02-06 Thread Smalyshev
Smalyshev added a comment. Moving to nodes didn't work either - when defining indexes for every property, it creates over 20K lucene files and tries to open all of them and gets out of filehandles, even though I've put a high limit on file descriptors (10) somehow it's not enough. Looks

[Wikidata-bugs] [Maniphest] [Commented On] T88571: Figure out if Neo4j is a possible alternative to Titan

2015-02-06 Thread Smalyshev
Smalyshev added a comment. Main problem with Neo4j so far is indexing - edge indexes are either manual addition (http://neo4j.com/docs/2.2.0-M03/indexing-add.html) or auto-indexing (http://neo4j.com/docs/2.2.0-M03/auto-indexing.html) which requires pre-defining all properties in configuration

[Wikidata-bugs] [Maniphest] [Commented On] T88549: Investigate ArangoDB for Wikidata Query

2015-02-06 Thread Smalyshev
Smalyshev added a comment. @Neunhoef In current data model, each edge carries a primary value, a boolean flag and a small set (usually well under 10, in most cases 1-3 or none) secondary values, each of which need to be indexed. It also can keep a set of auxiliary values for each of those

[Wikidata-bugs] [Maniphest] [Commented On] T88549: Investigate ArangoDB for Wikidata Query

2015-02-07 Thread Smalyshev
Smalyshev added a comment. In https://phabricator.wikimedia.org/T88549#1022562, @Fceller wrote: Still, I must admit that I'm not familiar with the internal data model of Wikipedia. I've checked with George Washington (Q23) that he as a lot of properties associated with him. However, I fail

[Wikidata-bugs] [Maniphest] [Commented On] T88549: Investigate ArangoDB for Wikidata Query

2015-02-05 Thread Smalyshev
Smalyshev added a comment. @Fceller for the test, I used two of the attributes that are present in each document for hash indexes, and two that may or may not be present, for skiplist ones. In typical situation, most of the indexable attributes would not be present, but some will be always

[Wikidata-bugs] [Maniphest] [Commented On] T88549: Investigate ArangoDB for Wikidata Query

2015-02-05 Thread Smalyshev
Smalyshev added a comment. @Fceller many thanks to you for your explanations! I certainly understand the reasons for your choices, however I'm still not sure how given our data sizes - ~16M vertices, ~100M edges, 2-3 thousands indexed fields - can be supported in current ArangoDB model. Maybe

[Wikidata-bugs] [Maniphest] [Changed Project Column] T86331: Implement sitelinks and badges

2015-01-15 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T86331 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T86786: Write example queries in different query languages

2015-01-15 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. TASK DETAIL https://phabricator.wikimedia.org/T86786 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T86833: Wikidata Query - add license

2015-01-14 Thread Smalyshev
Smalyshev added a comment. Apache is good for me. TASK DETAIL https://phabricator.wikimedia.org/T86833 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T86330: Implement import of references

2015-01-16 Thread Smalyshev
Smalyshev added a comment. References implementation: https://www.mediawiki.org/w/index.php?title=Wikibase/Indexing/Data_Model#Representing_references Not 100% happy with it but the best I could come up with so far. TASK DETAIL https://phabricator.wikimedia.org/T86330 REPLY HANDLER ACTIONS

[Wikidata-bugs] [Maniphest] [Changed Project Column] T86330: Implement import of references

2015-01-16 Thread Smalyshev
Smalyshev moved this task to Needs Review/Feedback on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T86330 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim

[Wikidata-bugs] [Maniphest] [Changed Project Column] T86524: Use Wikidata-Toolkit for import where possible

2015-01-16 Thread Smalyshev
Smalyshev moved this task to In Dev/Progress on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T86524 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim

[Wikidata-bugs] [Maniphest] [Updated] T87283: Wikidata dumps should have revision ID or other sequence mark

2015-01-20 Thread Smalyshev
Smalyshev added projects: Wikibase-DataModel, Wikibase-DataModel-JavaScript. TASK DETAIL https://phabricator.wikimedia.org/T87283 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T87283: Wikidata dumps should have revision ID or other sequence mark

2015-01-20 Thread Smalyshev
Smalyshev added a subscriber: Wikidata-bugs. Smalyshev set Security to none. TASK DETAIL https://phabricator.wikimedia.org/T87283 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Changed Project Column] T86524: Use Wikidata-Toolkit for import where possible

2015-01-20 Thread Smalyshev
Smalyshev moved this task to Backlog on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T86524 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe

[Wikidata-bugs] [Maniphest] [Created] T87307: Analyze indexes used for the engine

2015-01-20 Thread Smalyshev
Smalyshev created this task. Smalyshev claimed this task. Smalyshev added a subscriber: Smalyshev. Smalyshev added a project: wikidata-query-service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION Analyze indexes that exist for the query engine (https://www.mediawiki.org

[Wikidata-bugs] [Maniphest] [Claimed] T88717: Investigate BigData for WDQ

2015-02-11 Thread Smalyshev
Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T88717 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T88717: Investigate BigData for WDQ

2015-02-12 Thread Smalyshev
Smalyshev added a comment. @Beebs.systap this looks pretty good. How it is done - i.e. what is used to create the triples, how they are imported, etc. - is this code available? Also, I assume we'd want eventually to support qualifiers/references, i.e. queries like countries list by population

[Wikidata-bugs] [Maniphest] [Commented On] T85964: Install Java 8 on a Jenkins node

2015-02-12 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. Smalyshev added a comment. openjdk-8-jdk seems to be running ok on einsteinium but not sure if we actually need it or not. TASK DETAIL https://phabricator.wikimedia.org/T85964 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close

[Wikidata-bugs] [Maniphest] [Claimed] T50141: Implement RDF mapping for Claims/Statements

2015-02-19 Thread Smalyshev
Smalyshev claimed this task. Smalyshev set Security to None. TASK DETAIL https://phabricator.wikimedia.org/T50141 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T89949: RDF mapping should not assert that .../entity/Q123 is-a Wikidata item

2015-02-19 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. TASK DETAIL https://phabricator.wikimedia.org/T89949 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T89949: RDF mapping should not assert that .../entity/Q123 is-a Wikidata item

2015-02-19 Thread Smalyshev
Smalyshev added a comment. So, what should the dump look like? Nothing is a wikibase:Item, something else is a wikibase:Item ? If we think that entity:Q23 is the thing, should we have `entity:Q23 a entity:Q5`? Should we drop `data:Q23 a schema:Dataset`? TASK DETAIL https

[Wikidata-bugs] [Maniphest] [Changed Project Column] T86012: Port wikidata-gremlin domain specific language to Titan 0.9.0-M1

2015-02-20 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T86012 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Closed] T88821: Port Wikidata-Gremlin to test against Neo4j

2015-02-20 Thread Smalyshev
Smalyshev closed this task as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T88821 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed Project Column] T88577: Investigate hand building a gremlin implementation against Elasticsearch for wikidata query

2015-02-20 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T88577 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Changed Project Column] T85298: Figure out quantity representation

2015-02-20 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T85298 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Changed Project Column] T87307: Analyze indexes used for the engine

2015-02-20 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T87307 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Changed Project Column] T88821: Port Wikidata-Gremlin to test against Neo4j

2015-02-20 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T88821 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Changed Project Column] T88549: Investigate ArangoDB for Wikidata Query

2015-02-20 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T88549 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Changed Project Column] T88551: Find another TinkerPop 3 implementation to add to the Wikidata Query candidate list

2015-02-20 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T88551 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Changed Project Column] T88571: Figure out if Neo4j is a possible alternative to Titan

2015-02-20 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T88571 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Changed Project Column] T88299: Must use an order preserving data type

2015-02-20 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T88299 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Closed] T87307: Analyze indexes used for the engine

2015-02-20 Thread Smalyshev
Smalyshev closed this task as Resolved. Smalyshev added a comment. Closing as not relevant after the Titan's demise. TASK DETAIL https://phabricator.wikimedia.org/T87307 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL

[Wikidata-bugs] [Maniphest] [Declined] T87627: Update links/refs Data model according to discussed on MW Summit

2015-02-20 Thread Smalyshev
Smalyshev closed this task as Declined. TASK DETAIL https://phabricator.wikimedia.org/T87627 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed Project Column] T75813: Investigate OrientDB for WDQ

2015-02-20 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T75813 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Closed] T88549: Investigate ArangoDB for Wikidata Query

2015-02-20 Thread Smalyshev
Smalyshev closed this task as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T88549 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T90101: Confirm selection of BlazeGraph for wikidata query

2015-02-20 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. TASK DETAIL https://phabricator.wikimedia.org/T90101 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T90119: BlazeGraph Finalization: RDF Issues

2015-02-20 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. TASK DETAIL https://phabricator.wikimedia.org/T90119 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T90121: BlazeGraph Finalization: Prefix vs Suffix

2015-02-20 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. TASK DETAIL https://phabricator.wikimedia.org/T90121 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T90119: BlazeGraph Finalization: RDF Issues

2015-02-20 Thread Smalyshev
Smalyshev added a comment. Currently, we have two known issues with our RDF vs. BlazeGraph: 1. Date values (aka 13 billion BCE) 2. Geopoints notation TASK DETAIL https://phabricator.wikimedia.org/T90119 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T46580: Script for creating RDF dumps of all entities

2015-02-18 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. TASK DETAIL https://phabricator.wikimedia.org/T46580 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Claimed] T50143: Implement complete RDF mapping for entities (tracking)

2015-02-18 Thread Smalyshev
Smalyshev claimed this task. Smalyshev set Security to None. TASK DETAIL https://phabricator.wikimedia.org/T50143 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T87658: Make Wikidata labels, descriptions and coordinates (BBOX) api accessible

2015-01-27 Thread Smalyshev
Smalyshev added a subscriber: Smalyshev. TASK DETAIL https://phabricator.wikimedia.org/T87658 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T87658: Make Wikidata labels, descriptions and coordinates (BBOX) api accessible

2015-01-27 Thread Smalyshev
Smalyshev added a comment. We currently do not index descriptions, because they are big and are searchable by other means, but we could store the fact that certain item has description in certain language pretty cheaply. For labels, we store them so it's easy to check which items do not have

[Wikidata-bugs] [Maniphest] [Changed Project Column] T87627: Update links/refs Data model according to discussed on MW Summit

2015-01-28 Thread Smalyshev
Smalyshev moved this task to Needs Review/Feedback on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T87627 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim

[Wikidata-bugs] [Maniphest] [Changed Project Column] T87627: Update links/refs Data model according to discussed on MW Summit

2015-01-29 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T87627 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Changed Project Column] T86330: Implement import of references

2015-01-29 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T86330 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Changed Project Column] T87307: Analyze indexes used for the engine

2015-01-29 Thread Smalyshev
Smalyshev moved this task to In Dev/Progress on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T87307 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim

[Wikidata-bugs] [Maniphest] [Changed Project Column] T85957: Port wikidata-gremlin to Titan 0.9.0-M1

2015-01-29 Thread Smalyshev
Smalyshev moved this task to Done on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T85957 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign

[Wikidata-bugs] [Maniphest] [Created] T88192: Add labels to unknown/novalue nodes

2015-01-30 Thread Smalyshev
Smalyshev created this task. Smalyshev added a subscriber: Smalyshev. Smalyshev added a project: wikidata-query-service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION Currently, unknown/novalue nodes have no labels. This leads to various errors in queries which

[Wikidata-bugs] [Maniphest] [Created] T88193: Create label fallback mechanism

2015-01-30 Thread Smalyshev
Smalyshev created this task. Smalyshev added a subscriber: Smalyshev. Smalyshev added a project: wikidata-query-service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION Since many objects do not have labels in specific language (such as English), displaying arbitrary

[Wikidata-bugs] [Maniphest] [Changed Project Column] T87627: Update links/refs Data model according to discussed on MW Summit

2015-01-26 Thread Smalyshev
Smalyshev moved this task to In Dev/Progress on the wikidata-query-service workboard. TASK DETAIL https://phabricator.wikimedia.org/T87627 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim

[Wikidata-bugs] [Maniphest] [Triaged] T87627: Update links/refs Data model according to discussed on MW Summit

2015-01-26 Thread Smalyshev
Smalyshev triaged this task as High priority. Smalyshev added a project: wikidata-query-service. Smalyshev set Security to none. TASK DETAIL https://phabricator.wikimedia.org/T87627 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username

[Wikidata-bugs] [Maniphest] [Commented On] T86278: Define which data the query service would store

2015-01-11 Thread Smalyshev
Smalyshev added a comment. **This changes the structure, and the original structure is no longer represented and can no longer be faithfully recovered.** This is not correct, original structure can be recovered, though I see no reason why would you want to do so. Can you name one? **a mere 40

  1   2   3   4   5   6   7   8   9   10   >