[Wikidata-tech] rdfs:label referring to non-existent column causes Blazegraph high CPU/RAM and then MemoryManagerOutOfMemory

2017-04-24 Thread Miguel Paraz 
Hi,
I'm referring to a query like this, based on the Wikidata cats example:

SELECT ?item
WHERE
{
?item wdt:P31 wd:Q146 .
?crash rdfs:label ?_crashLabel
}

on my local Blazegraph, I can see that it uses up all the CPU cores and
goes up in RAM, and eventually kills the query with:
Caused by: com.bigdata.rwstore.sector.MemoryManagerOutOfMemory
at
com.bigdata.rwstore.sector.MemoryManager.getSectorFromFreeList(MemoryManager.java:646)

Could Blazegraph prevent this query from even starting?
Thanks!

Cheers
Miguel
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

[Wikidata-tech] Wikidata-Toolkit support for concurrent editing

2017-04-17 Thread Miguel Paraz 
Hi!
I'm looking to speed up my Wikidata-Toolkit code that updates a lot of
statements.

Does a single WikibaseDataEditor support multiple updateStatements running
in different threads?
If not, could I instead create a pool of WikibaseDataEditor instances and
then use them separately from different threads

Thanks!
Miguel
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

[Wikidata-tech] wikidata-query-rdf cannot munge a custom Wikidata instance

2017-03-19 Thread Miguel Paraz 
Hi!
I'm interested in using SPARQL for my own Wikibase installation. But, I was
getting errors in the munge step.

To reproduce the problem, I set up a new Wikibase instance and tried to
load it into wikidata-query-rdf.
It has only one item, one property, and one statement.

First I used the included dumpRdf.php to create the RDF file, attached.

Then I did the munge step. It produced the errors, attached.

I don't understand why there is "Unrecognized subjects" when the subjects
listed do match the patterns, "Expected only sitelinks
 and subjects starting with http://www.wikidata.org/wiki/Special:EntityData/
and
 http://www.wikidata.org/entity/;

I don't understand what is the revision id it is looking for.

What is the proper procedure for indexing?
If necessary, I can code in Java and fix the tools. I just need an overview
on how this system works.

Thanks!
Regards,
Miguel
13:35:21.016 [main] INFO  org.wikidata.query.rdf.tool.Munge - Switching to 
./data/split/wikidump-1.ttl.gz
13:35:21.101 [main] INFO  o.wikidata.query.rdf.tool.rdf.Munger - Unrecognized 
subjects: 
[http://www.wikidata.org/entity/statement/Q2-cefd4861-46ca-d3b3-0a09-20ac7dc6ab61,
 http://www.wikidata.org/wiki/index.php/Special:EntityData/P2, 
http://www.wikidata.org/wiki/index.php/Special:EntityData/Q2].  Expected only 
sitelinks and subjects starting with 
http://www.wikidata.org/wiki/Special:EntityData/ and 
http://www.wikidata.org/entity/
13:35:21.105 [main] WARN  org.wikidata.query.rdf.tool.Munge - Error munging null
org.wikidata.query.rdf.tool.exception.ContainedException: Didn't get a revision 
id for [(http://www.wikidata.org/prop/P2, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#ObjectProperty), 
(http://www.wikidata.org/prop/statement/value/P2, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#ObjectProperty), 
(http://www.wikidata.org/prop/qualifier/value/P2, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#ObjectProperty), 
(http://www.wikidata.org/prop/reference/value/P2, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#ObjectProperty), 
(http://www.wikidata.org/prop/statement/value-normalized/P2, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#ObjectProperty), 
(http://www.wikidata.org/prop/qualifier/value-normalized/P2, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#ObjectProperty), 
(http://www.wikidata.org/prop/reference/value-normalized/P2, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#ObjectProperty), 
(http://www.wikidata.org/prop/direct/P2, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#DatatypeProperty), 
(http://www.wikidata.org/prop/statement/P2, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#DatatypeProperty), 
(http://www.wikidata.org/prop/qualifier/P2, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#DatatypeProperty), 
(http://www.wikidata.org/prop/reference/P2, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#DatatypeProperty), 
(http://www.wikidata.org/prop/novalue/P2, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#Class), (http://www.wikidata.org/prop/novalue/P2, 
http://www.w3.org/2002/07/owl#complementOf, 
_:genid-1ba765b9b61745bc88a3bcfe76395418-genid1), 
(_:genid-1ba765b9b61745bc88a3bcfe76395418-genid1, 
http://www.w3.org/1999/02/22-rdf-syntax-ns#type, 
http://www.w3.org/2002/07/owl#Restriction), 
(_:genid-1ba765b9b61745bc88a3bcfe76395418-genid1, 
http://www.w3.org/2002/07/owl#onProperty, 
http://www.wikidata.org/prop/direct/P2), 
(_:genid-1ba765b9b61745bc88a3bcfe76395418-genid1, 
http://www.w3.org/2002/07/owl#someValuesFrom, 
http://www.w3.org/2002/07/owl#Thing)]
at 
org.wikidata.query.rdf.tool.rdf.Munger$MungeOperation.finishCommon(Munger.java:802)
 ~[wikidata-query-tools-0.2.4-SNAPSHOT-jar-with-dependencies.jar:na]
at 
org.wikidata.query.rdf.tool.rdf.Munger$MungeOperation.munge(Munger.java:421) 
~[wikidata-query-tools-0.2.4-SNAPSHOT-jar-with-dependencies.jar:na]
at org.wikidata.query.rdf.tool.rdf.Munger.munge(Munger.java:219) 
~[wikidata-query-tools-0.2.4-SNAPSHOT-jar-with-dependencies.jar:na]
at org.wikidata.query.rdf.tool.rdf.Munger.munge(Munger.java:247) 
~[wikidata-query-tools-0.2.4-SNAPSHOT-jar-with-dependencies.jar:na]
at 
org.wikidata.query.rdf.tool.Munge$EntityMungingRdfHandler.munge(Munge.java:363) 
[wikidata-query-tools-0.2.4-SNAPSHOT-jar-with-dependencies.jar:na]
at 
org.wikidata.query.rdf.tool.Munge$EntityMungingRdfHandler.endRDF(Munge.java:351)
 [wikidata-query-tools-0.2.4-SNAPSHOT-jar-with-dependencies.jar:na]
at 
org.wikidata.query.rdf.tool.rdf.DelegatingRdfHandler.endRDF(DelegatingRdfHandler.java:28)

[Wikidata-tech] Removing sitelinks when they aren't being used

2017-02-25 Thread Miguel Paraz 
Hi,
On my own install of Wikibase, I don't use sitelinks, but they still appear
with what seem to be the defaults; Wikipedia, wikibase-sitelinks-wikinews
...
Just like here: http://librarybase.wmflabs.org/wiki/Q264

How can I stop these from being shown?

Thanks!
Miguel
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

[Wikidata-tech] wikidata-query-rdf for other Wikibase installs

2017-02-24 Thread Miguel Paraz 
Hi,
Is there documentation on what needs to be done to run wikidata-query-rdf
on my custom Wikibase install, so that I could do SPARQL queries on it?

When following the instructions at:
https://github.com/wikimedia/wikidata-query-rdf/blob/master/docs/getting-started.md

but using my own RDF file, I get this in the munge:
02:08:14.099 [main] INFO  o.wikidata.query.rdf.tool.rdf.Munger -
Unrecognized subjects:
[
http://www.wikidata.org/entity/statement/Q1174-2A6EDE59-31EA-4585-B211-DCB751B0F577
...
 Expected only sitelinks and subjects starting with
http://www.wikidata.org/wiki/Special:EntityData/ and
http://www.wikidata.org/entity/

even if the all the URLs start with http://www.wikidata.org/entity/

They originally did not - the exported RDF was using my own namespace but I
did a substitution before running the munge.

Thanks!
Cheers
Miguel
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

[Wikidata-tech] Making items appear ahead of aliases when searching for entities

2017-01-07 Thread Miguel Paraz 
Hi,
I found that if I have items, the first one matching the alias is found,
instead of a later one which matches the label.
Example:
Q1 Label=SomethingElse, Alias=ABC
Q2 Label=ABC

Would it be possible to make Q2 appear first?
This is most useful if I only ask for one entity, thus this would be the
best match.

Thanks!

Cheers
Miguel
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

[Wikidata-tech] Searching through API and ignoring diacritics

2017-01-05 Thread Miguel Paraz 
Hi!
I'm starting to build my own Wikidata installation. Thanks for the great
work on this project!

One of my needs is to do a search and match ignoring diacritic marks.
For example: a search for "Comite" matching "Comité".

Is it feasible to change the Wikidata code to support this as an option?
Could you point me to the right place to make the change?

Thanks!
Regards,
Miguel
___
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech