Re: [Owlim-discussion] ERROR IN PREDICATE STATISTICS

damyan Thu, 27 Sep 2012 09:08:04 -0700

Hi,

- the predicate index should be rebuild while the repository is down, noOWLIM instance should use that storage folder during until it finish.For various technical reasons, predicates rebuild is intentionally notincluded into the auto recovery through which the context and predListsare recovered/rebuild ... one of the reasons is that these are notmandatory but 'predicates' is fro the 'normal' operation of the repository.


HTH
Damyan


On 9/27/2012 5:39 PM, Rakov, Simon wrote:

Hi,

Can I run the predicate index build in a separate process while OWLIM is 
running?  Or must it be shut down while com.ontotext.trree.BuildPredicates is 
running?  Does OWLIM *not* rebuild the predicate index when it restarts?

Thanks for your help!

Simon


--------------------------------------------------------------------------
Confidentiality Notice: This message is private and may contain confidential 
and proprietary information. If you have received this message in error, please 
notify us and remove it from your system and note that you must not copy, 
distribute or take any action in reliance on it. Any unauthorized use or 
disclosure of the contents of this message is not permitted and may be unlawful.

-----Original Message-----

From: damyan [mailto:dam...@sirma.bg]
Sent: Thursday, September 27, 2012 10:05 AM
To: Rakov, Simon
Cc: Blasetti, Luciano (CIOK); owlim-discussion@ontotext.com; rve...@yarcdata.com
Subject: Re: [Owlim-discussion] ERROR IN PREDICATE STATISTICS

Hi Simon,

if you are using/setup your repository through the sesame-workbench -
simply start your tomcat, open/access the workbench, select your
repository and click 'Query' or 'SPARQL Update' that will cause
openrdf-sesame app (to which the workbench is configured to connect) to
initialize the repository and during the initialization the missing
context or predLists index will be recreated using the data from pos/pso.

there is no way to recreate pos and pso indices.

Damyan

On 9/27/2012 4:52 PM, Rakov, Simon wrote:

Hi,

Thanks for the instructions...but can you tell me what you mean by "simply 
initialize it?"

What steps are required to initialize it?

Thanks for your prompt reply - I'd like to fix this asap.

Is there any way to rebuild pso and pos as well?

Thanks again,

Yours
Simon


--------------------------------------------------------------------------
Confidentiality Notice: This message is private and may contain confidential 
and proprietary information. If you have received this message in error, please 
notify us and remove it from your system and note that you must not copy, 
distribute or take any action in reliance on it. Any unauthorized use or 
disclosure of the contents of this message is not permitted and may be unlawful.

-----Original Message-----

From: damyan [mailto:dam...@sirma.bg]
Sent: Wednesday, September 26, 2012 4:46 AM
To: Blasetti, Luciano (CIOK)
Cc: Rakov, Simon; owlim-discussion@ontotext.com; rve...@yarcdata.com
Subject: Re: [Owlim-discussion] ERROR IN PREDICATE STATISTICS

Hi Luciano, Simon

rebuilding context indices requires first to delete them from the
storage folder (when repository is down) and then simply initialize it.

For the context indices, the files are:
pcso, pcso.index, pcos and pcos.index

For the predicate list index, the files are:
predLists and predLists.index

For the predicate statistics (files: predicates and predicates.index),
these can be rebuild with the aid of an additional tool that is included
within the distribution jar, class name is:
com.ontotext.trree.BuildPredicates
e.g. something like that should do it having preparing the right classpath:
   >java -cp <owlim-lib>/*.jar com.ontotext.trree.BuildPredicates
<repository's storage folder>

HTH,
Damyan Ognyanov
Ontotext AD

On 9/26/2012 8:55 AM, Blasetti, Luciano (CIOK) wrote:

...Very good question. How to rebuild indexes without reimport the whole 
repository?
________________________________________
From: owlim-discussion-boun...@ontotext.com 
[owlim-discussion-boun...@ontotext.com] On Behalf Of Rakov, Simon 
[simon.ra...@astrazeneca.com]
Sent: Tuesday, September 25, 2012 5:59 PM
To: damyan
Cc: owlim-discussion@ontotext.com; rve...@yarcdata.com
Subject: Re: [Owlim-discussion] ERROR IN PREDICATE STATISTICS

Hi,

I do have context indices enabled.  Sizes:

Pcos hits:  53663768
Pcso hits:  20127229
Pos hits:   482282752
Pso hits:   32098277
PL hits:    50179

It looks like pos is much larger than pcos or pso.  PL (Predicate Lists) is 
tiny.

How do I rebuild the indexes?  Please send me a list of steps.

Thanks
Simon

--------------------------------------------------------------------------
Confidentiality Notice: This message is private and may contain confidential 
and proprietary information. If you have received this message in error, please 
notify us and remove it from your system and note that you must not copy, 
distribute or take any action in reliance on it. Any unauthorized use or 
disclosure of the contents of this message is not permitted and may be unlawful.

-----Original Message-----
From: damyan [mailto:dam...@sirma.bg]
Sent: Tuesday, September 25, 2012 11:10 AM
To: Rakov, Simon
Cc: Barry Bishop; rve...@yarcdata.com; Pavel Mička; 
owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] ERROR IN PREDICATE STATISTICS

Hi Simon,

this is a message received when we detected a mismatch during estimation
of the query complexity for a particular statement pattern. And more
specifically when the collection size per that statement pattern is
calculated to be 0 but we found at least one statement satisfying the
pattern.
As a possible cause - there could be several but most likely one of the
indices is in inconsistent state (or have partial data).

Could you check if you are using context indices and the size of
these(pcso for instance) is close by (at same order) as for pso/pos ...

It could be easily caused when repository is initialized with context
indices enabled - then initialized without them but some data is added
and then these are again re-enabled. In such scenario, the third time
you initialize the repository it will find and use the old context
indices but they will not have the data from the previous updates, when
they were switched off since these were not maintained when switched off.

one way to recover is to rebuild them.

HTH
Damyan Ognyanov
Ontotext AD.

On 9/25/2012 5:26 PM, Rakov, Simon wrote:

Hi,

I am seeing a lot of these in catalina.out:

ERROR IN PREDICATE STATISTICS: 3546128 15939569 0, size = [D@4426e447
ERROR IN PREDICATE STATISTICS: 3546176 15939569 0, size = [D@55551445
ERROR IN PREDICATE STATISTICS: 3546307 15939569 0, size = [D@44a5e9c7
ERROR IN PREDICATE STATISTICS: 3546399 15939569 0, size = [D@68972b06
ERROR IN PREDICATE STATISTICS: 3546463 15939569 0, size = [D@1c89cc8f
ERROR IN PREDICATE STATISTICS: 3546546 15939569 0, size = [D@1ad4ceec
ERROR IN PREDICATE STATISTICS: 3546561 15939569 0, size = [D@95cf5b7
ERROR IN PREDICATE STATISTICS: 3546592 15939569 0, size = [D@3607c62e
ERROR IN PREDICATE STATISTICS: 3546607 15939569 0, size = [D@386411b8
ERROR IN PREDICATE STATISTICS: 3546663 15939569 0, size = [D@31c763f4
ERROR IN PREDICATE STATISTICS: 3546677 15939569 0, size = [D@197341b4
ERROR IN PREDICATE STATISTICS: 3546773 15939569 0, size = [D@5dde9d79
ERROR IN PREDICATE STATISTICS: 3546947 15939569 0, size = [D@4234a314

What do these mean?  More importantly, how do I fix them?

I'm running OWLIM 5.2.5512-20120921122819, JRE_HOME jdk1.6.0_32, Tomcat 6.0.35, 
CentOS with kernel Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010 
x86_64 x86_64 x86_64 GNU/Linux.

Thanks for your help
Simon Rakov


--------------------------------------------------------------------------
Confidentiality Notice: This message is private and may contain confidential 
and proprietary information. If you have received this message in error, please 
notify us and remove it from your system and note that you must not copy, 
distribute or take any action in reliance on it. Any unauthorized use or 
disclosure of the contents of this message is not permitted and may be unlawful.

-----Original Message-----
From: owlim-discussion-boun...@ontotext.com 
[mailto:owlim-discussion-boun...@ontotext.com] On Behalf Of Barry Bishop
Sent: Monday, September 24, 2012 6:23 PM
To: rve...@yarcdata.com; Pavel Mička
Cc: owlim-discussion@ontotext.com
Subject: Re: [Owlim-discussion] OWLIM ignores binding

Hi Pavel, Rob,

So it seems this is correct behaviour after all.

Perhaps I was too quick to assume something was wrong. I think for
anyone used to SQL then it just seems wrong, but is in fact correct.

Thanks for spotting this, Rob.

And thanks for raising awareness, Pavel.

All the best,
barry

On 24/09/12 18:34, Rob Vesse wrote:

I believe this is in fact the correct behavior and in line with the
specification, see
http://answers.semanticweb.com/questions/17410/semantics-of-sparql-aggregat
es for a recent discussion on this with an explanation from Andy Seaborne
of the working group as to why this is correct behavior

The specific example there was a query with MAX but the same argument
applies here (in fact his argument does include treatment of COUNT)

Rob



On 9/24/12 1:46 AM, "Barry Bishop" <barry.bis...@ontotext.com> wrote:

Hi Pavel,

Yes, the issue does seem to be in Sesame. I have raised this issue:

http://www.openrdf.org/issues/browse/SES-1108

I expect it will be taken care of fairly soon.

Thanks for letting us know.

barry

Barry Bishop
OWLIM Product Manager
Ontotext AD
Tel: +43 650 2000 237
email: barry.bis...@ontotext.com
skype: bazbishop
www.ontotext.com

On 24/09/12 10:17, Pavel Mička wrote:

Hi,

I have found the issue, which is probably in sesame. When I execute
the query, but the knowledge base does not contain any axiom, it
returns only the value from the aggregation (0) as a result. Which is
quite unexpected, as it (in my humble opinion) should return no result
(empty result set). When the query is executed and data are present,
than it works as expected.

Pavel


Dne 24.9.2012 8:22, Barry Bishop napsal(a):

Hello Pavel,

Can you provide some more details, please?

Do you mean that you see the expected variable names when calling
TupleQueryResult.getBindingNames(), but that calls to
TupleQueryResult.next().getBindingNames() do not have the expected
variable names? Is this true for every BindingSet?

Best
barry

Barry Bishop
OWLIM Product Manager
Ontotext AD
Tel: +43 650 2000 237
email: barry.bis...@ontotext.com
skype: bazbishop
www.ontotext.com

On 12/09/12 10:47, Pavel Mička wrote:

Hi,

I am using OWLIM-Lite edition (5.2.5331, Sesame 2.6.8) and I am
querying the system using the following SPARQL query:

PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX odra:
<http://kbss.felk.cvut.cz/ontologies/2012/OdraOntology.owl/>

SELECT ?type (COUNT(DISTINCT ?instance) AS ?count) WHERE {
         ?instance odra:hasType ?type .
}
GROUP BY ?type

In BindingSet I expect 2 bindings (type, count), however the result
contains only count. But the TupleQueryResult contains (correctly)
both of them.

Am I missing something or is this a bug?

Thanks

Pavel Mička
_______________________________________________
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

_______________________________________________
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

_______________________________________________
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion
_______________________________________________
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

_______________________________________________
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion


_______________________________________________
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

Re: [Owlim-discussion] ERROR IN PREDICATE STATISTICS

Reply via email to