Hi Barry, Thank you for investigating. That explains a lot of my current performance woes :)
So long as we know a fix is coming next week, I think we'll be able to cope, but just to explore other options so that we can present them to our customer: do you know when the bug was introduced? How many versions back would we have to revert, while waiting for the next release, to avoid it? Thanks, Jeni On 10 Jul 2012, at 15:49, Barry Bishop wrote: > Hi Jeni, > > This is indeed a bug in OWLIM-SE. It is quite specific and occurs only when a > FILTER contains two or more expressions OR'd together and these expressions > contain equalities. > > The problem causes the OWLIM specific query optimisation step to abort and > default back to the (unoptimised) Sesame query evaluation steps. This is why > you see a degradation in performance to something worse than OWLIM-Lite > (which uses the Sesame evaluation mechanism, but with optimisation). > > We are planning a new version of OWLIM-SE very soon, originally intended to > include Sesame 2.6.7 (which has other query optimisation fixes). However, we > have detected an unrelated bug in the SPARQL parser in this version of Sesame > and so we are working now to release Sesame 2.6.8 at the end of this week, > OWLIM next week. > > The question now is, how problematic is this current problem for you? Can you > wait until next week for a new release of OWLIM? > > All the best and thanks for reporting this, > barry > > On 10/07/12 10:48, Jeni Tennison wrote: >> Hi Barry2, >> >> I hadn't, but I just did and it had no discernible effect. >> >> Thanks, >> >> Jeni >> >> On 9 Jul 2012, at 22:26, Barry Norton wrote: >> >>> Just to throw in - you've tried this with a UNION rather than the property >>> path alternative, I guess? >>> >>> Barry2 >>> >>> >>> On 09/07/2012 21:07, Barry Bishop wrote: >>>> Hi Jeni, >>>> >>>> I suspect you have run in to a recent query optimisation problem, but to >>>> be sure, could you tell me which version of OWLIM-SE you are using >>>> (including build number)? >>>> >>>> I'll check with the developers in the morning. >>>> >>>> Regards, >>>> barry >>>> >>>> Barry Bishop >>>> OWLIM Product Manager >>>> Ontotext AD >>>> Tel: +43 650 2000 237 >>>> email: barry.bis...@ontotext.com >>>> skype: bazbishop >>>> www.ontotext.com >>>> >>>> On 09/07/12 17:31, Jeni Tennison wrote: >>>>> Hi, >>>>> >>>>> I have a query which is performing very poorly in OWLIM-SE, but very well >>>>> in OWLIM-Lite. This is a simplified version which displays the same >>>>> characteristics: >>>>> >>>>> PREFIX task: <http://www.legislation.gov.uk/def/task/> >>>>> SELECT ?process (COUNT(?task) AS ?count) >>>>> WHERE { >>>>> { >>>>> SELECT DISTINCT ?task ?process >>>>> WHERE { >>>>> ?task task:process ?process . >>>>> FILTER ( >>>>> ?process = >>>>> <http://www.legislation.gov.uk/id/process/prepare/effects> || >>>>> ?process = >>>>> <http://www.legislation.gov.uk/id/process/review/prepare/effects> >>>>> ) >>>>> OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . } >>>>> FILTER (!BOUND(?unassignedEnded)) >>>>> } >>>>> } >>>>> } >>>>> GROUP BY ?process >>>>> >>>>> This takes several seconds in OWLIM-SE but only about 100ms with >>>>> OWLIM-Lite (there are other differences in the servers of the two >>>>> repositories, but only ones that would lead me to imagine that the >>>>> OWLIM-SE machine would be faster). >>>>> >>>>> To give you an idea of size, the results are: >>>>> >>>>> <http://www.legislation.gov.uk/id/process/prepare/effects> 906 >>>>> <http://www.legislation.gov.uk/id/process/review/prepare/effects> 907 >>>>> >>>>> The lines which seem to be causing the problem are >>>>> >>>>> OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . } >>>>> FILTER (!BOUND(?unassignedEnded)) >>>>> >>>>> I have tried several rewrites of these lines, including using FILTER ( >>>>> NOT EXISTS { ... } ). This is the most performant that I've found, but >>>>> it's still too slow. If I remove these lines, the results are: >>>>> >>>>> <http://www.legislation.gov.uk/id/process/prepare/effects> 2594 >>>>> <http://www.legislation.gov.uk/id/process/review/prepare/effects> 2594 >>>>> >>>>> so it's having to do a bit of filtering, but it's not like there are tens >>>>> of millions of tasks for it to exclude. >>>>> >>>>> Does anyone have any suggestions for how I might rewrite the query to get >>>>> better performance, or why there might be this big difference in >>>>> performance between versions of OWLIM? >>>>> >>>>> Thanks, >>>>> >>>>> Jeni >>>>> _______________________________________________ >>>>> Owlim-discussion mailing list >>>>> Owlim-discussion@ontotext.com >>>>> http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion >>>> >>>> _______________________________________________ >>>> Owlim-discussion mailing list >>>> Owlim-discussion@ontotext.com >>>> http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion >>> >>> > > > -- Jeni Tennison http://www.jenitennison.com _______________________________________________ Owlim-discussion mailing list Owlim-discussion@ontotext.com http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion
