subject:"\[Owlim\-discussion\] Poor Performance in OWLIM\-SE"

[Owlim-discussion] Poor Performance in OWLIM-SE

2012-07-09 Thread Jeni Tennison


Hi,

I have a query which is performing very poorly in OWLIM-SE, but very 
well in OWLIM-Lite. This is a simplified version which displays the same 
characteristics:


PREFIX task: 
SELECT ?process (COUNT(?task) AS ?count)
WHERE {
  {
SELECT DISTINCT ?task ?process
WHERE {
  ?task task:process ?process .
  FILTER (
?process = 
 ||
?process = 


  )
  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
  FILTER (!BOUND(?unassignedEnded))
}
  }
}
GROUP BY ?process

This takes several seconds in OWLIM-SE but only about 100ms with 
OWLIM-Lite (there are other differences in the servers of the two 
repositories, but only ones that would lead me to imagine that the 
OWLIM-SE machine would be faster).


To give you an idea of size, the results are:

 906
 907

The lines which seem to be causing the problem are

  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
  FILTER (!BOUND(?unassignedEnded))

I have tried several rewrites of these lines, including using FILTER ( 
NOT EXISTS { ... } ). This is the most performant that I've found, but 
it's still too slow. If I remove these lines, the results are:


 2594
 2594

so it's having to do a bit of filtering, but it's not like there are 
tens of millions of tasks for it to exclude.


Does anyone have any suggestions for how I might rewrite the query to 
get better performance, or why there might be this big difference in 
performance between versions of OWLIM?


Thanks,

Jeni
___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

2012-07-09 Thread Barry Bishop


Hi Jeni,

I suspect you have run in to a recent query optimisation problem, but to 
be sure, could you tell me which version of OWLIM-SE you are using 
(including build number)?


I'll check with the developers in the morning.

Regards,
barry

Barry Bishop
OWLIM Product Manager
Ontotext AD
Tel: +43 650 2000 237
email: barry.bis...@ontotext.com
skype: bazbishop
www.ontotext.com

On 09/07/12 17:31, Jeni Tennison wrote:

Hi,

I have a query which is performing very poorly in OWLIM-SE, but very 
well in OWLIM-Lite. This is a simplified version which displays the 
same characteristics:


PREFIX task: 
SELECT ?process (COUNT(?task) AS ?count)
WHERE {
  {
SELECT DISTINCT ?task ?process
WHERE {
  ?task task:process ?process .
  FILTER (
?process = 
 ||
?process = 


  )
  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
  FILTER (!BOUND(?unassignedEnded))
}
  }
}
GROUP BY ?process

This takes several seconds in OWLIM-SE but only about 100ms with 
OWLIM-Lite (there are other differences in the servers of the two 
repositories, but only ones that would lead me to imagine that the 
OWLIM-SE machine would be faster).


To give you an idea of size, the results are:

 906
 907

The lines which seem to be causing the problem are

  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
  FILTER (!BOUND(?unassignedEnded))

I have tried several rewrites of these lines, including using FILTER ( 
NOT EXISTS { ... } ). This is the most performant that I've found, but 
it's still too slow. If I remove these lines, the results are:


 2594
 2594

so it's having to do a bit of filtering, but it's not like there are 
tens of millions of tasks for it to exclude.


Does anyone have any suggestions for how I might rewrite the query to 
get better performance, or why there might be this big difference in 
performance between versions of OWLIM?


Thanks,

Jeni
___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion



___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

2012-07-09 Thread Barry Norton



Just to throw in - you've tried this with a UNION rather than the 
property path alternative, I guess?


Barry2


On 09/07/2012 21:07, Barry Bishop wrote:

Hi Jeni,

I suspect you have run in to a recent query optimisation problem, but 
to be sure, could you tell me which version of OWLIM-SE you are using 
(including build number)?


I'll check with the developers in the morning.

Regards,
barry

Barry Bishop
OWLIM Product Manager
Ontotext AD
Tel: +43 650 2000 237
email: barry.bis...@ontotext.com
skype: bazbishop
www.ontotext.com

On 09/07/12 17:31, Jeni Tennison wrote:

Hi,

I have a query which is performing very poorly in OWLIM-SE, but very 
well in OWLIM-Lite. This is a simplified version which displays the 
same characteristics:


PREFIX task: 
SELECT ?process (COUNT(?task) AS ?count)
WHERE {
  {
SELECT DISTINCT ?task ?process
WHERE {
  ?task task:process ?process .
  FILTER (
?process = 
 ||
?process = 


  )
  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
  FILTER (!BOUND(?unassignedEnded))
}
  }
}
GROUP BY ?process

This takes several seconds in OWLIM-SE but only about 100ms with 
OWLIM-Lite (there are other differences in the servers of the two 
repositories, but only ones that would lead me to imagine that the 
OWLIM-SE machine would be faster).


To give you an idea of size, the results are:

 906
 907

The lines which seem to be causing the problem are

  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
  FILTER (!BOUND(?unassignedEnded))

I have tried several rewrites of these lines, including using FILTER 
( NOT EXISTS { ... } ). This is the most performant that I've found, 
but it's still too slow. If I remove these lines, the results are:


 2594
 2594

so it's having to do a bit of filtering, but it's not like there are 
tens of millions of tasks for it to exclude.


Does anyone have any suggestions for how I might rewrite the query to 
get better performance, or why there might be this big difference in 
performance between versions of OWLIM?


Thanks,

Jeni
___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion



___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion



___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

2012-07-10 Thread Jeni Tennison

Hi Barry2,

I hadn't, but I just did and it had no discernible effect.

Thanks,

Jeni

On 9 Jul 2012, at 22:26, Barry Norton wrote:

> 
> Just to throw in - you've tried this with a UNION rather than the property 
> path alternative, I guess?
> 
> Barry2
> 
> 
> On 09/07/2012 21:07, Barry Bishop wrote:
>> Hi Jeni,
>> 
>> I suspect you have run in to a recent query optimisation problem, but to be 
>> sure, could you tell me which version of OWLIM-SE you are using (including 
>> build number)?
>> 
>> I'll check with the developers in the morning.
>> 
>> Regards,
>> barry
>> 
>> Barry Bishop
>> OWLIM Product Manager
>> Ontotext AD
>> Tel: +43 650 2000 237
>> email: barry.bis...@ontotext.com
>> skype: bazbishop
>> www.ontotext.com
>> 
>> On 09/07/12 17:31, Jeni Tennison wrote:
>>> Hi,
>>> 
>>> I have a query which is performing very poorly in OWLIM-SE, but very well 
>>> in OWLIM-Lite. This is a simplified version which displays the same 
>>> characteristics:
>>> 
>>> PREFIX task: 
>>> SELECT ?process (COUNT(?task) AS ?count)
>>> WHERE {
>>>  {
>>>SELECT DISTINCT ?task ?process
>>>WHERE {
>>>  ?task task:process ?process .
>>>  FILTER (
>>>?process = 
>>>  ||
>>>?process = 
>>> 
>>>  )
>>>  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
>>>  FILTER (!BOUND(?unassignedEnded))
>>>}
>>>  }
>>> }
>>> GROUP BY ?process
>>> 
>>> This takes several seconds in OWLIM-SE but only about 100ms with OWLIM-Lite 
>>> (there are other differences in the servers of the two repositories, but 
>>> only ones that would lead me to imagine that the OWLIM-SE machine would be 
>>> faster).
>>> 
>>> To give you an idea of size, the results are:
>>> 
>>>  906
>>>  907
>>> 
>>> The lines which seem to be causing the problem are
>>> 
>>>  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
>>>  FILTER (!BOUND(?unassignedEnded))
>>> 
>>> I have tried several rewrites of these lines, including using FILTER ( NOT 
>>> EXISTS { ... } ). This is the most performant that I've found, but it's 
>>> still too slow. If I remove these lines, the results are:
>>> 
>>>  2594
>>>  2594
>>> 
>>> so it's having to do a bit of filtering, but it's not like there are tens 
>>> of millions of tasks for it to exclude.
>>> 
>>> Does anyone have any suggestions for how I might rewrite the query to get 
>>> better performance, or why there might be this big difference in 
>>> performance between versions of OWLIM?
>>> 
>>> Thanks,
>>> 
>>> Jeni
>>> ___
>>> Owlim-discussion mailing list
>>> Owlim-discussion@ontotext.com
>>> http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion
>> 
>> 
>> ___
>> Owlim-discussion mailing list
>> Owlim-discussion@ontotext.com
>> http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion
> 
> 
> 

-- 
Jeni Tennison
http://www.jenitennison.com

___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

2012-07-10 Thread Jeni Tennison

Hi Barry,

It's OWLIM-SE 5.1 release 5208.

Thanks,

Jeni

On 9 Jul 2012, at 21:07, Barry Bishop wrote:

> Hi Jeni,
> 
> I suspect you have run in to a recent query optimisation problem, but to be 
> sure, could you tell me which version of OWLIM-SE you are using (including 
> build number)?
> 
> I'll check with the developers in the morning.
> 
> Regards,
> barry
> 
> Barry Bishop
> OWLIM Product Manager
> Ontotext AD
> Tel: +43 650 2000 237
> email: barry.bis...@ontotext.com
> skype: bazbishop
> www.ontotext.com
> 
> On 09/07/12 17:31, Jeni Tennison wrote:
>> Hi,
>> 
>> I have a query which is performing very poorly in OWLIM-SE, but very well in 
>> OWLIM-Lite. This is a simplified version which displays the same 
>> characteristics:
>> 
>> PREFIX task: 
>> SELECT ?process (COUNT(?task) AS ?count)
>> WHERE {
>>  {
>>SELECT DISTINCT ?task ?process
>>WHERE {
>>  ?task task:process ?process .
>>  FILTER (
>>?process =  
>> ||
>>?process = 
>> 
>>  )
>>  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
>>  FILTER (!BOUND(?unassignedEnded))
>>}
>>  }
>> }
>> GROUP BY ?process
>> 
>> This takes several seconds in OWLIM-SE but only about 100ms with OWLIM-Lite 
>> (there are other differences in the servers of the two repositories, but 
>> only ones that would lead me to imagine that the OWLIM-SE machine would be 
>> faster).
>> 
>> To give you an idea of size, the results are:
>> 
>>  906
>>  907
>> 
>> The lines which seem to be causing the problem are
>> 
>>  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
>>  FILTER (!BOUND(?unassignedEnded))
>> 
>> I have tried several rewrites of these lines, including using FILTER ( NOT 
>> EXISTS { ... } ). This is the most performant that I've found, but it's 
>> still too slow. If I remove these lines, the results are:
>> 
>>  2594
>>  2594
>> 
>> so it's having to do a bit of filtering, but it's not like there are tens of 
>> millions of tasks for it to exclude.
>> 
>> Does anyone have any suggestions for how I might rewrite the query to get 
>> better performance, or why there might be this big difference in performance 
>> between versions of OWLIM?
>> 
>> Thanks,
>> 
>> Jeni
>> ___
>> Owlim-discussion mailing list
>> Owlim-discussion@ontotext.com
>> http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion
> 
> 
> 

-- 
Jeni Tennison
http://www.jenitennison.com

___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

2012-07-10 Thread Barry Bishop


Thanks for this, Jeni.

It sounds like a query optimisation problem similar to what has occurred 
recently.


The developers are investigating now and I will report back asap.

Thanks for your patience,
barry

On 10/07/12 10:47, Jeni Tennison wrote:

Hi Barry,

It's OWLIM-SE 5.1 release 5208.

Thanks,

Jeni

On 9 Jul 2012, at 21:07, Barry Bishop wrote:


Hi Jeni,

I suspect you have run in to a recent query optimisation problem, but to be 
sure, could you tell me which version of OWLIM-SE you are using (including 
build number)?

I'll check with the developers in the morning.

Regards,
barry

Barry Bishop
OWLIM Product Manager
Ontotext AD
Tel: +43 650 2000 237
email: barry.bis...@ontotext.com
skype: bazbishop
www.ontotext.com

On 09/07/12 17:31, Jeni Tennison wrote:

Hi,

I have a query which is performing very poorly in OWLIM-SE, but very well in 
OWLIM-Lite. This is a simplified version which displays the same 
characteristics:

PREFIX task: 
SELECT ?process (COUNT(?task) AS ?count)
WHERE {
  {
SELECT DISTINCT ?task ?process
WHERE {
  ?task task:process ?process .
  FILTER (
?process =  ||
?process = 

  )
  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
  FILTER (!BOUND(?unassignedEnded))
}
  }
}
GROUP BY ?process

This takes several seconds in OWLIM-SE but only about 100ms with OWLIM-Lite 
(there are other differences in the servers of the two repositories, but only 
ones that would lead me to imagine that the OWLIM-SE machine would be faster).

To give you an idea of size, the results are:

 906
 907

The lines which seem to be causing the problem are

  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
  FILTER (!BOUND(?unassignedEnded))

I have tried several rewrites of these lines, including using FILTER ( NOT 
EXISTS { ... } ). This is the most performant that I've found, but it's still 
too slow. If I remove these lines, the results are:

 2594
 2594

so it's having to do a bit of filtering, but it's not like there are tens of 
millions of tasks for it to exclude.

Does anyone have any suggestions for how I might rewrite the query to get 
better performance, or why there might be this big difference in performance 
between versions of OWLIM?

Thanks,

Jeni
___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion






___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

2012-07-10 Thread Barry Bishop


Hi Jeni,

This is indeed a bug in OWLIM-SE. It is quite specific and occurs only 
when a FILTER contains two or more expressions OR'd together and these 
expressions contain equalities.


The problem causes the OWLIM specific query optimisation step to abort 
and default back to the (unoptimised) Sesame query evaluation steps. 
This is why you see a degradation in performance to something worse than 
OWLIM-Lite (which uses the Sesame evaluation mechanism, but with 
optimisation).


We are planning a new version of OWLIM-SE very soon, originally intended 
to include Sesame 2.6.7 (which has other query optimisation fixes). 
However, we have detected an unrelated bug in the SPARQL parser in this 
version of Sesame and so we are working now to release Sesame 2.6.8 at 
the end of this week, OWLIM next week.


The question now is, how problematic is this current problem for you? 
Can you wait until next week for a new release of OWLIM?


All the best and thanks for reporting this,
barry

On 10/07/12 10:48, Jeni Tennison wrote:

Hi Barry2,

I hadn't, but I just did and it had no discernible effect.

Thanks,

Jeni

On 9 Jul 2012, at 22:26, Barry Norton wrote:


Just to throw in - you've tried this with a UNION rather than the property path 
alternative, I guess?

Barry2


On 09/07/2012 21:07, Barry Bishop wrote:

Hi Jeni,

I suspect you have run in to a recent query optimisation problem, but to be 
sure, could you tell me which version of OWLIM-SE you are using (including 
build number)?

I'll check with the developers in the morning.

Regards,
barry

Barry Bishop
OWLIM Product Manager
Ontotext AD
Tel: +43 650 2000 237
email: barry.bis...@ontotext.com
skype: bazbishop
www.ontotext.com

On 09/07/12 17:31, Jeni Tennison wrote:

Hi,

I have a query which is performing very poorly in OWLIM-SE, but very well in 
OWLIM-Lite. This is a simplified version which displays the same 
characteristics:

PREFIX task: 
SELECT ?process (COUNT(?task) AS ?count)
WHERE {
  {
SELECT DISTINCT ?task ?process
WHERE {
  ?task task:process ?process .
  FILTER (
?process =  ||
?process = 

  )
  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
  FILTER (!BOUND(?unassignedEnded))
}
  }
}
GROUP BY ?process

This takes several seconds in OWLIM-SE but only about 100ms with OWLIM-Lite 
(there are other differences in the servers of the two repositories, but only 
ones that would lead me to imagine that the OWLIM-SE machine would be faster).

To give you an idea of size, the results are:

 906
 907

The lines which seem to be causing the problem are

  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
  FILTER (!BOUND(?unassignedEnded))

I have tried several rewrites of these lines, including using FILTER ( NOT 
EXISTS { ... } ). This is the most performant that I've found, but it's still 
too slow. If I remove these lines, the results are:

 2594
 2594

so it's having to do a bit of filtering, but it's not like there are tens of 
millions of tasks for it to exclude.

Does anyone have any suggestions for how I might rewrite the query to get 
better performance, or why there might be this big difference in performance 
between versions of OWLIM?

Thanks,

Jeni
___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion


___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion






___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

2012-07-10 Thread Jeni Tennison

Hi Barry,

Thank you for investigating. That explains a lot of my current performance woes 
:)

So long as we know a fix is coming next week, I think we'll be able to cope, 
but just to explore other options so that we can present them to our customer: 
do you know when the bug was introduced? How many versions back would we have 
to revert, while waiting for the next release, to avoid it?

Thanks,

Jeni

On 10 Jul 2012, at 15:49, Barry Bishop wrote:

> Hi Jeni,
> 
> This is indeed a bug in OWLIM-SE. It is quite specific and occurs only when a 
> FILTER contains two or more expressions OR'd together and these expressions 
> contain equalities.
> 
> The problem causes the OWLIM specific query optimisation step to abort and 
> default back to the (unoptimised) Sesame query evaluation steps. This is why 
> you see a degradation in performance to something worse than OWLIM-Lite 
> (which uses the Sesame evaluation mechanism, but with optimisation).
> 
> We are planning a new version of OWLIM-SE very soon, originally intended to 
> include Sesame 2.6.7 (which has other query optimisation fixes). However, we 
> have detected an unrelated bug in the SPARQL parser in this version of Sesame 
> and so we are working now to release Sesame 2.6.8 at the end of this week, 
> OWLIM next week.
> 
> The question now is, how problematic is this current problem for you? Can you 
> wait until next week for a new release of OWLIM?
> 
> All the best and thanks for reporting this,
> barry
> 
> On 10/07/12 10:48, Jeni Tennison wrote:
>> Hi Barry2,
>> 
>> I hadn't, but I just did and it had no discernible effect.
>> 
>> Thanks,
>> 
>> Jeni
>> 
>> On 9 Jul 2012, at 22:26, Barry Norton wrote:
>> 
>>> Just to throw in - you've tried this with a UNION rather than the property 
>>> path alternative, I guess?
>>> 
>>> Barry2
>>> 
>>> 
>>> On 09/07/2012 21:07, Barry Bishop wrote:
 Hi Jeni,
 
 I suspect you have run in to a recent query optimisation problem, but to 
 be sure, could you tell me which version of OWLIM-SE you are using 
 (including build number)?
 
 I'll check with the developers in the morning.
 
 Regards,
 barry
 
 Barry Bishop
 OWLIM Product Manager
 Ontotext AD
 Tel: +43 650 2000 237
 email: barry.bis...@ontotext.com
 skype: bazbishop
 www.ontotext.com
 
 On 09/07/12 17:31, Jeni Tennison wrote:
> Hi,
> 
> I have a query which is performing very poorly in OWLIM-SE, but very well 
> in OWLIM-Lite. This is a simplified version which displays the same 
> characteristics:
> 
> PREFIX task: 
> SELECT ?process (COUNT(?task) AS ?count)
> WHERE {
>  {
>SELECT DISTINCT ?task ?process
>WHERE {
>  ?task task:process ?process .
>  FILTER (
>?process = 
>  ||
>?process = 
> 
>  )
>  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
>  FILTER (!BOUND(?unassignedEnded))
>}
>  }
> }
> GROUP BY ?process
> 
> This takes several seconds in OWLIM-SE but only about 100ms with 
> OWLIM-Lite (there are other differences in the servers of the two 
> repositories, but only ones that would lead me to imagine that the 
> OWLIM-SE machine would be faster).
> 
> To give you an idea of size, the results are:
> 
>  906
>  907
> 
> The lines which seem to be causing the problem are
> 
>  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
>  FILTER (!BOUND(?unassignedEnded))
> 
> I have tried several rewrites of these lines, including using FILTER ( 
> NOT EXISTS { ... } ). This is the most performant that I've found, but 
> it's still too slow. If I remove these lines, the results are:
> 
>  2594
>  2594
> 
> so it's having to do a bit of filtering, but it's not like there are tens 
> of millions of tasks for it to exclude.
> 
> Does anyone have any suggestions for how I might rewrite the query to get 
> better performance, or why there might be this big difference in 
> performance between versions of OWLIM?
> 
> Thanks,
> 
> Jeni
> ___
> Owlim-discussion mailing list
> Owlim-discussion@ontotext.com
> http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion
 
 ___
 Owlim-discussion mailing list
 Owlim-discussion@ontotext.com
 ht

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

2012-07-10 Thread Marek Šurek

Hi,
I experience the similar issues. The build b5123(OWLIM 5.0) was last build 
which didn't take advantage of new QueryOptimizer and it gave me expected 
performance. The new query optimizer(which causes the problems) seems to be 
fully functioning from build b5183 (at least it was the point where my query 
issues started to appear). I think there won't be much difference between your 
version (b5208) and b5123 as long as it suppose to be maintaince release mostly 
aimed on bugfixing.


Best regards,

Marek




 From: Jeni Tennison 
To: Barry Bishop  
Cc: owlim-discussion@ontotext.com; jeni.tenni...@tso.co.uk; 
amna.farh...@tso.co.uk 
Sent: Tuesday, 10 July 2012, 23:35
Subject: Re: [Owlim-discussion] Poor Performance in OWLIM-SE
 
Hi Barry,

Thank you for investigating. That explains a lot of my current performance woes 
:)

So long as we know a fix is coming next week, I think we'll be able to cope, 
but just to explore other options so that we can present them to our customer: 
do you know when the bug was introduced? How many versions back would we have 
to revert, while waiting for the next release, to avoid it?

Thanks,

Jeni

On 10 Jul 2012, at 15:49, Barry Bishop wrote:

> Hi Jeni,
> 
> This is indeed a bug in OWLIM-SE. It is quite specific and occurs only when a 
> FILTER contains two or more expressions OR'd together and these expressions 
> contain equalities.
> 
> The problem causes the OWLIM specific query optimisation step to abort and 
> default back to the (unoptimised) Sesame query evaluation steps. This is why 
> you see a degradation in performance to something worse than OWLIM-Lite 
> (which uses the Sesame evaluation mechanism, but with optimisation).
> 
> We are planning a new version of OWLIM-SE very soon, originally intended to 
> include Sesame 2.6.7 (which has other query optimisation fixes). However, we 
> have detected an unrelated bug in the SPARQL parser in this version of Sesame 
> and so we are working now to release Sesame 2.6.8 at the end of this week, 
> OWLIM next week.
> 
> The question now is, how problematic is this current problem for you? Can you 
> wait until next week for a new release of OWLIM?
> 
> All the best and thanks for reporting this,
> barry
> 
> On 10/07/12 10:48, Jeni Tennison wrote:
>> Hi Barry2,
>> 
>> I hadn't, but I just did and it had no discernible effect.
>> 
>> Thanks,
>> 
>> Jeni
>> 
>> On 9 Jul 2012, at 22:26, Barry Norton wrote:
>> 
>>> Just to throw in - you've tried this with a UNION rather than the property 
>>> path alternative, I guess?
>>> 
>>> Barry2
>>> 
>>> 
>>> On 09/07/2012 21:07, Barry Bishop wrote:
>>>> Hi Jeni,
>>>> 
>>>> I suspect you have run in to a recent query optimisation problem, but to 
>>>> be sure, could you tell me which version of OWLIM-SE you are using 
>>>> (including build number)?
>>>> 
>>>> I'll check with the developers in the morning.
>>>> 
>>>> Regards,
>>>> barry
>>>> 
>>>> Barry Bishop
>>>> OWLIM Product Manager
>>>> Ontotext AD
>>>> Tel: +43 650 2000 237
>>>> email: barry.bis...@ontotext.com
>>>> skype: bazbishop
>>>> www.ontotext.com
>>>> 
>>>> On 09/07/12 17:31, Jeni Tennison wrote:
>>>>> Hi,
>>>>> 
>>>>> I have a query which is performing very poorly in OWLIM-SE, but very well 
>>>>> in OWLIM-Lite. This is a simplified version which displays the same 
>>>>> characteristics:
>>>>> 
>>>>> PREFIX task: <http://www.legislation.gov.uk/def/task/>
>>>>> SELECT ?process (COUNT(?task) AS ?count)
>>>>> WHERE {
>>>>>  {
>>>>>    SELECT DISTINCT ?task ?process
>>>>>    WHERE {
>>>>>      ?task task:process ?process .
>>>>>      FILTER (
>>>>>        ?process = 
>>>>><http://www.legislation.gov.uk/id/process/prepare/effects> ||
>>>>>        ?process = 
>>>>><http://www.legislation.gov.uk/id/process/review/prepare/effects>
>>>>>      )
>>>>>      OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
>>>>>      FILTER (!BOUND(?unassignedEnded))
>>>>>    }
>>>>>  }
>>>>> }
>>>>> GROUP BY ?process
>>>>> 
>>>>> This takes several seconds in OWLIM-SE but only about 100ms with 
>>>>&

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

2012-07-11 Thread Barry Bishop


Hi Jeni,

I'm informed by the development team that this bug must have been in the 
optimisation logic for quite some time, i.e. long before OWLIM 5.X, so 
there isn't at the moment any reversion option.


barry

On 10/07/12 23:35, Jeni Tennison wrote:

Hi Barry,

Thank you for investigating. That explains a lot of my current performance woes 
:)

So long as we know a fix is coming next week, I think we'll be able to cope, 
but just to explore other options so that we can present them to our customer: 
do you know when the bug was introduced? How many versions back would we have 
to revert, while waiting for the next release, to avoid it?

Thanks,

Jeni

On 10 Jul 2012, at 15:49, Barry Bishop wrote:


Hi Jeni,

This is indeed a bug in OWLIM-SE. It is quite specific and occurs only when a 
FILTER contains two or more expressions OR'd together and these expressions 
contain equalities.

The problem causes the OWLIM specific query optimisation step to abort and 
default back to the (unoptimised) Sesame query evaluation steps. This is why 
you see a degradation in performance to something worse than OWLIM-Lite (which 
uses the Sesame evaluation mechanism, but with optimisation).

We are planning a new version of OWLIM-SE very soon, originally intended to 
include Sesame 2.6.7 (which has other query optimisation fixes). However, we 
have detected an unrelated bug in the SPARQL parser in this version of Sesame 
and so we are working now to release Sesame 2.6.8 at the end of this week, 
OWLIM next week.

The question now is, how problematic is this current problem for you? Can you 
wait until next week for a new release of OWLIM?

All the best and thanks for reporting this,
barry

On 10/07/12 10:48, Jeni Tennison wrote:

Hi Barry2,

I hadn't, but I just did and it had no discernible effect.

Thanks,

Jeni

On 9 Jul 2012, at 22:26, Barry Norton wrote:


Just to throw in - you've tried this with a UNION rather than the property path 
alternative, I guess?

Barry2


On 09/07/2012 21:07, Barry Bishop wrote:

Hi Jeni,

I suspect you have run in to a recent query optimisation problem, but to be 
sure, could you tell me which version of OWLIM-SE you are using (including 
build number)?

I'll check with the developers in the morning.

Regards,
barry

Barry Bishop
OWLIM Product Manager
Ontotext AD
Tel: +43 650 2000 237
email: barry.bis...@ontotext.com
skype: bazbishop
www.ontotext.com

On 09/07/12 17:31, Jeni Tennison wrote:

Hi,

I have a query which is performing very poorly in OWLIM-SE, but very well in 
OWLIM-Lite. This is a simplified version which displays the same 
characteristics:

PREFIX task: 
SELECT ?process (COUNT(?task) AS ?count)
WHERE {
  {
SELECT DISTINCT ?task ?process
WHERE {
  ?task task:process ?process .
  FILTER (
?process =  ||
?process = 

  )
  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
  FILTER (!BOUND(?unassignedEnded))
}
  }
}
GROUP BY ?process

This takes several seconds in OWLIM-SE but only about 100ms with OWLIM-Lite 
(there are other differences in the servers of the two repositories, but only 
ones that would lead me to imagine that the OWLIM-SE machine would be faster).

To give you an idea of size, the results are:

 906
 907

The lines which seem to be causing the problem are

  OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
  FILTER (!BOUND(?unassignedEnded))

I have tried several rewrites of these lines, including using FILTER ( NOT 
EXISTS { ... } ). This is the most performant that I've found, but it's still 
too slow. If I remove these lines, the results are:

 2594
 2594

so it's having to do a bit of filtering, but it's not like there are tens of 
millions of tasks for it to exclude.

Does anyone have any suggestions for how I might rewrite the query to get 
better performance, or why there might be this big difference in performance 
between versions of OWLIM?

Thanks,

Jeni
___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion








___
Owlim-discussion mailing list
Owlim-discussion@ontotext.com
http://ontomail.semdata.org/cgi-bin/mailman/listinfo/owlim-discussion

[Owlim-discussion] Poor Performance in OWLIM-SE

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

Re: [Owlim-discussion] Poor Performance in OWLIM-SE

10 matches

Site Navigation

Mail list logo

Footer information