buildbot success in ASF Buildbot on oak-trunk-win7
The Buildbot has detected a restored build on builder oak-trunk-win7 while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/oak-trunk-win7/builds/5306 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: bb-win7 Build Reason: scheduler Build Source Stamp: [branch jackrabbit/oak/trunk] 1587158 Blamelist: mduerig Build succeeded! sincerely, -The Buildbot
Re: Using Lucene indexes for property queries
Hi, In theory, the Lucene index could be used quite easily. As far as I see, we would only need to change the cost function of the Lucene index (return a reasonable cost even if there is no full-text constraint). One problem might be: the Lucene index is asynchronous, and the user might expect the result to be up-to-date. The user knows this already for full-text constraints, but not for property constraints. Should we let the user decide whether it's OK to use an asynchronous index for this case? For example by specifying an option in the query (for example similar to the order by, at the very end of the query, option async)? So a query that can use an asynchronous index would look like this: //*[@prop = 'x'] option async or //*[@prop = 'x'] order by @otherProperty option async or select [jcr:path] from [nt:base] as a where [prop] 1 option async Regards, Thomas On 14/04/14 06:54, Chetan Mehrotra chetan.mehro...@gmail.com wrote: Hi, In JR2 I believe Lucene was used for all types of queries and not only for full text searches. In Oak we have our own PropertyIndexes for handling queries involving constraints on properties. This I believe provides a more accurate result as its built on top of mvcc support so results obtained are consistent with session state/revision. However this involves creating a index for property to be queried. And the way currently property indexes are stored they consume quite a bit of state (at least in DocumentNodeStore). In comparison Lucene stores the index content in quite compact form. In quite a few cases (like user choice based query builder) it might not be known in advance which property the user would use. As we already have all string property indexed in Lucene. Would it be possible to use Lucene for performing such queries? Or allow the user to choose which types of index he wants to use depending on the usecase. Chetan Mehrotra
Re: Using Lucene indexes for property queries
Should we let the user decide whether it's OK to use an asynchronous index for this case +1 for that. It has been the case with JR2 (I may be wrong here). And when user is searching for say some asset via DAM in Adobe CQ then he would be ok if result is not for latest head. A small lag should be acceptable. This would enable scenarios where traversal would be too costly and Lucene can still be used to provide required results in a lot lesser time. Chetan Mehrotra On Mon, Apr 14, 2014 at 2:33 PM, Thomas Mueller muel...@adobe.com wrote: Hi, In theory, the Lucene index could be used quite easily. As far as I see, we would only need to change the cost function of the Lucene index (return a reasonable cost even if there is no full-text constraint). One problem might be: the Lucene index is asynchronous, and the user might expect the result to be up-to-date. The user knows this already for full-text constraints, but not for property constraints. Should we let the user decide whether it's OK to use an asynchronous index for this case? For example by specifying an option in the query (for example similar to the order by, at the very end of the query, option async)? So a query that can use an asynchronous index would look like this: //*[@prop = 'x'] option async or //*[@prop = 'x'] order by @otherProperty option async or select [jcr:path] from [nt:base] as a where [prop] 1 option async Regards, Thomas On 14/04/14 06:54, Chetan Mehrotra chetan.mehro...@gmail.com wrote: Hi, In JR2 I believe Lucene was used for all types of queries and not only for full text searches. In Oak we have our own PropertyIndexes for handling queries involving constraints on properties. This I believe provides a more accurate result as its built on top of mvcc support so results obtained are consistent with session state/revision. However this involves creating a index for property to be queried. And the way currently property indexes are stored they consume quite a bit of state (at least in DocumentNodeStore). In comparison Lucene stores the index content in quite compact form. In quite a few cases (like user choice based query builder) it might not be known in advance which property the user would use. As we already have all string property indexed in Lucene. Would it be possible to use Lucene for performing such queries? Or allow the user to choose which types of index he wants to use depending on the usecase. Chetan Mehrotra
buildbot failure in ASF Buildbot on oak-trunk-win7
The Buildbot has detected a new failure on builder oak-trunk-win7 while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/oak-trunk-win7/builds/5307 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: bb-win7 Build Reason: scheduler Build Source Stamp: [branch jackrabbit/oak/trunk] 1587212 Blamelist: reschke BUILD FAILED: failed compile sincerely, -The Buildbot
RE: Using Lucene indexes for property queries
In theory, the Lucene index could be used quite easily. As far as I see, we would only need to change the cost function of the Lucene index (return a reasonable cost even if there is no full-text constraint). +1 for allowing use of lucene indexes for property constraint, there advanced search use cases i.e. support GQL like search queries. Then some applications allow customers to perform ad hoc searches based on custom properties.. In such cases, searchable properties are not known in advance. A small lag should be acceptable in such cases. Regards, Amit -Original Message- From: Chetan Mehrotra [mailto:chetan.mehro...@gmail.com] Sent: 14 April 2014 14:48 To: oak-dev@jackrabbit.apache.org Subject: Re: Using Lucene indexes for property queries Should we let the user decide whether it's OK to use an asynchronous index for this case +1 for that. It has been the case with JR2 (I may be wrong here). And when user is searching for say some asset via DAM in Adobe CQ then he would be ok if result is not for latest head. A small lag should be acceptable. This would enable scenarios where traversal would be too costly and Lucene can still be used to provide required results in a lot lesser time. Chetan Mehrotra On Mon, Apr 14, 2014 at 2:33 PM, Thomas Mueller muel...@adobe.com wrote: Hi, In theory, the Lucene index could be used quite easily. As far as I see, we would only need to change the cost function of the Lucene index (return a reasonable cost even if there is no full-text constraint). One problem might be: the Lucene index is asynchronous, and the user might expect the result to be up-to-date. The user knows this already for full-text constraints, but not for property constraints. Should we let the user decide whether it's OK to use an asynchronous index for this case? For example by specifying an option in the query (for example similar to the order by, at the very end of the query, option async)? So a query that can use an asynchronous index would look like this: //*[@prop = 'x'] option async or //*[@prop = 'x'] order by @otherProperty option async or select [jcr:path] from [nt:base] as a where [prop] 1 option async Regards, Thomas On 14/04/14 06:54, Chetan Mehrotra chetan.mehro...@gmail.com wrote: Hi, In JR2 I believe Lucene was used for all types of queries and not only for full text searches. In Oak we have our own PropertyIndexes for handling queries involving constraints on properties. This I believe provides a more accurate result as its built on top of mvcc support so results obtained are consistent with session state/revision. However this involves creating a index for property to be queried. And the way currently property indexes are stored they consume quite a bit of state (at least in DocumentNodeStore). In comparison Lucene stores the index content in quite compact form. In quite a few cases (like user choice based query builder) it might not be known in advance which property the user would use. As we already have all string property indexed in Lucene. Would it be possible to use Lucene for performing such queries? Or allow the user to choose which types of index he wants to use depending on the usecase. Chetan Mehrotra
Re: svn commit: r1587286 - in /jackrabbit/oak/trunk: oak-core/pom.xml oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreService.java oak-parent/pom.xml
Hi Julian, On Tue, Apr 15, 2014 at 12:39 AM, resc...@apache.org wrote: - Embed-Dependencycommons-dbcp,commons-pool,h2,json-simple/Embed-Dependency + Embed-Dependencycommons-dbcp,commons-pool,h2,json-simple,postgresql,db2,db2-license/Embed-Dependency Embed-Transitivetrue/Embed-Transitive I believe this is a temporary change and would not be required for final implementation? Would be helpful if we add a TODO/FIXME there such that we remember to remove this later Instead of embedding all such types of drivers/dbcp/pool etc within oak-core it would be better to decouple them. For example one approach can be 1. Have a bundle which embeds common-dbcp and required dependencies. It would be responsible for registering a DataSource 2. Driver bundle would be fragments to the bundle #1 as host. With JDBC 4.0 the Driver classes are provided as part of META-INF/services/java.sql.Driver [1]. For such cases fragment bundles can be avoided by having #1 monitor for such drivers and register them programatically 3. DocumentNodeStoreService should only have a reference to DataSource and use that Chetan Mehrotra [1] http://docs.oracle.com/javase/7/docs/api/java/sql/DriverManager.html
buildbot failure in ASF Buildbot on oak-trunk-win7
The Buildbot has detected a new failure on builder oak-trunk-win7 while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/oak-trunk-win7/builds/5311 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: bb-win7 Build Reason: scheduler Build Source Stamp: [branch jackrabbit/oak/trunk] 1587399 Blamelist: jukka BUILD FAILED: failed compile sincerely, -The Buildbot
Re: buildbot failure in ASF Buildbot on oak-trunk-win7
Hi, On Tue, Apr 15, 2014 at 12:18 AM, build...@apache.org wrote: The Buildbot has detected a new failure on builder oak-trunk-win7 while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/oak-trunk-win7/builds/5311 Failed tests: testClockDrift(org.apache.jackrabbit.oak.stats.ClockTest): Clock.Fast unexpected drift: -41ms (estimated limit was 5ms, measured granularity was 1.0ms) Clock.Fast uses a background thread to update the reported time, so I wonder if the above failure was caused simply by a random delay in thread scheduling. Let's see if the problem occurs again before I look deeper into this. BR, Jukka Zitting
Re: Upgrade to Oak and Mandatory CommitHooks
Hi, On Thu, Apr 10, 2014 at 2:55 AM, Angela Schreiber anch...@adobe.com wrote: If my analysis for the issue described above was right and you are about to fix it, I would be very glad if you could verify that all the required commit hooks are in place. Yes, you're right. See OAK-1719 for a summary. All security-related hooks are now included and the rest I've reviewed/enabled on a case-by-case basis. BR, Jukka Zitting
