RE: Slide search performance issues
So I guess one wouldn't have to have an Exchange Server in order to implement those extensions? I'm not too happy about using proprietary extensions to a standard, but sometimes you have to be a realist and accept things... Anyway, we have solved to problem by modifying the source to Slide. This is not a good way to solve the problem, as upgrades to newer versions of Slide will be difficult but it was necessary to get the performance we need. Basically what we did was to add a MySQL-specific ComparableResourcesPool that adds the keyword limit to the select statement. This was a quick fix to a serious problem so it not thought through all the way yet so maybe we will do something else in the future. Regards, Pontus -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Monday, January 17, 2005 3:37 PM To: 'Slide Users Mailing List' Subject: AW: Slide search performance issues My favourite approach concerning this issue would be to implement the MS-Exchange extension that adds the ability to specify to limit the result set by specifying ranges of results. This is described at the MS WebDAV pages as far as I remember. But it is not that easy to implement that on server side. But in fact would lead a big step towards Exchange compatibility... I know that everybody hates MS to extend the standards, but in the area of WebDAV they've added some useful bits. So why not adopt it? Some ranges could be mapped to sql commands by retrieving only the first rows, others could at least be filtered at java result set level. Any ideas? Cheers, Daniel -Ursprüngliche Nachricht- Von: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] pache.org] Im Auftrag von Pontus Strand Gesendet: Montag, 17. Januar 2005 08:22 An: 'Slide Users Mailing List' Betreff: RE: Slide search performance issues Ok, found the reason to why the query below didn't work, I had failed to include a group by-clause. However, that doen't solve the problem. If I get lots of hits when doing a search it still takes a long time to execute. What I would like to do is to move the limititation from the Slide server to the DB-server (where it really belong). As I understand it the keyword limit isn't part of SQL-standard (Orcale for one doen't implement it) so I guess that this is difficult do in generic terms. This is, however, important for us as we need the performance so which class/classes would I have to override in order to modify the behaviour of the search method? Is this recommended at all? Or should we try to limit the client so we can't do to wide searches? /Pontus -Original Message- From: Pontus Strand [mailto:[EMAIL PROTECTED] Sent: Friday, January 14, 2005 2:51 PM To: 'Slide Users Mailing List' Subject: RE: Slide search performance issues A quick follow-up, when working with large number of files the number of hits when doing a search could be large. As I understand it, it is possible to limit the number of responses by using the DAV:limit XML element from the DASL specification. However, the specification also states that the server may disregard the requested limit. So my questions are: Does Slide support DAV:limit? And will it work when using the use-rdbms-expression-factory parameter? And, finally, am I using correct syntax in the example below? D:searchrequest xmlns:D =\DAV:\ D:basicsearch D:select D:allprop/ /D:select D:from D:scope D:href/D:href /D:scope /D:from D:where D:and D:eqD:propD:fileextension//D:prop D:literalxml/D:literal/D:eq /D:and /D:where D:limit D:nresults 10 /D:nresults /D:limit /D:basicsearch /D:searchrequest The reason for my questions is that I can't get it to work and I can't find any references to this. Is there perhaps another way to limit the number of search hits? Regards, Pontus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Slide search performance issues
I have been asked to post my changes, please note that I consider this a quick fix or even a hack. The basic idea behind it is that the ComparableResourcesPool used is specific to each database. In order to solve this the method getRequestedResourcePool() in RDBMSExpressionFactory was modified to check which RDBMS-adapter is used. To achieve that I hade to modify AbstractRDBMSStore to return the adapter, i.e. I added the method getAdapter(). I'm not sure this is a good solution, hence my designation of this as a hack. Finally I added the class MySql41ComparableResourcesPool, that is an extension of RDBMSComparableResourcesPool. The class is basically a copy of RDBMSComparableResourcesPool, but with a few small differences. The method getPool() check to see if a limit has been requested, if so it tries to retrieve objects matching the limit. Worth noting is that the SQL-query can return documents that the user doesn't have access to. So in order to get the right amount of documents the SQL-query is asked as many times as needed to get either the requested limit or until no more results can be found. The other major method that was changed is compileSQL(), this method was modified to include the limit-clause. Also, the retrieveObjects()-method where modified to take the current starting offset as a parameter, a parameter that is passed on to compileSQL(). I hope this desciption is good enough to understand the changes made and the logic behind those changes. Perhaps a seed has been planted so that someone can implement a more thought through solution. Best regards, Pontus -Original Message- From: Pontus Strand [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 18, 2005 10:04 AM To: 'Slide Users Mailing List' Subject: RE: Slide search performance issues So I guess one wouldn't have to have an Exchange Server in order to implement those extensions? I'm not too happy about using proprietary extensions to a standard, but sometimes you have to be a realist and accept things... Anyway, we have solved to problem by modifying the source to Slide. This is not a good way to solve the problem, as upgrades to newer versions of Slide will be difficult but it was necessary to get the performance we need. Basically what we did was to add a MySQL-specific ComparableResourcesPool that adds the keyword limit to the select statement. This was a quick fix to a serious problem so it not thought through all the way yet so maybe we will do something else in the future. Regards, Pontus -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Monday, January 17, 2005 3:37 PM To: 'Slide Users Mailing List' Subject: AW: Slide search performance issues My favourite approach concerning this issue would be to implement the MS-Exchange extension that adds the ability to specify to limit the result set by specifying ranges of results. This is described at the MS WebDAV pages as far as I remember. But it is not that easy to implement that on server side. But in fact would lead a big step towards Exchange compatibility... I know that everybody hates MS to extend the standards, but in the area of WebDAV they've added some useful bits. So why not adopt it? Some ranges could be mapped to sql commands by retrieving only the first rows, others could at least be filtered at java result set level. Any ideas? Cheers, Daniel -Ursprüngliche Nachricht- Von: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] pache.org] Im Auftrag von Pontus Strand Gesendet: Montag, 17. Januar 2005 08:22 An: 'Slide Users Mailing List' Betreff: RE: Slide search performance issues Ok, found the reason to why the query below didn't work, I had failed to include a group by-clause. However, that doen't solve the problem. If I get lots of hits when doing a search it still takes a long time to execute. What I would like to do is to move the limititation from the Slide server to the DB-server (where it really belong). As I understand it the keyword limit isn't part of SQL-standard (Orcale for one doen't implement it) so I guess that this is difficult do in generic terms. This is, however, important for us as we need the performance so which class/classes would I have to override in order to modify the behaviour of the search method? Is this recommended at all? Or should we try to limit the client so we can't do to wide searches? /Pontus -Original Message- From: Pontus Strand [mailto:[EMAIL PROTECTED] Sent: Friday, January 14, 2005 2:51 PM To: 'Slide Users Mailing List' Subject: RE: Slide search performance issues A quick follow-up, when working with large number of files the number of hits when doing a search could be large. As I understand it, it is possible
RE: Slide search performance issues
Many Thanks Pontus! :)
Miguel
-Original Message-
From: Pontus Strand [mailto:[EMAIL PROTECTED]
Sent: terça-feira, 18 de Janeiro de 2005 13:17
To: 'Slide Users Mailing List'
Subject: RE: Slide search performance issues
Ok, it seems that zip-files aren't approved by our mailserver ...
In the file AbstractRDBMSStore this was added:
// TODO: Added method. Maybe a bad idea to expose this variable?
public RDBMSAdapter getAdapter() {
return adapter;
}
In the file RDBMSExpressionFactory this method was modified:
protected ComparableResourcesPool getRequestedResourcePool() {
if (requestedResourcePool == null) {
// TODO: Add comment here.
// TODO: Add more pools here if need be ...
if (_store.getAdapter() instanceof MySql41RDBMSAdapter) {
requestedResourcePool = new
MySql41ComparableResourcesPool(_store, _context, getQuery());
}
else {
requestedResourcePool = new
RDBMSComparableResourcesPool(_store, _context, getQuery());
}
}
return requestedResourcePool;
}
--
The file MySql41ComparableResourcesPool.java was created and looks like
this:
/*
* $Header:
/home/cvs/jakarta-slide/src/stores/org/apache/slide/store/impl/rdbms/MySql41
ComparableResourcesPool.java,v 1.10.2.4 2004/10/27 12:58:41 unico Exp $
* $Revision: 1.10.2.4 $
* $Date: 2004/10/27 12:58:41 $
*
*
*
* Copyright 1999-2004 The Apache Software Foundation
*
* Licensed under the Apache License, Version 2.0 (the License);
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an AS IS BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.slide.store.impl.rdbms;
import java.lang.reflect.Constructor;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.Map;
import java.util.Set;
import org.apache.slide.common.PropertyName;
import org.apache.slide.common.RequestedProperties;
import org.apache.slide.common.RequestedProperty;
import org.apache.slide.common.RequestedPropertyImpl;
import org.apache.slide.common.ServiceAccessException;
import org.apache.slide.common.SlideException;
import org.apache.slide.common.SlideRuntimeException;
import org.apache.slide.content.NodeProperty;
import org.apache.slide.search.BadQueryException;
import org.apache.slide.search.PropertyProvider;
import org.apache.slide.search.QueryScope;
import org.apache.slide.search.SearchQuery;
import org.apache.slide.search.SearchToken;
import org.apache.slide.search.basic.ComparableResourceImpl;
import org.apache.slide.search.basic.ComparableResourcesPool;
import org.apache.slide.search.basic.IBasicQuery;
import org.apache.slide.security.AccessDeniedException;
import org.apache.slide.store.impl.rdbms.expression.RDBMSExpressionFactory;
import org.apache.slide.store.impl.rdbms.expression.RDBMSQueryContext;
import org.apache.slide.structure.ObjectNode;
import org.apache.slide.util.logger.Logger;
/**
*/
public class MySql41ComparableResourcesPool extends
RDBMSComparableResourcesPool {
private final AbstractRDBMSStore _store;
private final RDBMSQueryContext _context;
private final IBasicQuery _query;
private final SearchToken _token;
private final QueryScope _scope;
private final Map _selectProperties;
private final PropertyProvider _provider;
private Set _pool;
public MySql41ComparableResourcesPool(AbstractRDBMSStore store,
RDBMSQueryContext context,
IBasicQuery query) {
super(store, context, query);
_store = store;
_context = context;
_query = query;
_token = _query.getSearchToken();
_scope = _query.getScope();
_selectProperties = new HashMap();
_provider = new RDBMSPropertyProvider(_query.getPropertyProvider(),
_selectProperties);
if (_query instanceof SearchQuery) {
final RequestedProperties props = ((SearchQuery)
_query).requestedProperties();
if (!props.isAllProp()) {
final Iterator iter = props.getRequestedProperties();
while (iter.hasNext()) {
final RequestedProperty property
RE: Slide search performance issues
Ok, found the reason to why the query below didn't work, I had failed to include a group by-clause. However, that doen't solve the problem. If I get lots of hits when doing a search it still takes a long time to execute. What I would like to do is to move the limititation from the Slide server to the DB-server (where it really belong). As I understand it the keyword limit isn't part of SQL-standard (Orcale for one doen't implement it) so I guess that this is difficult do in generic terms. This is, however, important for us as we need the performance so which class/classes would I have to override in order to modify the behaviour of the search method? Is this recommended at all? Or should we try to limit the client so we can't do to wide searches? /Pontus -Original Message- From: Pontus Strand [mailto:[EMAIL PROTECTED] Sent: Friday, January 14, 2005 2:51 PM To: 'Slide Users Mailing List' Subject: RE: Slide search performance issues A quick follow-up, when working with large number of files the number of hits when doing a search could be large. As I understand it, it is possible to limit the number of responses by using the DAV:limit XML element from the DASL specification. However, the specification also states that the server may disregard the requested limit. So my questions are: Does Slide support DAV:limit? And will it work when using the use-rdbms-expression-factory parameter? And, finally, am I using correct syntax in the example below? D:searchrequest xmlns:D =\DAV:\ D:basicsearch D:select D:allprop/ /D:select D:from D:scope D:href/D:href /D:scope /D:from D:where D:and D:eqD:propD:fileextension//D:prop D:literalxml/D:literal/D:eq /D:and /D:where D:limit D:nresults 10 /D:nresults /D:limit /D:basicsearch /D:searchrequest The reason for my questions is that I can't get it to work and I can't find any references to this. Is there perhaps another way to limit the number of search hits? Regards, Pontus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Slide search performance issues
A quick follow-up, when working with large number of files the number of hits when doing a search could be large. As I understand it, it is possible to limit the number of responses by using the DAV:limit XML element from the DASL specification. However, the specification also states that the server may disregard the requested limit. So my questions are: Does Slide support DAV:limit? And will it work when using the use-rdbms-expression-factory parameter? And, finally, am I using correct syntax in the example below? D:searchrequest xmlns:D =\DAV:\ D:basicsearch D:select D:allprop/ /D:select D:from D:scope D:href/D:href /D:scope /D:from D:where D:and D:eqD:propD:fileextension//D:prop D:literalxml/D:literal/D:eq /D:and /D:where D:limit D:nresults 10 /D:nresults /D:limit /D:basicsearch /D:searchrequest The reason for my questions is that I can't get it to work and I can't find any references to this. Is there perhaps another way to limit the number of search hits? Regards, Pontus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Slide search performance issues
-Original Message- From: Guido Casper [mailto:[EMAIL PROTECTED] Sent: Thursday, January 13, 2005 8:44 AM To: Slide Users Mailing List Subject: Re: Slide search performance issues Pontus Strand wrote: Doing the same search as before on 1 files yielded a log file of 55 MB. Each one of the 1 files generated 18 different select-queries!! 18!!! There simply must be a better way to do this! snip/ Have you modified the search queries or search methods? Are there any parameters in Domain.xml or similar files that can be added and/or modified to increase performance? Any help at all would be welcome, even if it is just asking obvious questions! Did you put parameter name=use-rdbms-expression-factorytrue/parameter into your nodestore configuration? No, I didn't, mainly because I didn't know it existed. A Google-search on use-rdbms-expression-factory gave me only one hit so I wouldn't call this a well documented parameter ... However, maybe I should have found it given that the hit was from this mailing list and regarding searching and indexes ... What exactly is it that this parameter does? Because it does work exceptionally well, doing the same search as before on 4 files (which gave an OutOfMemoryException after 15 minutes) took about 1.4 seconds now. Now, given the performance boast shouldn't this parameter be on by default when using a JDBC store? After learning of this parameter I must ask this: Are there any more undocumented parameters that will boast performance? I guess this question is primarily directed to the developers of Slide. Guido, you saved our day!! Thank you! Regards, Pontus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Slide search performance issues
Guido, can you post a sample configuration? I'd like to add this to the dasl wiki page. Thanks, Stefan Guido Casper wrote: Pontus Strand wrote: Doing the same search as before on 1 files yielded a log file of 55 MB. Each one of the 1 files generated 18 different select-queries!! 18!!! There simply must be a better way to do this! snip/ Have you modified the search queries or search methods? Are there any parameters in Domain.xml or similar files that can be added and/or modified to increase performance? Any help at all would be welcome, even if it is just asking obvious questions! Did you put parameter name=use-rdbms-expression-factorytrue/parameter into your nodestore configuration? Guido -- Stefan Lützkendorf -- [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Slide search performance issues
I think the DASL-implementation concerning the RDBM-stores is not well supported at the moment. As Stefan is currently working on Lucene-based search in Slide HEAD it might be a choice to use this one and help him to find issues and make improvements. It would be great to have a lucene-based DASL implementation that can span multiple stores by hosting its own indexes. On the other hand all data has to be replecated and regarding memory consumption it would be much better to use the db directly. But it would take some combined effort from all people involved/interested to get this done. It's a bad idea to bypass Slide to improve search performance as this will not be supported by the community and has to be adopted for each Slide release... Daniel Slide Users Mailing List slide-user@jakarta.apache.org schrieb am 13.01.05 02:28:59: Surprise, but I too am seeing very slow search times! I'd greatly appreciate any insight the users of this list may have. I'm also seeing that when I do a getChildren on any collection with more than 10 entries, the times are very slow, even if it's just more collections. Thanks, Jeff From: Pontus Strand [EMAIL PROTECTED] Reply-To: Slide Users Mailing List slide-user@jakarta.apache.org To: 'Slide Users Mailing List' slide-user@jakarta.apache.org Subject: RE: Slide search performance issues Date: Wed, 12 Jan 2005 11:03:13 +0100 Hi Cédric, To add a custom property is really rather simple. First you must create a PropertyName object, like this: PropertyName propName = new PropertyName(DAV:, my_property_name); Next you have to use the setPath-method and the proppatchMethod in the WebdavResource class, like this: webdavResource.setPath(path_to_document_or_directory); webdavResource.proppatchMethod(propName, my_property_value, true); More details on these classes can be found in the javadoc. I hope this help you. The total lack of responses regarding the performance issues so far give me little hope that there is a solution, at least a simple solution. We are currently evaluating our options as how to proceed, currently we are looking into bypass Slide when searching. Regards, Pontus -Original Message- From: news [mailto:[EMAIL PROTECTED] Behalf Of Cédric Sent: Wednesday, January 12, 2005 10:25 AM To: slide-user@jakarta.apache.org Subject: Re: Slide search performance issues Hi Pontus , I have the same trouble, i work with 1 documents ... so if i found any solution or explication i will take you part. I have an another question , how do you created your own property , because when i created a property with proppatch i was enable to specify the Namespace webdav. Have you got a little exemple of specific property creation and how to have this property ? Thanks a lot ! CF - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] __ Verschicken Sie romantische, coole und witzige Bilder per SMS! Jetzt bei WEB.DE FreeMail: http://f.web.de/?mc=021193 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Slide search performance issues
Ok, forget about this mail... Great news to hear that DASL/RDBMS is working well :-) Slide Users Mailing List slide-user@jakarta.apache.org schrieb am 13.01.05 10:22:39: I think the DASL-implementation concerning the RDBM-stores is not well supported at the moment. As Stefan is currently working on Lucene-based search in Slide HEAD it might be a choice to use this one and help him to find issues and make improvements. It would be great to have a lucene-based DASL implementation that can span multiple stores by hosting its own indexes. On the other hand all data has to be replecated and regarding memory consumption it would be much better to use the db directly. But it would take some combined effort from all people involved/interested to get this done. It's a bad idea to bypass Slide to improve search performance as this will not be supported by the community and has to be adopted for each Slide release... Daniel Slide Users Mailing List slide-user@jakarta.apache.org schrieb am 13.01.05 02:28:59: Surprise, but I too am seeing very slow search times! I'd greatly appreciate any insight the users of this list may have. I'm also seeing that when I do a getChildren on any collection with more than 10 entries, the times are very slow, even if it's just more collections. Thanks, Jeff From: Pontus Strand [EMAIL PROTECTED] Reply-To: Slide Users Mailing List slide-user@jakarta.apache.org To: 'Slide Users Mailing List' slide-user@jakarta.apache.org Subject: RE: Slide search performance issues Date: Wed, 12 Jan 2005 11:03:13 +0100 Hi Cédric, To add a custom property is really rather simple. First you must create a PropertyName object, like this: PropertyName propName = new PropertyName(DAV:, my_property_name); Next you have to use the setPath-method and the proppatchMethod in the WebdavResource class, like this: webdavResource.setPath(path_to_document_or_directory); webdavResource.proppatchMethod(propName, my_property_value, true); More details on these classes can be found in the javadoc. I hope this help you. The total lack of responses regarding the performance issues so far give me little hope that there is a solution, at least a simple solution. We are currently evaluating our options as how to proceed, currently we are looking into bypass Slide when searching. Regards, Pontus -Original Message- From: news [mailto:[EMAIL PROTECTED] Behalf Of Cédric Sent: Wednesday, January 12, 2005 10:25 AM To: slide-user@jakarta.apache.org Subject: Re: Slide search performance issues Hi Pontus , I have the same trouble, i work with 1 documents ... so if i found any solution or explication i will take you part. I have an another question , how do you created your own property , because when i created a property with proppatch i was enable to specify the Namespace webdav. Have you got a little exemple of specific property creation and how to have this property ? Thanks a lot ! CF - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] __ Verschicken Sie romantische, coole und witzige Bilder per SMS! Jetzt bei WEB.DE FreeMail: http://f.web.de/?mc=021193 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] __ Verschicken Sie romantische, coole und witzige Bilder per SMS! Jetzt bei WEB.DE FreeMail: http://f.web.de/?mc=021193 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Slide search performance issues
Stefan Lützkendorf wrote: can you post a sample configuration? I'd like to add this to the dasl wiki page. Here is my store config: store name=mysql parameter name=cache-modefull/parameter nodestore classname=org.apache.slide.store.impl.rdbms.JDBCStore parameter name=adapterorg.apache.slide.store.impl.rdbms.MySqlRDBMSAdapter/parameter parameter name=drivercom.mysql.jdbc.Driver/parameter parameter name=urljdbc:mysql://@DB_HOST@:@DB_PORT@/@DB_DB@/parameter parameter name=user@DB_USER@/parameter parameter name=password@DB_PW@/parameter parameter name=dbcpPoolingtrue/parameter parameter name=maxPooledConnections100/parameter parameter name=isolationREAD_COMMITTED/parameter parameter name=SendStringParametersAsUnicodefalse/parameter parameter name=use-rdbms-expression-factorytrue/parameter /nodestore securitystorereference store=nodestore//securitystore lockstorereference store=nodestore//lockstore revisiondescriptorsstorereference store=nodestore//revisiondescriptorsstore revisiondescriptorstorereference store=nodestore//revisiondescriptorstore contentstorereference store=nodestore//contentstore contentindexer classname=org.apache.slide.index.TextContentIndexer parameter name=indexpath./index/parameter parameter name=includes/files/repo/staging/parameter parameter name=analyzerorg.apache.lucene.analysis.de.GermanAnalyzer/parameter /contentindexer /store scope match=/ store=mysql/ BTW, thanks for your great work on Lucene integration. I'll try it out, soon. Guido Thanks, Stefan Guido Casper wrote: Pontus Strand wrote: Doing the same search as before on 1 files yielded a log file of 55 MB. Each one of the 1 files generated 18 different select-queries!! 18!!! There simply must be a better way to do this! snip/ Have you modified the search queries or search methods? Are there any parameters in Domain.xml or similar files that can be added and/or modified to increase performance? Any help at all would be welcome, even if it is just asking obvious questions! Did you put parameter name=use-rdbms-expression-factorytrue/parameter into your nodestore configuration? Guido -- Freundliche Grüße / With kind regards Guido Casper SN AG Competence Center Open Source Klingenderstr. 5 D 33100 Paderborn voice +49 5251/1581-87 fax+49 5251/1581-71 eMail [EMAIL PROTECTED] Webhttp://www.s-und-n.de - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Slide search performance issues
Pontus Strand wrote: What exactly is it that this parameter does? It just tells the store to leverage the org.apache.slide.store.impl.rdbms.expression stuff and the database. Without it it simply loads and examines each and every resource within the search scope. Because it does work exceptionally well, doing the same search as before on 4 files (which gave an OutOfMemoryException after 15 minutes) took about 1.4 seconds now. Now, given the performance boast shouldn't this parameter be on by default when using a JDBC store? I'm not sure how well it's tested with other DB stores (besides MySQL). Guido -- Freundliche Grüße / With kind regards Guido Casper SN AG Competence Center Open Source Klingenderstr. 5 D 33100 Paderborn voice +49 5251/1581-87 fax+49 5251/1581-71 eMail [EMAIL PROTECTED] Webhttp://www.s-und-n.de - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Slide search performance issues
Hi Pontus , I have the same trouble, i work with 1 documents ... so if i found any solution or explication i will take you part. I have an another question , how do you created your own property , because when i created a property with proppatch i was enable to specify the Namespace webdav. Have you got a little exemple of specific property creation and how to have this property ? Thanks a lot ! CF - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Slide search performance issues
Hi Cdric, To add a custom property is really rather simple. First you must create a PropertyName object, like this: PropertyName propName = new PropertyName(DAV:, my_property_name); Next you have to use the setPath-method and the proppatchMethod in the WebdavResource class, like this: webdavResource.setPath(path_to_document_or_directory); webdavResource.proppatchMethod(propName, my_property_value, true); More details on these classes can be found in the javadoc. I hope this help you. The total lack of responses regarding the performance issues so far give me little hope that there is a solution, at least a simple solution. We are currently evaluating our options as how to proceed, currently we are looking into bypass Slide when searching. Regards, Pontus -Original Message- From: news [mailto:[EMAIL PROTECTED] Behalf Of Cdric Sent: Wednesday, January 12, 2005 10:25 AM To: slide-user@jakarta.apache.org Subject: Re: Slide search performance issues Hi Pontus , I have the same trouble, i work with 1 documents ... so if i found any solution or explication i will take you part. I have an another question , how do you created your own property , because when i created a property with proppatch i was enable to specify the Namespace webdav. Have you got a little exemple of specific property creation and how to have this property ? Thanks a lot ! CF - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Slide search performance issues
Surprise, but I too am seeing very slow search times! I'd greatly appreciate any insight the users of this list may have. I'm also seeing that when I do a getChildren on any collection with more than 10 entries, the times are very slow, even if it's just more collections. Thanks, Jeff From: Pontus Strand [EMAIL PROTECTED] Reply-To: Slide Users Mailing List slide-user@jakarta.apache.org To: 'Slide Users Mailing List' slide-user@jakarta.apache.org Subject: RE: Slide search performance issues Date: Wed, 12 Jan 2005 11:03:13 +0100 Hi Cédric, To add a custom property is really rather simple. First you must create a PropertyName object, like this: PropertyName propName = new PropertyName(DAV:, my_property_name); Next you have to use the setPath-method and the proppatchMethod in the WebdavResource class, like this: webdavResource.setPath(path_to_document_or_directory); webdavResource.proppatchMethod(propName, my_property_value, true); More details on these classes can be found in the javadoc. I hope this help you. The total lack of responses regarding the performance issues so far give me little hope that there is a solution, at least a simple solution. We are currently evaluating our options as how to proceed, currently we are looking into bypass Slide when searching. Regards, Pontus -Original Message- From: news [mailto:[EMAIL PROTECTED] Behalf Of Cédric Sent: Wednesday, January 12, 2005 10:25 AM To: slide-user@jakarta.apache.org Subject: Re: Slide search performance issues Hi Pontus , I have the same trouble, i work with 1 documents ... so if i found any solution or explication i will take you part. I have an another question , how do you created your own property , because when i created a property with proppatch i was enable to specify the Namespace webdav. Have you got a little exemple of specific property creation and how to have this property ? Thanks a lot ! CF - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Slide search performance issues
Pontus Strand wrote: Doing the same search as before on 1 files yielded a log file of 55 MB. Each one of the 1 files generated 18 different select-queries!! 18!!! There simply must be a better way to do this! snip/ Have you modified the search queries or search methods? Are there any parameters in Domain.xml or similar files that can be added and/or modified to increase performance? Any help at all would be welcome, even if it is just asking obvious questions! Did you put parameter name=use-rdbms-expression-factorytrue/parameter into your nodestore configuration? Guido -- Freundliche Grüße / With kind regards Guido Casper SN AG Competence Center Open Source Klingenderstr. 5 D 33100 Paderborn voice +49 5251/1581-87 fax+49 5251/1581-71 eMail [EMAIL PROTECTED] Webhttp://www.s-und-n.de - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
