RE: Search functionality: Verity, LIKE operator, or what?

[A.Little]

Hi,

We've had exactly the same problem with our document management system - the
way we originally got around it was to use 2 verity collections,
DocumentMetaData (a collection of the database entries) and DocumentFile (a
collection of the actual document files). Then when you do the verity search
you specify both collections. You'll then need to use a query on query of
the verity search result to strip out duplicates (where a record is returned
from DocumentMetaData and DocumentFile, but they are actually for the same
document) and you can use (another) query on query to strip out the
documents that the user doesn't have access to.

This method worked fine, though we found it was a bit too slow for our
liking. Having verity search more than 1 collection at once seemed to slow
it down quite a lot, plus the fact that you've then got the extra QoQ
manipulation.

We then changed how all our searching worked by making use of the full text
indexing within SQL and storing a copy of the file (using cfx_PutImage)
within the document table. Then we wrote a stored procedure to do the actual
searching and filtering out of documents that the user doesn't have access
to. We've found this to be much quicker and it gives us more flexibility to
alter the ranking - for example, if the search term is found in the title of
the document we multiply the ranking up (it's at x1.3 at the moment, but
we're still experimenting)

Hope that helps - I'm sure there's plenty of other solutions to this

Alex


> -Original Message-
> From: David Notik [mailto:dave@;digital202.com]
> Sent: 24 October 2002 00:03
> To: CF-Talk
> Subject: Search functionality: Verity, LIKE operator, or what?
> 
> 
> Hi all:
>  
> I haven't posted to this list for awhile.  I used to post all 
> the time,
> and used to write the TipWorld ColdFusion tips many of you received.
> Good to be back in the mix.
>  
> My question:
>  
> I built a document management system of sorts.  I have all 
> the files in
> a directory. there's a database record for every file with fields such
> as: title, filename, description, keywords, and more.  The filename
> field references the actual physical file and it's all very simple.
>  
> So now I want to provide the ability to search the files. 
> with a twist.
>  
> I have a box where the user can type in keywords.  I could 
> then write a
> query which uses the LIKE operator many times and searches the various
> relevant fields for that keyword.  But is that really the 
> most efficient
> way?
>  
> I know I could create a collection.  But doesn't a collection only
> search the actual content of the physical files indexed in the
> collection?  Meaning my title, description, and keywords fields (etc.)
> wouldn't be relevant.
>  
> What do I do?
>  
> That's question one. but then let's take it a step further:
>  
> I have a securitytype associated with each file record, which denotes
> whether the logged in user has access to the file.  When a 
> user searches
> for files, I DO NOT want to display files he does not have 
> access to.  I
> can easily accomplish this using my own query (LIKE operator 
> and a WHERE
> clause that only pulls records they have access to), but if I 
> do it the
> verity way, is this possible?
>  
> Just trying to decide the best way to go about this.
>  
> I am using ColdFusion Pro MX, SQLServer 2000.
>  
> --D
>  
> ###
> David Notik
> Digital202, LLC
> Imagination gone digital.
> Web:  <http://www.digital202.com/> www.digital202.com
> E-mail:  <mailto:dave@;digital202.com> [EMAIL PROTECTED]
> Office: (206) 575-1717
> Mobile: (206) 351-3948
> ###
>  
> 
> 
~|
Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4
Subscription: http://www.houseoffusion.com/index.cfm?sidebar=lists&body=lists/cf_talk
FAQ: http://www.thenetprofits.co.uk/coldfusion/faq
Signup for the Fusion Authority news alert and keep up with the latest news in 
ColdFusion and related topics. http://www.fusionauthority.com/signup.cfm

RE: Search functionality: Verity, LIKE operator, or what?

[Tilbrook, Peter]

David, email me off-list for help on this.

Cheers!

Peter Tilbrook ([EMAIL PROTECTED])
Project Officer 
Strategic Development 
Australian Building Codes Board 
GPO Box 9839 
CANBERRA ACT 2600 

Telephone: (02) 6213 6731 
Facsimile: (02) 6213 7287


-Original Message-
From: David Notik [mailto:dave@;digital202.com]
Sent: Thursday, 24 October 2002 9:03 AM
To: CF-Talk
Subject: Search functionality: Verity, LIKE operator, or what?


Hi all:
 
I haven't posted to this list for awhile.  I used to post all the time,
and used to write the TipWorld ColdFusion tips many of you received.
Good to be back in the mix.
 
My question:
 
I built a document management system of sorts.  I have all the files in
a directory. there's a database record for every file with fields such
as: title, filename, description, keywords, and more.  The filename
field references the actual physical file and it's all very simple.
 
So now I want to provide the ability to search the files. with a twist.
 
I have a box where the user can type in keywords.  I could then write a
query which uses the LIKE operator many times and searches the various
relevant fields for that keyword.  But is that really the most efficient
way?
 
I know I could create a collection.  But doesn't a collection only
search the actual content of the physical files indexed in the
collection?  Meaning my title, description, and keywords fields (etc.)
wouldn't be relevant.
 
What do I do?
 
That's question one. but then let's take it a step further:
 
I have a securitytype associated with each file record, which denotes
whether the logged in user has access to the file.  When a user searches
for files, I DO NOT want to display files he does not have access to.  I
can easily accomplish this using my own query (LIKE operator and a WHERE
clause that only pulls records they have access to), but if I do it the
verity way, is this possible?
 
Just trying to decide the best way to go about this.
 
I am using ColdFusion Pro MX, SQLServer 2000.
 
--D
 
###
David Notik
Digital202, LLC
Imagination gone digital.
Web:  <http://www.digital202.com/> www.digital202.com
E-mail:  <mailto:dave@;digital202.com> [EMAIL PROTECTED]
Office: (206) 575-1717
Mobile: (206) 351-3948
###
 


~|
Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4
Subscription: http://www.houseoffusion.com/index.cfm?sidebar=lists&body=lists/cf_talk
FAQ: http://www.thenetprofits.co.uk/coldfusion/faq
Get the mailserver that powers this list at http://www.coolfusion.com

Search functionality: Verity, LIKE operator, or what?

[David Notik]

Hi all:
 
I haven't posted to this list for awhile.  I used to post all the time,
and used to write the TipWorld ColdFusion tips many of you received.
Good to be back in the mix.
 
My question:
 
I built a document management system of sorts.  I have all the files in
a directory. there's a database record for every file with fields such
as: title, filename, description, keywords, and more.  The filename
field references the actual physical file and it's all very simple.
 
So now I want to provide the ability to search the files. with a twist.
 
I have a box where the user can type in keywords.  I could then write a
query which uses the LIKE operator many times and searches the various
relevant fields for that keyword.  But is that really the most efficient
way?
 
I know I could create a collection.  But doesn't a collection only
search the actual content of the physical files indexed in the
collection?  Meaning my title, description, and keywords fields (etc.)
wouldn't be relevant.
 
What do I do?
 
That's question one. but then let's take it a step further:
 
I have a securitytype associated with each file record, which denotes
whether the logged in user has access to the file.  When a user searches
for files, I DO NOT want to display files he does not have access to.  I
can easily accomplish this using my own query (LIKE operator and a WHERE
clause that only pulls records they have access to), but if I do it the
verity way, is this possible?
 
Just trying to decide the best way to go about this.
 
I am using ColdFusion Pro MX, SQLServer 2000.
 
--D
 
###
David Notik
Digital202, LLC
Imagination gone digital.
Web:   www.digital202.com
E-mail:   [EMAIL PROTECTED]
Office: (206) 575-1717
Mobile: (206) 351-3948
###
 

~|
Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4
Subscription: http://www.houseoffusion.com/index.cfm?sidebar=lists&body=lists/cf_talk
FAQ: http://www.thenetprofits.co.uk/coldfusion/faq
Get the mailserver that powers this list at http://www.coolfusion.com