Thanks for your replies.
As it happens, on this particular project, the information is in an Access document. As the data is already there, it means that I can not process the information before entering the database. I have also been considering a solution since I posted the message, and realized that the way to do it, may be to flag up any messages that do have a 'bad' word contained within the record. This would only have to be done once, and then the 'bad' messages could be dealt with as necessary at display time. Getting more into specifics, would the best be to have an array containing the 'bad' words and then check each word in each record against this? Or is there an sql command that may do this more easily. Your help is appreciated, B Date: Sat, 9 Jul 2005 08:17:32 +1000 From: "David Smart" <[EMAIL PROTECTED]> Subject: Re: RE: Censoring Records Yes. A very good point. Actually, to move further. The question that ran through my mind at the time was "why ask this on an ASP group, it's an off-line database processing question and should be being done in a 'real' language rather than on-line". Getting the database engine itself to do it is a further refinement. Taking it one step further ... why not censor the information BEFORE it gets to the database rather than afterwards? That way you're only doing the work once per database record, rather than each time it's accessed. Furthermore, one assumes the records are created individually, over time, so the processing overhead will not be so noticeable. Dave S ----- Original Message ----- From: Ben Sauer To: [email protected] Sent: Friday, July 08, 2005 11:49 PM Subject: Re: [ASP] RE: Censoring Records This isn't really the right way to do this... the processing can be horrendous. You need to do this as a database side process. Preferrably on load. Set a flag and query the flag. What DB are you on? On 7/8/05, David Smart <[EMAIL PROTECTED]> wrote: > There are some facilities you can use that spring to mind. > > But first, a comment ... > > You are going to have to parse the records to at least some extent (assuming > the records contain free text. For instance, it's not enough to look for > the string "bum", as you'll censor phrases like "bumper crop". > > One way to do this is to transform the record so that: > > - it contains no punctuation with the exception of a space between words > - it starts and ends with a space > > Once you've done this, you can do your search for " bum " without false > triggers on longer words. > > Now, I don't know regular expressions well enough to answer the next bit, > but I expect that they can help you to at least knock out the punctuation > and insert the spaces at each end. (Note that it isn't necessary to convert > multiple spaces back to single spaces.) > > Now, you need to check all your words against the sentence. At the very > least, you can use the VBScript / JScript "find sting inside string" > built-in facility, which will be pretty quick. > > But this will only do one word search at a time, so you'd be doing a lot of > them if you have a big list of bad words. > > You could always reverse the search - breaking the record into individual > words with a split facility (especially in JScript) and joining your list of > bad words into one "sentence" (again the JScript join is great for this). > Then you only need to do as many checks as there are words in the record - > certainly that's fewer iterations through the loop than iterating through > the bad word list. > > But, get advice from a regular expressions guru. I never fail to be amazed > at just how much you can achieve with them if you know how. For instance, > if you transform both your record and your bad words into a string each with > single spaces separating the words (something regular expressions can > certainly do), it might be possible to use a regular expression to look for > a match. > > Dave S [Non-text portions of this message have been removed] --------------------------------------------------------------------- Home : http://groups.yahoo.com/group/active-server-pages --------------------------------------------------------------------- Post : [email protected] Subscribe : [EMAIL PROTECTED] Unsubscribe: [EMAIL PROTECTED] --------------------------------------------------------------------- Yahoo! Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/active-server-pages/ <*> To unsubscribe from this group, send an email to: [EMAIL PROTECTED] <*> Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/
