Hi Mark, What we're doing is using a bunch of acceptance tests with JBehave to drive our testing. We run this in a clean room environment, clearing out the indexes before a test run and inserting the data we're interested in. As well as tests to ensure things "just work" we have a bunch of tests that insert data and check it comes out in the order we're expecting to - so unexpected changes to boosts etc. can be caught early.
Whereas what this doesn't tell us what a certain query will return with our live data set, it does affirm our assertions about the abstract case. You could use a similar technique to insert a bunch of data and then check your critical queries. > Hey guys, > > I'm wondering how people are managing regression testing, in particular with > things like text based search. > > I.e. if you change how fields are indexed or change boosts in dismax, > ensuring that doesn't mean that critical queries are showing bad data. > > The obvious answer to me was using unit tests. These may be brittle as some > index data can change over time, but I couldn't think of a better way. > > How is everyone else solving this problem? > > Cheers, > > Mark > > -- > E: mark.man...@gmail.com > T: http://www.twitter.com/neurotic > W: www.compoundtheory.com > > cf.Objective(ANZ) - Nov 17, 18 - Melbourne Australia > http://www.cfobjective.com.au > > Hands-on ColdFusion ORM Training > www.ColdFusionOrmTraining.com > > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ -- Colin Vipurs Server Team Lead Shazam Entertainment Ltd 26-28 Hammersmith Grove, London W6 7HA m: +44 (0) 0000 000 000 t: +44 (0) 20 8742 6820 w: www.shazam.com Please consider the environment before printing this document This e-mail and its contents are strictly private and confidential. It must not be disclosed, distributed or copied without our prior consent. If you have received this transmission in error, please notify Shazam Entertainment immediately on: +44 (0) 020 8742 6820 and then delete it from your system. Please note that the information contained herein shall additionally constitute Confidential Information for the purposes of any NDA between the recipient/s and Shazam Entertainment. Shazam Entertainment Limited is incorporated in England and Wales under company number 3998831 and its registered office is at 26-28 Hammersmith Grove, London W6 7HA. ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
