On Thu, Jun 07, 2007 at 05:19:26PM +0000, Daniel Einspanjer wrote: > I am looking at trying to use ferret/aaf to supplement my querying against a > medium and large table with lots of columns. Some facts first: > > Ferret 0.11.4 > AAF 0.4.0 > Ruby 1.8.6 > Rails 1.2.3 > > Medium table: > 105,464 rows > 168 columns (mostly varchar(20)) > 11 actual columns indexed in aaf plus > 40 virtual columns indexed in aaf (virtual is concat of two physical columns. > e.g. cast_first_name_1 + cast_last_name_1 through cast_first_name_20 + > cast_last_name_20) > > Large table: > 1,244,716 rows > same column/index structure > > These tables are not updated via Ruby, only read. I am trying to use > rebuild_index to bootstrap the medium sized table and it is taking a very long > time (running for about 4 hours, indicates 50% complete with 4 hours > remaining) > and creating a massive number of files in the index directory (currently about > 65k, was 90k earlier)
strange. Ferret is faster than that - I have a test script that builds an index of 100000 documents with 50 fields each containing a single random word in under 10 Minutes here on standard hardware. Maybe the problem is something else? For starters, change line 220 of local_index.rb from index << rec.to_doc if rec.ferret_enabled?(true) to doc = rec.to_doc if rec.ferret_enabled?(true) so nothing is added to the index. How long does that take? Jens -- Jens Krämer webit! Gesellschaft für neue Medien mbH Schnorrstraße 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 [EMAIL PROTECTED] | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa _______________________________________________ Ferret-talk mailing list [email protected] http://rubyforge.org/mailman/listinfo/ferret-talk
