I've managed to run the synonyms with 10 different synonyms file. Each of the synonym file size is 1MB, which consist of about 1000 tokens, and each token has about 40-50 words. These lists of files are more extreme, which I probably won't use for the real environment, except now for the testing purpose.
The QTime is about 100-200, as compared to about 50 for collection without synonyms configured. Is this timing consider fast or slow? Although the synonyms files are big, there's not that many index in my collection yet. Just afraid the performance will be affected when more index comes in. Regards, Edwin On 9 May 2015 00:14, "Zheng Lin Edwin Yeo" <edwinye...@gmail.com> wrote: > Thank you for your suggestions. > > I can't do a proper testing on that yet as I'm currently using a 4GB RAM > normal PC machine, and all these probably requires more RAM that what I > have. > I've tried running the setup with 20 synonyms file, and the system went > Out of Memory before I could test anything. > > For your option 2), do you mean that I'll need to download a synonym > database (like the one with over 20MB in size which I have), and index them > into an Ad Hoc Solr Core to manage them? > > I probably can only try them out properly when I can get the server > machine with more RAM. > > Regards, > Edwin > > > On 8 May 2015 at 22:16, Alessandro Benedetti <benedetti.ale...@gmail.com> > wrote: > >> This is a quite big Sinonym corpus ! >> If it's not feasible to have only 1 big synonym file ( I haven't checked, >> so I assume the 1 Mb limit is true, even if strange) >> I would do an experiment : >> 1) testing query time with a Solr Classic config >> 2) Use an Ad Hoc Solr Core to manage Synonyms ( in this way we can keep it >> updated and use it with a custom version of the Sysnonym filter that will >> get the Synonyms directly from another Solr instance). >> 2b) develop a Solr plugin to provide this approach >> >> If the synonym thesaurus is really big, I guess managing them through >> another Solr Core ( or something similar) locally , will be better than >> managing it with an external web service. >> >> Cheers >> >> 2015-05-08 12:16 GMT+01:00 Zheng Lin Edwin Yeo <edwinye...@gmail.com>: >> >> > So it means like having more than 10 or 20 synonym files locally will >> still >> > be faster than accessing external service? >> > >> > As I found out that zookeeper only allows the synonym.txt file to be a >> > maximum of 1MB, and as my potential synonym file is more than 20MB, I'll >> > need to split the file to more than 20 of them. >> > >> > Regards, >> > Edwin >> > >> >> >> >> -- >> -------------------------- >> >> Benedetti Alessandro >> Visiting card : http://about.me/alessandro_benedetti >> >> "Tyger, tyger burning bright >> In the forests of the night, >> What immortal hand or eye >> Could frame thy fearful symmetry?" >> >> William Blake - Songs of Experience -1794 England >> > >
