That was the original plan. However its important to preserve the originating field that loaded the value. The data is very fine and granular and each field stores a particular value. When searching the data against solr - it would be important to know what docs contain that particular data from that particular field. (fielda:value, fieldb:value ) where as searching field:value would hide what originating field loaded this value.
Im going to try loading all 3000 fields in the schema and see how that goes. Only concern is doing boolean searches and whether or not Ill run into URL length issues but I guess Ill find out soon. Thanks again! Sent from my iPhone > On Feb 6, 2014, at 1:02 PM, Erick Erickson <erickerick...@gmail.com> wrote: > > Sometimes you can spoof the many fields problem by using prefixes on the > data. Rather than fielda, fieldb... Have field and index values like > fielda_value, fieldb_value into a single field. Then do the right thing > when searching. Watch tokenization though. > > Best > Erick >> On Feb 5, 2014 4:59 AM, "Mike L." <javaone...@yahoo.com> wrote: >> >> >> Thanks Shawn. This is good to know. >> >> >> Sent from my iPhone >> >>>> On Feb 5, 2014, at 12:53 AM, Shawn Heisey <s...@elyograg.org> wrote: >>>> >>>> On 2/4/2014 8:00 PM, Mike L. wrote: >>>> I'm just wondering here if there is any defined limit to how many >> fields can be created within a schema? I'm sure the configuration >> maintenance of a schema like this would be a nightmare, but would like to >> know if its at all possible in the first place before It may be attempted. >>> >>> There are no hard limits on the number of fields, whether they are >>> dynamically defined or not. Several thousand fields should be no >>> problem. If you have enough system resources and you don't run into an >>> unlikely bug, there's no reason it won't work. As you've already been >>> told, there are potential performance concerns. Depending on the exact >>> nature of your queries, you might need to increase maxBooleanClauses. >>> >>> The only hard limitation that Lucene really has (and by extension, Solr >>> also has that limitation) is that a single index cannot have more than >>> about two billion documents in it - the inherent limitation on a Java >>> "int" type. Solr can use indexes larger than this through sharding. >>> >>> See the very end of this page: >> https://lucene.apache.org/core/4_6_0/core/org/apache/lucene/codecs/lucene46/package-summary.html#Limitations >>> >>> Thanks, >>> Shawn >>
