They reason for such a large number of fields: - users create dynamically 'classes' of documents, say one user creates 10 classes on average - for each 'class', the fields are created like this: "unique_id_"+fieldname - there are potentially hundreds of thousands of users.
There is faceting in each users' fields. So this will result in >1M fields, very sparsely populated. I warned them this did not sound like a good design to me, but apparently someone very knowledgeable in solr said this will work out fine. That is why I wanted to double check... On Sat, May 30, 2015 at 9:22 PM, Jack Krupansky <jack.krupan...@gmail.com> wrote: > Anything more than a few hundred seems very suspicious. > > Anything more than a few dozen or 50 or 75 seems suspicious as well. > > The point should not be how crazy can you get with Solr, but that craziness > should be avoided altogether! > > Solr's design is optimal for a large number of relatively small documents, > not large documents. > > > -- Jack Krupansky > > On Sat, May 30, 2015 at 3:05 PM, Erick Erickson <erickerick...@gmail.com> > wrote: > > > Nothing's really changed in that area lately. Your co-worker is > > perhaps confusing the statement that "Solr has no a-priori limit on > > the number of distinct fields that can be in a corpus" with supporting > > an infinite number of fields. Not having a built-in limit is much > > different than supporting.... > > > > Whether Solr breaks with thousands and thousands of fields is pretty > > dependent on what you _do_ with those fields. Simply doing keyword > > searches isn't going to put the same memory pressure on as, say, > > faceting on them all (even if in different queries). > > > > I'd really ask why so many fields are necessary though. > > > > Best, > > Erick > > > > On Sat, May 30, 2015 at 6:18 AM, xavi jmlucjav <jmluc...@gmail.com> > wrote: > > > Hi guys, > > > > > > someone I work with has been advised that currently Solr can support > > > 'infinite' number of fields. > > > > > > I thought there was a practical limitation of say thousands of fields > > (for > > > sure less than a million), or things can start to break (I think I > > > remember seeings memory issues reported on the mailing list by several > > > people). > > > > > > > > > Was there any change I missed lately that makes having say 1M fields in > > > Solr practical?? > > > > > > thanks > > >
