Re: Doing what does using SolrJ API

2020-09-17 Thread Steven White
Thank you all for your feedback. They are very helpful. @Walther, out of the 1000 fields in Solr's schema, only 5 are set as "required" fields and the Solr doc that I create and then send to Solr for indexing, contains only those fields that have data to be indexed. So some docs will have 10 fie

Re: Doing what does using SolrJ API

2020-09-17 Thread Erick Erickson
The script can actually be written an any number of scripting languages, python, groovy, javascript etc. but Alexandre’s comments about javascript are well taken. It all depends here on whether you every want to search the fields individually. If you do, you need to have them in your index as we

Re: Doing what does using SolrJ API

2020-09-17 Thread Walter Underwood
If you want to ignore a field being sent to Solr, you can set indexed=false and stored=false for that field in schema.xml. It will take up room in schema.xml but zero room on disk. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Sep 17, 2020, at 10:

Re: Doing what does using SolrJ API

2020-09-17 Thread Alexandre Rafalovitch
Solr has a whole pipeline that you can run during document ingesting before the actual indexing happens. It is called Update Request Processor (URP) and is defined in solrconfig.xml or in an override file. Obviously, since you are indexing from SolrJ client, you have even more flexibility, but it i

Re: Doing what does using SolrJ API

2020-09-17 Thread Steven White
Thanks Erick. Where can I learn more about "stateless script update processor factory". I don't know what you mean by this. Steven On Thu, Sep 17, 2020 at 1:08 PM Erick Erickson wrote: > 1000 fields is fine, you'll waste some cycles on bookkeeping, but I really > doubt you'll notice. That sai

Re: Doing what does using SolrJ API

2020-09-17 Thread Erick Erickson
1000 fields is fine, you'll waste some cycles on bookkeeping, but I really doubt you'll notice. That said, are these fields used for searching? Because you do have control over what gous into the index if you can put a "stateless script update processor factory" in your update chain. There you can

Re: Doing what does using SolrJ API

2020-09-17 Thread Steven White
Hi Eric, Yes, this is coming from a DB. Unfortunately I have no control over the list of fields. Out of the 1000 fields that there maybe, no document, that gets indexed into Solr will use more then about 50 and since i'm copying the values of those fields to the catch-all field and the catch-all

Re: Doing what does using SolrJ API

2020-09-17 Thread Erick Erickson
“there over 1000 of them[fields]” This is often a red flag in my experience. Solr will handle that many fields, I’ve seen many more. But this is often a result of “database thinking”, i.e. your mental model of how all this data is from a DB perspective rather than a search perspective. It’s unw

Re: Doing what does using SolrJ API

2020-09-16 Thread Steven White
Hi everyone, I figured it out. It is as simple as creating a List and using that as the value part for SolrInputDocument.addField() API. Thanks, Steven On Wed, Sep 16, 2020 at 9:13 PM Steven White wrote: > Hi everyone, > > I want to avoid creating a source="OneFieldOfMany"/> in my schema (

Doing what does using SolrJ API

2020-09-16 Thread Steven White
Hi everyone, I want to avoid creating a in my schema (there will be over 1000 of them and maybe more so managing it will be a pain). Instead, I want to use SolrJ API to do what does. Any example of how I can do this? If there is an example online, that would be great. Thanks in advance. Ste