You say your SQL database always has the values, but does the output from the SQL query you actually use have them? I've been fooled before by the query I form "somehow" doesn't have a value for all fields I expect.
You could also crank the logging level up enough to see the docs that are indexed, although that would probably only confirm that the fields weren't in the docs which you know already, not tell you why they are missing. Pull the SQL out and run it independently perhaps? I sound a bit like a broken record, but this is why I like SolrJ, I can actually debug that: https://lucidworks.com/2012/02/14/indexing-with-solrj/ Best, Erick On Tue, Apr 25, 2017 at 8:28 AM, AJ Lemke <aj.le...@securitylabs.com> wrote: > Thanks for the thought Alex! > The fields that have this happen most often are numeric and boolean fields. > These fields have real data (id numbers, true/false, etc.) > > AJ > > -----Original Message----- > From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] > Sent: Tuesday, April 25, 2017 8:27 AM > To: solr-user <solr-user@lucene.apache.org> > Subject: Re: DIH Issues > > Maybe the content gets simplified away between the database and the Solr > schema. For example if your field contains just spaces and you have > UpdateRequestProcessors to do trim and removal of empty fields? > > Schemaless mode will remove empty fields, but will not trim for example. > > Regards, > Alex. > ---- > http://www.solr-start.com/ - Resources for Solr users, new and experienced > > > On 25 April 2017 at 09:21, AJ Lemke <aj.le...@securitylabs.com> wrote: >> Hey all, >> >> We are using 6.3.0 and we have issues with DIH throwing errors. We are >> seeing an intermittent issue where on a full index a single error will be >> thrown. The error is always "missing required field: fieldname". >> Our SQL database always has data in the field that comes up with the error. >> Most of the errors are coming on fields that SQL has marked as required. >> >> Would anyone have any hints or ideas where to look to remedy this situation. >> >> As always if you need more information let me know. >> >> Thanks >> AJ
