Re: DataImportHandler dynamic fields clarification
Harry, could you please file a jira for this and I'll address this in a patch. I fixed related issue (SOLR-2102) and I think it's pretty similar. > Interesting, I was under the impression that case does not matter. > > From http://wiki.apache.org/solr/DataImportHandler#A_shorter_data-config : > "It is possible to totally avoid the field entries in entities if the names > of the fields are same (case does not matter) as those in Solr schema" > Yeah, case does not matter only for explicit mapping of sql columns to Solr fields. The reason is that DIH populates hash map for case insensitive match only for explicit mappings. You can also workaround this upper case column names in Oracle using the following SQL clause: = data-config.xml schema.xml = HTH, Alexey On Thu, Sep 30, 2010 at 9:10 PM, harrysmith wrote: > >> >>Two things, one are your DB column uppercase as this would effect the out. >> >> > > Interesting, I was under the impression that case does not matter. > > From http://wiki.apache.org/solr/DataImportHandler#A_shorter_data-config : > "It is possible to totally avoid the field entries in entities if the names > of the fields are same (case does not matter) as those in Solr schema" > > I confirmed that matching the schema.xml field case to the database table is > needed for dynamic fields, and the wiki statement above is incorrect, or at > the very least confusing, possibly a bug. > > My database is Oracle 10g and the column names have been created in all > uppercase in the database. > > In Oracle: > Table name: wide_table > Column names: COLUMN_1 ... COLUMN_100 (yes, uppercase) > > Please see following scenarios and results I found: > > data-config.xml > > > > > schema.xml > multiValued="true" /> > > Result: > Nothing Imported > > = > > data-config.xml > > > > > schema.xml > multiValued="true" /> > > Result: > Note query column names changed to uppercase. > Nothing Imported > > = > > > data-config.xml > > > > > schema.xml > multiValued="true" /> > > Result: > Note ONLY the field entry was changed to caps > > All records imported, with only COLUMN_100 id field. > > > > data-config.xml > > > > > schema.xml > multiValued="true" /> > > Result: > Note BOTH the field entry was changed to caps in data-config.xml, and the > dynamicField wildcard in schema.xml > > All records imported, with all fields specified. This is the behavior > desired. > > = > > > > > > > > > > > > > > > > > > > >> >>Second what does your db-data-config.xml look like >> >> > > The relevant data-config.xml is as follows: > > > > > > > > Ideally, I would rather have the query be 'select * from wide_table" with > the fields being dynamically matched by the column name from the > dynamicField wildcard from the schema.xml. > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/DataImportHandler-dynamic-fields-clarification-tp1606159p1609578.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: DataImportHandler dynamic fields clarification
> >Two things, one are your DB column uppercase as this would effect the out. > > Interesting, I was under the impression that case does not matter. >From http://wiki.apache.org/solr/DataImportHandler#A_shorter_data-config : "It is possible to totally avoid the field entries in entities if the names of the fields are same (case does not matter) as those in Solr schema" I confirmed that matching the schema.xml field case to the database table is needed for dynamic fields, and the wiki statement above is incorrect, or at the very least confusing, possibly a bug. My database is Oracle 10g and the column names have been created in all uppercase in the database. In Oracle: Table name: wide_table Column names: COLUMN_1 ... COLUMN_100 (yes, uppercase) Please see following scenarios and results I found: data-config.xml schema.xml Result: Nothing Imported = data-config.xml schema.xml Result: Note query column names changed to uppercase. Nothing Imported = data-config.xml schema.xml Result: Note ONLY the field entry was changed to caps All records imported, with only COLUMN_100 id field. data-config.xml schema.xml Result: Note BOTH the field entry was changed to caps in data-config.xml, and the dynamicField wildcard in schema.xml All records imported, with all fields specified. This is the behavior desired. = > >Second what does your db-data-config.xml look like > > The relevant data-config.xml is as follows: Ideally, I would rather have the query be 'select * from wide_table" with the fields being dynamically matched by the column name from the dynamicField wildcard from the schema.xml. -- View this message in context: http://lucene.472066.n3.nabble.com/DataImportHandler-dynamic-fields-clarification-tp1606159p1609578.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: DataImportHandler dynamic fields clarification
Two things, one are your DB column uppercase as this would effect the out. Second what does your db-data-config.xml look like Regards, Dave On 30 Sep 2010, at 03:01, harrysmith wrote: > > Looking for some clarification on DIH to make sure I am interpreting this > correctly. > > I have a wide DB table, 100 columns. I'd rather not have to add 100 values > in schema.xml and data-config.xml. I was under the impression that if the > column name matched a dynamic Field name, it would be added. I am not > finding this is the case, but only works when the column name is explicitly > listed as a static field. > > Example: 100 column table, columns named 'COLUMN_1, COLUMN_2 ... COLUMN_100' > > If I add something like: > > to schema.xml, and don't reference the column in data-config entity/field > tag, it gets imported, as expected. > > However, if I use: > stored="true"/> > It does not get imported into Solr, I would expect it would. > > > Is this the expected behavior? > -- > View this message in context: > http://lucene.472066.n3.nabble.com/DataImportHandler-dynamic-fields-clarification-tp1606159p1606159.html > Sent from the Solr - User mailing list archive at Nabble.com.
DataImportHandler dynamic fields clarification
Looking for some clarification on DIH to make sure I am interpreting this correctly. I have a wide DB table, 100 columns. I'd rather not have to add 100 values in schema.xml and data-config.xml. I was under the impression that if the column name matched a dynamic Field name, it would be added. I am not finding this is the case, but only works when the column name is explicitly listed as a static field. Example: 100 column table, columns named 'COLUMN_1, COLUMN_2 ... COLUMN_100' If I add something like: to schema.xml, and don't reference the column in data-config entity/field tag, it gets imported, as expected. However, if I use: It does not get imported into Solr, I would expect it would. Is this the expected behavior? -- View this message in context: http://lucene.472066.n3.nabble.com/DataImportHandler-dynamic-fields-clarification-tp1606159p1606159.html Sent from the Solr - User mailing list archive at Nabble.com.
