I think if you look closely you'll find the date quoted in the Exception report doesn't match any of the declared formats in the schema. I would suggest, as a first step, hunting through your data to see where that date is coming from.
-Mike > -----Original Message----- > From: Ken Stanley [mailto:doh...@gmail.com] > Sent: Monday, October 18, 2010 7:40 AM > To: solr-user@lucene.apache.org > Subject: Re: SOLR DateTime and SortableLongField field type problems > > Just following up to see if anybody might have some words of > wisdom on the issue? > > Thank you, > > Ken > > It looked like something resembling white marble, which was > probably what it was: something resembling white marble. > -- Douglas Adams, "The Hitchhikers Guide to > the Galaxy" > > > On Fri, Oct 15, 2010 at 6:42 PM, Ken Stanley <doh...@gmail.com> wrote: > > > Hello all, > > > > I am using SOLR-1.4.1 with the DataImportHandler, and I am > trying to > > follow the advice from > > > http://www.mail-archive.com/solr-user@lucene.apache.org/msg11887.htmla > > bout converting date fields to SortableLong fields for > better memory > > efficiency. However, whenever I try to do this using the > DateFormater, > > I get exceptions when indexing for every row that tries to > create my sortable fields. > > > > In my schema.xml, I have the following definitions for the > fieldType > > and > > dynamicField: > > > > <fieldType name="sdate" class="solr.SortableLongField" > indexed="true" > > stored="false" sortMissingLast="true" omitNorms="true" /> > > <dynamicField name="sort_date_*" type="sdate" > stored="false" indexed="true" > > /> > > > > In my dih.xml, I have the following definitions: > > > > <dataConfig> > > <dataSource type="FileDataSource" encoding="UTF-8" /> > > <entity > > name="xml_stories" > > rootEntity="false" > > dataSource="null" > > processor="FileListEntityProcessor" > > fileName="legacy_stories.*\.xml$" > > recursive="false" > > baseDir="/usr/local/extracts" > > newerThan="${dataimporter.xml_stories.last_index_time}" > > > > > <entity > > name="stories" > > pk="id" > > dataSource="xml_stories" > > processor="XPathEntityProcessor" > > url="${xml_stories.fileAbsolutePath}" > > forEach="/RECORDS/RECORD" > > stream="true" > > > > > transformer="DateFormatTransformer,HTMLStripTransformer,RegexT > ransformer,TemplateTransformer" > > onError="continue" > > > > > <field column="_modified_date" > > xpath="/RECORDS/RECORD/pr...@name='R_ModifiedTime']/PVAL" /> > > <field column="modified_date" > > sourceColName="_modified_date" > > dateTimeFormat="yyyy-MM-dd'T'hh:mm:ss'Z'" /> > > > > <field column="_df_date_published" > > xpath="/RECORDS/RECORD/pr...@name='R_StoryDate']/PVAL" /> > > <field column="df_date_published" > > sourceColName="_df_date_published" > dateTimeFormat="yyyy-MM-dd'T'hh:mm:ss'Z'" > > /> > > > > <field column="sort_date_modified" > > sourceColName="modified_date" dateTimeFormat="yyyyMMddhhmmss" /> > > <field column="sort_date_published" > > sourceColName="df_date_published" dateTimeFormat="yyyyMMddhhmmss" /> > > </entity> > > </entity> > > </document> > > </dataConfig> > > > > The fields in question are in the formats: > > > > <RECORDS> > > <RECORD> > > <PROP NAME="R_StoryDate"> > > <PVAL>2001-12-04T00:00:00Z</PVAL> > > </PROP> > > <PROP NAME="R_ModifiedTime"> > > <PVAL>2001-12-04T19:38:01Z</PVAL> > > </PROP> > > </RECORD> > > </RECORDS> > > > > The exception that I am receiving is: > > > > Oct 15, 2010 6:23:24 PM > > org.apache.solr.handler.dataimport.DateFormatTransformer > transformRow > > WARNING: Could not parse a Date field > > java.text.ParseException: Unparseable date: "Wed Nov 28 > 21:39:05 EST 2007" > > at java.text.DateFormat.parse(DateFormat.java:337) > > at > > > org.apache.solr.handler.dataimport.DateFormatTransformer.proce > ss(DateFormatTransformer.java:89) > > at > > > org.apache.solr.handler.dataimport.DateFormatTransformer.trans > formRow(DateFormatTransformer.java:69) > > at > > > org.apache.solr.handler.dataimport.EntityProcessorWrapper.appl > yTransformer(EntityProcessorWrapper.java:195) > > at > > > org.apache.solr.handler.dataimport.EntityProcessorWrapper.next > Row(EntityProcessorWrapper.java:241) > > at > > > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(Do > cBuilder.java:357) > > at > > > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(Do > cBuilder.java:383) > > at > > > org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBu > ilder.java:242) > > at > > > org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuild > er.java:180) > > at > > > org.apache.solr.handler.dataimport.DataImporter.doFullImport(D > ataImporter.java:331) > > at > > > org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImp > orter.java:389) > > at > > > org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.jav > > a:370) > > > > I know that it has to be the SortableLong fields, because > if I remove > > just those two lines from my dih.xml, everything imports as > I expect > > it to. Am I doing something wrong? Mis-using the > SortableLong and/or > > DateTransformer? Is this not supported in my version of > SOLR? I'm not > > very experienced with Java, so digging into the code would > be a lost > > cause for me right now. I was hoping that somebody here > might be able > > to help point me in the right/correct direction. > > > > It should be noted that the modified_date and > df_date_published fields > > index just fine (so long as I do it as I've defined above). > > > > Thank you, > > > > - Ken > > > > It looked like something resembling white marble, which was > probably > > what it was: something resembling white marble. > > -- Douglas Adams, "The Hitchhikers Guide to > the Galaxy" > > >
