Noble, thanks for the quick fix! I just grabbed the latest patch from jira, and I think that getIdCacheData() also needs to use getFromRowCacheTransformed() -- currently the first retrieved row is not transformed.
> From: Noble Paul നോബിള് नोब्ळ् <[EMAIL PROTECTED]> > Reply-To: <[email protected]> > Date: Wed, 23 Jul 2008 16:40:05 +0530 > To: <[email protected]> > Subject: Re: [jira] Issue Comment Edited: (SOLR-469) Data Import > RequestHandler > > thanks Jonathan. I overlooked the transformer part altogether for > CachedSqlEntityProcessor. I shall add a testcase and cut a patch for > the same. > > On Wed, Jul 23, 2008 at 4:29 PM, Jonathan Lee (JIRA) <[EMAIL PROTECTED]> > wrote: >> >> [ >> https://issues.apache.org/jira/browse/SOLR-469?page=com.atlassian.jira.plugin >> .system.issuetabpanels:comment-tabpanel&focusedCommentId=12615952#action_1261 >> 5952 ] >> >> jonjlee edited comment on SOLR-469 at 7/23/08 3:59 AM: >> ------------------------------------------------------------ >> >> This patch has been a wonderful addition to solr - thanks for all the work! >> >> I believe that there is a bug in CachedSqlEntityProcessor that causes >> transformers to be ignored. Here is a patch that worked for me, but I am not >> sure it is entirely correct: >> >> {noformat} >> --- >> contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/Ca >> chedSqlEntityProcessor.java >> +++ >> contrib.new/dataimporthandler/src/main/java/org/apache/solr/handler/dataimpor >> t/CachedSqlEntityProcessor.java >> @@ -43,18 +43,23 @@ >> } >> >> public Map<String, Object> nextRow() { >> - if (rowcache != null) >> - return getFromRowCache(); >> - if (!isFirst) >> + Map<String, Object> r; >> + if (rowcache != null) { >> + r = getFromRowCache(); >> + } else if (!isFirst) { >> return null; >> - String query = >> resolver.replaceTokens(context.getEntityAttribute("query")); >> - isFirst = false; >> - if (simpleCache != null) { >> - return getSimplCacheData(query); >> } else { >> - return getIdCacheData(query); >> + String query = >> resolver.replaceTokens(context.getEntityAttribute("query")); >> + isFirst = false; >> + if (simpleCache != null) { >> + r = getSimplCacheData(query); >> + } else { >> + r = getIdCacheData(query); >> + } >> } >> - >> + if (r == null) >> + return null; >> + return applyTransformer(r); >> } >> >> protected List<Map<String, Object>> getAllNonCachedRows() { >> {noformat} >> >> was (Author: jonjlee): >> This patch has been a wonderful addition to solr - thanks for all the >> work! >> >> I believe that there is a bug in CachedSqlEntityProcessor that causes >> transformers to be ignored. Here is a patch that worked for me, but I am not >> sure it is entirely correct: >> >> --- >> contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/Ca >> chedSqlEntityProcessor.java >> +++ >> contrib.new/dataimporthandler/src/main/java/org/apache/solr/handler/dataimpor >> t/CachedSqlEntityProcessor.java >> @@ -43,18 +43,23 @@ >> } >> >> public Map<String, Object> nextRow() { >> - if (rowcache != null) >> - return getFromRowCache(); >> - if (!isFirst) >> + Map<String, Object> r; >> + if (rowcache != null) { >> + r = getFromRowCache(); >> + } else if (!isFirst) { >> return null; >> - String query = >> resolver.replaceTokens(context.getEntityAttribute("query")); >> - isFirst = false; >> - if (simpleCache != null) { >> - return getSimplCacheData(query); >> } else { >> - return getIdCacheData(query); >> + String query = >> resolver.replaceTokens(context.getEntityAttribute("query")); >> + isFirst = false; >> + if (simpleCache != null) { >> + r = getSimplCacheData(query); >> + } else { >> + r = getIdCacheData(query); >> + } >> } >> - >> + if (r == null) >> + return null; >> + return applyTransformer(r); >> } >> >> protected List<Map<String, Object>> getAllNonCachedRows() { >> >> >>> Data Import RequestHandler >>> -------------------------- >>> >>> Key: SOLR-469 >>> URL: https://issues.apache.org/jira/browse/SOLR-469 >>> Project: Solr >>> Issue Type: New Feature >>> Components: update >>> Affects Versions: 1.3 >>> Reporter: Noble Paul >>> Assignee: Grant Ingersoll >>> Fix For: 1.3 >>> >>> Attachments: SOLR-469-contrib.patch, SOLR-469-contrib.patch, >>> SOLR-469-contrib.patch, SOLR-469-contrib.patch, SOLR-469-contrib.patch, >>> SOLR-469-contrib.patch, SOLR-469-contrib.patch, SOLR-469-contrib.patch, >>> SOLR-469.patch, SOLR-469.patch, SOLR-469.patch, SOLR-469.patch, >>> SOLR-469.patch, SOLR-469.patch, SOLR-469.patch, SOLR-469.patch, >>> SOLR-469.patch >>> >>> >>> We need a RequestHandler Which can import data from a DB or other >>> dataSources into the Solr index .Think of it as an advanced form of >>> SqlUpload Plugin (SOLR-103). >>> The way it works is as follows. >>> * Provide a configuration file (xml) to the Handler which takes in the >>> necessary SQL queries and mappings to a solr schema >>> - It also takes in a properties file for the data source >>> configuraution >>> * Given the configuration it can also generate the solr schema.xml >>> * It is registered as a RequestHandler which can take two commands >>> do-full-import, do-delta-import >>> - do-full-import - dumps all the data from the Database into the >>> index (based on the SQL query in configuration) >>> - do-delta-import - dumps all the data that has changed since last >>> import. (We assume a modified-timestamp column in tables) >>> * It provides a admin page >>> - where we can schedule it to be run automatically at regular >>> intervals >>> - It shows the status of the Handler (idle, full-import, >>> delta-import) >> >> -- >> This message is automatically generated by JIRA. >> - >> You can reply to this email to add a comment to the issue online. >> >> > > > > -- > --Noble Paul
