[
https://issues.apache.org/jira/browse/SOLR-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12604257#action_12604257
]
shalinmangar edited comment on SOLR-469 at 6/11/08 8:58 AM:
---------------------------------------------------------------------
A new patch file (SOLR-469.patch) consisting of some important bug fixes and
minor enhancements. The changes and the corresponding classes are given below
*Changes*
* Set fetch size to Integer.MIN_VALUE if batchSize in configuration is -1 as
per Patrick's suggestion -- JdbcDataSource
* Transformers can add a boost to a document by adding a key/value pair
row.put("$docBoost", 2.0f) from any entity -- DocBuilder,SolrWriter and
DataImportHandler
* Fixes for infinite loop in SqlEntityProcessor when delta query fails for some
reason and NullPointerException is thrown in EntityProcessorBase --
EntityProcessorBase
* Fix for NullPointerException in TemplateTransformer and corresponding test --
TemplateTransformer, TestTemplateTransformer
* Enhancement for specifying table.column syntax for pk attribute in entity as
per issue reported by Chris Moser and Olivier Poitrey --
SqlEntityProcessor,TestSqlEntityProcessor2
* Fix for NullPointerException in XPathRecordReader when attribute specified
through xpath is null as per issue reported by Nicolas Pastorino in solr-user
-- XPathRecordReader, TestXPathRecordReader
* Enhancement to DataSource interface to provide a close method -- DataSource,
FileDataSource, HttpDataSource, MockDataSource
* Context interface has a new method getDataSource(String name) for getting a
new DataSource instance as per the name specified in solrconfig.xml or
data-config.xml -- Context, ContextImpl, DataImporter, DocBuilder
* FileListEntityProcessor implements olderThan and newerThan filtering
parameters -- FileListEntityProcessor, TestFileListEntityProcessor
* Debug Mode can be disabled from solrconfig.xml by enableDebug=false --
DataImporter, DataImportHandler
* Running statistics are exposed on the Solr Statistics page in addition to
cumulative statictics -- DataImportHandler, DocBuilder
* The dataSource attribute can be null when using certain EntityProcessors such
as FileListEntityProcessor which does not need a dataSource. So when
dataSource="null", no attempt is made to create a DataSource instance --
DataImporter
Updated as per Noble's comment below.
was (Author: shalinmangar):
A new patch file (SOLR-469.patch) consisting of some important bug fixes
and minor enhancements. The changes and the corresponding classes are given
below
*Changes*
* Set fetch size to Integer.MIN_VALUE if batchSize in configuration is -1 as
per Patrick's suggestion -- JdbcDataSource
* Transformers can add a boost to a document by adding a key/value pair
row.put("$docBoost", 2.0f) from any entity -- DocBuilder,SolrWriter and
DataImportHandler
* Fixes for infinite loop in SqlEntityProcessor when delta query fails for some
reason and NullPointerException is thrown in EntityProcessorBase --
EntityProcessorBase
* Fix for NullPointerException in TemplateTransformer and corresponding test --
TemplateTransformer, TestTemplateTransformer
* Enhancement for specifying table.column syntax for pk attribute in entity as
per issue reported by Chris Moser and Olivier Poitrey --
SqlEntityProcessor,TestSqlEntityProcessor2
* Fix for NullPointerException in XPathRecordReader when attribute specified
through xpath is null -- XPathRecordReader, TestXPathRecordReader
* Enhancement to DataSource interface to provide a close method -- DataSource,
FileDataSource, HttpDataSource, MockDataSource
* Context interface has a new method getDataSource(String entityName) for
getting a new DataSource instance for the given entity -- Context, ContextImpl,
DataImporter, DocBuilder
* FileListEntityProcessor implements olderThan and newerThan filtering
parameters -- FileListEntityProcessor, TestFileListEntityProcessor
* Debug Mode can be disabled from solrconfig.xml by enableDebug=false --
DataImporter, DataImportHandler
* Running statistics are exposed on the Solr Statistics page in addition to
cumulative statictics -- DataImportHandler, DocBuilder
> Data Import RequestHandler
> --------------------------
>
> Key: SOLR-469
> URL: https://issues.apache.org/jira/browse/SOLR-469
> Project: Solr
> Issue Type: New Feature
> Components: update
> Affects Versions: 1.3
> Reporter: Noble Paul
> Assignee: Grant Ingersoll
> Fix For: 1.3
>
> Attachments: SOLR-469-contrib.patch, SOLR-469.patch, SOLR-469.patch,
> SOLR-469.patch, SOLR-469.patch, SOLR-469.patch, SOLR-469.patch,
> SOLR-469.patch, SOLR-469.patch, SOLR-469.patch
>
>
> We need a RequestHandler Which can import data from a DB or other dataSources
> into the Solr index .Think of it as an advanced form of SqlUpload Plugin
> (SOLR-103).
> The way it works is as follows.
> * Provide a configuration file (xml) to the Handler which takes in the
> necessary SQL queries and mappings to a solr schema
> - It also takes in a properties file for the data source
> configuraution
> * Given the configuration it can also generate the solr schema.xml
> * It is registered as a RequestHandler which can take two commands
> do-full-import, do-delta-import
> - do-full-import - dumps all the data from the Database into the
> index (based on the SQL query in configuration)
> - do-delta-import - dumps all the data that has changed since last
> import. (We assume a modified-timestamp column in tables)
> * It provides a admin page
> - where we can schedule it to be run automatically at regular
> intervals
> - It shows the status of the Handler (idle, full-import,
> delta-import)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
