Re: cas-crawler wierdness

[Brian Foster]

hey tom,

actually, there is a bug in cmd-line-options.xml... ya like you noticed... change MoveMetadataToFailureDir (without 'File) to MoveMetadataFileToFailureDir... good catch... wanna create the issue and add the patch?

-brian

On Mar 14, 2012, at 02:40 PM, Brian Foster <holeno...@mac.com> wrote:

hey tom,

what does your action-beans.xml file look like... it appears you are missing the action: MoveMetadataToFailureDir

-brian

On Mar 14, 2012, at 06:19 AM, Thomas Bennett <lmzxq....@gmail.com> wrote:

Sorry I actually ran:

$ java -Djava.util.logging.config.file=/var/kat/katconfig/static/oodt/etc/logging.properties -Djava.ext.dirs=/usr/local/oodt/cas-crawler/lib -Dorg.apache.oodt.cas.crawl.bean.repo=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/crawler-config.xml  -Dorg.apache.oodt.cas.cli.action.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-actions.xml -Dorg.apache.oodt.cas.cli.option.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-options.xml org.apache.oodt.cas.crawl.CrawlerLauncher --operation --launchAutoCrawler --productPath /var/kat/data --filemgrUrl http://127.0.0.1:9101--mimeExtractorRepo /var/kat/katconfig/static/oodt/cas-crawler/policy/mime-extractor-map.xml --failureDir /var/kat/failed --actionIds MoveDataFileToFailureDir

The output looks like:

Setting property 'AutoDetectProductCrawler.mimeExtractorRepo'

Setting property 'StdProductCrawler.productPath'

Setting property 'MetExtractorProductCrawler.productPath'

Setting property 'AutoDetectProductCrawler.productPath'

Setting property 'MoveMetadataToFailureDir.toDir'

Setting property 'MovePushpullMetFileToFailureDir.toDir'

Setting property 'MoveDataFileToFailureDir.toDir'

Setting property 'StdProductCrawler.filemgrUrl'

Setting property 'MetExtractorProductCrawler.filemgrUrl'

Setting property 'AutoDetectProductCrawler.filemgrUrl'

ERROR: Could not process key 'MoveMetadataToFailureDir.toDir' in PropertyOverrideConfigurer; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'MoveMetadataToFailureDir' is defined

I'm a bit uncertain how to properly define failed to ingest actions with the new interface.

I see when calling help:

 -fd,    --failureDir <directory>                           Directory where files will be moved on failure

                                                              Requirement Rules: 

                                                               [launchStdCrawler : OPTIONAL, launchMetCrawler : 

                                                               OPTIONAL, launchAutoCrawler : OPTIONAL] 

                                                              Handler: 

                                                               Set the following bean properties: 

                                                               [MoveMetadataToFailureDir.toDir, 

                                                               MovePushpullMetFileToFailureDir.toDir, 

                                                               MoveDataFileToFailureDir.toDir] 

How do I set the bean properties from the command line :)

I see there are two definitions - MoveMetadataFileToFailureDir and MoveMetadataToFailureDir, should these be the same?

Cheers,

Tom