RE: cas-crawler wierdness

[Brian Foster]

hey tom,

ya Review Board isn't intuitive at all... took me like 30 mins of trial and error to figure it all out... basically after you click "New Review Request" you pick "oodt" from the "Repository" drop down, then for "Base Directory" put a relative path (relative to oodt svn base directory: https://svn.apache.org/repos/asf/oodt/)... so say you are creating a crawler patch your path would be "trunk/crawler"... then just upload your patch and all should work... in your case though your patch is relative to the trunk so just put "trunk" for "Base Directory"

-brian

On Mar 14, 2012, at 06:18 AM, Thomas Bennett <lmzxq....@gmail.com> wrote:

Hi Brian + Chris,

Thanks, for you're help! All sorted.

I attempted to try out Review Board, I was unable to load a diff, it kept on rejecting my diff files.

I've attached the diff (to update logging from WARNING to SEVERE) to the JIRA issue.

For #2, what chris said about the file://** makes sense... try that and let me know how that works ou

Yup that did the trick, although I needed three slashes file:/// :)

I also had to modify the constructor-arg values in cmd-line-options.xml. 

<constructor-arg value="file:///var/kat/katconfig/static/oodt/cas-crawler/policy/precondition-beans.xml" /> 

All working fine now except when I execute:

$ java -Djava.util.logging.config.file=/var/kat/katconfig/static/oodt/etc/logging.properties -Djava.ext.dirs=/usr/local/oodt/cas-crawler/lib -Dorg.apache.oodt.cas.crawl.bean.repo=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/crawler-config.xml  -Dorg.apache.oodt.cas.cli.action.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-actions.xml -Dorg.apache.oodt.cas.cli.option.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-options.xml org.apache.oodt.cas.crawl.CrawlerLauncher --operation --launchAutoCrawler --productPath /var/kat/data --filemgrUrl http://127.0.0.1:9101 --mimeExtractorRepo /var/kat/katconfig/static/oodt/cas-crawler/policy/mime-extractor-map.xml --failureDir /var/kat/failed

The output looks like:

Setting property 'AutoDetectProductCrawler.mimeExtractorRepo'

Setting property 'StdProductCrawler.productPath'

Setting property 'MetExtractorProductCrawler.productPath'

Setting property 'AutoDetectProductCrawler.productPath'

Setting property 'MoveMetadataToFailureDir.toDir'

Setting property 'MovePushpullMetFileToFailureDir.toDir'

Setting property 'MoveDataFileToFailureDir.toDir'

Setting property 'StdProductCrawler.filemgrUrl'

Setting property 'MetExtractorProductCrawler.filemgrUrl'

Setting property 'AutoDetectProductCrawler.filemgrUrl'

ERROR: Could not process key 'MoveMetadataToFailureDir.toDir' in PropertyOverrideConfigurer; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'MoveMetadataToFailureDir' is defined

I'm a bit uncertain how to properly define failed to ingest actions with the new interface.

I see when calling help:

 -fd,    --failureDir <directory>                           Directory where files will be moved on failure

                                                              Requirement Rules: 

                                                               [launchStdCrawler : OPTIONAL, launchMetCrawler : 

                                                               OPTIONAL, launchAutoCrawler : OPTIONAL] 

                                                              Handler: 

                                                               Set the following bean properties: 

                                                               [MoveMetadataToFailureDir.toDir, 

                                                               MovePushpullMetFileToFailureDir.toDir, 

                                                               MoveDataFileToFailureDir.toDir] 

How do I set the bean properties from the command line :)

I see there are two definitions - MoveMetadataFileToFailureDir and MoveMetadataToFailureDir, should these be the same?

Cheers,

Tom