[
https://issues.apache.org/jira/browse/OODT-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238787#comment-13238787
]
[email protected] commented on OODT-426:
----------------------------------------------------
bq. On 2012-03-26 15:50:17, Chris Mattmann wrote:
bq. >
trunk/crawler/src/main/java/org/apache/oodt/cas/crawl/ProductCrawler.java, line
326
bq. > <https://reviews.apache.org/r/4444/diff/2/?file=95394#file95394line326>
bq. >
bq. > Are all of these @VisibleForTesting coupling our test system too
much with the code? Just wondering...
I just changing the visibility on these methods from private to package
level... then i notated them with the @VisibleForTesting to then make this
clear to other developers that the only reason for these methods being package
level is to make them unit-test-able
bq. On 2012-03-26 15:50:17, Chris Mattmann wrote:
bq. >
trunk/crawler/src/main/java/org/apache/oodt/cas/crawl/StdProductCrawler.java,
line 71
bq. > <https://reviews.apache.org/r/4444/diff/2/?file=95395#file95395line71>
bq. >
bq. > Should we augment the ProductCrawler super class to declare this
function as an abstract method since all sub class crawlers implement it?
it is... this method does need a @Override above it to make it clear... i'll
add this
- brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4444/#review6349
-----------------------------------------------------------
On 2012-03-25 01:55:32, brian Foster wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4444/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-25 01:55:32)
bq.
bq.
bq. Review request for oodt, Chris Mattmann, Ricky Nguyen, Paul Ramirez, and
Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. - Introduced NamingConvention support, which for
MetExtractorProductCrawler now takes an ID for the NamingConvention to use and
AutoDetectProductCrawler has a new element <namingConvention class=""/> in
MimeExtractorRepo
bq. - Also cleaned up handleFile(File)... documented better and is now public
and returns the IngestResult for what happened when called.
bq.
bq.
bq. This addresses bug OODT-426.
bq. https://issues.apache.org/jira/browse/OODT-426
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/crawler/src/main/resources/naming-beans.xml PRE-CREATION
bq. trunk/crawler/src/test/org/apache/oodt/cas/crawl/TestProductCrawler.java
PRE-CREATION
bq. trunk/crawler/src/main/resources/cmd-line-options.xml 1302790
bq. trunk/crawler/src/main/resources/crawler-config.xml 1302790
bq. trunk/crawler/src/main/resources/examples/mime-extractor-map.xml 1302790
bq.
trunk/crawler/src/main/java/org/apache/oodt/cas/crawl/typedetection/MimeExtractorRepo.java
1302790
bq.
trunk/crawler/src/main/java/org/apache/oodt/cas/crawl/StdProductCrawler.java
1302790
bq.
trunk/crawler/src/main/java/org/apache/oodt/cas/crawl/typedetection/MimeExtractorConfigMetKeys.java
1302790
bq.
trunk/crawler/src/main/java/org/apache/oodt/cas/crawl/typedetection/MimeExtractorConfigReader.java
1302790
bq.
trunk/crawler/src/main/java/org/apache/oodt/cas/crawl/ProductCrawler.java
1302790
bq. trunk/crawler/pom.xml 1302790
bq.
trunk/crawler/src/main/java/org/apache/oodt/cas/crawl/AutoDetectProductCrawler.java
1302790
bq.
trunk/crawler/src/main/java/org/apache/oodt/cas/crawl/MetExtractorProductCrawler.java
1302790
bq.
bq. Diff: https://reviews.apache.org/r/4444/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Still need to unit-test up cas-crawler
bq.
bq.
bq. Thanks,
bq.
bq. brian
bq.
bq.
> Introduce a CAS-Metadata based renaming interface
> -------------------------------------------------
>
> Key: OODT-426
> URL: https://issues.apache.org/jira/browse/OODT-426
> Project: OODT
> Issue Type: Sub-task
> Components: crawler, metadata container, pge wrapper framework
> Affects Versions: 0.3
> Environment: none
> Reporter: Brian Foster
> Assignee: Brian Foster
> Priority: Minor
> Fix For: 0.4
>
> Attachments: OODT-426.2012-03-20.cas-crawler.patch.txt,
> OODT-426.2012-03-20.cas-metadata.patch.txt,
> OODT-426.2012-03-24.cas-crawler.patch.txt
>
>
> The idea here is that CAS-Metadata will introduce a new NamingConvention
> interface, which will allow for renaming of files. CAS-Crawler will then be
> modified to support specified NamingConventions which will be run after all
> preconditions have passed for a given file. This will then allow CAS-PGE to
> then use AutoDetectProductCrawler instead of StdProductCrawler, which will
> standardize across the board for file extraction (currently CAS-PGE has it's
> own file extraction interface which uses regular expression to determine
> files which should be extracted and ingested). The only missing feature in
> CAS-Crawler which CAS-PGE supports is file renaming, which this new
> NamingConvention interface will introduce. Here is what the NamingConvention
> interface will look like:
> {code}
> public interface NamingConvention {
> public File rename(File file, Metadata metadata)
> throws NamingConventionException;
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
