[
https://issues.apache.org/jira/browse/OODT-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247970#comment-13247970
]
[email protected] commented on OODT-426:
----------------------------------------------------
bq. On 2012-04-04 02:12:41, Paul Ramirez wrote:
bq. > trunk/pge/src/main/resources/examples/Crawler/action-beans.xml, lines
29-37
bq. > <https://reviews.apache.org/r/4628/diff/1/?file=98806#file98806line29>
bq. >
bq. > I'd define these properties in another file and then include them
here. This is only a suggestion and not a just but I see the properties as
something that could likely be changed or set to a fixed value and if we factor
it out of here we can keep people from touching this file too much. I think
this file just makes peoples heads spin at first but the properties don't (i.e.
it hides the Spring goodness in a good way).
done
bq. On 2012-04-04 02:12:41, Paul Ramirez wrote:
bq. > trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml,
lines 42-43
bq. > <https://reviews.apache.org/r/4628/diff/1/?file=98813#file98813line42>
bq. >
bq. > Put these examples inside comment tags as they wouldn't work as they
existed anyhow. Also putting a longer description in the comment would help
(i.e. one or more of these is not as helpful as what it does functionally. Why
did we remove the files tag? Is this no longer supported? If it is then I
recommend putting it back in but commented out.
bq. >
bq. > For instance, I'd expect that instead of metadata keys you want to
set more of what will be done with that custom metadata would be of use. Also
an example of multivalued metadata.
Added a TODO at the top of this file... The reader for this file still needs to
be updated... so when i update it i'll make this file a working example when i
write the unit-tests for it
- brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4628/#review6670
-----------------------------------------------------------
On 2012-04-03 21:56:17, brian Foster wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4628/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-03 21:56:17)
bq.
bq.
bq. Review request for oodt, Chris Mattmann, Ricky Nguyen, Paul Ramirez, and
Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. CAS-PGE Changes to this issue...
bq. - Renaming and Metadata extraction removed from CAS-PGE and instead
CAS-PGE now uses AutoDetectProductCrawler instead of StdProductCrawler
bq.
bq.
bq. This addresses bug OODT-426.
bq. https://issues.apache.org/jira/browse/OODT-426
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/pge/pom.xml 1302648
bq. trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java
1302648
bq. trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java
1302648
bq. trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfig.java
1302648
bq.
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigBuilder.java
1302648
bq.
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigMetKeys.java
1302648
bq.
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RegExprOutputFiles.java
1302648
bq. trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RenamingConv.java
1302648
bq.
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/XmlFilePgeConfigBuilder.java
1302648
bq.
trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PgeTaskMetKeys.java
1302648
bq.
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/ExternExtractorMetWriter.java
1302648
bq.
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/FilenameExtractorWriter.java
1302648
bq.
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/PcsMetFileWriter.java
1302648
bq.
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/SciPgeConfigFileWriter.java
1302648
bq.
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/metlist/MetadataListPcsMetFileWriter.java
1302648
bq.
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/xslt/XslTransformWriter.java
1302648
bq. trunk/pge/src/main/resources/examples/Crawler/action-beans.xml
PRE-CREATION
bq. trunk/pge/src/main/resources/examples/Crawler/crawler-config.xml
PRE-CREATION
bq. trunk/pge/src/main/resources/examples/Crawler/mime-extractor-map.xml
PRE-CREATION
bq. trunk/pge/src/main/resources/examples/Crawler/mime-types.xml
PRE-CREATION
bq. trunk/pge/src/main/resources/examples/Crawler/naming-beans.xml
PRE-CREATION
bq. trunk/pge/src/main/resources/examples/Crawler/precondition-beans.xml
PRE-CREATION
bq.
trunk/pge/src/main/resources/examples/MetadataOutputFiles/metadata-output.xml
1302648
bq. trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml
1302648
bq. trunk/pge/src/test/org/apache/oodt/cas/pge/TestPGETaskInstance.java
1302781
bq.
bq. Diff: https://reviews.apache.org/r/4628/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Several Unit-tests
bq.
bq.
bq. Thanks,
bq.
bq. brian
bq.
bq.
> Introduce a CAS-Metadata based renaming interface
> -------------------------------------------------
>
> Key: OODT-426
> URL: https://issues.apache.org/jira/browse/OODT-426
> Project: OODT
> Issue Type: Sub-task
> Components: crawler, metadata container, pge wrapper framework
> Affects Versions: 0.3
> Environment: none
> Reporter: Brian Foster
> Assignee: Brian Foster
> Priority: Minor
> Fix For: 0.5
>
> Attachments: OODT-426.2012-03-20.cas-crawler.patch.txt,
> OODT-426.2012-03-20.cas-metadata.patch.txt,
> OODT-426.2012-03-24.cas-crawler.patch.txt, OODT-426.2012-04-03.cas-pge.txt
>
>
> The idea here is that CAS-Metadata will introduce a new NamingConvention
> interface, which will allow for renaming of files. CAS-Crawler will then be
> modified to support specified NamingConventions which will be run after all
> preconditions have passed for a given file. This will then allow CAS-PGE to
> then use AutoDetectProductCrawler instead of StdProductCrawler, which will
> standardize across the board for file extraction (currently CAS-PGE has it's
> own file extraction interface which uses regular expression to determine
> files which should be extracted and ingested). The only missing feature in
> CAS-Crawler which CAS-PGE supports is file renaming, which this new
> NamingConvention interface will introduce. Here is what the NamingConvention
> interface will look like:
> {code}
> public interface NamingConvention {
> public File rename(File file, Metadata metadata)
> throws NamingConventionException;
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
