Re: How to process files in a sorted order

2013-11-13 Thread Cameron Goodale 
Konstantinos,

My name is Cameron and I am a committer on the Apache OODT project.  I am
not familiar with the internals of crawler, but I don't believe there is a
way to accomplish your goal of enforcing a sorting algorithm within the
crawler config.  I think you will have to write your own crawler that will
implement your sorting logic.



Sincerely,


Cameron Goodale


On Thu, Nov 7, 2013 at 7:44 PM, Konstantinos Mavrommatis <
kmavromma...@celgene.com> wrote:

> Hi,
> In my environment I am using cas-crawler to process directories of 1000s
> of files. The metadata for these files are extracted automatically using
> the mimetypes definitions and small wrapper scripts.
> In these directories some of the files are derived from other files and
> metadata from the older files need to be transferred to the newer file.
> In order to achieve this I need to have the files processed by the
> cas-crawler starting from the older file to the newer file or in other
> cases in alphabetical order..
> Any ideas how this can be achieved?
>
> The crawler command I currently use is:
> ./crawler_launcher --operation --launchAutoCrawler --productPath $FILEPATH
> --filemgrUrl $FMURL --clientTransferer
> org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferFactory
>  --mimeExtr
> actorRepo ../policy/mime-extractor-map.xml
>
> Thanks in advance for your help
> Konstantinos
>
> *
> THIS ELECTRONIC MAIL MESSAGE AND ANY ATTACHMENT IS
> CONFIDENTIAL AND MAY CONTAIN LEGALLY PRIVILEGED
> INFORMATION INTENDED ONLY FOR THE USE OF THE INDIVIDUAL
> OR INDIVIDUALS NAMED ABOVE.
> If the reader is not the intended recipient, or the
> employee or agent responsible to deliver it to the
> intended recipient, you are hereby notified that any
> dissemination, distribution or copying of this
> communication is strictly prohibited. If you have
> received this communication in error, please reply to the
> sender to notify us of the error and delete the original
> message. Thank You.
> *
>
>


-- 

Sent from a Tin Can attached to a String

Build failed in Jenkins: oodt-trunk #1152

2013-11-13 Thread Apache Jenkins Server 
See 

--
[...truncated 6906 lines...]
[INFO] [assembly:single {execution: default}]
[INFO] Reading assembly descriptor: src/main/assembly/assembly.xml
[INFO] Processing DependencySet (output=lib)
[INFO] Building tar : 

[INFO] Processing DependencySet (output=lib)
[INFO] Building zip: 

[INFO] [install:install {execution: default-install}]
[INFO] Installing 

 to 

[INFO] Installing 

 to 

[INFO] Installing 

 to 

[INFO] 
[INFO] Building CAS Curation Interface
[INFO]task-segment: [install]
[INFO] 
Downloading: 
http://repo1.maven.org/maven2/org/apache/maven/plugins/maven-war-plugin/2.1-alpha-2/maven-war-plugin-2.1-alpha-2.pom
7K downloaded  (maven-war-plugin-2.1-alpha-2.pom)
Downloading: 
http://repo1.maven.org/maven2/org/apache/maven/plugins/maven-plugins/11/maven-plugins-11.pom
10K downloaded  (maven-plugins-11.pom)
Downloading: 
http://repo1.maven.org/maven2/org/apache/maven/plugins/maven-war-plugin/2.1-alpha-2/maven-war-plugin-2.1-alpha-2.jar
72K downloaded  (maven-war-plugin-2.1-alpha-2.jar)
[INFO] [remote-resources:process {execution: default}]
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/commons-fileupload/commons-fileupload/1.2.1/commons-fileupload-1.2.1.pom
7K downloaded  (commons-fileupload-1.2.1.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.pom
12K downloaded  (commons-collections-3.2.1.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/net/sf/json-lib/json-lib/2.3/json-lib-2.3.pom
13K downloaded  (json-lib-2.3.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/commons-beanutils/commons-beanutils/1.8.0/commons-beanutils-1.8.0.pom
10K downloaded  (commons-beanutils-1.8.0.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/net/sf/ezmorph/ezmorph/1.0.6/ezmorph-1.0.6.pom
6K downloaded  (ezmorph-1.0.6.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/cxf/cxf-rt-frontend-jaxrs/2.6.0/cxf-rt-frontend-jaxrs-2.6.0.pom
5K downloaded  (cxf-rt-frontend-jaxrs-2.6.0.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/cxf/cxf-parent/2.6.0/cxf-parent-2.6.0.pom
79K downloaded  (cxf-parent-2.6.0.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/cxf/cxf/2.6.0/cxf-2.6.0.pom
33K downloaded  (cxf-2.6.0.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/cxf/cxf-api/2.6.0/cxf-api-2.6.0.pom
14K downloaded  (cxf-api-2.6.0.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/org/codehaus/woodstox/woodstox-core-asl/4.1.2/woodstox-core-asl-4.1.2.pom
[JENKINS] Recording test results1K downloaded  (woodstox-core-asl-4.1.2.pom)

Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/org/codehaus/woodstox/stax2-api/3.1.1/stax2-api-3.1.1.pom
1K downloaded  (stax2-api-3.1.1.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/ws/xmlschema/xmlschema-core/2.0.2/xmlschema-core-2.0.2.pom
5K downloaded  (xmlschema-core-2.0.2.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/ws/xmlschema/xmlschema/2.0.2/xmlschema-2.0.2.pom
28K downloaded  (xmlschema-2.0.2.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/geronimo/specs/geronimo-javamail_1.4_spec/1.7.1/geronimo-javamail_1.4_spec-1.7.1.pom
6K downloaded  (geronimo-javamail_1.4_spec-1.7.1.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/wsdl4j/wsdl4j/1.6.2/wsdl4j-1.6.2.pom
939b downloaded  (wsdl4j-1.6.2.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/org/apache/cxf/cxf-rt-core/2.6.0/cxf-rt-core-2.6.0.pom
4K downloaded  (cxf-rt-core-2.6.0.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirrors/maven2/com/sun/xml/bind/jaxb-impl/2.1.13/jaxb-impl-2.1.13.pom
2K downloaded  (jaxb-impl-2.1.13.pom)
Downloading: 
http://mirrors.ibiblio.org/pub/mirro