Re: FileConcatenator not generating output

[Ravikumar Jeyaraman]

Hi Professor,

               I tried setting JobDir to
$PGE_ROOT/file_concatenator/output/jobs manually.

After executing JobDir is still a empty dir has nothing in it.

I have attached my PGEConfig.xml with this mail in case you need it.


On Sun, Nov 2, 2014 at 1:52 PM, Mattmann, Chris A (3980) <
chris.a.mattm...@jpl.nasa.gov> wrote:

>
> Thanks Ravi can u show what the job dir has in it?
>
> Sent from my iPhone
>
> On Nov 2, 2014, at 1:48 PM, "Ravikumar Jeyaraman" <jeyar...@usc.edu>
> wrote:
>
>    Hi Professor,
>                     We are still trying out file concatenate work-flow
> example . We were able to execute it with out error. But concatenated
> output file is not generated at the end of execution.
>
>  We are doubting some path setting in PGEConfig.xml .
>
>  I remember you suggested one of the student to change createBeforeExe to
> true. I tried that also. But still files are not generated. I also tried
> setting manually JobDir env variable. Still no luck . Any suggestions ?
>
>    <exe dir="[JobDir]" shell="/bin/bash">
>     <!-- cd to PGE root -->
>     <cmd>cd [PGE_ROOT]/file_concatenator</cmd>
>     <cmd>cp [InputFile1] [OutputFile]</cmd>
>     <cmd>cat [InputFile2] >> [OutputFile]</cmd>
>   </exe>
>
>   <!-- Files to ingest -->
>   <output>
>     <!-- one or more of these -->
>     *<dir path="[JobDir]" createBeforeExe="false">*
>
>  Thanks
>  Ravi
>
>

<?xml version="1.0" encoding="UTF-8"?>
<pgeConfig>

        <!-- How to run the PGE -->
        <exe dir="[JobDir]" shell="/bin/bash">
          <!-- cd to PGE root -->
          <cmd>cd [PGE_ROOT]/file_concatenator</cmd>
                <cmd>cp [InputFile1] [OutputFile]</cmd>
                <cmd>cat [InputFile2] >> [OutputFile]</cmd>
        </exe>

            <!-- Files to ingest -->
            <output>
                <!-- one or more of these -->
                <dir path="[JobDir]" createBeforeExe="true">

                <!-- one or more of these ** regExp or name can be used-->
                <files regExp=".*\.txt" metFileWriterClass="org.apache.oodt.pge.examples.fileconcatenator.writers.ConcactenatingFilenameExtractorWriter"  args="[PGE_ROOT]/file_concatenator/extractors/concatenatingfilename.extractor.config.xml"/>
                <files regExp=".*\.txt" metFileWriterClass="org.apache.oodt.cas.pge.writers.metlist.MetadataListPcsMetFileWriter" args="[PGE_ROOT]/file_concatenator/extractors/metlistwriter/metout.xml"/>
            </dir>
        </output>

        <!-- Custom metadata to add to output files -->
        <customMetadata>
                <!-- helpful keys -->
                <metadata key="LessThan" val="&#x3C;"/>
                <metadata key="LessThanOrEqualTo" val="[LessThan]="/>
                <metadata key="GreaterThan" val="&#x3E;"/>
                <metadata key="GreaterThanOrEqualTo" val="[GreaterThan]="/>
                <metadata key="Exclamation" val="&#33;"/>
                <metadata key="Ampersand" val="&#38;"/>
                <metadata key="NotEqualTo" val="[Ampersand]="/>
                <metadata key="LogicalAnd" val="[Ampersand][Ampersand]"/>
                <metadata key="CshPipeToStdOutAndError" val="[GreaterThan][Ampersand][Exclamation]"/>

                                <metadata key="ProductionDateTime" val="[DATE.UTC]"/>
                                <metadata key="JobDir" val="[PGE_ROOT]/file_concatenator/output/jobs/job-[ProductionDateTime]"/>
                                <metadata key="InputFile1" val="[PGE_ROOT]/file_concatenator/files/concatenatingInputFile1.txt"/>
				<metadata key="InputFile2" val="[PGE_ROOT]/file_concatenator/files/concatenatingInputFile2.txt"/>
                                <metadata key="OutputFile" val="[PGE_ROOT]/file_concatenator/output/jobs/concatenatedOutputFile-[ProductionDateTime].txt"/>
      </customMetadata>

</pgeConfig>