Re: FileConcatenator not generating output

[Ravikumar Jeyaraman]

Hi Professor,
                       It works now.Looks like the wiki page was recently
updated and when I followed that steps (which includes downloading .7
version of oodt ) it works fine .

Thanks for checking with us.

We were wondering how the overall picture looks like ?

As per our current understanding

   - Json files will be injected into the filemgr
   - Workflow will be used to execute ETLlib which indexe JSON files from
   the filemgr archive into solr.
      - Solr's searh algo will be modified as per our requirement.

But if this is the case how come filemgr will have metric on solr
ingestion? Its little confusing here.


Thanks

Ravikumar Jeyaraman
On Nov 5, 2014 4:59 PM, "Mattmann, Chris A (3980)" <
chris.a.mattm...@jpl.nasa.gov> wrote:

> Hi Ravi,
> are you still having trouble here?
>
> Cheers,
> Chris
>
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattm...@nasa.gov
> WWW:  http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>
>
> -----Original Message-----
> From: Ravikumar Jeyaraman <jeyar...@usc.edu>
> Date: Sunday, November 2, 2014 at 10:00 PM
> To: Chris Mattmann <mattm...@usc.edu>
> Cc: Chris Mattmann <chris.a.mattm...@jpl.nasa.gov>, "rmura...@usc.edu"
> <rmura...@usc.edu>, "saish...@usc.edu" <saish...@usc.edu>, Vineet Ghatge
> Hemantkumar <heman...@usc.edu>, Gouthami Kondakindi <konda...@usc.edu>,
> Preethi Ramesh <pram...@usc.edu>, dev <dev@oodt.apache.org>
> Subject: Re: FileConcatenator not generating output
>
> >Hi Professor,
> >
> >                   I have defined the PGE_ROOT. Infact I have set them in
> >my .bashrc file.so it ll be set always
> >
> >My PGE_ROOT is defined as below.
> >
> >root@homework1:~# echo $PGE_ROOT
> >/root/oodt-deploy/cas-pge-0.7
> >
> >
> >cas-pge-0.7 folder has filecontenator folder where jobs output should
> >reside.
> >
> >
> >Not sure what I am missing
> >
> >
> >Thanks a lot helping us with this debugging.
> >
> >
> >Thanks
> >
> >Ravi
> >
> >
> >On Sun, Nov 2, 2014 at 8:39 PM, Christian Alan Mattmann
> ><mattm...@usc.edu> wrote:
> >
> >I think your PGE_ROOT environment variable is not defined.
> >Can you try echo $PGE_ROOT and tell me what it shows? If it¹s
> >not defined, please define it, then restart OODT and try again.
> >
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >Chris Mattmann, Ph.D.
> >Adjunct Associate Professor, Computer Science Department
> >University of Southern California
> >Los Angeles, CA 90089 USA
> >Email: mattm...@usc.edu
> >WWW: http://sunset.usc.edu/~mattmann/
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> >
> >
> >
> >-----Original Message-----
> >From: Ravikumar Jeyaraman <jeyar...@usc.edu>
> >Date: Sunday, November 2, 2014 at 3:13 PM
> >To: Chris Mattmann <chris.a.mattm...@jpl.nasa.gov>
> >Cc: Chris Mattmann <mattm...@usc.edu>, "rmura...@usc.edu"
> ><rmura...@usc.edu>, "saish...@usc.edu" <saish...@usc.edu>, Vineet Ghatge
> >Hemantkumar <heman...@usc.edu>, Gouthami Kondakindi <konda...@usc.edu>,
> >Preethi Ramesh <pram...@usc.edu>, dev <dev@oodt.apache.org>
> >Subject: Re: FileConcatenator not generating output
> >
> >>Hi Professor,
> >>
> >>               I tried setting JobDir to
> >>$PGE_ROOT/file_concatenator/output/jobs manually.
> >>
> >>
> >>After executing JobDir is still a empty dir has nothing in it.
> >>
> >>
> >>I have attached my PGEConfig.xml with this mail in case you need it.
> >>
> >>
> >>
> >>On Sun, Nov 2, 2014 at 1:52 PM, Mattmann, Chris A (3980)
> >><chris.a.mattm...@jpl.nasa.gov> wrote:
> >>
> >>
> >>Thanks Ravi can u show what the job dir has in it?
> >>
> >>Sent from my iPhone
> >>
> >>On Nov 2, 2014, at 1:48 PM, "Ravikumar Jeyaraman" <jeyar...@usc.edu>
> >>wrote:
> >>
> >>
> >>
> >>Hi Professor,
> >>
> >>                   We are still trying out file concatenate work-flow
> >>example . We were able to execute it with out error. But concatenated
> >>output file is not generated at the end of execution.
> >>
> >>
> >>
> >>We are doubting some path setting in PGEConfig.xml .
> >>
> >>
> >>I remember you suggested one of the student to change createBeforeExe to
> >>true. I tried that also. But still files are not generated. I also tried
> >>setting manually JobDir env variable. Still no luck . Any suggestions ?
> >>
> >>  <exe dir="[JobDir]" shell="/bin/bash">
> >>    <!-- cd to PGE root -->
> >>    <cmd>cd [PGE_ROOT]/file_concatenator</cmd>
> >>    <cmd>cp [InputFile1] [OutputFile]</cmd>
> >>    <cmd>cat [InputFile2] >> [OutputFile]</cmd>
> >>  </exe>
> >>
> >>  <!-- Files to ingest -->
> >>  <output>
> >>    <!-- one or more of these -->
> >>    <dir path="[JobDir]" createBeforeExe="false">
> >>
> >>
> >>Thanks
> >>
> >>Ravi
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
> >
> >
> >
> >
> >
> >
>
>