Re: RADiX CAS-Crawler cannot trigger workflow event
Dear Prof. Mattmann,
Yes, you are right. To solve the problem, we have to change the
[WORKFLOWMGR_URL]
to [WORKFLOW_URL] in the action-beans.xml file.
The default CAS-crawler has been changed to [WORKFLOW_URL] in
https://issues.apache.org/jira/browse/OODT-724.
I have created a new JIRA ticket (
https://issues.apache.org/jira/browse/OODT-783) to change the RADiX crawler.
Also the path file has been uploaded, and a pull request has been committed
(https://github.com/apache/oodt/pull/9). Thanks.
Best,
Mengying (Angela) Wang
On Wed, Nov 5, 2014 at 12:37 PM, Mattmann, Chris A (3980) <
chris.a.mattm...@jpl.nasa.gov> wrote:
> That’s the problem - the policy is wrong in the RADIX crawler
> policy. Can you open up a JIRA ticket to fix this? It’s probably
> wrong too in the default CAS-crawler policy. The value inside of
> that file should be [WORKFLOW_URL].
>
> Please open up a JIRA and submit a patch. Thanks Angela!
>
> Cheers,
> Chris
>
> ++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattm...@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++
>
>
>
>
>
>
> -Original Message-
> From: MengYing Wang
> Date: Tuesday, November 4, 2014 at 11:35 AM
> To: Chris Mattmann
> Cc: "dev@oodt.apache.org" , Chris Mattmann
>
> Subject: Re: RADiX CAS-Crawler cannot trigger workflow event
>
> >Dear Prof. Mattmann,
> >
> >
> >In the default property, there is a line " >key="crawler.workflowmgr.url">[WORKFLOWMGR_URL]" while no
> >definition of the [WORKFLOW_URL].
> >
> >
> >
> >
> >
> >[FAILURE_DIR]
> >[BACKUP_DIR]
> > >key="crawler.workflowmgr.url">[WORKFLOWMGR_URL]
> >[FILEMGR_URL]
> > >key="crawler.client.transferer">org.apache.oodt.cas.filemgr.datatransfer.L
> >ocalDataTransferFactory
> >met
> > >key="crawler.pushpull.met.file.ext">[PUSHPULL_MET_FILE_EXT]
> >anc
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >Below is the definition of the TriggerPostIngestWorkflow event with the
> >workflowMgrUrl.
> >
> >
> >
> > >class="org.apache.oodt.cas.crawl.action.WorkflowMgrStatusUpdate">
> >
> >
> > >value="${crawler.workflowmgr.url}" />
> >
> >
> >postIngestSuccess
> >
> >
> >
> >
> >
> >
> >Thanks!
> >
> >
> >Best,
> >Mengying (Angela) Wang
> >
> >
> >On Tue, Nov 4, 2014 at 10:21 AM, Mattmann, Chris A (3980)
> > wrote:
> >
> >Thanks Angela. What do you see in terms of your policy in
> >$OODT_HOME/crawler/policy in terms of action-beans.xml - do you
> >see a [WORKFLOW_URL] defined in there?
> >
> >++
> >Chris Mattmann, Ph.D.
> >Chief Architect
> >Instrument Software and Science Data Systems Section (398)
> >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> >Office: 168-519, Mailstop: 168-527
> >Email: chris.a.mattm...@nasa.gov
> >WWW: http://sunset.usc.edu/~mattmann/
> >++
> >Adjunct Associate Professor, Computer Science Department
> >University of Southern California, Los Angeles, CA 90089 USA
> >++
> >
> >
> >
> >
> >
> >
> >-Original Message-
> >From: MengYing Wang
> >Date: Tuesday, November 4, 2014 at 11:07 AM
> >To: Chris Mattmann
> >Cc: "dev@oodt.apache.org" , Chris Mattmann
> >
> >Subject: Re: RADiX CAS-Crawler cannot trigger workflow event
> >
> >>Dear Prof. Mattmann,
> >>
> &
Re: [RADiX File Manager] query-tool script sets the wrong values to the OODT_HOME and FILEMGR_HOME variables
Dear Prof. Mattmann, Ok. Then this problem has been solved a few days ago by OODT-763 ( https://issues.apache.org/jira/browse/OODT-763). Also I guess the open issue OODT-768 ( https://issues.apache.org/jira/browse/OODT-768) may be saying the same problem. Thanks! Best, Mengying (Angela) Wang On Wed, Nov 5, 2014 at 5:43 PM, Chris Mattmann wrote: > Dear Angela, > > Yes that¹s the correct location. > > See my instructions on Github now for > submitting a Pull Request to OODT: > > http://github.com/apache/oodt > > You can also submit a patch the traditional > way (via SVN) as well. > > Cheers, > Chris > > > Chris Mattmann > chris.mattm...@gmail.com > > > > > -Original Message- > From: MengYing Wang > Reply-To: > Date: Wednesday, November 5, 2014 at 6:14 PM > To: Chris Mattmann > Cc: Chris Mattmann , "dev@oodt.apache.org" > > Subject: Re: [RADiX File Manager] query-tool script sets the wrong values > to the OODT_HOME and FILEMGR_HOME variables > > >Dear Prof. Mattmann, > > > >OODT-781 is about the filemgr-client script, while this one is about the > >query-tool script. They may be different issues. > > > >Also I would like to submit a patch file, but I am not so sure about where > >to get the RADiX OODT 0.7 source code? > > > >This one > > > http://svn.apache.org/repos/asf/oodt/trunk/mvn/archetypes/radix/src/main/r > >esources/archetype-resources/ > >is correct. Thanks. > > > >Best, > >Mengying (Angela) Wang > > > > > >On Wed, Nov 5, 2014 at 4:39 PM, Mattmann, Chris A (3980) < > >chris.a.mattm...@jpl.nasa.gov> wrote: > > > >> Hi Angela, > >> > >> Awesome, can you please submit a patch to the issue you just > >> created (OODT-781) and can you create a Review Board: > >> > >> http://reviews.apache.org/ > >> > >> Thanks, > >> Chris > >> > >> ++ > >> Chris Mattmann, Ph.D. > >> Chief Architect > >> Instrument Software and Science Data Systems Section (398) > >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > >> Office: 168-519, Mailstop: 168-527 > >> Email: chris.a.mattm...@nasa.gov > >> WWW: http://sunset.usc.edu/~mattmann/ > >> ++ > >> Adjunct Associate Professor, Computer Science Department > >> University of Southern California, Los Angeles, CA 90089 USA > >> ++ > >> > >> > >> > >> > >> > >> > >> -Original Message- > >> From: MengYing Wang > >> Date: Wednesday, November 5, 2014 at 4:54 PM > >> To: Chris Mattmann > >> Cc: "dev@oodt.apache.org" > >> Subject: [RADiX File Manager] query-tool script sets the wrong values to > >> the OODT_HOME and FILEMGR_HOME variables > >> > >> >Hey everyone, > >> > > >> > > >> >After a text file was ingested into my file manager, I run the command: > >> > > >> > > >> >./query-tool --url http://localhost:9000 --sql -query "SELECT * FROM > >> >GenericFile" > >> > > >> > > >> >to verify the new file. However, it raised such an error: > >> > > >> > > >> >$ ./query-tool --url http://localhost:9000 --sql -query "SELECT * FROM > >> >GenericFile" > >> >./query-tool: line 73: cd: > >> >/Users/AngelaWang/Documents/programs/components/filemgr/bin: No such > >>file > >> >or directory > >> >./query-tool: line 75: : command not found > >> > > >> > > >> >This is due to the wrong values of the OODT_HOME and FILEMGR_HOME > >> >variables in the query-tool script. > >> > > >> > > >> >To solve the problem, please > >> > > >> > > >> >(1) Line 45, change the code from > >> > > >> > > >> ># Only set OODT_HOME if not already set > >> > > >> >[ -z "$OODT_HOME" ] && OODT_HOME=`cd "$PRGDIR/../../.." ; pwd` > >> > > >> > > >> > > >> >to > >> > > >> > > >> ># Only set OODT_HOME if not already set > >> > > >> >[ -z "$OODT_HOME" ] && OODT_HOME=`cd "$PRGDIR/../.." ; pwd` > >> > > >> > > >> > > >> >(2) Line 54, change the code from > >> > > >> > > >> ># Only set FILEMGR_HOME if not already set > >> >if [ -z "$FILEMGR_HOME" ]; then > >> > FILEMGR_HOME="$OODT_HOME"/components/filemgr > >> > export FILEMGR_HOME > >> >fi > >> > > >> > > >> > > >> >to > >> > > >> > > >> ># Only set FILEMGR_HOME if not already set > >> >if [ -z "$FILEMGR_HOME" ]; then > >> > FILEMGR_HOME="$OODT_HOME"/filemgr > >> > export FILEMGR_HOME > >> >fi > >> > > >> > > >> > > >> >Now if you run the same command, then everything is ok. > >> > > >> > > >> >$ ./query-tool --url http://localhost:9000 --sql -query "SELECT * FROM > >> >GenericFile" > >> > >>>/Users/AngelaWang/Documents/programs/oodt-0.7/data/archive/test.txt,text > >>>/p > >> > >>>lain,text,plain,Flat,79a7bbc0-6542-11e4-82c3-33fd0e4b4523,GenericFile,20 > >>>14 > >> >-11-05T15:21:31.129-08:00,test.txt,test.txt > >> > > >> > > >> > > >> >Thanks. > >> > > >> > > >> >Best, > >> >Mengying (Angela) Wang > >> > > >> > > >> > > >> > > >> > >> > > > > > >-- > >Best, > >Mengying (Angela) Wang > > > -- Best, Mengying (Angela) Wang
Re: [RADiX File Manager] query-tool script sets the wrong values to the OODT_HOME and FILEMGR_HOME variables
Dear Prof. Mattmann, OODT-781 is about the filemgr-client script, while this one is about the query-tool script. They may be different issues. Also I would like to submit a patch file, but I am not so sure about where to get the RADiX OODT 0.7 source code? This one http://svn.apache.org/repos/asf/oodt/trunk/mvn/archetypes/radix/src/main/resources/archetype-resources/ is correct. Thanks. Best, Mengying (Angela) Wang On Wed, Nov 5, 2014 at 4:39 PM, Mattmann, Chris A (3980) < chris.a.mattm...@jpl.nasa.gov> wrote: > Hi Angela, > > Awesome, can you please submit a patch to the issue you just > created (OODT-781) and can you create a Review Board: > > http://reviews.apache.org/ > > Thanks, > Chris > > ++ > Chris Mattmann, Ph.D. > Chief Architect > Instrument Software and Science Data Systems Section (398) > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 168-519, Mailstop: 168-527 > Email: chris.a.mattm...@nasa.gov > WWW: http://sunset.usc.edu/~mattmann/ > ++ > Adjunct Associate Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++ > > > > > > > -Original Message- > From: MengYing Wang > Date: Wednesday, November 5, 2014 at 4:54 PM > To: Chris Mattmann > Cc: "dev@oodt.apache.org" > Subject: [RADiX File Manager] query-tool script sets the wrong values to > the OODT_HOME and FILEMGR_HOME variables > > >Hey everyone, > > > > > >After a text file was ingested into my file manager, I run the command: > > > > > >./query-tool --url http://localhost:9000 --sql -query "SELECT * FROM > >GenericFile" > > > > > >to verify the new file. However, it raised such an error: > > > > > >$ ./query-tool --url http://localhost:9000 --sql -query "SELECT * FROM > >GenericFile" > >./query-tool: line 73: cd: > >/Users/AngelaWang/Documents/programs/components/filemgr/bin: No such file > >or directory > >./query-tool: line 75: : command not found > > > > > >This is due to the wrong values of the OODT_HOME and FILEMGR_HOME > >variables in the query-tool script. > > > > > >To solve the problem, please > > > > > >(1) Line 45, change the code from > > > > > ># Only set OODT_HOME if not already set > > > >[ -z "$OODT_HOME" ] && OODT_HOME=`cd "$PRGDIR/../../.." ; pwd` > > > > > > > >to > > > > > ># Only set OODT_HOME if not already set > > > >[ -z "$OODT_HOME" ] && OODT_HOME=`cd "$PRGDIR/../.." ; pwd` > > > > > > > >(2) Line 54, change the code from > > > > > ># Only set FILEMGR_HOME if not already set > >if [ -z "$FILEMGR_HOME" ]; then > > FILEMGR_HOME="$OODT_HOME"/components/filemgr > > export FILEMGR_HOME > >fi > > > > > > > >to > > > > > ># Only set FILEMGR_HOME if not already set > >if [ -z "$FILEMGR_HOME" ]; then > > FILEMGR_HOME="$OODT_HOME"/filemgr > > export FILEMGR_HOME > >fi > > > > > > > >Now if you run the same command, then everything is ok. > > > > > >$ ./query-tool --url http://localhost:9000 --sql -query "SELECT * FROM > >GenericFile" > >/Users/AngelaWang/Documents/programs/oodt-0.7/data/archive/test.txt,text/p > >lain,text,plain,Flat,79a7bbc0-6542-11e4-82c3-33fd0e4b4523,GenericFile,2014 > >-11-05T15:21:31.129-08:00,test.txt,test.txt > > > > > > > >Thanks. > > > > > >Best, > >Mengying (Angela) Wang > > > > > > > > > > -- Best, Mengying (Angela) Wang
[RADiX File Manager] query-tool script sets the wrong values to the OODT_HOME and FILEMGR_HOME variables
Hey everyone, After a text file was ingested into my file manager, I run the command: ./query-tool --url http://localhost:9000 --sql -query "SELECT * FROM GenericFile" to verify the new file. However, it raised such an error: $ ./query-tool --url http://localhost:9000 --sql -query "SELECT * FROM GenericFile" ./query-tool: line 73: cd: /Users/AngelaWang/Documents/programs/components/filemgr/bin: No such file or directory ./query-tool: line 75: : command not found This is due to the wrong values of the OODT_HOME and FILEMGR_HOME variables in the query-tool script. To solve the problem, please (1) Line 45, change the code from # Only set OODT_HOME if not already set *[ -z "$OODT_HOME" ] && OODT_HOME=`cd "$PRGDIR/../../.." ; pwd`* to # Only set OODT_HOME if not already set *[ -z "$OODT_HOME" ] && OODT_HOME=`cd "$PRGDIR/../.." ; pwd`* (2) Line 54, change the code from # Only set FILEMGR_HOME if not already set if [ -z "$FILEMGR_HOME" ]; then * FILEMGR_HOME="$OODT_HOME"/components/filemgr* export FILEMGR_HOME fi to # Only set FILEMGR_HOME if not already set if [ -z "$FILEMGR_HOME" ]; then * FILEMGR_HOME="$OODT_HOME"/filemgr* export FILEMGR_HOME fi Now if you run the same command, then everything is ok. $ ./query-tool --url http://localhost:9000 --sql -query "SELECT * FROM GenericFile" /Users/AngelaWang/Documents/programs/oodt-0.7/data/archive/test.txt,text/plain,text,plain,Flat,79a7bbc0-6542-11e4-82c3-33fd0e4b4523,GenericFile,2014-11-05T15:21:31.129-08:00,test.txt,test.txt Thanks. Best, Mengying (Angela) Wang
Re: Made OODT workflow manager take the event
Hi Meghamsh, The cas-pge-0.x.jar is in the PGE_ROOT/lib directory, where 0.x represents your CAS-PGE version. Thanks! Mengying (Angela) Wang On Wed, Nov 5, 2014 at 10:58 AM, Christian Alan Mattmann wrote: > Thank you would really appreciate it. Do you have access? > If not, what¹s your wiki username and I can grant you > permissions. > > Thanks, > Chris > > > Chris Mattmann, Ph.D. > Adjunct Associate Professor, Computer Science Department > University of Southern California > Los Angeles, CA 90089 USA > Email: mattm...@usc.edu > WWW: http://sunset.usc.edu/~mattmann/ > > > > > > -Original Message- > From: Meghamsh Utkur > Date: Tuesday, November 4, 2014 at 9:35 PM > To: Chris Mattmann > Subject: Made OODT workflow manager take the event > > >Professor, > > > > > >I finally figured out how to send an event "fileconcatenator" with an > >additional step. > >It is mentioned in the wiki to copy cas-pge-0.7.jar into lib folders. But > >I spent sometime looking around for this jar in the oodt directory and > >couldn't found. > > > > > >But I found this in my maven repository. > >(/home/meghamsh/.m2/repository/org/apache/oodt/cas-pge/0.7) > > > > > >I will update the wiki. Thank you. > >-- > >Meghamsh UtkurGraduate Student,University of Southern California > >ut...@usc.edu > > > >(503)881-4791 > > > > > > > > > > -- Best, Mengying (Angela) Wang
Re: Could not mvn install OODT from the trunk
Dear Michael and Tyler, No OODT related environment variables are set. *-* Ok, maybe it is time to upgrade my maven version. Thanks again. Best, Mengying (Angela) Wang On Sun, Nov 2, 2014 at 10:10 AM, Tyler Palsulich wrote: > > I use maven 3.2 to build OODT. This might help. > +1 > -- Best, Mengying (Angela) Wang
Re: RADiX filemgr-client script cannot delete product by id or name
Dear Prof. Mattmann and Paul, @Mattmann, Retried, still the same error. -_-! @Paul, the jira issue has been created ( https://issues.apache.org/jira/browse/OODT-781). Thanks. Best, Mengying (Angela) Wang On Sun, Nov 2, 2014 at 11:47 AM, Ramirez, Paul M (398M) < paul.m.rami...@jpl.nasa.gov> wrote: > Sounds like a bug in RADiX. MengYing would you mind adding an issue. The > filemgr-client script should source the env.sh so that this variable would > be set. > > --Paul > > Sent from my iPhone > > > On Nov 1, 2014, at 7:52 PM, Mattmann, Chris A (3980) < > chris.a.mattm...@jpl.nasa.gov> wrote: > > > > Yes, you have to reingest your data after restarting your file manager > > after deleting > > the catalog. Then try the query tool > > > > ++ > > Chris Mattmann, Ph.D. > > Chief Architect > > Instrument Software and Science Data Systems Section (398) > > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > > Office: 168-519, Mailstop: 168-527 > > Email: chris.a.mattm...@nasa.gov > > WWW: http://sunset.usc.edu/~mattmann/ > > ++ > > Adjunct Associate Professor, Computer Science Department > > University of Southern California, Los Angeles, CA 90089 USA > > ++ > > > > > > > > > > > > > > -Original Message- > > From: MengYing Wang > > Date: Saturday, November 1, 2014 at 1:47 PM > > To: Chris Mattmann > > Cc: "dev@oodt.apache.org" , Chris Mattmann > > > > Subject: Re: RADiX filemgr-client script cannot delete product by id or > > name > > > >> Dear Prof. Mattmann, > >> > >> > >> Now the ./query-tool sql query does work too. -_-! > >> $ ./query-tool --url http://localhost:9000 --sql -query "SELECT * FROM > >> GenericFile" > >> org.apache.xmlrpc.XmlRpcException: java.lang.Exception: > >> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed > >> to perform complex query : null > >> at > >> > org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClie > >> ntResponseProcessor.java:104) > >> at > >> > org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClien > >> tResponseProcessor.java:71) > >> at > >> org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73) > >> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194) > >> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185) > >> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178) > >> at > >> > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(Xm > >> lRpcFileManagerClient.java:1003) > >> at > >> > org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.java > >> :252) > >> at org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:242) > >> Exception in thread "main" > >> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: > >> java.lang.Exception: > >> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed > >> to perform complex query : null > >> at > >> > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(Xm > >> lRpcFileManagerClient.java:1009) > >> at > >> > org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.java > >> :252) > >> at org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:242) > >> Best, > >> Mengying (Angela) Wang > >> > >> > >> > >> On Sat, Nov 1, 2014 at 1:09 PM, Mattmann, Chris A (3980) > >> wrote: > >> > >> Can you please remove your FILEMGR_HOME/catalog (or your > >> RADIX_HOME/data/catalog) > >> directory, restart file manager, and then try again? > >> > >> ++ > >> Chris Mattmann, Ph.D. > >> Chief Architect > >> Instrument Software and Science Data Systems Section (398) > >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > >> Office: 168-519, Mailstop: 168-527 > >> Email: chris.a.mattm...@nasa.gov > >> WWW: http://sunset.usc.edu/~mattmann/ > >> ++ > >> Adjunc
Re: RADiX CAS-Crawler cannot trigger workflow event
Dear Prof. Mattmann, In the default property, there is a line "[WORKFLOWMGR_URL]" while no definition of the [WORKFLOW_URL]. [FAILURE_DIR] [BACKUP_DIR] [WORKFLOWMGR_URL] [FILEMGR_URL] org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory met [PUSHPULL_MET_FILE_EXT] anc Below is the definition of the TriggerPostIngestWorkflow event with the workflowMgrUrl. postIngestSuccess Thanks! Best, Mengying (Angela) Wang On Tue, Nov 4, 2014 at 10:21 AM, Mattmann, Chris A (3980) < chris.a.mattm...@jpl.nasa.gov> wrote: > Thanks Angela. What do you see in terms of your policy in > $OODT_HOME/crawler/policy in terms of action-beans.xml - do you > see a [WORKFLOW_URL] defined in there? > > ++ > Chris Mattmann, Ph.D. > Chief Architect > Instrument Software and Science Data Systems Section (398) > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 168-519, Mailstop: 168-527 > Email: chris.a.mattm...@nasa.gov > WWW: http://sunset.usc.edu/~mattmann/ > ++ > Adjunct Associate Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++ > > > > > > > -Original Message- > From: MengYing Wang > Date: Tuesday, November 4, 2014 at 11:07 AM > To: Chris Mattmann > Cc: "dev@oodt.apache.org" , Chris Mattmann > > Subject: Re: RADiX CAS-Crawler cannot trigger workflow event > > >Dear Prof. Mattmann, > > > > > >I have set all the required environment variables. > > > > > >export OODT_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7" > >export > >CRAWLER_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/crawler" > >export > >FILEMGR_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/filemgr" > >export > >WORKFLOW_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/workflow" > >export PGE_ROOT="/Users/AngelaWang/Documents/programs/oodt-0.7/pge" > >export WORKFLOW_URL="http://localhost:9200"; > >export FILEMGR_URL="http://localhost:9000"; > > > > > > > >It seems the RADiX OODT may not as stable as the original OODT? Thanks. > > > > > >Mengying (Angela) Wang > > > > > >On Sat, Nov 1, 2014 at 10:43 AM, Mattmann, Chris A (3980) > > wrote: > > > >Hi Angela, > > > >This is an issue with you not having your WORKFLOW_URL environment > >variable set. Can you make sure it¹s set before you start the workflow > >manager and RADIX? Please make sure it¹s getting passed down.. > > > >Cheers, > >Chris > > > >++ > >Chris Mattmann, Ph.D. > >Chief Architect > >Instrument Software and Science Data Systems Section (398) > >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > >Office: 168-519, Mailstop: 168-527 > >Email: chris.a.mattm...@nasa.gov > >WWW: http://sunset.usc.edu/~mattmann/ > >++ > >Adjunct Associate Professor, Computer Science Department > >University of Southern California, Los Angeles, CA 90089 USA > >++ > > > > > > > > > > > > > >-Original Message- > >From: MengYing Wang > >Date: Thursday, October 30, 2014 at 9:30 PM > >To: "dev@oodt.apache.org" > >Cc: Chris Mattmann > >Subject: RADiX CAS-Crawler cannot trigger workflow event > > > >>Hi Everyone, > >> > >> > >>I am trying to trigger some workflow evert using the crawler. However, it > >>raises a "no protocol" error: > >> > >> > >>INFO: Performing action (id = TriggerPostIngestWorkflow : description = > >>Triggers workflow event with the name [ProductType]Ingest) > >>Oct 30, 2014 4:41:45 PM org.apache.oodt.cas.crawl.ProductCrawler > >>performProductCrawlerActions > >>WARNING: Failed to perform crawler action : Failed to update workflow > >>manag
Re: RADiX CAS-Crawler cannot trigger workflow event
Dear Prof. Mattmann, I have set all the required environment variables. export OODT_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7" export CRAWLER_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/crawler" export FILEMGR_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/filemgr" export WORKFLOW_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/workflow" export PGE_ROOT="/Users/AngelaWang/Documents/programs/oodt-0.7/pge" export WORKFLOW_URL="http://localhost:9200"; export FILEMGR_URL="http://localhost:9000"; It seems the RADiX OODT may not as stable as the original OODT? Thanks. Mengying (Angela) Wang On Sat, Nov 1, 2014 at 10:43 AM, Mattmann, Chris A (3980) < chris.a.mattm...@jpl.nasa.gov> wrote: > Hi Angela, > > This is an issue with you not having your WORKFLOW_URL environment > variable set. Can you make sure it¹s set before you start the workflow > manager and RADIX? Please make sure it¹s getting passed down.. > > Cheers, > Chris > > ++ > Chris Mattmann, Ph.D. > Chief Architect > Instrument Software and Science Data Systems Section (398) > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 168-519, Mailstop: 168-527 > Email: chris.a.mattm...@nasa.gov > WWW: http://sunset.usc.edu/~mattmann/ > ++ > Adjunct Associate Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++ > > > > > > > -Original Message- > From: MengYing Wang > Date: Thursday, October 30, 2014 at 9:30 PM > To: "dev@oodt.apache.org" > Cc: Chris Mattmann > Subject: RADiX CAS-Crawler cannot trigger workflow event > > >Hi Everyone, > > > > > >I am trying to trigger some workflow evert using the crawler. However, it > >raises a "no protocol" error: > > > > > >INFO: Performing action (id = TriggerPostIngestWorkflow : description = > >Triggers workflow event with the name [ProductType]Ingest) > >Oct 30, 2014 4:41:45 PM org.apache.oodt.cas.crawl.ProductCrawler > >performProductCrawlerActions > >WARNING: Failed to perform crawler action : Failed to update workflow > >manager : no protocol: null > >org.apache.oodt.cas.crawl.structs.exceptions.CrawlerActionException: > >Failed to update workflow manager : no protocol: null > >at > >org.apache.oodt.cas.crawl.action.WorkflowMgrStatusUpdate.performAction(Wor > >kflowMgrStatusUpdate.java:59) > >at > >org.apache.oodt.cas.crawl.ProductCrawler.performProductCrawlerActions(Prod > >uctCrawler.java:357) > >at > >org.apache.oodt.cas.crawl.ProductCrawler.performPostIngestOnSuccessActions > >(ProductCrawler.java:331) > >at > >org.apache.oodt.cas.crawl.ProductCrawler.handleFile(ProductCrawler.java:19 > >4) > >at org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:108) > >at org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:75) > >at > >org.apache.oodt.cas.crawl.cli.action.CrawlerLauncherCliAction.execute(Craw > >lerLauncherCliAction.java:58) > >at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331) > >at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187) > >at > >org.apache.oodt.cas.crawl.CrawlerLauncher.main(CrawlerLauncher.java:36) > > > > > >It seems some thing wrong with my crawler. Should I do some configuration > >to my crawler? And how? Thank you for your time and help! > > > > > >-- > >Best, > >Mengying (Angela) Wang > > > > > > > > > > -- Best, Mengying (Angela) Wang
Re: RADiX filemgr-client script cannot delete product by id or name
Dear Prof. Mattmann, Now the ./query-tool sql query does work too. -_-! $ ./query-tool --url http://localhost:9000 --sql -query "SELECT * FROM GenericFile" org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed to perform complex query : null at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104) at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71) at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178) at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(XmlRpcFileManagerClient.java:1003) at org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.java:252) at org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:242) Exception in thread "main" org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed to perform complex query : null at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(XmlRpcFileManagerClient.java:1009) at org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.java:252) at org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:242) Best, Mengying (Angela) Wang On Sat, Nov 1, 2014 at 1:09 PM, Mattmann, Chris A (3980) < chris.a.mattm...@jpl.nasa.gov> wrote: > Can you please remove your FILEMGR_HOME/catalog (or your > RADIX_HOME/data/catalog) > directory, restart file manager, and then try again? > > ++ > Chris Mattmann, Ph.D. > Chief Architect > Instrument Software and Science Data Systems Section (398) > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 168-519, Mailstop: 168-527 > Email: chris.a.mattm...@nasa.gov > WWW: http://sunset.usc.edu/~mattmann/ > ++ > Adjunct Associate Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++++++ > > > > > > > -Original Message- > From: MengYing Wang > Date: Saturday, November 1, 2014 at 1:03 PM > To: "dev@oodt.apache.org" > Cc: Chris Mattmann > Subject: Re: RADiX filemgr-client script cannot delete product by id or > name > > >Dear Prof. Mattmann, > > > > > >Thank you for your help! Actually, I have set the FILEMGR_URL. Below are > >the variables that I have set: > > > > > >export OODT_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7" > >export > >CRAWLER_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/crawler" > >export > >FILEMGR_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/filemgr" > >export > >WORKFLOW_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/workflow" > >export PGE_ROOT="/Users/AngelaWang/Documents/programs/oodt-0.7/pge" > >export WORKFLOW_URL="http://localhost:9200"; > >export FILEMGR_URL="http://localhost:9000"; > > > > > > > >Best, > >Mengying (Angela) Wang > > > > > >On Sat, Nov 1, 2014 at 10:46 AM, Mattmann, Chris A (3980) > > wrote: > > > >This is an environment variable issue - your FILEMGR_URL is not set. > >Can you check it Angela? > > > >Cheers, > >Chris > > > > > >++ > >Chris Mattmann, Ph.D. > >Chief Architect > >Instrument Software and Science Data Systems Section (398) > >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > >Office: 168-519, Mailstop: 168-527 > >Email: chris.a.mattm...@nasa.gov > >WWW: http://sunset.usc.edu/~mattmann/ > >++ > >Adjunct Associate Professor, Computer Science Department > >University of Southern California, Los Angeles, CA 90089 USA > >++ > > > > > > > > > > > > > >-Original Message- > >From: MengYing Wang > >Reply-To: "dev@oodt.apache.org" > >Date: Thursday, October 30, 2014 at 3:02 PM > >To: "dev@oodt.apache.org" > >Subject: RADiX filemgr-client script canno
Re: RADiX filemgr-client script cannot delete product by id or name
Dear Prof. Mattmann, Thank you for your help! Actually, I have set the FILEMGR_URL. Below are the variables that I have set: export OODT_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7" export CRAWLER_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/crawler" export FILEMGR_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/filemgr" export WORKFLOW_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/workflow" export PGE_ROOT="/Users/AngelaWang/Documents/programs/oodt-0.7/pge" export WORKFLOW_URL="http://localhost:9200"; export FILEMGR_URL="http://localhost:9000"; Best, Mengying (Angela) Wang On Sat, Nov 1, 2014 at 10:46 AM, Mattmann, Chris A (3980) < chris.a.mattm...@jpl.nasa.gov> wrote: > This is an environment variable issue - your FILEMGR_URL is not set. > Can you check it Angela? > > Cheers, > Chris > > > ++ > Chris Mattmann, Ph.D. > Chief Architect > Instrument Software and Science Data Systems Section (398) > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 168-519, Mailstop: 168-527 > Email: chris.a.mattm...@nasa.gov > WWW: http://sunset.usc.edu/~mattmann/ > ++ > Adjunct Associate Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++ > > > > > > > -Original Message- > From: MengYing Wang > Reply-To: "dev@oodt.apache.org" > Date: Thursday, October 30, 2014 at 3:02 PM > To: "dev@oodt.apache.org" > Subject: RADiX filemgr-client script cannot delete product by id or name > > >Hey everyone, > > > >I am trying to delete some product in the OODT File Manager with its > >product id or name. However, the system raises > >"java.lang.IllegalArgumentException: URI is not absolute" error. Please > >help me! Thank you! > > > >My commands are: > > > >./filemgr-client --op --dbyid --productId > >2b0e7582-6077-11e4-ac24-8d456d2e5b9b --url http://localhost:9000 > > > >./filemgr-client -op --deleteProductByName --productName > >concatenatedOutputFile-2014-10-30T20:56:02.189Z.txt --url > >http://localhost:9000 > > > >The logs of the file manager is: > >.. > > > >Oct 30, 2014 2:56:21 PM > >org.springframework.beans.factory.support.AbstractBeanFactory doGetBean > > > >FINE: Returning cached instance of singleton bean 'sqlQuery' > > > >Oct 30, 2014 2:56:21 PM > >org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient > > > >INFO: Loading File Manager Configuration Properties from: > >[/Users/AngelaWang/Documents/programs/oodt-0.7/filemgr/etc/filemgr.propert > >ies] > > > >Oct 30, 2014 2:56:21 PM > >org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient > > > >INFO: Loading File Manager Configuration Properties from: > >[/Users/AngelaWang/Documents/programs/oodt-0.7/filemgr/etc/filemgr.propert > >ies] > > > >ERROR: Failed to delete product > >'org.apache.oodt.cas.filemgr.structs.Product@5433d17' : > >java.lang.Exception: java.lang.IllegalArgumentException: URI is not > >absolute > > > >-- > >Best, > >Mengying (Angela) Wang > > -- Best, Mengying (Angela) Wang
RADiX CAS-Crawler cannot trigger workflow event
Hi Everyone, I am trying to trigger some workflow evert using the crawler. However, it raises a "no protocol" error: INFO: Performing action (id = TriggerPostIngestWorkflow : description = Triggers workflow event with the name [ProductType]Ingest) Oct 30, 2014 4:41:45 PM org.apache.oodt.cas.crawl.ProductCrawler performProductCrawlerActions WARNING: Failed to perform crawler action : Failed to update workflow manager : no protocol: null org.apache.oodt.cas.crawl.structs.exceptions.CrawlerActionException: Failed to update workflow manager : no protocol: null at org.apache.oodt.cas.crawl.action.WorkflowMgrStatusUpdate.performAction(WorkflowMgrStatusUpdate.java:59) at org.apache.oodt.cas.crawl.ProductCrawler.performProductCrawlerActions(ProductCrawler.java:357) at org.apache.oodt.cas.crawl.ProductCrawler.performPostIngestOnSuccessActions(ProductCrawler.java:331) at org.apache.oodt.cas.crawl.ProductCrawler.handleFile(ProductCrawler.java:194) at org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:108) at org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:75) at org.apache.oodt.cas.crawl.cli.action.CrawlerLauncherCliAction.execute(CrawlerLauncherCliAction.java:58) at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331) at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187) at org.apache.oodt.cas.crawl.CrawlerLauncher.main(CrawlerLauncher.java:36) It seems some thing wrong with my crawler. Should I do some configuration to my crawler? And how? Thank you for your time and help! -- Best, Mengying (Angela) Wang
RADiX OPSUI Webapp Page Expired Error
Hey Rishi, I see you have fixed this error in the OODT 0.6 https://issues.apache.org/jira/browse/OODT-596. However, currently I am using the OODT 0.7 with RADiX, and keep receiving the error. Could you tell me what should I do to fix it, or is there any patch file that I could apply? Many thanks! -- Best, Mengying (Angela) Wang
[Problem Solved] Could not mvn install OODT from the trunk
Hey, After the Jenkins build is back to stable : oodt-trunk #1553, I have tried again. And now, it works. Many thanks. Best, Mengying (Angela) Wang On Wed, Oct 29, 2014 at 4:31 PM, MengYing Wang wrote: > Hey, > > $ mvn -version > > Apache Maven 2.2.1 (r801777; 2009-08-06 12:16:01-0700) > > Java version: 1.7.0_51 > > Java home: > /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre > > Default locale: en_US, platform encoding: UTF-8 > > OS name: "mac os x" version: "10.9.2" arch: "x86_64" Family: "mac" > > > Yes. The error is repeatable on my pc. I have tried several times, all > failed with the same error. > > > Best, > > Mengying (Angela) Wang > > On Wed, Oct 29, 2014 at 4:06 PM, Michael Starch > wrote: > >> MengYing, >> >> I cannot duplicate it on Mac OS X 10.9.5, Java 7 nor CentOS with Java 7. >> >> Is the error repeatable on your end? >> >> Michael >> On Oct 29, 2014 4:01 PM, "MengYing Wang" wrote: >> >> > Hi Michael, >> > >> > $ java -version >> > java version "1.7.0_51" >> > Java(TM) SE Runtime Environment (build 1.7.0_51-b13) >> > Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode). >> > >> > Thanks. >> > >> > Best, >> > Mengying Wang >> > >> > On Wed, Oct 29, 2014 at 6:52 AM, Michael Starch >> > wrote: >> > >> > > Mengying, >> > > >> > > I just tested this on Mac 1.9.5 using Java 7 and had no issues.Can >> > you >> > > tell me your java version? In the meantime I will test with Java 6 >> as I >> > > suspect that this might be an issue. >> > > >> > > --Michael >> > > Hi Everyone, >> > > >> > > This is my command sequence and some log: >> > > >> > > $svn co http://svn.apache.org/repos/asf/oodt/trunk/ oodt_trunk >> > > >> > > $cd oodt_trunk/ >> > > >> > > $mvn clean install >> > > >> > > .. >> > > >> > > --- >> > > >> > > T E S T S >> > > >> > > --- >> > > >> > > Running org.apache.oodt.cas.resource.mux.TestQueueMuxMonitor >> > > >> > > Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.277 >> sec >> > > <<< FAILURE! >> > > >> > > Running org.apache.oodt.cas.resource.queuerepo.TestXmlQueueRepository >> > > >> > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.182 >> sec >> > > >> > > Running >> org.apache.oodt.cas.resource.monitor.TestGangliaResourceMonitor >> > > >> > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.41 >> sec >> > > >> > > Running org.apache.oodt.cas.resource.cli.TestResourceCli >> > > >> > > Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.848 >> > sec >> > > >> > > Running org.apache.oodt.cas.resource.jobqueue.TestJobStack >> > > >> > > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 >> sec >> > > >> > > Running org.apache.oodt.cas.resource.system.TestXmlRpcResourceManager >> > > >> > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.346 >> sec >> > > >> > > Running org.apache.oodt.cas.resource.monitor.TestGangliaXMLParser >> > > >> > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.327 >> sec >> > > >> > > Running org.apache.oodt.cas.resource.mux.TestQueueMuxBatchmgr >> > > >> > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.057 >> sec >> > > >> > > Running org.apache.oodt.cas.resource.monitor.TestAssignmentMonitor >> > > >> > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.248 >> sec >> > > >> > > Running org.apache.oodt.cas.resource.util.TestUlimit >> > > >> > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.046 >> sec >> > > >> > > >> > > Results : >> > > >> > > >> > > Failed tests: >> > > >> > > >> > > Tests run: 48, Failures: 1, Errors: 0, Skipped: 0 >> > > >> > > >> > > [INFO] >> > > >> >> > > >> > > [ERROR] BUILD FAILURE >> > > >> > > [INFO] >> > > >> >> > > >> > > [INFO] There are test failures. >> > > >> > > Please refer to >> > > >> /Users/AngelaWang/Downloads/oodt_trunk/resource/target/surefire-reports >> > for >> > > the individual test results. >> > > .. >> > > >> > > Attached is the detailed test result for the failed >> TestQueueMuxMonitor >> > > test. I am using Mac OS X 10.9.2. Thank you for your help! >> > > >> > > -- >> > > Best, >> > > Mengying (Angela) Wang >> > > >> > >> > >> > >> > -- >> > Best, >> > Mengying (Angela) Wang >> > >> > > > > -- > Best, > Mengying (Angela) Wang > -- Best, Mengying (Angela) Wang
RADiX filemgr-client script cannot delete product by id or name
Hey everyone, I am trying to delete some product in the OODT File Manager with its product id or name. However, the system raises "java.lang.IllegalArgumentException: URI is not absolute" error. Please help me! Thank you! My commands are: ./filemgr-client --op --dbyid --productId 2b0e7582-6077-11e4-ac24-8d456d2e5b9b --url http://localhost:9000 ./filemgr-client -op --deleteProductByName --productName concatenatedOutputFile-2014-10-30T20:56:02.189Z.txt --url http://localhost:9000 The logs of the file manager is: .. Oct 30, 2014 2:56:21 PM org.springframework.beans.factory.support.AbstractBeanFactory doGetBean FINE: Returning cached instance of singleton bean 'sqlQuery' Oct 30, 2014 2:56:21 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient INFO: Loading File Manager Configuration Properties from: [/Users/AngelaWang/Documents/programs/oodt-0.7/filemgr/etc/filemgr.properties] Oct 30, 2014 2:56:21 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient INFO: Loading File Manager Configuration Properties from: [/Users/AngelaWang/Documents/programs/oodt-0.7/filemgr/etc/filemgr.properties] ERROR: Failed to delete product 'org.apache.oodt.cas.filemgr.structs.Product@5433d17' : java.lang.Exception: java.lang.IllegalArgumentException: URI is not absolute -- Best, Mengying (Angela) Wang
Re: Could not mvn install OODT from the trunk
Hey, $ mvn -version Apache Maven 2.2.1 (r801777; 2009-08-06 12:16:01-0700) Java version: 1.7.0_51 Java home: /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre Default locale: en_US, platform encoding: UTF-8 OS name: "mac os x" version: "10.9.2" arch: "x86_64" Family: "mac" Yes. The error is repeatable on my pc. I have tried several times, all failed with the same error. Best, Mengying (Angela) Wang On Wed, Oct 29, 2014 at 4:06 PM, Michael Starch wrote: > MengYing, > > I cannot duplicate it on Mac OS X 10.9.5, Java 7 nor CentOS with Java 7. > > Is the error repeatable on your end? > > Michael > On Oct 29, 2014 4:01 PM, "MengYing Wang" wrote: > > > Hi Michael, > > > > $ java -version > > java version "1.7.0_51" > > Java(TM) SE Runtime Environment (build 1.7.0_51-b13) > > Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode). > > > > Thanks. > > > > Best, > > Mengying Wang > > > > On Wed, Oct 29, 2014 at 6:52 AM, Michael Starch > > wrote: > > > > > Mengying, > > > > > > I just tested this on Mac 1.9.5 using Java 7 and had no issues.Can > > you > > > tell me your java version? In the meantime I will test with Java 6 as > I > > > suspect that this might be an issue. > > > > > > --Michael > > > Hi Everyone, > > > > > > This is my command sequence and some log: > > > > > > $svn co http://svn.apache.org/repos/asf/oodt/trunk/ oodt_trunk > > > > > > $cd oodt_trunk/ > > > > > > $mvn clean install > > > > > > .. > > > > > > --- > > > > > > T E S T S > > > > > > --- > > > > > > Running org.apache.oodt.cas.resource.mux.TestQueueMuxMonitor > > > > > > Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.277 > sec > > > <<< FAILURE! > > > > > > Running org.apache.oodt.cas.resource.queuerepo.TestXmlQueueRepository > > > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.182 > sec > > > > > > Running org.apache.oodt.cas.resource.monitor.TestGangliaResourceMonitor > > > > > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.41 > sec > > > > > > Running org.apache.oodt.cas.resource.cli.TestResourceCli > > > > > > Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.848 > > sec > > > > > > Running org.apache.oodt.cas.resource.jobqueue.TestJobStack > > > > > > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 > sec > > > > > > Running org.apache.oodt.cas.resource.system.TestXmlRpcResourceManager > > > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.346 > sec > > > > > > Running org.apache.oodt.cas.resource.monitor.TestGangliaXMLParser > > > > > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.327 > sec > > > > > > Running org.apache.oodt.cas.resource.mux.TestQueueMuxBatchmgr > > > > > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.057 > sec > > > > > > Running org.apache.oodt.cas.resource.monitor.TestAssignmentMonitor > > > > > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.248 > sec > > > > > > Running org.apache.oodt.cas.resource.util.TestUlimit > > > > > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.046 > sec > > > > > > > > > Results : > > > > > > > > > Failed tests: > > > > > > > > > Tests run: 48, Failures: 1, Errors: 0, Skipped: 0 > > > > > > > > > [INFO] > > > > > > > > > > [ERROR] BUILD FAILURE > > > > > > [INFO] > > > > > > > > > > [INFO] There are test failures. > > > > > > Please refer to > > > /Users/AngelaWang/Downloads/oodt_trunk/resource/target/surefire-reports > > for > > > the individual test results. > > > .. > > > > > > Attached is the detailed test result for the failed TestQueueMuxMonitor > > > test. I am using Mac OS X 10.9.2. Thank you for your help! > > > > > > -- > > > Best, > > > Mengying (Angela) Wang > > > > > > > > > > > -- > > Best, > > Mengying (Angela) Wang > > > -- Best, Mengying (Angela) Wang
Profile with id: 'fm-solr-catalog' has not been activated error
-- Forwarded message -- From: MengYing Wang Date: Thu, Oct 23, 2014 at 10:00 PM Subject: Re: Directed Research Weekly Report from 2014/09/29 - 2014/10/05 To: "Verma, Rishi (398M)" Cc: Christian Alan Mattmann , "Mcgibbney, Lewis J (398M)" , "Bryant, Ann C (398G-Affiliate)" < anniebry...@gmail.com>, "Ramirez, Paul M (398M)" < paul.m.rami...@jpl.nasa.gov>, "Mattmann, Chris A (3980)" < chris.a.mattm...@jpl.nasa.gov>, Tyler Palsulich , " u...@oodt.apache.org" Dear Rishi, I followed the new steps to use the OODT RADiX. Unfortunately, I got the same "Profile with id: 'fm-solr-catalog' has not been activated" error. Below are my commands, and some terminal output. Please check it to see if I have made some mistakes, or is it possible that something wrong with the source code? Really appreciate for your help! Step 1: $svn co http://svn.apache.org/repos/asf/oodt/trunk/ oodt_radix Aoodt_radix/curator Aoodt_radix/curator/pom.xml Aoodt_radix/curator/src Aoodt_radix/curator/src/test .. Checked out revision 1633738. Step 2: $cd oodt_radix/ Step 3: $mvn clean install [INFO] Scanning for projects... [INFO] Reactor build order: [INFO] OODT Core [INFO] Common Utilities [INFO] CAS Command Line Interface .. [INFO] BUILD SUCCESSFUL [INFO] [INFO] Total time: 5 minutes 29 seconds [INFO] Finished at: Wed Oct 22 20:40:38 PDT 2014 [INFO] Final Memory: 133M/254M Step 4: $mvn archetype:generate [INFO] Scanning for projects... [INFO] Reactor build order: [INFO] OODT Core [INFO] Common Utilities .. [INFO] project created from Old (1.x) Archetype in dir: /Users/AngelaWang/Downloads/oodt_radix/radix-archetype [INFO] [INFO] BUILD SUCCESSFUL [INFO] [INFO] Total time: 1 minute 40 seconds [INFO] Finished at: Wed Oct 22 20:52:46 PDT 2014 [INFO] Final Memory: 36M/84M [INFO] Step 5: $cd radix-archetype/ Step 6: $mvn clean package -Pfm-solr-catalog [INFO] Scanning for projects... [WARNING] Profile with id: 'fm-solr-catalog' has not been activated. [INFO] [INFO] Building radix-archetype [INFO]task-segment: [clean, package] [INFO] . Best, Mengying (Angela) Wang On Sat, Oct 18, 2014 at 3:49 PM, Verma, Rishi (398M) < rishi.ve...@jpl.nasa.gov> wrote: > Hi MengYing, > > Your CMD1 should not have the ‘-Pfm-solr-catalog’ argument. The reason > is because that command *generates* a new project for you, whereas, the > ‘-Pfm-solr-catalog’ should only be used to *build* the project once it > has already been generated. You might want to read up a bit on Maven > archetypes, which is what OODT RADiX is. > http://maven.apache.org/guides/introduction/introduction-to-archetypes.html > > Let me explain in this way, here’s the steps to using OODT RADiX: > 1. Get a hold of the latest OODT RADiX Maven Archetype (you might have > already done this if you have the full OODT source) > i.e. download the full OODT source and invoke ‘mvn install’ so that > you can use the latest RADiX archetype > http://svn.apache.org/repos/asf/oodt/trunk/ > 2. Use the OODT RADiX Maven Archetype to *generate* a new OODT project > source folder structure for you (this is the source for your new project!) > i.e. invoke the command: > > mvn archetype:generate > (select RADiX from the list of archetypes you see, and follow the prompts) > 3. Change into the newly generated directory from above, and *build* a > tar-ball distribution of OODT that you can run from the source folder > structure you generated earlier > > mvn clean package -Pfm-solr-catalog > 4. Take the build tar-ball distribution, and extract it somewhere else for > launching OODT > > tar zxf distribution/target/oodt-*.jar -C /usr/local/my-oodt-project > 5. Run OODT > > cd /usr/local/my-oodt-project/bin > > ./oodt start > > That’s the typical workflow for using RADiX. So the key here is, only > use the ‘-Pfm-solr-catalog’ argument when *building* OODT, not when > *generating* the folder structure. > > *If you’re starting from scratch:* > 1. Use Vagrant Virtual Machine technology to get a pre-built OODT > deployment connected to Solr in one command: > https://cwiki.apache.org/confluence/display/OODT/Vagrant+Powered+OODT > > [ I didn't try this approach ] > > > You should try this! Because all five steps above are automated for > y
Re: Could not mvn install OODT from the trunk
Hi Michael, $ java -version java version "1.7.0_51" Java(TM) SE Runtime Environment (build 1.7.0_51-b13) Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode). Thanks. Best, Mengying Wang On Wed, Oct 29, 2014 at 6:52 AM, Michael Starch wrote: > Mengying, > > I just tested this on Mac 1.9.5 using Java 7 and had no issues.Can you > tell me your java version? In the meantime I will test with Java 6 as I > suspect that this might be an issue. > > --Michael > Hi Everyone, > > This is my command sequence and some log: > > $svn co http://svn.apache.org/repos/asf/oodt/trunk/ oodt_trunk > > $cd oodt_trunk/ > > $mvn clean install > > .. > > --- > > T E S T S > > --- > > Running org.apache.oodt.cas.resource.mux.TestQueueMuxMonitor > > Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.277 sec > <<< FAILURE! > > Running org.apache.oodt.cas.resource.queuerepo.TestXmlQueueRepository > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.182 sec > > Running org.apache.oodt.cas.resource.monitor.TestGangliaResourceMonitor > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.41 sec > > Running org.apache.oodt.cas.resource.cli.TestResourceCli > > Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.848 sec > > Running org.apache.oodt.cas.resource.jobqueue.TestJobStack > > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec > > Running org.apache.oodt.cas.resource.system.TestXmlRpcResourceManager > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.346 sec > > Running org.apache.oodt.cas.resource.monitor.TestGangliaXMLParser > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.327 sec > > Running org.apache.oodt.cas.resource.mux.TestQueueMuxBatchmgr > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.057 sec > > Running org.apache.oodt.cas.resource.monitor.TestAssignmentMonitor > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.248 sec > > Running org.apache.oodt.cas.resource.util.TestUlimit > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.046 sec > > > Results : > > > Failed tests: > > > Tests run: 48, Failures: 1, Errors: 0, Skipped: 0 > > > [INFO] > > > [ERROR] BUILD FAILURE > > [INFO] > > > [INFO] There are test failures. > > Please refer to > /Users/AngelaWang/Downloads/oodt_trunk/resource/target/surefire-reports for > the individual test results. > .. > > Attached is the detailed test result for the failed TestQueueMuxMonitor > test. I am using Mac OS X 10.9.2. Thank you for your help! > > -- > Best, > Mengying (Angela) Wang > -- Best, Mengying (Angela) Wang
Could not mvn install OODT from the trunk
Hi Everyone, This is my command sequence and some log: $svn co http://svn.apache.org/repos/asf/oodt/trunk/ oodt_trunk $cd oodt_trunk/ $mvn clean install .. --- T E S T S --- Running org.apache.oodt.cas.resource.mux.TestQueueMuxMonitor Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.277 sec <<< FAILURE! Running org.apache.oodt.cas.resource.queuerepo.TestXmlQueueRepository Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.182 sec Running org.apache.oodt.cas.resource.monitor.TestGangliaResourceMonitor Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.41 sec Running org.apache.oodt.cas.resource.cli.TestResourceCli Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.848 sec Running org.apache.oodt.cas.resource.jobqueue.TestJobStack Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec Running org.apache.oodt.cas.resource.system.TestXmlRpcResourceManager Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.346 sec Running org.apache.oodt.cas.resource.monitor.TestGangliaXMLParser Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.327 sec Running org.apache.oodt.cas.resource.mux.TestQueueMuxBatchmgr Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.057 sec Running org.apache.oodt.cas.resource.monitor.TestAssignmentMonitor Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.248 sec Running org.apache.oodt.cas.resource.util.TestUlimit Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.046 sec Results : Failed tests: Tests run: 48, Failures: 1, Errors: 0, Skipped: 0 [INFO] [ERROR] BUILD FAILURE [INFO] [INFO] There are test failures. Please refer to /Users/AngelaWang/Downloads/oodt_trunk/resource/target/surefire-reports for the individual test results. .. Attached is the detailed test result for the failed TestQueueMuxMonitor test. I am using Mac OS X 10.9.2. Thank you for your help! -- Best, Mengying (Angela) Wang --- Test set: org.apache.oodt.cas.resource.mux.TestQueueMuxMonitor --- Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.277 sec <<< FAILURE! testGetNodeByURL(org.apache.oodt.cas.resource.mux.TestQueueMuxMonitor) Time elapsed: 0.216 sec <<< FAILURE! junit.framework.AssertionFailedError: expected: but was: at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.failNotEquals(Assert.java:280) at junit.framework.Assert.assertEquals(Assert.java:64) at junit.framework.Assert.assertEquals(Assert.java:71) at org.apache.oodt.cas.resource.mux.TestQueueMuxMonitor.testGetNodeByURL(TestQueueMuxMonitor.java:122)
Re: Two Questions about the "Integrating Solr with OODT RADiX" tutorial page
Dear Rishi,
Thank you for your help and detailed explanation! Yes, I have tried your
new method to use the OODT RADiX, but still received the "Profile with id:
'fm-solr-catalog' has not been activated" error. I have replied your
previous email with my command sequences and logs. It seems you have not
received my message. I will resend my previous email. Wish you could check
what is wrong with my installation process. Many thanks. :)
Best,
Mengying (Angela) Wang
On Mon, Oct 27, 2014 at 10:32 AM, Verma, Rishi (398M) <
rishi.ve...@jpl.nasa.gov> wrote:
> Hi Mengying,
>
> Thanks for the questions and for pushing through with this. Responses
> below:
>
> >> (1) In 2(a), it says "Create a new directory under webapps, with the
> >> following directory structure:"
> >> bash-3.2$ tree
> >> .
> >> ├── pom.xml
> >> └── src
> >> └── main
> >> ├── assembly
> >> │ └── assembly.xml
> >> └── webapp
> >> ├── META-INF
> >> │ └── context.xml
> >> └── WEB-INF
> >> └── web.xml
> >> However, according to the development trunk:
> >>
> >>
> http://svn.apache.org/repos/asf/oodt/trunk/mvn/archetypes/radix/src/main/r
> >> esources/archetype-resources/
> >> <
> http://svn.apache.org/repos/asf/oodt/trunk/mvn/archetypes/radix/src/main/
> >> resources/archetype-resources/>, there is no "assembly" directory under
> >> the "main" folder. Instead, the "assembly" directory should be in the
> >> "[RADiX project home]/solr/src/main/" directory.
>
> The article you are reading “Integrating Solr with OODT RADiX” is a bit
> out of date, because the integration process has been automated to be part
> of OODT RADiX. Were you still having problems running OODT RADiX via the
> -Pfm-solr-catalog flag?
>
> That being said, you shouldn’t need an “assembly” directory under the
> solr-webapp. I’ve updated the wiki to reflect this incorrect suggestion.
> Thanks for spotting it.
>
>
> >> (2) The content of the assembly.xml file is a little different from that
> >> in the development trunk. For example:
> >> In the confluence tutorial:
> >> false
> >> solr
> >> false
> >>
> >>
> >> ${basedir}/bin
> >> solr/bin
> >>
> >> 775
> >>
> >>
> >>
> >> While in the development trunk:
> >> false
> >> solr
> >> false
> >>
> >>
> >> ${basedir}/src/main/resources/bin
> >> solr/bin
> >>
> >> **.*
> >>
> >> 775
> >>
> >>
>
> Use the latter, it is more explicit - all files under
> solr/src/main/resources/bin will be included in the solr tar-ball (and
> final OODT RADiX tar ball).
>
> Thanks much, keep us posted on your progress,
> Rishi
>
>
> On Oct 23, 2014, at 11:23 PM, Mattmann, Chris A (3980) <
> chris.a.mattm...@jpl.nasa.gov> wrote:
>
> > CC’ing dev@oodt
> >
> > ++++++
> > Chris Mattmann, Ph.D.
> > Chief Architect
> > Instrument Software and Science Data Systems Section (398)
> > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> > Office: 168-519, Mailstop: 168-527
> > Email: chris.a.mattm...@nasa.gov
> > WWW: http://sunset.usc.edu/~mattmann/
> > ++
> > Adjunct Associate Professor, Computer Science Department
> > University of Southern California, Los Angeles, CA 90089 USA
> > ++
> >
> >
> >
> >
> >
> >
> > -Original Message-
> > From: MengYing Wang
> > Date: Thursday, October 23, 2014 at 11:21 PM
> > To: Chris Mattmann , "Mcgibbney, Lewis J (398M)"
> > , "Bryant, Ann C (398G-Affiliate)"
> > , "Ramirez, Paul M (398M)"
> > , Chris Mattmann
> > , Tyler Palsulich
> > Subject: Two Questions about the "Integrating Solr with OODT RADiX"
> > tutorial page
> >
> >> Dear Rishi,
> >>
> >>
> >> Following the steps in the tutorial
> >>
> https://cwiki.apache.org/confluence/display/OODT/Integrating+Solr+with+OOD
> >> T+RADiX
> >> <
> https://cwiki.apache.org/confluence/display/OODT/Integrating+Solr+with+OO
> >> DT+RADiX>, I h
Re: Error occurs when using the query_tool script of the file manager for a new product type
Hi Prof Chris, Done with Jira: https://issues.apache.org/jira/browse/OODT-766. This happens with the SQL Query only, not with the Lucene. At least no exception is thrown out in Lucene queries. By the way, maybe I am using the wrong syntax, but it seems the Lucene is much difficult to use than Sql. For example: Suppose I have 3 files with the GenericFile type: $ ./query_tool --url http://localhost:9000 --sql -query "SELECT CAS.ProductReceivedTime,CAS.ProductName,CAS.ProductId,ProductType,ProductTypeId,ProductTypeName,ProductStructure,Filename,FileLocation,MimeType FROM GenericFile" -sortBy 'CAS.ProductReceivedTime' -outputFormat '$CAS.ProductReceivedTime,$CAS.ProductName,$CAS.ProductId,$ProductType,$ProductTypeId,$ProductTypeName,$ProductStructure,$Filename,$FileLocation,$MimeType' Oct 28, 2014 3:51:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager complexQuery INFO: Query returned 3 results 2014-09-15T12:41:31.241-07:00,blah.txt,4ad79161-3d10-11e4-b2be-cf9a90c3077d,GenericFile,GenericFileId,GenericFileName,Flat,blah.txt,/Users/AngelaWang/files/blah.txt,text/plain,text,plain 2014-10-13T22:04:28.703-07:00,hello.txt,93586611-535f-11e4-9209-4d1d47528f7d,GenericFile,GenericFileId,GenericFileName,Flat,hello.txt,/Users/AngelaWang/files/hello.txt,text/plain,text,plain 2014-10-27T23:31:54.319-07:00,concatenatedOutputFile-2014-10-28T06:31:53.597Z.txt,1bc212ba-5e6c-11e4-84b4-f5653df4b171,GenericFile,FileConcatenatorProductType,GenericFile,FileConcatenatorProductTypeId,GenericFile,FileConcatenatorProductTypeName,Flat,concatenatedOutputFile-2014-10-28T06:31:53.597Z.txt,concatenatedOutputFile-2014-10-28T06:31:53.597Z.txt,/Users/AngelaWang/files/concatenatedOutputFile-2014-10-28T06:31:53.597Z.txt,text/plain,text,plain All the flowing 4 Lucene queries return 0 record. $./query_tool --url http://localhost:9000 --lucene -query 'CAS.ProductTypeName:GenericFile' $./query_tool --url http://localhost:9000 --lucene -query 'ProductTypeName:GenericFile' $./query_tool --url http://localhost:9000 --lucene -query 'CAS.ProductTypeName:GenericFileName' $./query_tool --url http://localhost:9000 --lucene -query 'ProductTypeName:GenericFileName' Oct 28, 2014 3:57:09 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog paginateQuery WARNING: Query: [q=CAS.ProductTypeName:GenericFile] for Product Type: [urn:oodt:GenericFile] returned no results Only this one gives me the correct result: $./query_tool --url http://localhost:9000 --lucene -query 'CAS.ProductName:blah.txt' 4ad79161-3d10-11e4-b2be-cf9a90c3077d Thanks. Best, Mengying (Angela) Wang On Tue, Oct 28, 2014 at 7:24 AM, Chris Mattmann wrote: > Thanks for reporting this Angela. It would be great if you can > file an issue for this in JIRA? Does this happen with the SQL > Query only, or does it do so also with Lucene? > > Cheers, > Chris > > ---- > Chris Mattmann > chris.mattm...@gmail.com > > > > > -Original Message- > From: Mengying Wang > Reply-To: > Date: Tuesday, October 28, 2014 at 12:10 AM > To: > Subject: Error occurs when using the query_tool script of the file manager > for a new product type > > >Hi Everyone, > > > >According to the tutorial > >http://oodt.apache.org/components/maven/curator/user/basic.html, I added > a > >new JSON product type to my file manager. However, if I run the query_tool > >script before any json files ingested, it will raise some errors, as shown > >here: > > > >$ ./query_tool --url http://localhost:9000 --sql -query "SELECT * FROM > >JSON" > >Oct 27, 2014 11:52:07 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog > >paginateQuery > >WARNING: Query: [q=] for Product Type: [urn:example:JSON] returned no > >results > >java.lang.NullPointerException > >at > >org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.complexQuery(XmlRpcFi > >leManager.java:602) > >at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) > >at > >sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm > >pl.java:43) > >at java.lang.reflect.Method.invoke(Method.java:606) > >at org.apache.xmlrpc.Invoker.execute(Invoker.java:130) > >at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84) > >at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146) > >at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139) > >at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125) > >at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761) > >at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642) > >at java.lang.Thread.run(Thread.java:744) > >org.apache.xmlrpc.XmlRpcException: java.lang.Excepti
[File Manager] Users could ingest the same file multiple times, but error occurs if they delete the file more than once
Hi Everyone, Users could use the filemgr-client script to ingest the same file multiple times, which leads to several records in the query_tool result. However, if you delete the same file more than once, then error occurs. Below is the scenario: Step 1: Ingest the test.txt file twice. $ ./query_tool --url http://localhost:9000 --sql -query "SELECT * FROM GenericFile" Oct 28, 2014 12:23:07 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager complexQuery INFO: Query returned 2 results /Users/AngelaWang/files/test.txt,text/plain,text,plain,Flat,9caff30e-5ae3-11e4-ada3-4d034e11a919,GenericFile,2014-10-23T11:37:16.085-07:00,test.txt,test.txt /Users/AngelaWang/files/test.txt,text/plain,text,plain,Flat,77e80905-476d-11e4-83cb-fd04a1088b13,GenericFile,2014-09-28T17:13:41.669-07:00,test.txt,test.txt Step 2: Succeed in deleting the test.txt file for the first time $ ./filemgr-client --op --dbyid --productId 9caff30e-5ae3-11e4-ada3-4d034e11a919 --url http://localhost:9000 Oct 28, 2014 12:28:25 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties] Oct 28, 2014 12:28:25 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties] Successfully deleted product 'test.txt' Step 3: Failed in deleting the test.txt file for the second time. $ ./filemgr-client --op --dbyid --productId 77e80905-476d-11e4-83cb-fd04a1088b13 --url http://localhost:9000 Oct 28, 2014 12:31:50 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties] Oct 28, 2014 12:31:50 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties] ERROR: Failed to delete product 'org.apache.oodt.cas.filemgr.structs.Product@2226a790' : Failed to delete file 'file:/Users/AngelaWang/files/test.txt/test.txt' You have to manually create the test.txt file under the archive directory, then you could successfully delete the test.txt file once again. Thanks. Mengying (Angela) Wang
Error occurs when using the query_tool script of the file manager for a new product type
Hi Everyone, According to the tutorial http://oodt.apache.org/components/maven/curator/user/basic.html, I added a new JSON product type to my file manager. However, if I run the query_tool script before any json files ingested, it will raise some errors, as shown here: $ ./query_tool --url http://localhost:9000 --sql -query "SELECT * FROM JSON" Oct 27, 2014 11:52:07 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog paginateQuery WARNING: Query: [q=] for Product Type: [urn:example:JSON] returned no results java.lang.NullPointerException at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.complexQuery(XmlRpcFileManager.java:602) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.xmlrpc.Invoker.execute(Invoker.java:130) at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84) at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146) at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139) at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125) at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761) at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642) at java.lang.Thread.run(Thread.java:744) org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed to perform complex query : null at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104) at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71) at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178) at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(XmlRpcFileManagerClient.java:990) at org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.java:252) at org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:242) Exception in thread "main" org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed to perform complex query : null at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(XmlRpcFileManagerClient.java:996) at org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.java:252) at org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:242) By the way, if some files with the product type JSON are already ingested in my file manager, then the query_tool script works very well. Thanks. Mengying (Angela) Wang
Re: PushPull directory
Dear Prof. Mattmann, Done. I have updated the confluence pages: https://cwiki.apache.org/confluence/display/OODT/OODT+Push-Pull+User+Guide https://cwiki.apache.org/confluence/display/OODT/OODT+Push+Pull+Plugins And send the solution email back to you and dev@oodt.apache.org through the page https://www.mail-archive.com/dev@oodt.apache.org/msg04276.html. It may take some days to update the mail archive? Thanks, Mengying Wang On Thu, Oct 16, 2014 at 4:47 PM, Mattmann, Chris A (3980) < chris.a.mattm...@jpl.nasa.gov> wrote: > Please send out how you solved it Angela thanks > > Sent from my iPhone > > On Oct 16, 2014, at 4:38 PM, "MengYing Wang" > wrote: > > Dear Tyler, > > The pushpull problem has been solved. Now I could use pushpull to crawl > some remote ftp directories. Thank you. > > Best, > Mengying Wang > > > > > > On Thu, Oct 16, 2014 at 2:47 PM, Tyler Palsulich > wrote: > >> Hi Mengying, >> >> Did you ever make any progress on this? What exactly were you trying to >> do, again? What error are you seeing? >> >> Thanks, >> Tyler >> >> On Sun, Oct 5, 2014 at 10:26 AM, MengYing Wang >> wrote: >> >>> Dear Prof. Mattmann and Tyler, >>> >>> This is the pushpull zip file. >>> cas-pushpull-0.6.zip >>> <https://docs.google.com/file/d/0B7PYVKDpy0jlSDR4eGdrekkyc2s/edit?usp=drive_web> >>> >>> Thank you for your help! >>> >>> Best, >>> Mengying (Angela) Wang >>> >>> On Sun, Oct 5, 2014 at 9:56 AM, Tyler Palsulich >>> wrote: >>> >>>> > MY COMMENT: Please send me your zipped up. I will >>>> >take a look - Tyler can you also look? >>>> >>>> Yup! We'll get it sorted out. >>>> >>> >>> >>> >>> -- >>> Best, >>> Mengying (Angela) Wang >>> >> >> > > > -- > Best, > Mengying (Angela) Wang > > -- Best, Mengying (Angela) Wang
Re: No data files are downloaded using Apache OODT Pushpull
Dear Mattmann, Problem solved. I have made the following 2 changes: 1: Change the active status of the daemon from ’no’ to ‘yes’. That is, [CAS_PP_HOME]/etc/examples/RemoteSpecsFiles/RemoteSpecs.xml epsilon="20s" runOnReboot="yes"/> .. 2: It is mainly caused by the protocol issues of the PushPull ftp plugins. So I used the other PushPull ftp plugins. For the details please refer to OODT Push Pull Plugins. Then everything is ok. Thank you for your help! Best, Angela Wang
Re: Where is the TikaCmdLineMetExtractor?
Dear Prof. Mattmann, Yes, the TikaCmdLineMetExtractor is available in the Apache OODT Cas-Crawler-0.7, and it works very well. Thank you! Best, Angela Wang On Sun, Sep 21, 2014 at 7:46 PM, Mattmann, Chris A (3980) < chris.a.mattm...@jpl.nasa.gov> wrote: > Hi Angela, > > The TikaExtractor is available here: > > svn.apache.org/repos/asf/oodt/trunk/metadata/src/main/java/org/apache/oodt/ > cas/metadata/extractors/ > > Looking at CHANGES.txt looks like this only showed up in 0.7, so you'll > need to upgrade to 0.7 (just released). > > Cheers, > Chris > > ++ > Chris Mattmann, Ph.D. > Chief Architect > Instrument Software and Science Data Systems Section (398) > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 168-519, Mailstop: 168-527 > Email: chris.a.mattm...@nasa.gov > WWW: http://sunset.usc.edu/~mattmann/ > ++ > Adjunct Associate Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++++++ > > > > > > > -Original Message- > From: Mengying Wang > Date: Friday, September 19, 2014 12:46 PM > To: > Subject: Where is the TikaCmdLineMetExtractor? > > >Dear everyone, > >I am trying to integrate the Apache OODT Crawler with the Apache Tika. > >According to the Apache OODT Crawler Help > >(https://cwiki.apache.org/confluence/display/OODT/OODT+Crawler+Help), I > >can use the TikaCmdLineMetExtractor directly. However, when I run the > >command: > > > >./crawler_launcher > >--filemgrUrl http://localhost:9000 > >--operation --launchMetCrawler > >--clientTransferer > >org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory > >--productPath /usr/local/meerkat/data/staging/products/hdf5 > >--metExtractor > >org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor > >--metExtractorConfig > >/usr/local/meerkat/extractors/tikaextractor/tikaextractor.config > > > >It raises an error: > >ERROR: Validation Failures: - Value > >'org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor' for > >option metExtractor is not a valid class > > > > > >I am wondering is this a software version problem? Which version of the > >Apache OODT Crawler and the Apache Tika should I use to run such a > >command? Currently, I am using the Apache OODT Cas-Crawler-0.6 and the > >Apache Tika-1.6. > > > >Thank you very much for your time and help! > > > >Best, > >Angela Wang > > > > -- Best, Mengying (Angela) Wang
Where is the TikaCmdLineMetExtractor?
Dear everyone, I am trying to integrate the Apache OODT Crawler with the Apache Tika. According to the Apache OODT Crawler Help ( https://cwiki.apache.org/confluence/display/OODT/OODT+Crawler+Help), I can use the TikaCmdLineMetExtractor directly. However, when I run the command: ./crawler_launcher --filemgrUrl http://localhost:9000 --operation --launchMetCrawler --clientTransferer org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory --productPath /usr/local/meerkat/data/staging/products/hdf5 --metExtractor org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor --metExtractorConfig /usr/local/meerkat/extractors/tikaextractor/tikaextractor.config It raises an error: ERROR: Validation Failures: - Value 'org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor' for option metExtractor is not a valid class I am wondering is this a software version problem? Which version of the Apache OODT Crawler and the Apache Tika should I use to run such a command? Currently, I am using the Apache OODT Cas-Crawler-0.6 and the Apache Tika-1.6. Thank you very much for your time and help! Best, Angela Wang
No data files are downloaded using Apache OODT Pushpull
Dear Everyone, I am trying to use the Apache OODT Pushpull to download some remote data files to my local machine. According to the Apache OODT Pushpull user guide (https://cwiki.apache.org/confluence/display/OODT/OODT+Push-Pull+User+Guide), I have completed all steps. However, when I run the command ./pushpull, no data files are downloaded to my staging area after 15 minutes. I have used FileZilla to login to the remove server and confirmed that there are some data files. So I am not sure what is wrong with my configurations? Could you please help me out? Many thanks! Following are my running screenshot and configuration details: $ ./pushpull TRANSFER: org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory ^C *[CAS_PP_HOME]/etc/push_pull_framework.properties* line 21 #external configuration files 22 org.apache.oodt.cas.pushpull.config.external.properties.files=[CAS_PP_HOME]/etc/default.properties 35 # ingester filemgr url 36 org.apache.oodt.cas.filemgr.url=http://localhost:9000 61 #protocolfactory specification for protocol types 62 org.apache.oodt.cas.pushpull.config.protocolfactory.info.files=[CAS_PP_HOME]/policy/ProtocolFactoryInfo.xml 69 #parser to retrievalmethod map 70 org.apache.oodt.cas.pushpull.config.parser.info.files=[CAS_PP_HOME]/policy/ParserToRetrievalMethodMap.xml 71 72 #unique metadata element info 73 org.apache.oodt.cas.pushpull.config.type.detection.file=[CAS_PP_HOME]/policy/mimetypes.xml 74 75 #directory below which all data file will be downloaded to 76 org.apache.oodt.cas.pushpull.data.files.base.staging.area=[CAS_PP_HOME]/staging *[CAS_PP_HOME]/policy/mimetypes.xml* <_comment> *[CAS_PP_HOME]/etc/examples/ExternalSourcesFiles/ExternalSources.xml* anonymous u...@host.com *[CAS_PP_HOME]/etc/examples/RemoteSpecsFiles/RemoteSpecs.xml* *[CAS_PP_HOME]/etc/examples/DirStructXmlParserFiles/RTG_SST.xml* Thanks again! Best, Angela Wang
HttpClient NoClassDefFoundError For the url-downloader Script of the Apache OODT Crawler
Dear Everyone, I am trying to use the url-downloader script of the Apache OODT Crawler. However, it raises the following "no class definition" error, I am not sure what is going on. Could you please give me some hints? Many thanks. $ ./url-downloader http://pds-imaging.jpl.nasa.gov/data/msl/MSLHAZ_0XXX/CATALOG/CATINFO.TXT . Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/oodt/cas/pushpull/protocol/http/HttpClient Caused by: java.lang.ClassNotFoundException: org.apache.oodt.cas.pushpull.protocol.http.HttpClient at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) Best, Angela Wang
