[NOTICE] Added Links to static component level user guides
Hey All, Since I get asked this question a lot, I decided to make a wiki page about it. Here's some links to the Component-Level (static) user guides for OODT published along with the current versions of the components: http://s.apache.org/nFO Comments/updates, welcome. At some point, we could transition these guides more to the wiki, but until someone has the cycles to do that, we'll keep pointing to them here. Now you can use the above URL if someone asks you. Cheers, Chris ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Jenkins build became unstable: oodt-trunk #270
See https://builds.apache.org/job/oodt-trunk/270/
Re: Review Request: XMLPS should be able to stream large results
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3543/#review5227 --- Ship it! After testing I believe this is good to go. trunk/grid/src/main/java/org/apache/oodt/grid/ProductQueryServlet.java https://reviews.apache.org/r/3543/#comment11427 I remember receiving partial responses. Also the servlet API says to flush the output stream to commit the response. This is especially needed since we don't know the length of the response. http://tomcat.apache.org/tomcat-5.5-doc/servletapi/javax/servlet/ServletResponse.html#getOutputStream() trunk/grid/src/main/java/org/apache/oodt/grid/ProductQueryServlet.java https://reviews.apache.org/r/3543/#comment11428 Quoted from servlet API here http://tomcat.apache.org/tomcat-5.5-doc/servletapi/javax/servlet/http/HttpServlet.html#doGet(javax.servlet.http.HttpServletRequest,%20javax.servlet.http.HttpServletResponse): Where possible, set the Content-Length header (with the ServletResponse.setContentLength(int) method), to allow the servlet container to use a persistent connection to return its response to the client, improving performance. The content length is automatically set if the entire response fits inside the response buffer. When using HTTP 1.1 chunked encoding (which means that the response has a Transfer-Encoding header), do not set the Content-Length header. - Thus, we don't set Content-Length and always use chunked Transfer-Encoding. Tested using cURL to watch the header values. Tomcat 5.5.35 : works Tomcat 6.0.35: works Tomcat 7.0.25: works trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/DBMSExecutor.java https://reviews.apache.org/r/3543/#comment11425 it is organized. OODT first, JDK second, removed unused imports. trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/XMLPSProductHandler.java https://reviews.apache.org/r/3543/#comment11426 yep organized. - Ricky On 2012-01-19 02:54:44, Ricky Nguyen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3543/ --- (Updated 2012-01-19 02:54:44) Review request for oodt. Summary --- See OODT-341: https://issues.apache.org/jira/browse/OODT-341 * CDEResult extends org.apache.oodt.xmlquery.Result * CDEResult mimetype is always text/plain * CDEResult size is always 0 * CDEResult inputstream is CDEResultInputStream * CDEResultInputStream has IO methods and wraps a CDEResult * CDEResult wraps a ResultSet and returns rows as Strings, applying MappingFuncs if a Mapping is provided, and appending constant fields if a List of CDEValues is provided. * ProductQueryServlet relies on servlet container to handle Content-Length (and possibly Transfer-Encoding: chunked) This addresses bug OODT-341. https://issues.apache.org/jira/browse/OODT-341 Diffs - trunk/grid/src/main/java/org/apache/oodt/grid/ProductQueryServlet.java 1183564 trunk/xmlps/pom.xml 1233127 trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/DBMSExecutor.java 1233127 trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/XMLPSProductHandler.java 1233127 trunk/xmlps/src/main/java/org/apache/oodt/xmlps/structs/CDEResult.java 1233127 trunk/xmlps/src/main/java/org/apache/oodt/xmlps/structs/CDEResultInputStream.java PRE-CREATION trunk/xmlps/src/test/java/org/apache/oodt/xmlps/structs/TestCDEResult.java PRE-CREATION trunk/xmlps/src/test/java/org/apache/oodt/xmlps/structs/TestCDEResultInputStream.java PRE-CREATION Diff: https://reviews.apache.org/r/3543/diff Testing --- Runs in Tomcat 7. I've used it at CHLA. NOT tested in other app servers. Which other app servers should I test? NOT tested MappingFuncs. Is it necessary? Added 2 unit tests: TestCDEResult and TestCDEResultInputStream, which both pass. Thanks, Ricky
Re: Review Request: XMLPS should be able to stream large results
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3543/#review5230 --- Ship it! Looks good! Ship it! - Chris On 2012-01-19 02:54:44, Ricky Nguyen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3543/ --- (Updated 2012-01-19 02:54:44) Review request for oodt. Summary --- See OODT-341: https://issues.apache.org/jira/browse/OODT-341 * CDEResult extends org.apache.oodt.xmlquery.Result * CDEResult mimetype is always text/plain * CDEResult size is always 0 * CDEResult inputstream is CDEResultInputStream * CDEResultInputStream has IO methods and wraps a CDEResult * CDEResult wraps a ResultSet and returns rows as Strings, applying MappingFuncs if a Mapping is provided, and appending constant fields if a List of CDEValues is provided. * ProductQueryServlet relies on servlet container to handle Content-Length (and possibly Transfer-Encoding: chunked) This addresses bug OODT-341. https://issues.apache.org/jira/browse/OODT-341 Diffs - trunk/grid/src/main/java/org/apache/oodt/grid/ProductQueryServlet.java 1183564 trunk/xmlps/pom.xml 1233127 trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/DBMSExecutor.java 1233127 trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/XMLPSProductHandler.java 1233127 trunk/xmlps/src/main/java/org/apache/oodt/xmlps/structs/CDEResult.java 1233127 trunk/xmlps/src/main/java/org/apache/oodt/xmlps/structs/CDEResultInputStream.java PRE-CREATION trunk/xmlps/src/test/java/org/apache/oodt/xmlps/structs/TestCDEResult.java PRE-CREATION trunk/xmlps/src/test/java/org/apache/oodt/xmlps/structs/TestCDEResultInputStream.java PRE-CREATION Diff: https://reviews.apache.org/r/3543/diff Testing --- Runs in Tomcat 7. I've used it at CHLA. NOT tested in other app servers. Which other app servers should I test? NOT tested MappingFuncs. Is it necessary? Added 2 unit tests: TestCDEResult and TestCDEResultInputStream, which both pass. Thanks, Ricky
Issue with query_tool when all product types are not populated
I first noticed this behavior in release 0.3 and just reaffirmed it in a latest
and greatest build of 0.4-SNAPSHOT. To the best of my knowledge, this was not
the case in previous versions. When querying a File Manager instance with a
Lucene Catalog on the back end, the query_tool will throw an exception unless
there is a product ingested for each product type listed in the policy (see the
stack trace below).
I assume I am not the first person to notice this behavior. Is this worthy of a
JIRA issue?
Thanks,
Sean
bash-3.2$ ./query_tool --url ${FILEMGR_URL} --sql -query SELECT * FROM *
Feb 20, 2012 5:52:26 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog
paginateQuery
WARNING: Query: [q=] for Product Type: [urn:pds:CatalogObject] returned no
results
java.lang.NullPointerException
at
org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.complexQuery(XmlRpcFileManager.java:602)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
org.apache.xmlrpc.XmlRpcException: java.lang.Exception:
org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed to
perform complex query : null
at
org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at
org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at
org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(XmlRpcFileManagerClient.java:974)
at
org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.java:252)
at org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:242)
Exception in thread main
org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException:
java.lang.Exception:
org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed to
perform complex query : null
at
org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(XmlRpcFileManagerClient.java:980)
at
org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.java:252)
at org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:242)
Issue with File Manager initiating a Lucene Catalog
Sorry for spamming the list today but I have been doing some comparative tests today with the File Manager. When populating a newly installed version (0.4-SNAPSHOT) of the File Manager, I was unable to ingest a product until I supplied an existing (yet empty of products) Lucene Catalog for the File Manager to populate. I have included the stack trace from both the client and the server below. This isn't the behavior seen in the 0.3 release. If the Lucene Catalog does not exist in 0.3, it is created upon the first file ingestion. Once again, I am not sure whether this is worthy of a JIRA issue. I am just pointing out differing behavior. Thanks, Sean Stack trace from filemgr-client: bash-3.2$ ./filemgr-client --url http://localhost:9000 --operation --ingestProduct --productName Test --productStructure Flat --productTypeName CatalogFile --metadataFile file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt.met --refs file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt Feb 20, 2012 6:49:30 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient init INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties] org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104) at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71) at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178) at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct(XmlRpcFileManagerClient.java:1162) at org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(IngestProductCliAction.java:102) at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:296) at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:179) at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFileManagerClient.java:1307) Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct SEVERE: Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) -- rolling back ingest Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct SEVERE: Failed to rollback ingest of product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) ERROR: Failed to ingest product 'Test' : Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) Stack trace from filemgr: Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for product type: [CatalogFile] Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor] for product type: [CatalogFile] Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor] for product type: [CatalogFile] Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog addCompleteProductToIndex WARNING: Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) java.io.FileNotFoundException: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) at java.io.RandomAccessFile.open(Native Method) at
Re: Review Request: XMLPS should be able to stream large results
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3543/ --- (Updated 2012-02-21 04:09:58.286025) Review request for oodt. Changes --- CDEResult.getSize() overridden to return -1. ProductQueryServlet only outputs Content-Length header when Result.getSize() = 0, in case other implementations of ProductHandler return Results that shouldn't be chunked. This is safer than always using chunked Transfer-Encoding (since the behavior is changed only for CDEResults). Summary --- See OODT-341: https://issues.apache.org/jira/browse/OODT-341 * CDEResult extends org.apache.oodt.xmlquery.Result * CDEResult mimetype is always text/plain * CDEResult size is always 0 * CDEResult inputstream is CDEResultInputStream * CDEResultInputStream has IO methods and wraps a CDEResult * CDEResult wraps a ResultSet and returns rows as Strings, applying MappingFuncs if a Mapping is provided, and appending constant fields if a List of CDEValues is provided. * ProductQueryServlet relies on servlet container to handle Content-Length (and possibly Transfer-Encoding: chunked) This addresses bug OODT-341. https://issues.apache.org/jira/browse/OODT-341 Diffs (updated) - trunk/grid/src/main/java/org/apache/oodt/grid/ProductQueryServlet.java 1291543 trunk/xmlps/pom.xml 1291543 trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/DBMSExecutor.java 1291543 trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/XMLPSProductHandler.java 1291543 trunk/xmlps/src/main/java/org/apache/oodt/xmlps/structs/CDEResult.java 1291543 trunk/xmlps/src/main/java/org/apache/oodt/xmlps/structs/CDEResultInputStream.java PRE-CREATION trunk/xmlps/src/test/java/org/apache/oodt/xmlps/structs/TestCDEResult.java PRE-CREATION trunk/xmlps/src/test/java/org/apache/oodt/xmlps/structs/TestCDEResultInputStream.java PRE-CREATION Diff: https://reviews.apache.org/r/3543/diff Testing --- Runs in Tomcat 7. I've used it at CHLA. NOT tested in other app servers. Which other app servers should I test? NOT tested MappingFuncs. Is it necessary? Added 2 unit tests: TestCDEResult and TestCDEResultInputStream, which both pass. Thanks, Ricky
Re: Issue with File Manager initiating a Lucene Catalog
Hey Sean, Looking at the stack trace below, it looks like you had an existing catalog directory, but it didn't have anything inside of it. I think you need to *first* delete the catalog directory, restart the file manager, and then try again. It should work at that point, and that was the same behavior from 0.3. Let me know if that addresses it. Cheers, Chris On Feb 20, 2012, at 6:57 PM, Hardman, Sean H (388J) wrote: Sorry for spamming the list today but I have been doing some comparative tests today with the File Manager. When populating a newly installed version (0.4-SNAPSHOT) of the File Manager, I was unable to ingest a product until I supplied an existing (yet empty of products) Lucene Catalog for the File Manager to populate. I have included the stack trace from both the client and the server below. This isn't the behavior seen in the 0.3 release. If the Lucene Catalog does not exist in 0.3, it is created upon the first file ingestion. Once again, I am not sure whether this is worthy of a JIRA issue. I am just pointing out differing behavior. Thanks, Sean Stack trace from filemgr-client: bash-3.2$ ./filemgr-client --url http://localhost:9000 --operation --ingestProduct --productName Test --productStructure Flat --productTypeName CatalogFile --metadataFile file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt.met --refs file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt Feb 20, 2012 6:49:30 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient init INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties] org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104) at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71) at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178) at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct(XmlRpcFileManagerClient.java:1162) at org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(IngestProductCliAction.java:102) at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:296) at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:179) at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFileManagerClient.java:1307) Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct SEVERE: Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) -- rolling back ingest Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct SEVERE: Failed to rollback ingest of product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) ERROR: Failed to ingest product 'Test' : Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4] : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory) Stack trace from filemgr: Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for product type: [CatalogFile] Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor] for product type: [CatalogFile] Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors INFO: Running Met Extractor:
Re: Issue with query_tool when all product types are not populated
Hey Sean,
Interesting. Can you try with 0.3 and see if it gives you a different behavior?
If it's a regression I'm happy to file an issue
and/or create a unit test for it.
Cheers,
Chris
On Feb 20, 2012, at 6:30 PM, Hardman, Sean H (388J) wrote:
I first noticed this behavior in release 0.3 and just reaffirmed it in a
latest and greatest build of 0.4-SNAPSHOT. To the best of my knowledge, this
was not the case in previous versions. When querying a File Manager instance
with a Lucene Catalog on the back end, the query_tool will throw an exception
unless there is a product ingested for each product type listed in the policy
(see the stack trace below).
I assume I am not the first person to notice this behavior. Is this worthy of
a JIRA issue?
Thanks,
Sean
bash-3.2$ ./query_tool --url ${FILEMGR_URL} --sql -query SELECT * FROM *
Feb 20, 2012 5:52:26 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog
paginateQuery
WARNING: Query: [q=] for Product Type: [urn:pds:CatalogObject] returned no
results
java.lang.NullPointerException
at
org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.complexQuery(XmlRpcFileManager.java:602)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
org.apache.xmlrpc.XmlRpcException: java.lang.Exception:
org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed to
perform complex query : null
at
org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at
org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at
org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(XmlRpcFileManagerClient.java:974)
at
org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.java:252)
at org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:242)
Exception in thread main
org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException:
java.lang.Exception:
org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed to
perform complex query : null
at
org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(XmlRpcFileManagerClient.java:980)
at
org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.java:252)
at org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:242)
++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++
Re: Issue with query_tool when all product types are not populated
Hey Sean,
Try using the new SqlQuery action in 0.4 filemgr
-Brian
On Feb 20, 2012, at 6:30 PM, Hardman, Sean H (388J)
sean.h.hard...@jpl.nasa.gov wrote:
I first noticed this behavior in release 0.3 and just reaffirmed it in a
latest and greatest build of 0.4-SNAPSHOT. To the best of my knowledge, this
was not the case in previous versions. When querying a File Manager instance
with a Lucene Catalog on the back end, the query_tool will throw an exception
unless there is a product ingested for each product type listed in the policy
(see the stack trace below).
I assume I am not the first person to notice this behavior. Is this worthy of
a JIRA issue?
Thanks,
Sean
bash-3.2$ ./query_tool --url ${FILEMGR_URL} --sql -query SELECT * FROM *
Feb 20, 2012 5:52:26 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog
paginateQuery
WARNING: Query: [q=] for Product Type: [urn:pds:CatalogObject] returned no
results
java.lang.NullPointerException
at
org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.complexQuery(XmlRpcFileManager.java:602)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
at java.lang.Thread.run(Thread.java:680)
org.apache.xmlrpc.XmlRpcException: java.lang.Exception:
org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed to
perform complex query : null
at
org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
at
org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
at
org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(XmlRpcFileManagerClient.java:974)
at
org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.java:252)
at org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:242)
Exception in thread main
org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException:
java.lang.Exception:
org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed to
perform complex query : null
at
org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery(XmlRpcFileManagerClient.java:980)
at
org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.java:252)
at org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:242)
