Re: recovering potentially corrupt data

[Rohit Mathur]

Hi Todd Below is the exception i see in catalina.out . However, i get this exception when i try to access the globalJobs1 through the db browser on my internet browser. I have been executing the export command using the core xindice install (i.e. not the webapp install) I think its the same error though, occurs when parsing through the colleciton. I tried to run xindice.sh in the webapps/xindice/WEB-INF dir for exporting, but couldnt get it to work. after setting JAVA_HOME, export XINDICE_HOME=/software/jakarta-tomcat-5.0.30/webapps/xindice/WEB-INF/ [EMAIL PROTECTED] WEB-INF]# ./xindice.sh export -v -c  xmldb:xindice://localhost:8080/db -n globalJobs1 -f globalJobs1 bash: ./xindice.sh: Permission denied and after changing permissions [EMAIL PROTECTED] WEB-INF]# ./xindice.sh export -v -c  xmldb:xindice://localhost:8080/db -n globalJobs1 -f globalJobs1 : bad interpreter: No such file or directory regards, Rohit hope this is good enough May 16, 2006 2:56:40 PM org.apache.xindice.core.filer.BTree getBTreeNode WARNING: Ignored exception java.io.EOFException         at java.io.DataInputStream.readShort(DataInputStream.java:377)         at org.apache.xindice.core.filer.BTree$BTreeNode.read(BTree.java:495)         at org.apache.xindice.core.filer.BTree.getBTreeNode(BTree.java:372)         at org.apache.xindice.core.filer.BTree.access$800(BTree.java:71)         at org.apache.xindice.core.filer.BTree$BTreeNode.getChildNode(BTree.java:536)         at org.apache.xindice.core.filer.BTree$BTreeNode.query(BTree.java:937)         at org.apache.xindice.core.filer.BTree$BTreeNode.query(BTree.java:937)         at org.apache.xindice.core.filer.BTree.query(BTree.java:221)         at org.apache.xindice.core.filer.BTreeFiler$BTreeFilerRecordSet.<init>(BTreeFiler.java:191)         at org.apache.xindice.core.filer.BTreeFiler.getRecordSet(BTreeFiler.java:174)         at org.apache.xindice.core.Collection.listDocuments(Collection.java:1007)         at org.apache.xindice.server.UglyBrowser.getHierarchy(UglyBrowser.java:157)         at org.apache.xindice.server.UglyBrowser.doGet(UglyBrowser.java:78)         at org.apache.xindice.server.XindiceServlet.doGet(XindiceServlet.java:82)         at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)         at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)         at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252 )         at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)         at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:214)         at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)         at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)         at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)         at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)         at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)         at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)         at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)         at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)         at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)         at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)         at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)         at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)         at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)         at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)         at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)         at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)         at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)         at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol .java:705)         at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)         at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)         at java.lang.Thread.run(Thread.java:534) Todd Byrne wrote: Hi Rohit, So I need you to run that export command again and send me the expection thats going to be printed in your catalina.log . It should look like : Ignored exception (stack trace) If I can get that stack trace maybe I can write up a patch and you can still recover your data. Todd Rohit Mathur wrote: Hi Todd, sorry about that last message, since i saw the error i was expecting, i didnt think to check the export command syntax. xindice.sh export -v -c xmldb:xindice://localhost:8080/db -n globalJobs1 -f globalJobs1 trying to register database Creating directory globalJobs1/db Extracting 0 files from xmldb:xindice://localhost:8080/db Creating directory globalJobs1/db/products Extracting 0 files from xmldb:xindice://localhost:8080/db/products ............... Creating directory globalJobs1/db/site Extracting 1 files from xmldb:xindice://localhost:8080/db/site Creating directory globalJobs1/db/jobs Extracting 0 files from xmldb:xindice://localhost:8080/db/jobs Creating directory globalJobs1/db/jobs/globalJobs1 XMLDB Exception 0: Failed to execute command 'ListDocuments' on server: http://localhost:8080/xindice/,message: java.lang.Exception: java.lang.NullPointerException org.xmldb.api.base.XMLDBException: Failed to execute command 'ListDocuments' on server: http://localhost:8080/xindice/, message: java.lang.Exception: java.lang.NullPointerException at org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:124) at org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488) ..... Caused by: org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java) ... 8 more [[EMAIL PROTECTED]: java.lang.Exception: java.lang.NullPointerException .... at org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:148) at org.apache.xindice.tools.command.ExportTree.execute(ExportTree.java:71) at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293) at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171) at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82) and listing the document is where i encountered the null pointer error in the first place. I am able to list other documents/collections in this db xindice.sh ld -v -c xmldb:xindice://localhost:8080/db/jobs/globalJobs1 trying to register database XMLDB Exception 0: Failed to execute command 'ListDocuments' on server: http://localhost:8080/xindice/,message: java.lang.Exception: java.lang.NullPointerException org.xmldb.api.base.XMLDBException: Failed to execute command 'ListDocuments' on server: http://localhost:8080/xindice/, message: java.lang.Exception: java.lang.NullPointerException at org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:124) at org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488) at org.apache.xindice.tools.command.ListCollectionDocuments.execute(ListCollectionDocuments.java:55) at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293) at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171) at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82) Caused by: org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java) at org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:112) ... 5 more [[EMAIL PROTECTED]: java.lang.Exception: java.lang.NullPointerException at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java) at org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:112) at org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488) at org.apache.xindice.tools.command.ListCollectionDocuments.execute(ListCollectionDocuments.java:55) at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293) at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171) at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82) hope this helps/is fixable thanks a lot for your quick responses. i'm new to this project, and i havent worked with xindice before so i'm all at sea here. Rohit Todd Byrne wrote: I wasn't quite clear but try this: xindice.sh export -v -c xmldb:xindice://localhost:8080/db -n globalJobs1 -f globalJobs1 Also try listing the documents from database and retriving them one by one. $XINDICE_HOME/bin/xindice ld -c xmldb:xindice://localhost:8080/db/globalJobs1 Todd Rohit Mathur wrote: Hi Todd, So the original db was running off an application that was a wrapper around xindice, which is why i didnt want to send details. I have since copied the db over to another machine where I'm running the xindice 1.1b4. I tried what you recommended on a subset of the db (i.e. db/jobs). Here is the error i get $ xindice.sh export -v -c xmldb:xindice://localhost:8080/db/jobs -n test -f test trying to register database Creating directory test/jobs Extracting 0 files from xmldb:xindice://localhost:8080/db/jobs Creating directory test/jobs/globalJobs1 XMLDB Exception 0: Failed to execute command 'ListDocuments' on server: http://localhost:8080/xindice/,message: java.lang.Exception: java.lang.NullPointerException org.xmldb.api.base.XMLDBException: Failed to execute command 'ListDocuments' on server: http://localhost:8080/xindice/, message: java.lang.Exception: java.lang.NullPointerException at org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:124) at org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488) at org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:111) at org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:148) at org.apache.xindice.tools.command.ExportTree.execute(ExportTree.java:71) at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293) at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171) at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82) Caused by: org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.NullPointerException at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java) at org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:112) ... 7 more [[EMAIL PROTECTED]: java.lang.Exception: java.lang.NullPointerException at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java) at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java) at org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:112) at org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488) at org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:111) at org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:148) at org.apache.xindice.tools.command.ExportTree.execute(ExportTree.java:71) at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293) at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171) at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82) thanks, Rohit P.S. I am able to add/remove collections from db/jobs, but not from db/globalJobs1 [EMAIL PROTECTED] wrote: You could try running xindice command line tool like this to export the data and then reload it. $XINDICE_HOME/bin/xindice export -c xmldb:xindice://localhost:8080/db/ -n test -f test Maybe I can work with you to narrow down where this null pointer is coming from. Are you using the latest release? Todd On Fri, 12 May 2006, Rohit Mathur wrote: Hi, Is there any utility out there that can recover the data from a collection. Is it at all possible to do this at all? One particular collection has "gone bad". The rest of the db is working great. I am unable to query it or create a collection within it. I get a null pointer exception from the data. thanks Rohit java.lang.NullPointerException at org.apache.xindice.core.filer.BTree$BTreeNode.query(BTree.java:937) at org.apache.xindice.core.filer.BTree$BTreeNode.query(BTree.java:937) at org.apache.xindice.core.filer.BTree.query(BTree.java:221) at org.apache.xindice.core.filer.BTreeFiler$BTreeFilerRecordSet.(BTreeFiler.java:191) at org.apache.xindice.core.filer.BTreeFiler.getRecordSet(BTreeFiler.java:174) at org.apache.xindice.core.Collection.listDocuments(Collection.java:1007) at org.apache.xindice.server.UglyBrowser.getHierarchy(UglyBrowser.java:157) at org.apache.xindice.server.UglyBrowser.doGet(UglyBrowser.java:78) at org.apache.xindice.server.XindiceServlet.doGet(XindiceServlet.java:82) at javax.servlet.http.HttpServlet.service(HttpServlet.java:740) at javax.servlet.http.HttpServlet.service(HttpServlet.java:853) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:260) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2415) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643) at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:170) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:509) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174) at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:223) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:432) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:386) at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:534) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:530) at java.lang.Thread.run(Thread.java:534)