Hi,
I am doing a test ingest of ~ 100 pairs of objects. The first object of the
pairs of objects ingest successfully, but the second object of the pairs of
objects have about a 40% error rate randomly occurring.
The second objects of the pairs (having thing 40% error rate) have a
DC,RELS-EXT, two inline xml datastreams, and a managed tiff datastream.
When ingesting one of the objects that failed in batch on its own with the
admin client, it was successfully ingested.
Pasted below are portions of the fedora.log for a successfully ingested object,
and one that fails. What I don't understand with the one that fails is how 3ms
after "committing addition" of pid there is a "committing removal" of that same
pid.
Perhaps someone has a solution or can clarify what is happening?
Thanks, Eric
----------------
successful
INFO 2009-07-02 16:19:03.248 [http-8032-Processor25] (DefaultDOManager) New
object PID is walpole:256
INFO 2009-07-02 16:19:03.259 [http-8032-Processor25] (DefaultDOManager)
Committing addition of walpole:256
INFO 2009-07-02 16:19:03.263 [http-8032-Processor25] (DefaultDOManager) Getting
managed datastream from remote location:
http://dpres-dev.library.yale.edu:8032/dpres/RR_Walpole_Images_Bag_lwlpr00348/data/Component_lwlpr00348/lwlpr00348.tif
INFO 2009-07-02 16:19:03.975 [http-8032-Processor25] (DefaultDOManager)
Replaced managed datastream location with internal id:
walpole:256+DPRES_FILEID5+DPRES_FILEID5.0
INFO 2009-07-02 16:19:03.976 [http-8032-Processor25] (DefaultDOManager) Adding
to ResourceIndex
INFO 2009-07-02 16:19:03.986 [http-8032-Processor25] (DefaultDOManager)
Updating dissemination index
INFO 2009-07-02 16:19:03.989 [http-8032-Processor25] (DefaultDOManager)
Updating FieldSearch index
INFO 2009-07-02 16:19:03.991 [http-8032-Processor25] (DefaultManagement)
Completed ingest(objectXML, format: info:fedora/fedora-system:FOXML-1.0,
encoding: UTF-8, newPid: true, logMessage: )
error
INFO 2009-07-02 16:19:08.185 [http-8032-Processor11] (DefaultDOManager) New
object PID is walpole:258
INFO 2009-07-02 16:19:08.197 [http-8032-Processor11] (DefaultDOManager)
Committing addition of walpole:258
INFO 2009-07-02 16:19:08.200 [http-8032-Processor11] (DefaultDOManager)
Committing removal of walpole:258
INFO 2009-07-02 16:19:08.200 [http-8032-Processor11] (DefaultDOManager)
Deleting managed datastream: walpole:258+DPRES_FILEID6+DPRES_FILEID6.0
WARN 2009-07-02 16:19:08.201 [http-8032-Processor11] (DefaultDOManager) Error
attempting removal of managed content datastream:
fedora.server.errors.ObjectNotInLowlevelStorageException: no path in db
registry for [walpole:258+DPRES_FILEID6+DPRES_FILEID6.0]
at fedora.server.storage.lowlevel.DBPathRegistry.get(DBPathRegistry.java:75)
at
fedora.server.storage.lowlevel.DefaultLowlevelStorage$Store.remove(DefaultLowlevelStorage.java:328)
at
fedora.server.storage.lowlevel.DefaultLowlevelStorage.removeDatastream(DefaultLowlevelStorage.java:113)
at
fedora.server.storage.lowlevel.DefaultLowlevelStorageModule.removeDatastream(DefaultLowlevelStorageModule.java:163)
at
fedora.server.storage.DefaultDOManager.doCommit(DefaultDOManager.java:1065)
at
fedora.server.storage.DefaultDOManager.doCommit(DefaultDOManager.java:1429)
at fedora.server.storage.SimpleDOWriter.commit(SimpleDOWriter.java:443)
at
fedora.server.management.DefaultManagement.ingest(DefaultManagement.java:152)
at sun.reflect.GeneratedMethodAccessor54.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
fedora.server.messaging.NotificationInvocationHandler.invoke(NotificationInvocationHandler.java:70)
at $Proxy0.ingest(Unknown Source)
at
fedora.server.management.ManagementModule.ingest(ManagementModule.java:338)
at
fedora.server.management.FedoraAPIMBindingSOAPHTTPImpl.ingest(FedoraAPIMBindingSOAPHTTPImpl.java:89)
at
fedora.server.management.FedoraAPIMBindingSOAPHTTPSkeleton.ingest(FedoraAPIMBindingSOAPHTTPSkeleton.java:342)
at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)
at
org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)
at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
at
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:453)
at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at
org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:525)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:595)
WARN 2009-07-02 16:19:08.203 [http-8032-Processor11] (DefaultDOManager) Error
while cleaning up after failed add
fedora.server.errors.ObjectNotInLowlevelStorageException: no path in db
registry for [walpole:258]
at fedora.server.storage.lowlevel.DBPathRegistry.get(DBPathRegistry.java:75)
at
fedora.server.storage.lowlevel.DefaultLowlevelStorage$Store.retrieve(DefaultLowlevelStorage.java:295)
at
fedora.server.storage.lowlevel.DefaultLowlevelStorage.retrieveObject(DefaultLowlevelStorage.java:82)
at
fedora.server.storage.lowlevel.DefaultLowlevelStorageModule.retrieveObject(DefaultLowlevelStorageModule.java:132)
at
fedora.server.storage.DefaultDOManager.doCommit(DefaultDOManager.java:1080)
at
fedora.server.storage.DefaultDOManager.doCommit(DefaultDOManager.java:1429)
at fedora.server.storage.SimpleDOWriter.commit(SimpleDOWriter.java:443)
at
fedora.server.management.DefaultManagement.ingest(DefaultManagement.java:152)
at sun.reflect.GeneratedMethodAccessor54.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
fedora.server.messaging.NotificationInvocationHandler.invoke(NotificationInvocationHandler.java:70)
at $Proxy0.ingest(Unknown Source)
at
fedora.server.management.ManagementModule.ingest(ManagementModule.java:338)
at
fedora.server.management.FedoraAPIMBindingSOAPHTTPImpl.ingest(FedoraAPIMBindingSOAPHTTPImpl.java:89)
at
fedora.server.management.FedoraAPIMBindingSOAPHTTPSkeleton.ingest(FedoraAPIMBindingSOAPHTTPSkeleton.java:342)
at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)
at
org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)
at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
at
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:453)
at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at
org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:525)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:595)
INFO 2009-07-02 16:19:08.203 [http-8032-Processor11] (DefaultManagement)
Completed ingest(objectXML, format: info:fedora/fedora-system:FOXML-1.0,
encoding: UTF-8, newPid: true, logMessage: )
ERROR 2009-07-02 16:19:08.204 [http-8032-Processor11]
(FedoraAPIMBindingSOAPHTTPImpl) Error ingesting
fedora.server.errors.GeneralException: Error getting
http://dpres-dev.library.yale.edu:8032/dpres/RR_Walpole_Images_Bag_lwlpr00052/data/Component_lwlpr00052/lwlpr00052.tif
at
fedora.server.storage.DefaultExternalContentManager.get(DefaultExternalContentManager.java:129)
at
fedora.server.storage.DefaultExternalContentManager.getExternalContent(DefaultExternalContentManager.java:202)
at
fedora.server.storage.DefaultDOManager.doCommit(DefaultDOManager.java:1209)
at fedora.server.storage.SimpleDOWriter.commit(SimpleDOWriter.java:443)
at
fedora.server.management.DefaultManagement.ingest(DefaultManagement.java:152)
at sun.reflect.GeneratedMethodAccessor54.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
fedora.server.messaging.NotificationInvocationHandler.invoke(NotificationInvocationHandler.java:70)
at $Proxy0.ingest(Unknown Source)
at
fedora.server.management.ManagementModule.ingest(ManagementModule.java:338)
at
fedora.server.management.FedoraAPIMBindingSOAPHTTPImpl.ingest(FedoraAPIMBindingSOAPHTTPImpl.java:89)
at
fedora.server.management.FedoraAPIMBindingSOAPHTTPSkeleton.ingest(FedoraAPIMBindingSOAPHTTPSkeleton.java:342)
at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)
at
org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)
at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
at
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:453)
at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at
org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
fedora.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:234)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:525)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.IOException: Request failed [404
/dpres/RR_Walpole_Images_Bag_lwlpr00052/data/Component_lwlpr00052/lwlpr00052.tif]
at fedora.common.http.WebClient.get(WebClient.java:178)
at fedora.common.http.WebClient.get(WebClient.java:102)
at
fedora.server.storage.DefaultExternalContentManager.get(DefaultExternalContentManager.java:121)
... 57 more
------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge
This is your chance to win up to $100,000 in prizes! For a limited time,
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
Fedora-commons-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
