Hi, Chris -- Yes, I had hoped that I could find the java.nio.file package for Java 1.6, but no such luck.
I did download the latest Akubra from trunk, and I made some modifications to FSBlob to implement a fairly failsafe copy-and-delete function within moveTo(), which I'm now testing. I do have a few developer administrivia questions for you, which I'll ask offline. thanks, -- Scott Chris Wilper wrote: > Hi Scott, > > Good find. Yes, please submit it to the Akubra tracker. It appears > that java.nio.file.Path is a Java 1.7+ class, so it doesn't look like > we can use it, but I'm sure we can muster up a suitable 1.6-compatible > workaround. > > - Chris > > On Tue, Oct 5, 2010 at 5:03 PM, Scott Prater <[email protected]> wrote: >> I think I may have found the source of the problem. >> >> In org.akubraproject.fs.FSBlob, this is what happens when an object gets >> moved: >> >> if (!file.renameTo(other)) { >> if (!file.exists()) >> throw new MissingBlobException(getId()); >> >> throw new IOException("Rename failed for an unknown reason."); >> } >> >> file.renameTo() is a java.io.File method that may or may not move files >> across symlinks and filesystems, depending on your platform (I'm running >> Solaris 10, so go figure): >> >> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4073756 >> >> Googling turns up a rich history of problems with the java.io.File >> renameTo() method when other filesystems and symlinks are involved. Best >> practice these days seems to be to use java.nio.file instead: >> >> http://download.oracle.com/javase/tutorial/essential/io/legacy.html >> >> Chris: shall I open to a bug report in the Akubra JIRA issue tracker? >> >> -- Scott >> >> On 10/05/10, "Scott Prater" <[email protected]> wrote: >> >>> Another item of information: Ingesting FOXML files works without a >>> problem, induicating that this may be a problem with the copy/replace >>> functionality? >>> >>> Also, I simplified the process somewhat: really, the new data is spread >>> across three different filesystems, with the symlinks occuring at the >>> top-level hash character directories: >>> >>> /fedora/data/objects/0 -> /fedora/db01/objects/0 >>> /fedora/data/objects/1 -> /fedora/db02/objects/1 >>> etc. >>> >>> I mention this, as it occurs to me that perhaps Akubra, as part of its safe >>> move/rename function, is attempting to create temporary hard links across >>> filesystems? Just a guess... >>> >>> -- Scott >>> >>> On 10/05/10, "Scott Prater" <[email protected]> wrote: >>> >>>> We encountered a worrisome problem today after filling up a filesystem and >>>> being forced to move our data to another filesystem. After the move and >>>> rebuild, we can no longer add datastreams to objects. >>>> >>>> I'm running Fedora 3.4, with the default Akubra low-level storage module. >>>> >>>> Here's what happened: >>>> >>>> 1. filesystem /fedora/data filled up, Fedora (and Tomcat) hung. >>>> 2. I shut down Fedora, Tomcat, Postgres. >>>> 3. I moved the directory /fedora/data/objects to another filesystem, >>>> /fedora/db01/objects. >>>> 4. I moved the directory /fedora/data/datastreams to another filesystem, >>>> /fedora/db01/datastreams. >>>> 5. I made symlinks from the original locations to the new locations: >>>> /fedora/data/objects -> /fedora/db01/objects >>>> /fedora/data/datastreams -> /fedora/db01/datastreams >>>> 6. I started up Postgres, and ran fedora-rebuild.sh twice, once to >>>> rebuild the database and once to rebuild the resource index. The rebuild >>>> completed successfully. >>>> 7. I then started up Tomcat + Fedora, and I was able to view all the >>>> objects. >>>> 8. I then did a test ingest of an object, and a stub object was >>>> successfully created, with a DC datastream. I verified that I could find >>>> the object and the datastream on the filesystem (both through the direct >>>> path and through the symlinked path). I verified that permsissions were >>>> correct at every directory level, and at the file level. >>>> >>>> I then try to add a datastream to the test object, and I receive this >>>> error (truncated): >>>> >>>> ERROR 2010-10-05 14:51:16.424 [http-9090-1] (WebApplicationImpl) Internal >>>> server error >>>> javax.ws.rs.WebApplicationException: >>>> org.fcrepo.server.errors.GeneralException: Unable to add or modify object >>>> (commit canceled) >>>> [...] >>>> Caused by: java.io.IOException: Rename failed for an unknown reason. >>>> at org.akubraproject.fs.FSBlob.moveTo(FSBlob.java:151) >>>> [akubra-fs-0.3.jar:na] >>>> >>>> I've tested this with the REST API, through the web admin interface, and >>>> through the java GUI admin tool, and the results are the same in all three >>>> cases. This error occurs regardless of whether I'm adding a datastream to >>>> a newly-created object or attempting to add a datastream to an older >>>> object. >>>> >>>> This may properly be an Akubra matter, but I thought I would bring it up >>>> here, first. Any suggestions or help would be greatly appreciated. >>>> >>>> thanks, >>>> >>>> -- Scott >>>> >>>> Following is the full stack trace: >>>> >>>> ERROR 2010-10-05 14:51:16.424 [http-9090-1] (WebApplicationImpl) Internal >>>> server error >>>> javax.ws.rs.WebApplicationException: >>>> org.fcrepo.server.errors.GeneralException: Unable to add or modify object >>>> (commit canceled) >>>> at >>>> org.fcrepo.server.rest.BaseRestResource.handleException(BaseRestResource.java:168) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.fcrepo.server.rest.DatastreamResource.addOrUpdateDatastream(DatastreamResource.java:583) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.fcrepo.server.rest.DatastreamResource.addDatastream(DatastreamResource.java:358) >>>> [fcrepo-server-3.4.jar:na] >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> [na:1.6.0_21] >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>>> [na:1.6.0_21] >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>> [na:1.6.0_21] >>>> at java.lang.reflect.Method.invoke(Method.java:597) [na:1.6.0_21] >>>> at >>>> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:175) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at >>>> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:67) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at >>>> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:163) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at >>>> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at >>>> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:71) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at >>>> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at >>>> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:63) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at >>>> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:689) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at >>>> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:647) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at >>>> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:638) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at >>>> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:309) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at >>>> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:425) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at >>>> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:590) >>>> [jersey-bundle-1.0.3.1.jar:1.0.3.1] >>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) >>>> [servlet-api.jar:na] >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.fcrepo.server.security.servletfilters.FilterRestApiFlash.doFilter(FilterRestApiFlash.java:66) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.fcrepo.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:235) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.fcrepo.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:235) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.fcrepo.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:235) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.fcrepo.server.security.servletfilters.FilterSetup.doFilter(FilterSetup.java:235) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:563) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.valves.RequestFilterValve.process(RequestFilterValve.java:269) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.valves.RemoteAddrValve.invoke(RemoteAddrValve.java:81) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) >>>> [catalina.jar:6.0.29] >>>> at >>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) >>>> [tomcat-coyote.jar:6.0.29] >>>> at >>>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) >>>> [tomcat-coyote.jar:6.0.29] >>>> at >>>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) >>>> [tomcat-coyote.jar:6.0.29] >>>> at java.lang.Thread.run(Thread.java:619) [na:1.6.0_21] >>>> Caused by: org.fcrepo.server.errors.GeneralException: Unable to add or >>>> modify object (commit canceled) >>>> at >>>> org.fcrepo.server.storage.DefaultDOManager.doCommit(DefaultDOManager.java:1438) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.fcrepo.server.storage.SimpleDOWriter.commit(SimpleDOWriter.java:508) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.fcrepo.server.management.DefaultManagement.addDatastream(DefaultManagement.java:572) >>>> [fcrepo-server-3.4.jar:na] >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> [na:1.6.0_21] >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>>> [na:1.6.0_21] >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>> [na:1.6.0_21] >>>> at java.lang.reflect.Method.invoke(Method.java:597) [na:1.6.0_21] >>>> at >>>> org.fcrepo.server.messaging.NotificationInvocationHandler.invoke(NotificationInvocationHandler.java:68) >>>> [fcrepo-server-3.4.jar:na] >>>> at $Proxy0.addDatastream(Unknown Source) [na:na] >>>> at >>>> org.fcrepo.server.management.ManagementModule.addDatastream(ManagementModule.java:227) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.fcrepo.server.rest.DatastreamResource.addOrUpdateDatastream(DatastreamResource.java:467) >>>> [fcrepo-server-3.4.jar:na] >>>> ... 50 common frames omitted >>>> Caused by: org.fcrepo.common.FaultException: java.io.IOException: Rename >>>> failed for an unknown reason. >>>> at >>>> org.fcrepo.server.storage.lowlevel.akubra.AkubraLowlevelStorage.rename(AkubraLowlevelStorage.java:363) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.fcrepo.server.storage.lowlevel.akubra.AkubraLowlevelStorage.safeOverwrite(AkubraLowlevelStorage.java:306) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.fcrepo.server.storage.lowlevel.akubra.AkubraLowlevelStorage.replace(AkubraLowlevelStorage.java:244) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.fcrepo.server.storage.lowlevel.akubra.AkubraLowlevelStorage.replaceObject(AkubraLowlevelStorage.java:150) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.fcrepo.server.storage.lowlevel.akubra.AkubraLowlevelStorageModule.replaceObject(AkubraLowlevelStorageModule.java:79) >>>> [fcrepo-server-3.4.jar:na] >>>> at >>>> org.fcrepo.server.storage.DefaultDOManager.doCommit(DefaultDOManager.java:1334) >>>> [fcrepo-server-3.4.jar:na] >>>> ... 60 common frames omitted >>>> Caused by: java.io.IOException: Rename failed for an unknown reason. >>>> at org.akubraproject.fs.FSBlob.moveTo(FSBlob.java:151) >>>> [akubra-fs-0.3.jar:na] >>>> at >>>> org.akubraproject.map.IdMappingBlob.moveTo(IdMappingBlob.java:69) >>>> [akubra-map-0.3.jar:na] >>>> at >>>> org.fcrepo.server.storage.lowlevel.akubra.AkubraLowlevelStorage.rename(AkubraLowlevelStorage.java:361) >>>> [fcrepo-server-3.4.jar:na] >>>> ... 65 common frames omitted >>>> -- >>>> Scott Prater >>>> Library, Instructional, and Research Applications (LIRA) >>>> Division of Information Technology (DoIT) >>>> University of Wisconsin - Madison >>>> [email protected] >> ------------------------------------------------------------------------------ >> Beautiful is writing same markup. Internet Explorer 9 supports >> standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. >> Spend less time writing and rewriting code and more time creating great >> experiences on the web. Be a part of the beta today. >> http://p.sf.net/sfu/beautyoftheweb >> _______________________________________________ >> Fedora-commons-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users >> -- Scott Prater Library, Instructional, and Research Applications (LIRA) Division of Information Technology (DoIT) University of Wisconsin - Madison [email protected] ------------------------------------------------------------------------------ Beautiful is writing same markup. Internet Explorer 9 supports standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. Spend less time writing and rewriting code and more time creating great experiences on the web. Be a part of the beta today. http://p.sf.net/sfu/beautyoftheweb _______________________________________________ Fedora-commons-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
