Hi All, I'm only just now "re-discovering" this thread, as I've found this same behavior in DSpace OAI-PMH servers. Here's how to see this behavior (and I've double checked it on http://demo.dspace.org, running 5.2):
1. Create an Item 2. Run the OAI-PMH import (./dspace oai import). 3. The item now appears in OAI-PMH 4. Withdraw that Item 5. Re-Run the OAI-PMH import (./dspace oai import) 6. The Item still appears in OAI-PMH and is NEVER flagged as withdrawn. So, you can still access all its metadata, etc. The ONLY way to trigger an update to the OAI-PMH record (of the withdrawn item) is to re-import EVERYTHING (./dspace oai import -c). However, as noted, since withdrawn items are not included in the XOAI "indexAll()" command, the item will now disappear from OAI-PMH entirely. Here's that indexAll() command: https://github.com/DSpace/DSpace/blob/master/dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java#L191 This behavior seems to be counter to what is documented at: https://wiki.duraspace.org/pages/viewpage.action?pageId=45548245#OAI-PMHDataProvider2.0%28Internals%29-Deletions The documentation specifically states that "DSpace keeps a permanent record of withdrawn items". It also states that a request for a withdrawn item "will yield the 'record deleted' header". This sounds like a bug to me. The documentation definitely does not match with the behavior. I've created a new bug ticket for this. It will need a volunteer to resolve. https://jira.duraspace.org/browse/DS-2593 - Tim On 3/6/2015 5:21 AM, helix84 wrote: > Yes, that makes sense. Still, it needs a brief verification because > sometimes our understanding of a concept and its implementation may > differ. > > > On Fri, Mar 6, 2015 at 10:12 AM, Kristian Roberto Salcedo > <[email protected]> wrote: >> Hi Ivan, >> >> I might have overlooked something, but from your comments on >> https://jira.duraspace.org/browse/DS-2491 wouldnt it be meaningful >> to do both your declaration change and Ondřej's suggestion in order to match >> both what is declared and what the documentation says about OAI and deleted >> items? >> >> regards, >> Kristian >> >>> -----Original Message----- >>> From: [email protected] [mailto:[email protected]] On Behalf Of >>> helix84 >>> Sent: Thursday, March 05, 2015 3:46 PM >>> To: Kristian Roberto Salcedo >>> Cc: [email protected]; João Melo >>> Subject: Re: [Dspace-tech] OAI-PMH data provider 2.0 not persistent? >>> >>> Hi Kristian, >>> >>> I think you're right that the declared >>> <deletedRecord>persistent</deletedRecord> doesn't match how DSpace >>> behaves. I filed a Jira issue and created a pull request to change the >>> declared >>> status to <deletedRecord>transient</deletedRecord>. >>> >>> https://jira.duraspace.org/browse/DS-2491 >>> >>> >>> Regards, >>> ~~helix84 >>> >>> Compulsory reading: DSpace Mailing List Etiquette >>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >>> >>> >>> On Thu, Mar 5, 2015 at 2:52 PM, Kristian Roberto Salcedo >>> <[email protected]> wrote: >>>> Hi all, >>>> >>>> In Dspace 4.2 we are currently seeing that our OAI feeds are not being >>> persistent as per these definitions when it comes to deleted (withdrawn) >>> items: >>>> >>>> http://www.openarchives.org/OAI/openarchivesprotocol.html#deletion >>>> https://wiki.duraspace.org/pages/viewpage.action?pageId=34640887#OAI- >>> P >>>> MHDataProvider2.0(Internals)-Deletions >>>> >>>> Withdrawn items do not get the <header status=deleted> as they should, >>> like in this example: >>>> >>>> >>> http://webservices.itcs.umich.edu/mediawiki/oaibp/index.php/Deleted_Re >>>> cord_Example_1 >>>> >>>> Withdrawing an item is not reflected in the OAI entry in any way. >>>> >>>> ---------------------------------------------- >>>> >>>> This is our Identify page at the moment: >>>> >>>> https://www.duo.uio.no/oai/request?verb=Identify >>>> >>>> ---------------------------------------------- >>>> >>>> I believe we're doing everything right regarding config parameters and >>> maintenance of the solr oai index. >>>> >>>> The only place I can find a "persistent"-parameter is in this config file: >>>> /www/var/data/dspace/config/oaicat.properties >>>> which I thought was deprecated, but we still set it just to be sure: >>>> Identify.deletedRecord=persistent >>>> >>>> All other relevant config parameters are set in these two files as far as >>>> I can >>> tell: >>>> >>>> /www/var/data/dspace/config/modules/oai.cfg >>>> /www/var/data/dspace/config/crosswalks/oai/description.xml >>>> >>>> We are running the oai import -o command nightly. >>>> >>>> ---------------------------------------------- >>>> >>>> The only way we've found to remove withdrawn items from the OAI feed is >>> by running a complete re-indexing of the oai solr index: >>>> >>>> /www/var/data/dspace/bin/dspace oai import -c >>>> >>>> with a subsequent >>>> >>>> /www/var/data/dspace/bin/dspace oai clean-cache >>>> >>>> This removes a withdrawn item: >>>> >>>> https://www.duo.uio.no/handle/10852/42670 >>>> >>>> completely from the feed: >>>> >>>> >>> https://www.duo.uio.no/oai/request?verb=GetRecord&metadataPrefix=oai >>> _d >>>> c&identifier=oai:localhost:10852/42670 >>>> >>>> which is not what we want... >>>> >>>> ---------------------------------------------- >>>> >>>> >>>> Is anyone else experiencing the same problem? >>>> >>>> Maybe I've missed something - If this actually works differently than we >>> expect or has been addressed in OAI 2.1, please let me know. >>>> >>>> >>>> regards, >>>> Kristian Salcedo >>>> Universitetet of Oslo Library >>>> Department of digital services >>>> >>>> >>>> ---------------------------------------------------------------------- >>>> -------- Dive into the World of Parallel Programming The Go Parallel >>>> Website, sponsored by Intel and developed in partnership with Slashdot >>>> Media, is your hub for all things parallel software development, from >>>> weekly thought leadership blogs to news, videos, case studies, >>>> tutorials and more. Take a look and join the conversation now. >>>> http://goparallel.sourceforge.net/ >>>> _______________________________________________ >>>> DSpace-tech mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech >>>> List Etiquette: >>>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming The Go Parallel Website, sponsored > by Intel and developed in partnership with Slashdot Media, is your hub for all > things parallel software development, from weekly thought leadership blogs to > news, videos, case studies, tutorials and more. Take a look and join the > conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > DSpace-tech mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/dspace-tech > List Etiquette: > https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette > ------------------------------------------------------------------------------ _______________________________________________ DSpace-tech mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
