[
https://jira.duraspace.org/browse/DS-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20330#action_20330
]
Robin Taylor commented on DS-866:
---------------------------------
Can I retract that last comment ?! Looking at the code provided with DS-829 I
think it may exacerbate the problems with Resumption Tokens.
> OAI requests with resumption token start at the wrong offset if non-public
> items are included and there are withdrawn items
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: DS-866
> URL: https://jira.duraspace.org/browse/DS-866
> Project: DSpace
> Issue Type: Bug
> Components: DSpace API
> Affects Versions: 1.6.1, 1.6.2, 1.7.0, 1.7.1
> Reporter: Andrea Schweer
> Priority: Major
> Fix For: 1.8.0
>
> Attachments: oai-offset.patch
>
>
> In some circumstances, items are not disseminated via OAI.
> OAI responds to listRecords requests with batches of up to 100 (or other
> number set via oai.didl.maxresponse property) records. Requests can include a
> resumption token to specify which batch is required. The resumption token is
> then translated into an offset to the response from the database.
> OAI listRecords responses contain withdrawn items (marked as "deleted").
> If harvest.includerestricted.oai = false is set in dspace.cfg, only publicly
> readable items are included in the response. The offset into the database
> results is recalculated to skip over restricted items. If a withdrawn item is
> found in the database response, this item will also be skipped over by the
> offset recalculation code (because it is not publicly readable) even though
> it shouldn't because it will still be included in the OAI response. The code
> that actually adds items to the response, in contrast, does not skip over
> withdrawn items. The next batch will start n items later in the database
> response than it should, where n is the number of withdrawn items before the
> start of the batch.
> This means that a full OAI harvest via consecutive listRecords requests will
> miss as many items as there are withdrawn items.
> I first found this in the 1.6.1 code and it is still present in 1.7.1. I
> didn't check whether this affects earlier versions too. A patch against 1.7.1
> is attached.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.duraspace.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel
