Hey Tim, No reason for not submitting a pull request and I for sure can move the code around a bit. However, I don't see a way around adding dspace-oai classes as a dependency. At the end of the day the consumer just calls org.dspace.xoai.app.XOAI.index.
xmlui webapp does not have dspace-oai jar/classes in it's WEB-INF dir, the same is probably true for jspui webapp. Perhaps there is another approach? Regards, OK 2015-05-27 16:23 GMT+02:00 Tim Donohue <tdono...@duraspace.org>: > Hi Ondřej, > > I wonder if you'd be willing to submit a Pull Request of these changes > (the new OAIIndexEventConsumer, etc)? These seem like potentially useful > changes overall, and would be something we'd consider adding to DSpace. > > My only (small) recommendation would be to simply put the > OAIIndexEventConsumer into the "dspace-api", so that it can be easily > enabled/disabled from the "event.dispatcher.default.consumers" setting. > Plus, if it's in the dspace-api, that ensures it'd also be triggered by any > updates from other interface (XMLUI, JSPUI, REST, etc) > > - Tim > > On 5/27/2015 5:08 AM, Ondřej Košarko wrote: > >> >> ---------- Forwarded message ---------- >> From: *Ondrej Kosarko* <ko...@centrum.cz <mailto:ko...@centrum.cz>> >> Date: 2015-05-27 12:05 GMT+02:00 >> Subject: Re: [Dspace-tech] OAI-PMH data provider 2.0 not persistent? >> To: Christian Scheible <christian.schei...@uni-konstanz.de >> <mailto:christian.schei...@uni-konstanz.de>> >> Cc: "dspace-tech@lists.sourceforge.net >> <mailto:dspace-tech@lists.sourceforge.net>" >> <dspace-tech@lists.sourceforge.net >> <mailto:dspace-tech@lists.sourceforge.net>> >> >> >> Hi Christian, >> >> I'm not sure I remember all the setup, but here goes... >> >> in dspace.cfg there is >> >> event.dispatcher.default.consumers = versioning, discovery, eperson, >> harvester, eudatreplication, *xoai* >> >> >> and >> >> # consumer to maintain the oai index >> event.consumer.xoai.class = cz.cuni.mff.ufal.event.OAIIndexEventConsumer >> event.consumer.xoai.filters = >> >> Community|Collection|Item|Bundle|Bitstream+Add|Create|Modify|Modify_Metadata|Delete|Remove >> >> >> and you need to make sure the class is found by the event dispatcher, we >> solve it by adding dspace-oai(where the consumer lives) as a dependency >> to dspace-xmlui >> >> diff --git a/dspace-xmlui/pom.xml b/dspace-xmlui/pom.xml >> index 17d75c0..f19257f 100644 >> --- a/dspace-xmlui/pom.xml >> +++ b/dspace-xmlui/pom.xml >> @@ -93,6 +93,22 @@ >> </profiles> >> <dependencies> >> + <!-- because of event listener that updates oai --> >> + <dependency> >> + <groupId>org.dspace</groupId> >> + <artifactId>dspace-oai</artifactId> >> + <classifier>classes</classifier> >> + </dependency> >> >> I think that's about it... >> >> Regards, >> OK >> >> >> 2015-05-27 9:05 GMT+02:00 Christian Scheible >> <christian.schei...@uni-konstanz.de >> <mailto:christian.schei...@uni-konstanz.de>>: >> >> Hi Ondřej, >> >> I have seen that you are using your own OAIIndexEventConsumer. We >> did the same but I added It in the module additions (which leads to >> code duplication). >> How do you register your Consumer? Is it the same way as for other >> consumers? Because I couldn't solve that problem. >> >> Best Regards, >> Christian >> >> >> Am 27.05.2015 um 08:41 schrieb Ondřej Košarko: >> >>> In short no... >>> >>> As I've suggested in the thread the indexAll should include also >>> the withdrawn, that's what we do >>> >>> https://github.com/ufal/lindat-dspace/blob/lindat/dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java#L193 >>> We use our Event consumer to trigger the update (it currently >>> doesn't handle caching and changes in discoverable flag) >>> >>> https://github.com/ufal/lindat-dspace/blob/lindat/dspace-oai/src/main/java/cz/cuni/mff/ufal/event/OAIIndexEventConsumer.java >>> And by the way dspace changed the deletion mode to transient in >>> https://jira.duraspace.org/browse/DS-2491 so maybe the right way >>> for dspace is updating the documentation... >>> >>> >>> OK >>> >>> >>> >>> 2015-05-26 23:19 GMT+02:00 Jozef Misutka <misu...@ufal.mff.cuni.cz >>> <mailto:misu...@ufal.mff.cuni.cz>>: >>> >>> Ondrej, is this happening to us as well? >>> >>> jm >>> >>> >>> ------------------------------------------------------------------------ >>> *From: *"Tim Donohue" <tdono...@duraspace.org >>> <mailto:tdono...@duraspace.org>> >>> *To: *dspace-tech@lists.sourceforge.net >>> <mailto:dspace-tech@lists.sourceforge.net> >>> *Sent: *Tuesday, 26 May, 2015 23:09:20 >>> >>> *Subject: *Re: [Dspace-tech] OAI-PMH data provider 2.0 not >>> >>> persistent? >>> >>> Hi All, >>> >>> I'm only just now "re-discovering" this thread, as I've found >>> this same >>> behavior in DSpace OAI-PMH servers. Here's how to see this >>> behavior (and >>> I've double checked it on http://demo.dspace.org, running 5.2): >>> >>> 1. Create an Item >>> 2. Run the OAI-PMH import (./dspace oai import). >>> 3. The item now appears in OAI-PMH >>> 4. Withdraw that Item >>> 5. Re-Run the OAI-PMH import (./dspace oai import) >>> 6. The Item still appears in OAI-PMH and is NEVER flagged as >>> withdrawn. >>> >>> So, you can still access all its metadata, etc. The ONLY way >>> to trigger >>> an update to the OAI-PMH record (of the withdrawn item) is to >>> re-import >>> EVERYTHING (./dspace oai import -c). However, as noted, since >>> withdrawn >>> items are not included in the XOAI "indexAll()" command, the >>> item will >>> now disappear from OAI-PMH entirely. >>> >>> Here's that indexAll() command: >>> >>> https://github.com/DSpace/DSpace/blob/master/dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java#L191 >>> >>> This behavior seems to be counter to what is documented at: >>> >>> https://wiki.duraspace.org/pages/viewpage.action?pageId=45548245#OAI-PMHDataProvider2.0%28Internals%29-Deletions >>> >>> The documentation specifically states that "DSpace keeps a >>> permanent >>> record of withdrawn items". It also states that a request for a >>> withdrawn item "will yield the 'record deleted' header". >>> >>> This sounds like a bug to me. The documentation definitely >>> does not >>> match with the behavior. >>> >>> I've created a new bug ticket for this. It will need a >>> volunteer to resolve. >>> >>> https://jira.duraspace.org/browse/DS-2593 >>> >>> - Tim >>> >>> On 3/6/2015 5:21 AM, helix84 wrote: >>> > Yes, that makes sense. Still, it needs a brief verification >>> because >>> > sometimes our understanding of a concept and its >>> implementation may >>> > differ. >>> > >>> > >>> > On Fri, Mar 6, 2015 at 10:12 AM, Kristian Roberto Salcedo >>> > <k.r.salc...@ub.uio.no <mailto:k.r.salc...@ub.uio.no>> wrote: >>> >> Hi Ivan, >>> >> >>> >> I might have overlooked something, but from your comments >>> on https://jira.duraspace.org/browse/DS-2491 wouldnt it be >>> meaningful >>> >> to do both your declaration change and Ondřej's suggestion >>> in order to match both what is declared and what the >>> documentation says about OAI and deleted items? >>> >> >>> >> regards, >>> >> Kristian >>> >> >>> >>> -----Original Message----- >>> >>> From: ivan.ma...@gmail.com <mailto:ivan.ma...@gmail.com> >>> [mailto:ivan.ma...@gmail.com <mailto:ivan.ma...@gmail.com>] On >>> Behalf Of >>> >>> helix84 >>> >>> Sent: Thursday, March 05, 2015 3:46 PM >>> >>> To: Kristian Roberto Salcedo >>> >>> Cc: dspace-tech@lists.sourceforge.net >>> <mailto:dspace-tech@lists.sourceforge.net>; João Melo >>> >>> Subject: Re: [Dspace-tech] OAI-PMH data provider 2.0 not >>> persistent? >>> >>> >>> >>> Hi Kristian, >>> >>> >>> >>> I think you're right that the declared >>> >>> <deletedRecord>persistent</deletedRecord> doesn't match >>> how DSpace >>> >>> behaves. I filed a Jira issue and created a pull request >>> to change the declared >>> >>> status to <deletedRecord>transient</deletedRecord>. >>> >>> >>> >>> https://jira.duraspace.org/browse/DS-2491 >>> >>> >>> >>> >>> >>> Regards, >>> >>> ~~helix84 >>> >>> >>> >>> Compulsory reading: DSpace Mailing List Etiquette >>> >>> >>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >>> >>> >>> >>> >>> >>> On Thu, Mar 5, 2015 at 2:52 PM, Kristian Roberto Salcedo >>> >>> <k.r.salc...@ub.uio.no <mailto:k.r.salc...@ub.uio.no>> >>> wrote: >>> >>>> Hi all, >>> >>>> >>> >>>> In Dspace 4.2 we are currently seeing that our OAI feeds >>> are not being >>> >>> persistent as per these definitions when it comes to >>> deleted (withdrawn) >>> >>> items: >>> >>>> >>> >>>> >>> >>> http://www.openarchives.org/OAI/openarchivesprotocol.html#deletion >>> >>>> >>> >>> https://wiki.duraspace.org/pages/viewpage.action?pageId=34640887#OAI- >>> >>> P >>> >>>> MHDataProvider2.0(Internals)-Deletions >>> >>>> >>> >>>> Withdrawn items do not get the <header status=deleted> as >>> they should, >>> >>> like in this example: >>> >>>> >>> >>>> >>> >>> >>> >>> http://webservices.itcs.umich.edu/mediawiki/oaibp/index.php/Deleted_Re >>> >>>> cord_Example_1 >>> >>>> >>> >>>> Withdrawing an item is not reflected in the OAI entry in >>> any way. >>> >>>> >>> >>>> ---------------------------------------------- >>> >>>> >>> >>>> This is our Identify page at the moment: >>> >>>> >>> >>>> https://www.duo.uio.no/oai/request?verb=Identify >>> >>>> >>> >>>> ---------------------------------------------- >>> >>>> >>> >>>> I believe we're doing everything right regarding config >>> parameters and >>> >>> maintenance of the solr oai index. >>> >>>> >>> >>>> The only place I can find a "persistent"-parameter is in >>> this config file: >>> >>>> /www/var/data/dspace/config/oaicat.properties >>> >>>> which I thought was deprecated, but we still set it just >>> to be sure: >>> >>>> Identify.deletedRecord=persistent >>> >>>> >>> >>>> All other relevant config parameters are set in these two >>> files as far as I can >>> >>> tell: >>> >>>> >>> >>>> /www/var/data/dspace/config/modules/oai.cfg >>> >>>> /www/var/data/dspace/config/crosswalks/oai/description.xml >>> >>>> >>> >>>> We are running the oai import -o command nightly. >>> >>>> >>> >>>> ---------------------------------------------- >>> >>>> >>> >>>> The only way we've found to remove withdrawn items from >>> the OAI feed is >>> >>> by running a complete re-indexing of the oai solr index: >>> >>>> >>> >>>> /www/var/data/dspace/bin/dspace oai import -c >>> >>>> >>> >>>> with a subsequent >>> >>>> >>> >>>> /www/var/data/dspace/bin/dspace oai clean-cache >>> >>>> >>> >>>> This removes a withdrawn item: >>> >>>> >>> >>>> https://www.duo.uio.no/handle/10852/42670 >>> >>>> >>> >>>> completely from the feed: >>> >>>> >>> >>>> >>> >>> >>> >>> https://www.duo.uio.no/oai/request?verb=GetRecord&metadataPrefix=oai >>> >>> _d >>> >>>> c&identifier=oai:localhost:10852/42670 >>> >>>> >>> >>>> which is not what we want... >>> >>>> >>> >>>> ---------------------------------------------- >>> >>>> >>> >>>> >>> >>>> Is anyone else experiencing the same problem? >>> >>>> >>> >>>> Maybe I've missed something - If this actually works >>> differently than we >>> >>> expect or has been addressed in OAI 2.1, please let me know. >>> >>>> >>> >>>> >>> >>>> regards, >>> >>>> Kristian Salcedo >>> >>>> Universitetet of Oslo Library >>> >>>> Department of digital services >>> >>>> >>> >>>> >>> >>>> >>> >>> ---------------------------------------------------------------------- >>> >>>> -------- Dive into the World of Parallel Programming The >>> Go Parallel >>> >>>> Website, sponsored by Intel and developed in partnership >>> with Slashdot >>> >>>> Media, is your hub for all things parallel software >>> development, from >>> >>>> weekly thought leadership blogs to news, videos, case >>> studies, >>> >>>> tutorials and more. Take a look and join the conversation >>> now. >>> >>>> http://goparallel.sourceforge.net/ >>> >>>> _______________________________________________ >>> >>>> DSpace-tech mailing list >>> >>>> DSpace-tech@lists.sourceforge.net >>> <mailto:DSpace-tech@lists.sourceforge.net> >>> >>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech >>> >>>> List Etiquette: >>> >>>> >>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >>> > >>> > >>> >>> ------------------------------------------------------------------------------ >>> > Dive into the World of Parallel Programming The Go Parallel >>> Website, sponsored >>> > by Intel and developed in partnership with Slashdot Media, >>> is your hub for all >>> > things parallel software development, from weekly thought >>> leadership blogs to >>> > news, videos, case studies, tutorials and more. Take a look >>> and join the >>> > conversation now. http://goparallel.sourceforge.net/ >>> > _______________________________________________ >>> > DSpace-tech mailing list >>> > DSpace-tech@lists.sourceforge.net >>> <mailto:DSpace-tech@lists.sourceforge.net> >>> > https://lists.sourceforge.net/lists/listinfo/dspace-tech >>> > List Etiquette: >>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >>> > >>> >>> >>> ------------------------------------------------------------------------------ >>> _______________________________________________ >>> DSpace-tech mailing list >>> DSpace-tech@lists.sourceforge.net >>> <mailto:DSpace-tech@lists.sourceforge.net> >>> https://lists.sourceforge.net/lists/listinfo/dspace-tech >>> List Etiquette: >>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >>> >>> >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> _______________________________________________ >>> DSpace-tech mailing list >>> DSpace-tech@lists.sourceforge.net <mailto: >>> DSpace-tech@lists.sourceforge.net> >>> https://lists.sourceforge.net/lists/listinfo/dspace-tech >>> List Etiquette: >>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >>> >> >> >> -- >> Christian Scheible >> Softwareentwickler / Abt. Content-basierte Dienste >> Kommunikations-, Informations- und Medienzentrum (KIM) >> Universität Konstanz >> 78457 Konstanz >> +49 (0)7531 / 88-2857 >> Raum B 703 >> >> >> >> ------------------------------------------------------------------------------ >> >> _______________________________________________ >> DSpace-tech mailing list >> DSpace-tech@lists.sourceforge.net >> <mailto:DSpace-tech@lists.sourceforge.net> >> https://lists.sourceforge.net/lists/listinfo/dspace-tech >> List Etiquette: >> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> >> >> _______________________________________________ >> DSpace-tech mailing list >> DSpace-tech@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/dspace-tech >> List Etiquette: >> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >> >>
------------------------------------------------------------------------------
_______________________________________________ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
