[Dspace-tech] [KE1019161] Embargo settings on item import
Hi all, is it possible to set embergo settings during the import? The aim is, getting items metadata into DSpace 3.0 not accessable through the OAI and update the access state later through a second import or batch modification. Any ideas? Regards Marco -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] [KE1019161] Embargo settings on item import
On Thu, Jan 3, 2013 at 12:41 PM, wrote: > is it possible to set embergo settings during the import? What method of import are you using? > The aim is, getting items metadata into DSpace 3.0 not accessable > through the OAI and update the access state later through a second > import or batch modification. Can you elaborate a little? What do you mean not accessible through OAI? Do you wish to use the original 1.6 embargo or the new 3.0 embargo? Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] [KE1019161] Embargo settings on item import
Hi Helix, at the moment i tested the item import but as my system is not productive today i don't mind if i have to use another method. Oh sorry, i mean that it is not harvestable by the OAI (ListRecord, GetRecord) I think I have to use the 3.0 embargo because my DSpace are 3.0 or? Regards, Marco -- Kessler GmbH - Netzwerke und mehr Gewerbegebiet 6 82399 Raisting Tel.: 08807 / 94 67 7 - 0 Fax: 08807 / 94 67 7 - 99 Web: http://www.kesslernetworks.de Sitz der Gesellschaft: Raisting Geschäftsführer: Florian Kessler Prokura: Marco Weiß HRB 163710 -Ursprüngliche Nachricht- Von: ivan.ma...@gmail.com [mailto:ivan.ma...@gmail.com] Im Auftrag von helix84 Gesendet: Donnerstag, 3. Januar 2013 13:04 An: marco.we...@kesslernetworks.de Cc: dspace-tech@lists.sourceforge.net Betreff: Re: [Dspace-tech] [KE1019161] Embargo settings on item import On Thu, Jan 3, 2013 at 12:41 PM, wrote: > is it possible to set embergo settings during the import? What method of import are you using? > The aim is, getting items metadata into DSpace 3.0 not accessable > through the OAI and update the access state later through a second > import or batch modification. Can you elaborate a little? What do you mean not accessible through OAI? Do you wish to use the original 1.6 embargo or the new 3.0 embargo? Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] [KE1019161] Embargo settings on item import
On Thu, Jan 3, 2013 at 2:23 PM, Marco Weiß wrote: > at the moment i tested the item import but as my system is not productive > today i don't mind if i have to use another method. If you mean the old importer ([dspace]/bin/dspace import), this type of package (Simple Archive Format) does not support storing the embargo information (resource policies) [1]. I think the AIP format does preserve it [2] [3]. I recommend you to create a testing item manually from the admin interface with embargo, then export it to AIP and examine the AIP contents manualy. There should be resource policies in the METS file. I'm not completely sure about embargo terms, but I didn't hear that the new embargo would not be compatible with AIP. For the old embargo, that's not a problem because it doesn't use any special fields (it uses standard metadata fields and custom setter/lifter code). > I think I have to use the 3.0 embargo because my DSpace are 3.0 or? In DSpace 3.0 both are supported. The new embargo is easier to use for common use cases (e.g. change bistream access policy for the specified date range), the old one still allows you to write a custom embargo setter/lifeter code so it's more flexible. More information is in the official documentation. [1] https://wiki.duraspace.org/display/DSDOC3x/Importing+and+Exporting+Items+via+Simple+Archive+Format [2] https://wiki.duraspace.org/display/DSDOC3x/AIP+Backup+and+Restore [3] https://wiki.duraspace.org/display/DSDOC3x/Importing+and+Exporting+Content+via+Packages Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] [KE1019161] Embargo settings on item import
Hi Helix, it tooks a time but now i tested the embargo and the different methods to import that settings. In short till now it wont work. Now what have i done. I created an item and imported it in OAI Solr search. Searched by the OAI and got the expected output. I exported it as a simple archive by item exporter, as a csv with batch metadata editor (-a option) and an AIP by packager. In DSpace it self the item was searchable and browsable. After checking that i put the item into private state via XMLUI. Deleting the Solr search index, recreating the search index and clearing the cache and the item was no longer browsable or getable by OAI. I exported it again as a simple archive by item exporter, as a csv with batch metadata editor (-a option) and an AIP by packager. Now i did a diff between the two states for the three different ways of export. The simple archive and the csv file shows no difference between the two states except of the dc.description.provenance. The unpacked AIP shows a difference between the two mets.xml files. That tells me, that only over AIP i can handle the embargo settings, or? So i had the idea now to restore/replace it with the AIP i created as the item was in "not private" state and it have to be in that state after replacing it. Unfortunately it replaces the item but the "not private" state was not applied to the item. I set the "not private" state in XMLUI manually and imported the AIP i created as the item was in the "private" state and this works in parts. In parts means the item is withdrawn but in "not private" state after that. That means replacing an item that is in "not private" state with the "private" state item works the other way round not. To see the difference between the withdrawn item that is in "not private" state i put it manually in "private" state, exported it as AIP, unzipped, and diff the two mets.xml ... no changes that shows me what i have to set so get the item in "private"state. Importing an item with AIP and withdraw options will cause an error like this. Replacing DSpace object(s) with package located at aip24.zip org.dspace.authorize.AuthorizeException: To withdraw item must be COLLECTION_ADMIN or have REMOVE authorization on owning Collection at org.dspace.app.util.AuthorizeUtil.authorizeWithdrawItem(AuthorizeUtil.java:574) at org.dspace.content.Item.withdraw(Item.java:1870) at org.dspace.content.crosswalk.AIPTechMDCrosswalk.ingest(AIPTechMDCrosswalk.java:452) at org.dspace.content.crosswalk.AIPTechMDCrosswalk.ingest(AIPTechMDCrosswalk.java:350) at org.dspace.content.packager.METSManifest.crosswalkXmd(METSManifest.java:1151) at org.dspace.content.packager.METSManifest.crosswalkObjectSourceMD(METSManifest.java:1093) at org.dspace.content.packager.AbstractMETSIngester.ingestObject(AbstractMETSIngester.java:456) at org.dspace.content.packager.AbstractMETSIngester.replace(AbstractMETSIngester.java:1134) at org.dspace.app.packager.Packager.replace(Packager.java:766) at org.dspace.app.packager.Packager.main(Packager.java:373) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:183) org.dspace.authorize.AuthorizeException: To withdraw item must be COLLECTION_ADMIN or have REMOVE authorization on owning Collection Puh so, till now i have no way to import items in private state. I can import it and, replace it afterwards to get it withdrawn but not in state "private" Can you please confirm if i'm right here or enlighten me ;) Best regards, Marco On Thu, Jan 3, 2013 at 2:23 PM, Marco Weiß wrote: > at the moment i tested the item import but as my system is not > productive today i don't mind if i have to use another method. If you mean the old importer ([dspace]/bin/dspace import), this type of package (Simple Archive Format) does not support storing the embargo information (resource policies) [1]. I think the AIP format does preserve it [2] [3]. I recommend you to create a testing item manually from the admin interface with embargo, then export it to AIP and examine the AIP contents manualy. There should be resource policies in the METS file. I'm not completely sure about embargo terms, but I didn't hear that the new embargo would not be compatible with AIP. For the old embargo, that's not a problem because it doesn't use any special fields (it uses standard metadata fields and custom setter/lifter code). > I think I have to use the 3.0 embargo because my DSpace are 3.0 or? In DSpace 3.0 both are supported. The new embargo is easier to use for common use cases (e.g. change
Re: [Dspace-tech] [KE1019161] Embargo settings on item import
On Fri, Jan 18, 2013 at 11:36 AM, wrote: > Now what have i done. I created an item and imported it in OAI Solr search. What do you mean OAI Solr search? I assume you mean the "oai" Solr core? "search" is a different Solr core, used for Discovery. > The simple archive and the csv file shows no difference between the two > states except of the dc.description.provenance. > The unpacked AIP shows a difference between the two mets.xml files. > That tells me, that only over AIP i can handle the embargo settings, or? Yes, as I wrote in my previous email. Old embargo = bitstream authorizations + embargo terms in metadata (so SAF import should recognize it) 3.0 embargo = bitstream resource policies (so probably only AIP would recognize it - it's part of METS) > So i had the idea now to restore/replace it with the AIP i created as the > item was in "not private" state and it have to be in that state after > replacing it. > Unfortunately it replaces the item but the "not private" state was not > applied to the item. > I set the "not private" state in XMLUI manually and imported the AIP i > created as the item was in the "private" state and this works in parts. > In parts means the item is withdrawn but in "not private" state after that. > That means replacing an item that is in "not private" state with the > "private" state item works the other way round not. > To see the difference between the withdrawn item that is in "not private" > state i put it manually in "private" state, exported it as AIP, unzipped, > and diff the two mets.xml ... no changes that shows me what i have to set so > get the item in "private"state. Thanks for the testing, you may have found a bug. I didn't try it myself (exporting/importing items with embargo via AIP), so I can't explain it. If noone else answers in the next few days, you should file a Jira issue to make sure it's addressed (either explained if it's supposed to work that way, or fixed if it's not). > org.dspace.authorize.AuthorizeException: To withdraw item must be > COLLECTION_ADMIN or have REMOVE authorization on owning Collection This sounds pretty straightforward - did you use a correct value for the -e flag? E.g. a site admin eperson? Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and much more. Get web development skills now with LearnDevNow - 350+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122812 ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] [KE1019161] Embargo settings on item import
Hi Helix, it's a bit late my answer but here is it. Am 18.01.2013 12:52, schrieb helix84: > On Fri, Jan 18, 2013 at 11:36 AM, > wrote: >> Now what have i done. I created an item and imported it in OAI Solr >> search. > > What do you mean OAI Solr search? I assume you mean the "oai" Solr > core? "search" is a different Solr core, used for Discovery. With OAI Solr search i meant i imported the items in Solr so that a query on OAI can deliver the data getting it from the Solr ... hope i'm right here. >> The simple archive and the csv file shows no difference between the >> two >> states except of the dc.description.provenance. >> The unpacked AIP shows a difference between the two mets.xml files. >> That tells me, that only over AIP i can handle the embargo settings, >> or? > > Yes, as I wrote in my previous email. > Old embargo = bitstream authorizations + embargo terms in metadata > (so > SAF import should recognize it) > 3.0 embargo = bitstream resource policies (so probably only AIP would > recognize it - it's part of METS) Ok so the old embargo should work, but what is SAF ? Can't find something in the documentation. Unfortunately i don't have enough time to test it because it was just nice to have and not must have criteria. Maybe in a later step i will get back to this. The step is planed but i can not tell when. >> So i had the idea now to restore/replace it with the AIP i created >> as the >> item was in "not private" state and it have to be in that state >> after >> replacing it. >> Unfortunately it replaces the item but the "not private" state was >> not >> applied to the item. >> I set the "not private" state in XMLUI manually and imported the AIP >> i >> created as the item was in the "private" state and this works in >> parts. >> In parts means the item is withdrawn but in "not private" state >> after that. >> That means replacing an item that is in "not private" state with the >> "private" state item works the other way round not. >> To see the difference between the withdrawn item that is in "not >> private" >> state i put it manually in "private" state, exported it as AIP, >> unzipped, >> and diff the two mets.xml ... no changes that shows me what i have >> to set so >> get the item in "private"state. > > Thanks for the testing, you may have found a bug. I didn't try it > myself (exporting/importing items with embargo via AIP), so I can't > explain it. If noone else answers in the next few days, you should > file a Jira issue to make sure it's addressed (either explained if > it's supposed to work that way, or fixed if it's not). Ok no one had answered to this mail so i will file a Jira issue. >> org.dspace.authorize.AuthorizeException: To withdraw item must be >> COLLECTION_ADMIN or have REMOVE authorization on owning Collection > > This sounds pretty straightforward - did you use a correct value for > the -e flag? E.g. a site admin eperson? Yes, sure i only have one account the site admin i created following the installation documentation. And yes i used the -e flag but in long version --eperson=my.m...@adress.de Ok, i hope i can find a bit spare time to place a Jira issue the next day for that and i'll post the link to that. Regards Marco -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] [KE1019161] Embargo settings on item import
On Mon, Jan 28, 2013 at 8:29 AM, wrote: > Am 18.01.2013 12:52, schrieb helix84: >> On Fri, Jan 18, 2013 at 11:36 AM, wrote: >>> Now what have i done. I created an item and imported it in OAI Solr >>> search. >> >> What do you mean OAI Solr search? I assume you mean the "oai" Solr >> core? "search" is a different Solr core, used for Discovery. > > With OAI Solr search i meant i imported the items in Solr so that a query on > OAI can deliver the data getting it from the Solr ... hope i'm right here. Yes, that is the "oai" Solr core and you've done well. >>> The simple archive and the csv file shows no difference between the two >>> states except of the dc.description.provenance. >>> The unpacked AIP shows a difference between the two mets.xml files. >>> That tells me, that only over AIP i can handle the embargo settings, or? >> >> Yes, as I wrote in my previous email. >> Old embargo = bitstream authorizations + embargo terms in metadata (so >> SAF import should recognize it) >> 3.0 embargo = bitstream resource policies (so probably only AIP would >> recognize it - it's part of METS) > > Ok so the old embargo should work, but what is SAF ? Can't find something in > the documentation. > Unfortunately i don't have enough time to test it because it was just nice > to have and not must have criteria. > Maybe in a later step i will get back to this. The step is planed but i can > not tell when. Sorry, SAF is just a name for the format used in the original importer ([dspace]/bin/dspace import/export). >>> So i had the idea now to restore/replace it with the AIP i created as the >>> item was in "not private" state and it have to be in that state after >>> replacing it. >>> Unfortunately it replaces the item but the "not private" state was not >>> applied to the item. >>> I set the "not private" state in XMLUI manually and imported the AIP i >>> created as the item was in the "private" state and this works in parts. >>> In parts means the item is withdrawn but in "not private" state after >>> that. >>> That means replacing an item that is in "not private" state with the >>> "private" state item works the other way round not. >>> To see the difference between the withdrawn item that is in "not private" >>> state i put it manually in "private" state, exported it as AIP, unzipped, >>> and diff the two mets.xml ... no changes that shows me what i have to set >>> so >>> get the item in "private"state. >> >> >> Thanks for the testing, you may have found a bug. I didn't try it >> myself (exporting/importing items with embargo via AIP), so I can't >> explain it. If noone else answers in the next few days, you should >> file a Jira issue to make sure it's addressed (either explained if >> it's supposed to work that way, or fixed if it's not). > > > Ok no one had answered to this mail so i will file a Jira issue. > > >>> org.dspace.authorize.AuthorizeException: To withdraw item must be >>> COLLECTION_ADMIN or have REMOVE authorization on owning Collection >> >> >> This sounds pretty straightforward - did you use a correct value for >> the -e flag? E.g. a site admin eperson? > > > Yes, sure i only have one account the site admin i created following the > installation documentation. > And yes i used the -e flag but in long version > --eperson=my.m...@adress.de Sorry, I can't think of anything else that could have gone wrong there. > Ok, i hope i can find a bit spare time to place a Jira issue the next day > for that and i'll post the link to that. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] [KE1019161] Embargo settings on item import
Hi all, i now tested i again in 3.1 and its the same as in 3.0 I placed a Jira issue https://jira.duraspace.org/browse/DS-1514 like helix advice a few mails ago. Regards, Marco Am 28.01.2013 09:43, schrieb helix84: > On Mon, Jan 28, 2013 at 8:29 AM, > wrote: >> Am 18.01.2013 12:52, schrieb helix84: >>> On Fri, Jan 18, 2013 at 11:36 AM, >>> wrote: Now what have i done. I created an item and imported it in OAI Solr search. >>> >>> What do you mean OAI Solr search? I assume you mean the "oai" Solr >>> core? "search" is a different Solr core, used for Discovery. >> >> With OAI Solr search i meant i imported the items in Solr so that a >> query on >> OAI can deliver the data getting it from the Solr ... hope i'm right >> here. > > Yes, that is the "oai" Solr core and you've done well. > The simple archive and the csv file shows no difference between the two states except of the dc.description.provenance. The unpacked AIP shows a difference between the two mets.xml files. That tells me, that only over AIP i can handle the embargo settings, or? >>> >>> Yes, as I wrote in my previous email. >>> Old embargo = bitstream authorizations + embargo terms in metadata >>> (so >>> SAF import should recognize it) >>> 3.0 embargo = bitstream resource policies (so probably only AIP >>> would >>> recognize it - it's part of METS) >> >> Ok so the old embargo should work, but what is SAF ? Can't find >> something in >> the documentation. >> Unfortunately i don't have enough time to test it because it was >> just nice >> to have and not must have criteria. >> Maybe in a later step i will get back to this. The step is planed >> but i can >> not tell when. > > Sorry, SAF is just a name for the format used in the original > importer > ([dspace]/bin/dspace import/export). > So i had the idea now to restore/replace it with the AIP i created as the item was in "not private" state and it have to be in that state after replacing it. Unfortunately it replaces the item but the "not private" state was not applied to the item. I set the "not private" state in XMLUI manually and imported the AIP i created as the item was in the "private" state and this works in parts. In parts means the item is withdrawn but in "not private" state after that. That means replacing an item that is in "not private" state with the "private" state item works the other way round not. To see the difference between the withdrawn item that is in "not private" state i put it manually in "private" state, exported it as AIP, unzipped, and diff the two mets.xml ... no changes that shows me what i have to set so get the item in "private"state. >>> >>> >>> Thanks for the testing, you may have found a bug. I didn't try it >>> myself (exporting/importing items with embargo via AIP), so I can't >>> explain it. If noone else answers in the next few days, you should >>> file a Jira issue to make sure it's addressed (either explained if >>> it's supposed to work that way, or fixed if it's not). >> >> >> Ok no one had answered to this mail so i will file a Jira issue. >> >> org.dspace.authorize.AuthorizeException: To withdraw item must be COLLECTION_ADMIN or have REMOVE authorization on owning Collection >>> >>> >>> This sounds pretty straightforward - did you use a correct value >>> for >>> the -e flag? E.g. a site admin eperson? >> >> >> Yes, sure i only have one account the site admin i created following >> the >> installation documentation. >> And yes i used the -e flag but in long version >> --eperson=my.m...@adress.de > > Sorry, I can't think of anything else that could have gone wrong > there. > >> Ok, i hope i can find a bit spare time to place a Jira issue the >> next day >> for that and i'll post the link to that. > > > Regards, > ~~helix84 > > Compulsory reading: DSpace Mailing List Etiquette > https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
