[Dspace-tech] Setting file descriptions on batch import
I've been importing digitized manuscripts into our test DSpace instance, one item per manuscript, each page in its own file. Everything's working fine, but I can't figure out a way to set the file descriptions during this process. Is there one? I've also tried exporting metadata, but file descriptions aren't part of the item metadata, so I can't set them by editing and reimporting it. Thanks for your help, Vika Zafrin Institutional Repository Librarian Boston University +1.617.358.6370 -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] Creative Common License Does not Show Properly in xmlui
Dear friends, If a Creatice Common license is attached to an item (in xmlui), the page showing the license does not show properly (looks like an unformatted HTML with no image) and some of the links inside the page (including the Legal Code which is important) do not work and show this error: org.apache.cocoon.ResourceNotFoundException: Unable to locate bitstream map:read type=BitstreamReader - context:/jndi:/localhost/xmlui/sitemap.xmap - 268:70 I know that there has been a known bug (http://dspace.2283337.n4.nabble.com/dspace-Bugs-2086708-xmlui-Creative-Commons-not-properly-displayed-td3294853.html ) but I just wonder if somebody in the meantime knows a solution to this. Things are OK in jspui and I wonder why the xmlui tries to show the license as an internal page not just as a link to the CC website. I have manually copied the necessary css files from the Creative Common website into the similar DSpace folders, now the look is a bit better but the links still don't work. Please try http://elogeo.nottingham.ac.uk/xmlui/bitstream/handle/url/37/license_text to see what I mean! Thanks, Amir. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation.-- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Creative Common License Does not Show Properly in xmlui
Hello Amir The format associated with the license text is not correct, which is why it does not display. There is a patch to address that particular issue for 1.6. See this patch https://jira.duraspace.org/browse/DS-295. I just completed some work on CCLicense that I hope will be in the upcoming 1.8 release. As part of that, the XMLUI will contain a link to the actual license, rather than storing the license text and trying to display it internally as you point out. I will leave it to the DSpace committers/Release Coordinator to comment on the schedule of the latter. Thanks! ..\Wendy I am putting myself to the fullest possible use, which is all I think that any conscious entity can ever hope to do. Wendy Bossons Senior Software Engineer MIT Libraries Software Analysis and Development 77 Massachusetts Avenue Cambridge, MA 02139-4307 617-253-0770 wboss...@mit.edumailto:wboss...@mit.edu On Jul 27, 2011, at 1:42 PM, Amir Pourabdollah wrote: Dear friends, If a Creatice Common license is attached to an item (in xmlui), the page showing the license does not show properly (looks like an unformatted HTML with no image) and some of the links inside the page (including the Legal Code which is important) do not work and show this error: org.apache.cocoon.ResourceNotFoundException: Unable to locate bitstream map:read type=BitstreamReader - context:/jndi:/localhost/xmlui/sitemap.xmap - 268:70 I know that there has been a known bug (http://dspace.2283337.n4.nabble.com/dspace-Bugs-2086708-xmlui-Creative-Commons-not-properly-displayed-td3294853.html ) but I just wonder if somebody in the meantime knows a solution to this. Things are OK in jspui and I wonder why the xmlui tries to show the license as an internal page not just as a link to the CC website. I have manually copied the necessary css files from the Creative Common website into the similar DSpace folders, now the look is a bit better but the links still don't work. Please try http://elogeo.nottingham.ac.uk/xmlui/bitstream/handle/url/37/license_text to see what I mean! Thanks, Amir. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. ATT1..cATT2..c -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Creative Common License Does not Show Properly in xmlui
Thanks Wendy for your reply. I noticed that there is a bitstream format of CC License already defined and associated to License_text. Does this mean that the patch is already applied? I have manually changed the MIME type of CCLicense to text/html; charset=utf-8 so the page is now rendered as HTML but it is still not what it should be, particularly some links (like the legal codes) are not working. I think the problem is beyond formatting and rendering issues. Do you think if I apply the patch, the links will also work? Yours, Amir. From: Wendy J Bossons [wboss...@mit.edu] Sent: 27 July 2011 19:54 To: Amir Pourabdollah Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Creative Common License Does not Show Properly in xmlui Hello Amir The format associated with the license text is not correct, which is why it does not display. There is a patch to address that particular issue for 1.6. See this patch https://jira.duraspace.org/browse/DS-295. I just completed some work on CCLicense that I hope will be in the upcoming 1.8 release. As part of that, the XMLUI will contain a link to the actual license, rather than storing the license text and trying to display it internally as you point out. I will leave it to the DSpace committers/Release Coordinator to comment on the schedule of the latter. Thanks! ..\Wendy I am putting myself to the fullest possible use, which is all I think that any conscious entity can ever hope to do. Wendy Bossons Senior Software Engineer MIT Libraries Software Analysis and Development 77 Massachusetts Avenue Cambridge, MA 02139-4307 617-253-0770 wboss...@mit.edumailto:wboss...@mit.edu On Jul 27, 2011, at 1:42 PM, Amir Pourabdollah wrote: Dear friends, If a Creatice Common license is attached to an item (in xmlui), the page showing the license does not show properly (looks like an unformatted HTML with no image) and some of the links inside the page (including the Legal Code which is important) do not work and show this error: org.apache.cocoon.ResourceNotFoundException: Unable to locate bitstream map:read type=BitstreamReader - context:/jndi:/localhost/xmlui/sitemap.xmap - 268:70 I know that there has been a known bug (http://dspace.2283337.n4.nabble.com/dspace-Bugs-2086708-xmlui-Creative-Commons-not-properly-displayed-td3294853.html ) but I just wonder if somebody in the meantime knows a solution to this. Things are OK in jspui and I wonder why the xmlui tries to show the license as an internal page not just as a link to the CC website. I have manually copied the necessary css files from the Creative Common website into the similar DSpace folders, now the look is a bit better but the links still don't work. Please try http://elogeo.nottingham.ac.uk/xmlui/bitstream/handle/url/37/license_text to see what I mean! Thanks, Amir. This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. ATT1..cATT2..c -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] Batch metadata corrections question: does anyone know why the limit is set to just 20 items at a time?
Hello, We're experimenting with making batch corrections to metadata using the Import Metadata feature in the jsp. We'd like to raise the limit on the number of items that may be changed at a time. I can see the file: http://scm.dspace.org/svn/repo/dspace/tags/dspace-1.6.2/dspace-jspui/dspace-jspui-api/src/main/java/org/dspace/app/webui/servlet/MetadataImportServlet.java Where it says this: // Set the lmimt to the number of items that may be changed in one go, default to 20 limit = ConfigurationManager.getIntProperty(bulkedit.gui-item-limit, 20); log.debug(Setting bulk edit limit to + limit + items); …We’d like to up it from 20 to maybe 500 as an experiment -- but potentially much higher. Does anyone know if that's a really bad idea? We just don’t know what the consequence is of making this limit larger, but 20 seems way too low for a typical batch of changes. Thanks, Irene Berry, MLIS Digital Services Librarian Dudley Knox Library, Naval Postgraduate School -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] batch import of Bagit-formated collections and/or conversion script for Bagit to SAF
Thanks, Mark, that code from MIT looks interesting, I will look into it more. I did notice that the Bagit spec is supported by the SWORD protocol, and when I mentioned this to our archivist, he went and looked and it does appear that the BIL 3.9 can send a bag using SWORD (see output of the BIL -h command, pasted below). So, it looks like Bagger and/or BIL + turning on SWORD for our repository will get us what we want. Huzzah! * BagIt Library (BIL) Version 3.9 Usage: bag operation [operation arguments] [--help] Parameters: operation Valid operations are: baginplace, bob, checkpayloadoxum, create, fillholey, generatepayloadoxum, makecomplete, makeholey, retrieve, splitbagbyfiletype, splitbagbysize, splitbagbysizeandfiletype, sword, update, updatetagmanifests, verifycomplete, verifypayloadmanifests, verifytagmanifests and verifyvalid. Operation explanations: baginplace: Creates a bag-in-place. The source must be a directory on a filesystem and may already have a data directory. bob: Sends a bag using BOB. checkpayloadoxum: Generates Payload-Oxum and checks against Payload-Oxum in bag-info.txt. create: Creates a bag from supplied files/directories, completes the bag, and then writes in a specified format. fillholey: Retrieves any missing pieces of a local bag. generatepayloadoxum: Generates and returns the Payload-Oxum for the bag. makecomplete: Completes a bag and then writes in a specified format. Completing a bag fills in any missing parts. makeholey: Generates a fetch.txt and then writes bag in a specified format. retrieve: Retrieves a bag exposed by a web server. A local holey bag is not required. splitbagbyfiletype: Splits a bag by file types. splitbagbysize: Splits a bag by size. splitbagbysizeandfiletype: Splits a bag by size and file types. sword: Sends a bag using SWORD. update: Updates the manifests and (if it exists) the bag-info.txt for a bag. updatetagmanifests: Updates the tag manifests for a bag. The bag must be unserialized. verifycomplete: Verifies the completeness of a bag. verifypayloadmanifests: Verifies the checksums in all payload manifests. verifytagmanifests: Verifies the checksums in all tag manifests. verifyvalid: Verifies the validity of a bag. [--version] Prints version of BIL and exits. [--help] Prints usage message for the operation. Examples: bag verifyvalid --help Prints help for the verifyvalid operation. -- HARDY POTTINGER pottinge...@umsystem.edu University of Missouri Library Systems http://lso.umsystem.edu/~pottingerhj/ No matter how far down the wrong road you've gone, turn back. --Turkish proverb On 7/26/11 5:31 PM, Mark Diggory mdigg...@atmire.com wrote: Hardy, Be aware that MIT / Richard Rodgers also has some Bagit work available, currently nested within the modules directory here: http://scm.dspace.org/svn/repo/modules/dspace-replicate/trunk/src/main/jav a/org/dspace/pack/ http://scm.dspace.org/svn/repo/modules/dspace-replicate/trunk/src/main/ja va/org/dspace/pack/Mark On Tue, Jul 26, 2011 at 2:33 PM, Pottinger, Hardy J. pottinge...@umsystem.edu wrote: Hi, I've done a bit of googling on Bagit, and I see that Dryad (and @mire) have done some work with Bagit as a repository interchange mechanism. I am interested in something a bit more mundane. There exists a very nice tool for constructing a bag, called Bagger: http://sourceforge.net/projects/loc-xferutils/files/loc-bagger/ Which would be ideal for adapting for our needs--we need a tool that a scanner technician can use to feed scanned images into our repository. Bags, in my mind, are not much different than SAF packages. It would be trivial to script a converter between the two formats, though I'm thinking someone is likely to have walked this path already. If so, and if you can share any code, or just talk about your approach, I'd love to hear from you. Thanks! -- HARDY POTTINGER pottinge...@umsystem.edu University of Missouri Library Systems http://lso.umsystem.edu/~pottingerhj/ No matter how far down the wrong road you've gone, turn back. --Turkish proverb -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ DSpace-tech mailing list
Re: [Dspace-tech] batch import of Bagit-formated collections and/or conversion script for Bagit to SAF
Hi Hardy, SWORD is completely agnostic about what packages it transports, however out the box, DSpace does not know how to ingest bags via SWORD. You might therefore need to write a bag ingester than knows how to unpack and ingest the contents of the bag. This would make an excellent addition to DSpace :) Thanks, Stuart Lewis Digital Development Manager Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: +64 (0)9 373 7599 x81928 On 28/07/2011, at 9:29 AM, Pottinger, Hardy J. wrote: Thanks, Mark, that code from MIT looks interesting, I will look into it more. I did notice that the Bagit spec is supported by the SWORD protocol, and when I mentioned this to our archivist, he went and looked and it does appear that the BIL 3.9 can send a bag using SWORD (see output of the BIL -h command, pasted below). So, it looks like Bagger and/or BIL + turning on SWORD for our repository will get us what we want. Huzzah! * BagIt Library (BIL) Version 3.9 Usage: bag operation [operation arguments] [--help] Parameters: operation Valid operations are: baginplace, bob, checkpayloadoxum, create, fillholey, generatepayloadoxum, makecomplete, makeholey, retrieve, splitbagbyfiletype, splitbagbysize, splitbagbysizeandfiletype, sword, update, updatetagmanifests, verifycomplete, verifypayloadmanifests, verifytagmanifests and verifyvalid. Operation explanations: baginplace: Creates a bag-in-place. The source must be a directory on a filesystem and may already have a data directory. bob: Sends a bag using BOB. checkpayloadoxum: Generates Payload-Oxum and checks against Payload-Oxum in bag-info.txt. create: Creates a bag from supplied files/directories, completes the bag, and then writes in a specified format. fillholey: Retrieves any missing pieces of a local bag. generatepayloadoxum: Generates and returns the Payload-Oxum for the bag. makecomplete: Completes a bag and then writes in a specified format. Completing a bag fills in any missing parts. makeholey: Generates a fetch.txt and then writes bag in a specified format. retrieve: Retrieves a bag exposed by a web server. A local holey bag is not required. splitbagbyfiletype: Splits a bag by file types. splitbagbysize: Splits a bag by size. splitbagbysizeandfiletype: Splits a bag by size and file types. sword: Sends a bag using SWORD. update: Updates the manifests and (if it exists) the bag-info.txt for a bag. updatetagmanifests: Updates the tag manifests for a bag. The bag must be unserialized. verifycomplete: Verifies the completeness of a bag. verifypayloadmanifests: Verifies the checksums in all payload manifests. verifytagmanifests: Verifies the checksums in all tag manifests. verifyvalid: Verifies the validity of a bag. [--version] Prints version of BIL and exits. [--help] Prints usage message for the operation. Examples: bag verifyvalid --help Prints help for the verifyvalid operation. -- HARDY POTTINGER pottinge...@umsystem.edu University of Missouri Library Systems http://lso.umsystem.edu/~pottingerhj/ No matter how far down the wrong road you've gone, turn back. --Turkish proverb On 7/26/11 5:31 PM, Mark Diggory mdigg...@atmire.com wrote: Hardy, Be aware that MIT / Richard Rodgers also has some Bagit work available, currently nested within the modules directory here: http://scm.dspace.org/svn/repo/modules/dspace-replicate/trunk/src/main/jav a/org/dspace/pack/ http://scm.dspace.org/svn/repo/modules/dspace-replicate/trunk/src/main/ja va/org/dspace/pack/Mark On Tue, Jul 26, 2011 at 2:33 PM, Pottinger, Hardy J. pottinge...@umsystem.edu wrote: Hi, I've done a bit of googling on Bagit, and I see that Dryad (and @mire) have done some work with Bagit as a repository interchange mechanism. I am interested in something a bit more mundane. There exists a very nice tool for constructing a bag, called Bagger: http://sourceforge.net/projects/loc-xferutils/files/loc-bagger/ Which would be ideal for adapting for our needs--we need a tool that a scanner technician can use to feed scanned images into our repository. Bags, in my mind, are not much different than SAF packages. It would be trivial to script a converter between the two formats, though I'm thinking someone is likely to have walked this path already. If so, and if you can share any code, or just talk
Re: [Dspace-tech] Batch metadata corrections question: does anyone know why the limit is set to just 20 items at a time?
Hi Irene, We're experimenting with making batch corrections to metadata using the Import Metadata feature in the jsp. We'd like to raise the limit on the number of items that may be changed at a time. I can see the file: http://scm.dspace.org/svn/repo/dspace/tags/dspace-1.6.2/dspace-jspui/dspace-jspui-api/src/main/java/org/dspace/app/webui/servlet/MetadataImportServlet.java Where it says this: // Set the lmimt to the number of items that may be changed in one go, default to 20 limit = ConfigurationManager.getIntProperty(bulkedit.gui-item-limit, 20); log.debug(Setting bulk edit limit to + limit + items); Correct - adjust the setting 'bulkedit.gui-item-limit' in dspace.cfg. …We’d like to up it from 20 to maybe 500 as an experiment -- but potentially much higher. Does anyone know if that's a really bad idea? We just don’t know what the consequence is of making this limit larger, but 20 seems way too low for a typical batch of changes. We set it to 20 initially so that there is a low risk of anything going wrong. If you are are happy with the tool, and the way it works, then it is fine to set it higher (but of course it carries the risk of potentially wrecking 500 records at once instead of 20!). We don't recommend setting it too high as the changes are all made in one hit, and this can cause timeouts or memory problems on the server. 500 should be fine. We've heard of problems when people start going over 1 or 2 thousand records at a time. Thanks, Stuart Lewis Digital Development Manager Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: +64 (0)9 373 7599 x81928 -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] batch import of Bagit-formated collections and/or conversion script for Bagit to SAF
BTW, the replicate code does all this - serializes and deserializes bags to DSpace objects Richard On Jul 27, 2011, at 5:56 PM, Stuart Lewis wrote: Hi Hardy, SWORD is completely agnostic about what packages it transports, however out the box, DSpace does not know how to ingest bags via SWORD. You might therefore need to write a bag ingester than knows how to unpack and ingest the contents of the bag. This would make an excellent addition to DSpace :) Thanks, Stuart Lewis Digital Development Manager Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: +64 (0)9 373 7599 x81928 On 28/07/2011, at 9:29 AM, Pottinger, Hardy J. wrote: Thanks, Mark, that code from MIT looks interesting, I will look into it more. I did notice that the Bagit spec is supported by the SWORD protocol, and when I mentioned this to our archivist, he went and looked and it does appear that the BIL 3.9 can send a bag using SWORD (see output of the BIL -h command, pasted below). So, it looks like Bagger and/or BIL + turning on SWORD for our repository will get us what we want. Huzzah! * BagIt Library (BIL) Version 3.9 Usage: bag operation [operation arguments] [--help] Parameters: operation Valid operations are: baginplace, bob, checkpayloadoxum, create, fillholey, generatepayloadoxum, makecomplete, makeholey, retrieve, splitbagbyfiletype, splitbagbysize, splitbagbysizeandfiletype, sword, update, updatetagmanifests, verifycomplete, verifypayloadmanifests, verifytagmanifests and verifyvalid. Operation explanations: baginplace: Creates a bag-in-place. The source must be a directory on a filesystem and may already have a data directory. bob: Sends a bag using BOB. checkpayloadoxum: Generates Payload-Oxum and checks against Payload-Oxum in bag-info.txt. create: Creates a bag from supplied files/directories, completes the bag, and then writes in a specified format. fillholey: Retrieves any missing pieces of a local bag. generatepayloadoxum: Generates and returns the Payload-Oxum for the bag. makecomplete: Completes a bag and then writes in a specified format. Completing a bag fills in any missing parts. makeholey: Generates a fetch.txt and then writes bag in a specified format. retrieve: Retrieves a bag exposed by a web server. A local holey bag is not required. splitbagbyfiletype: Splits a bag by file types. splitbagbysize: Splits a bag by size. splitbagbysizeandfiletype: Splits a bag by size and file types. sword: Sends a bag using SWORD. update: Updates the manifests and (if it exists) the bag-info.txt for a bag. updatetagmanifests: Updates the tag manifests for a bag. The bag must be unserialized. verifycomplete: Verifies the completeness of a bag. verifypayloadmanifests: Verifies the checksums in all payload manifests. verifytagmanifests: Verifies the checksums in all tag manifests. verifyvalid: Verifies the validity of a bag. [--version] Prints version of BIL and exits. [--help] Prints usage message for the operation. Examples: bag verifyvalid --help Prints help for the verifyvalid operation. -- HARDY POTTINGER pottinge...@umsystem.edu University of Missouri Library Systems http://lso.umsystem.edu/~pottingerhj/ No matter how far down the wrong road you've gone, turn back. --Turkish proverb On 7/26/11 5:31 PM, Mark Diggory mdigg...@atmire.com wrote: Hardy, Be aware that MIT / Richard Rodgers also has some Bagit work available, currently nested within the modules directory here: http://scm.dspace.org/svn/repo/modules/dspace-replicate/trunk/src/main/jav a/org/dspace/pack/ http://scm.dspace.org/svn/repo/modules/dspace-replicate/trunk/src/main/ja va/org/dspace/pack/Mark On Tue, Jul 26, 2011 at 2:33 PM, Pottinger, Hardy J. pottinge...@umsystem.edu wrote: Hi, I've done a bit of googling on Bagit, and I see that Dryad (and @mire) have done some work with Bagit as a repository interchange mechanism. I am interested in something a bit more mundane. There exists a very nice tool for constructing a bag, called Bagger: http://sourceforge.net/projects/loc-xferutils/files/loc-bagger/ Which would be ideal for adapting for our needs--we need a tool that a scanner technician can use to feed scanned images into our repository. Bags, in my mind, are not much different than SAF packages. It would be trivial to script a converter between
Re: [Dspace-tech] Batch metadata corrections question: does anyone know why the limit is set to just 20 items at a time?
Hi Irene, We've wondered that too at my university, Ohio State University, so we've upped our setting to 600 which we feel is safe, but users typically do smaller batches. I'm guessing the too-low-for-practical-use limit of 20 is to be overly conservative by default, so that there is no risk of data loss or interruption. Doing batch changes with a large number of changes will keep your system busy, and the reindexing can take a while. I've noticed that when we set the limit to be really high, it appears that nothing will happen from the user's browser for 20+ minutes, so I've connected to the server from the command line, and noticed that the reindexing task was taking a long time, but still running. So you might be safe with setting this to a really high number (several thousand), you'll just have to have the patience to not disrupt it. But smaller / more manageable batch sizes will complete in a reasonable amount of time. With this set to 1000 or more, I'm guessing your more likely to run into Out-Of-Memory errors There is a note in: https://wiki.duraspace.org/display/DSDOC/System+Administration#SystemAdministration-BatchMetadataEditing that cautions against doing too large of batch edits. Peter Dietz On Wed, Jul 27, 2011 at 5:25 PM, Berry, Irene (CIV) icbe...@nps.edu wrote: Hello, We're experimenting with making batch corrections to metadata using the Import Metadata feature in the jsp. We'd like to raise the limit on the number of items that may be changed at a time. I can see the file: http://scm.dspace.org/svn/repo/dspace/tags/dspace-1.6.2/dspace-jspui/dspace-jspui-api/src/main/java/org/dspace/app/webui/servlet/MetadataImportServlet.java Where it says this: // Set the lmimt to the number of items that may be changed in one go, default to 20 limit = ConfigurationManager.getIntProperty(bulkedit.gui-item-limit, 20); log.debug(Setting bulk edit limit to + limit + items); …We’d like to up it from 20 to maybe 500 as an experiment -- but potentially much higher. Does anyone know if that's a really bad idea? We just don’t know what the consequence is of making this limit larger, but 20 seems way too low for a typical batch of changes. Thanks, Irene Berry, MLIS Digital Services Librarian Dudley Knox Library, Naval Postgraduate School -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Batch metadata corrections question: does anyone know why the limit is set to just 20 items at a time?
Hi, On 28/07/11 10:19, Peter Dietz wrote: Doing batch changes with a large number of changes will keep your system busy, and the reindexing can take a while. I've noticed that when we set the limit to be really high, it appears that nothing will happen from the user's browser for 20+ minutes, so I've connected to the server from the command line, and noticed that the reindexing task was taking a long time, but still running. So you might be safe with setting this to a really high number (several thousand), you'll just have to have the patience to not disrupt it. But smaller / more manageable batch sizes will complete in a reasonable amount of time. With this set to 1000 or more, I'm guessing your more likely to run into Out-Of-Memory errors With one of 'my' repositories, when we increased the limit (to 1000 I think), completing the changes took so long that the Apache-Tomcat connection timed out. This meant that the user saw an error in their browser even though the changes actually went through fine. In our case we decided to stick with a lower limit to avoid confusion. Though 20 really does feel very low. cheers, Andrea -- Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
