[Dspace-devel] [DuraSpace JIRA] Created: (DS-736) ItemImport - nicer handling of no contents file, and more efficient handling of no handle file
ItemImport - nicer handling of no contents file, and more efficient handling of no handle file -- Key: DS-736 URL: https://jira.duraspace.org/browse/DS-736 Project: DSpace Issue Type: Improvement Reporter: Graham Triggs Assignee: Graham Triggs Fix For: 1.7.0 Improves contents file handling, so that if the contents file is missing, but the directory only contains metadata files, then continue to import the item as a metadata record. If there are any non-metadata (and non-handle) files, then fail to import if the contents file is missing. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Achieve Improved Network Security with IP and DNS Reputation. Defend against bad network traffic, including botnets, malware, phishing sites, and compromised hosts - saving your company time, money, and embarrassment. Learn More! http://p.sf.net/sfu/hpdev2dev-nov ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Work started: (DS-736) ItemImport - nicer handling of no contents file, and more efficient handling of no handle file
[ https://jira.duraspace.org/browse/DS-736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on DS-736 started by Graham Triggs. ItemImport - nicer handling of no contents file, and more efficient handling of no handle file -- Key: DS-736 URL: https://jira.duraspace.org/browse/DS-736 Project: DSpace Issue Type: Improvement Reporter: Graham Triggs Assignee: Graham Triggs Fix For: 1.7.0 Improves contents file handling, so that if the contents file is missing, but the directory only contains metadata files, then continue to import the item as a metadata record. If there are any non-metadata (and non-handle) files, then fail to import if the contents file is missing. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Achieve Improved Network Security with IP and DNS Reputation. Defend against bad network traffic, including botnets, malware, phishing sites, and compromised hosts - saving your company time, money, and embarrassment. Learn More! http://p.sf.net/sfu/hpdev2dev-nov ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Resolved: (DS-619) Item exports not deleted on deletion of eperson
[ https://jira.duraspace.org/browse/DS-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Taylor resolved DS-619. - Resolution: Fixed Fix Version/s: 1.7.0 Commited to trunk revision 5444, Item exports not deleted on deletion of eperson --- Key: DS-619 URL: https://jira.duraspace.org/browse/DS-619 Project: DSpace Issue Type: Bug Components: DSpace API Affects Versions: 1.6.0, 1.6.1, 1.6.2 Reporter: Claudia Jürgen Assignee: Robin Taylor Fix For: 1.7.0 Attachments: DS-619.patch An eperson's item exports are not deleted when the eperson is deleted. This results in orphaned entries in [dspace.dir]/exports/download/[EpersonID] as the exports are only deleted if an eperson creates a new export and got old exports which exceed the timespan configured. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Achieve Improved Network Security with IP and DNS Reputation. Defend against bad network traffic, including botnets, malware, phishing sites, and compromised hosts - saving your company time, money, and embarrassment. Learn More! http://p.sf.net/sfu/hpdev2dev-nov ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Resolved: (DS-401) Wrong date issued during submission
[ https://jira.duraspace.org/browse/DS-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Taylor resolved DS-401. - Documentation Status: (was: Needed) Resolution: Fixed Commited to trunk revision 5719. Wrong date issued during submission --- Key: DS-401 URL: https://jira.duraspace.org/browse/DS-401 Project: DSpace Issue Type: Bug Components: JSPUI, XMLUI Affects Versions: 1.6.0 Reporter: Claudia Jürgen Assignee: Robin Taylor Priority: Major Fix For: 1.7.0 Attachments: Check-DCDate-parameters.patch During the submission of an item, which has been published before, it is possible to enter an invalid day, e.g. month: January day: 43 year: 2009 This will result in dc.date.issued set to 2009-02-12. There should be a validity check or calendar to choose from. The transformation of the date makes fault entries difficult to discover. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Achieve Improved Network Security with IP and DNS Reputation. Defend against bad network traffic, including botnets, malware, phishing sites, and compromised hosts - saving your company time, money, and embarrassment. Learn More! http://p.sf.net/sfu/hpdev2dev-nov ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] Java checked vs unchecked exceptions
Hi all, A couple of us have been having our own 'checked vs unchecked exceptions' debate on the IRC #dspace channel so I thought I would widen the audience to get some more opinions. I just changed DCDate to thow a checked exception if it is fed an invalid parameter eg 31st February. There are currently two places that call the code and hence now have to deal with the exception - DescribeStep and InstallItem. A checked exception seems the right thing in DescribeStep as you want to know about the problem and return an error message to the user, but in InstallItem you know the parameters will be valid because they are generated using java.util.Date rather then being user entered, so it's clumsy to have to catch an exception you know will never be thrown. So the questions is, should the exception be checked or unchecked in this case ? Any comments welcome. Thanks, Robin. Robin Taylor Main Library University of Edinburgh Tel. 0131 6513808 -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Achieve Improved Network Security with IP and DNS Reputation. Defend against bad network traffic, including botnets, malware, phishing sites, and compromised hosts - saving your company time, money, and embarrassment. Learn More! http://p.sf.net/sfu/hpdev2dev-nov ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
Re: [Dspace-devel] Java checked vs unchecked exceptions
Doh ! Ever been struggling with something, shared your thoughts and then immediately realised where you were going wrong ? Well that's what just happened to me. I just need to move the validation code to its own method that can be called if required. Apologies for adding to your spam. Cheers, Robin. Robin Taylor Main Library University of Edinburgh Tel. 0131 6513808 -Original Message- From: TAYLOR Robin Sent: 03 November 2010 11:44 To: dspace-devel@lists.sourceforge.net Subject: [Dspace-devel] Java checked vs unchecked exceptions Hi all, A couple of us have been having our own 'checked vs unchecked exceptions' debate on the IRC #dspace channel so I thought I would widen the audience to get some more opinions. I just changed DCDate to thow a checked exception if it is fed an invalid parameter eg 31st February. There are currently two places that call the code and hence now have to deal with the exception - DescribeStep and InstallItem. A checked exception seems the right thing in DescribeStep as you want to know about the problem and return an error message to the user, but in InstallItem you know the parameters will be valid because they are generated using java.util.Date rather then being user entered, so it's clumsy to have to catch an exception you know will never be thrown. So the questions is, should the exception be checked or unchecked in this case ? Any comments welcome. Thanks, Robin. Robin Taylor Main Library University of Edinburgh Tel. 0131 6513808 -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Achieve Improved Network Security with IP and DNS Reputation. Defend against bad network traffic, including botnets, malware, phishing sites, and compromised hosts - saving your company time, money, and embarrassment. Learn More! http://p.sf.net/sfu/hpdev2dev-nov ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- Achieve Improved Network Security with IP and DNS Reputation. Defend against bad network traffic, including botnets, malware, phishing sites, and compromised hosts - saving your company time, money, and embarrassment. Learn More! http://p.sf.net/sfu/hpdev2dev-nov ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
Re: [Dspace-devel] DSpace 1.7 Coming Soon -- Testing and Feedback starting Nov 8
I'd like to add that, if you have the time and resources and inclination, your experience with the installation and upgrade processes themselves is also valuable. Likewise, a critical reading of the documentation can be very helpful. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Balance your desire for bells and whistles with the reality that only a little more than 2 percent of world population has broadband. -- Ledford and Tyler, _Google Analytics 2.0_ pgpYBniAubo8P.pgp Description: PGP signature -- Achieve Improved Network Security with IP and DNS Reputation. Defend against bad network traffic, including botnets, malware, phishing sites, and compromised hosts - saving your company time, money, and embarrassment. Learn More! http://p.sf.net/sfu/hpdev2dev-nov___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
Re: [Dspace-devel] Removal of [dspace]/bin/ scripts
On Tue, Nov 02, 2010 at 06:31:45PM -0400, Peter Dietz wrote: I would vote to remove the perl scripts: dspace-info.pl, dspace_migrate, log-reporter Well, dspace-info.pl would be nice to have, but it doesn't work on a properly secured server (since tomcat and postgres shouldn't have access to each other's internal data), and I don't see how it could conveniently be made to, unless run as root. Those truly heroic 'sed' pipelines in dspace_migrate could probably be turned into an XSL transform and the whole thing redone in Java. Then the launcher can invoke it. That would also make this operation available to Windows sites. I took a quick look at log-reporter and didn't see anything obvious that would make it difficult to translate to Java. This and dspace-info.pl are the last two bits of Perl in the product. I've no objection to Perl myself, but there seems to be some movement to remove it from DSpace's requirements, and that wouldn't be a Bad Thing. Or they could all move to a contrib directory: nice things to have, but breakage would not be considered cause to delay a release. I don't know what to say about the handle shell scripts though, since we had to create 1.6.1 specifically to address starting the handle server. IMO we shouldn't be shipping scripts for configuring and starting someone else's product. We should just say what needs to be customized, and how, to link it with DSpace. make-handle-config has fallen out of sync. with the product in the past, and time taken to maintain start-handle-server would be better spent on e.g. Commons Daemon-izing the Handle server to make it more readily usable in a production environment. The current scripts couple DSpace too tightly to the Handle System -- if we need for it to be easier to use with DSpace, we should put our effort there, not here. It looks to me as though the bulk of the work in start-handle-server is very like some of what the launcher does. Why not invent 'dspace classpath' to facilitate minor customization of a more generic initscript for the Handle server (when it has one) and then drop start-handle-server? [type, type] OK, I invented it: see DS-737. Hmmm, a look at the launcher shows that it has a make-handle-config command, so maybe we don't need bin/make-handle-config anyway? -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Balance your desire for bells and whistles with the reality that only a little more than 2 percent of world population has broadband. -- Ledford and Tyler, _Google Analytics 2.0_ pgp8v9CTgF0n5.pgp Description: PGP signature -- Achieve Improved Network Security with IP and DNS Reputation. Defend against bad network traffic, including botnets, malware, phishing sites, and compromised hosts - saving your company time, money, and embarrassment. Learn More! http://p.sf.net/sfu/hpdev2dev-nov___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
Re: [Dspace-devel] Suggest Dependency Upgrade for 1.7: commons-dbcp 1.2.2 = 1.4
If no one has any opposition to this, I'd like to upgrade commons-dbcp to the latest version before 1.7 is cut. -- sands fish Senior Software Engineer MIT Libraries Technology Research Development sa...@mit.edumailto:sa...@mit.edu E25-131 On Sep 23, 2010, at 4:28 PM, Sands Alden Fish wrote: So Graham, is that a yes? :) I am not familiar with the side-effects or refactoring/adjusting that would be necessary for such a change so I can't quite comment on how feasible it is for 1.7. -- sands fish Software Engineer MIT Libraries Technology Research Development sa...@mit.edumailto:sa...@mit.edu E25-131 On Sep 23, 2010, at 4:19 PM, Graham Triggs wrote: If you want stability, then all JDBC classes - including DBCP (which is itself a DataSource / driver wrapper) - should be pushed down / delegated to the common Tomcat classloader. This would be necessary for JNDI configurations, but more importantly all JDBC drivers register into the JVM wider DriverManager. In itself, you might get away without it being the cause of stability issues, but it will prevent you from ensuring the web application classloader can be disposed of cleanly - which will prevent you finding other likely stability problems. G On 23 September 2010 21:05, Sands Alden Fish sa...@mit.edumailto:sa...@mit.edu wrote: There are quite a significant amount of bugs fixed in the 1.4 version of commons-dbcp. http://commons.apache.org/dbcp/changes-report.html#a1.4 Considering it is such a central library to DSpace, and since we are, pursuing enhanced stability, it would seem to make sense to pick up this version for 1.7. Peter and others? Does anyone have a reason we shouldn't use the more recent version of this dependency? -- sands fish Software Engineer MIT Libraries Technology Research Development sa...@mit.edumailto:sa...@mit.edu E25-131 -- Nokia and ATT present the 2010 Calling All Innovators-North America contest Create new apps games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.netmailto:Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel ATT1..cATT2..c ATT1..cATT2..c -- Achieve Improved Network Security with IP and DNS Reputation. Defend against bad network traffic, including botnets, malware, phishing sites, and compromised hosts - saving your company time, money, and embarrassment. Learn More! http://p.sf.net/sfu/hpdev2dev-nov___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Commented: (DS-731) add the capability to indicate a withdraw reason to an item ( tombstone )
[
https://jira.duraspace.org/browse/DS-731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=17976#action_17976
]
Jose Blanco commented on DS-731:
I noticed that in the file BitstreamServlet.java you need to add the following
( + ):
+ // The item has to be set for the tombstone.jsp to render
+ // the tombstone page
+ request.setAttribute( item, item );
String groupids =
ConfigurationManager.getProperty(tombstone.groupid);
add the capability to indicate a withdraw reason to an item ( tombstone )
-
Key: DS-731
URL: https://jira.duraspace.org/browse/DS-731
Project: DSpace
Issue Type: New Feature
Components: JSPUI
Reporter: Jose Blanco
Attachments: tombstone.zip
This patch will give jspui implementations the ability to configure their
instance to display a
tombstone page that shows the metadata and the reason for which the item was
withdrawn, or to
display a message indicating the item was withdrawn. In either case the
system
will output a HttpServletResponse.SC_NOT_FOUND status.
If you want users to see the tombstone page, you place the user in a group
and indicate that
group's id in the parameter tombstone.groupid. If you set the groupid to 0 (
the anonymous
group, for which all users belong to), then all the user will get the
tombstone page.
you need to add the following dc filed:
description.withdralreason
You will need the following new dspace.cfg parameter:
Tombstone Groups ##
# Groups for wich to display the tombstone page, rather than the error page
# this is a comma separated integer, can have more than one.
tombstone.groupid =
Files java files affected:
EditItemServlet.java
dspace-tag.tld
Message.properties
BitstreamServlet.java
HandleServlet.java
JSPManager.java
ItemTaga.java - so many changes I'm including the file.
The jsp files affected are:
webapp/withdrawnerror.jsp - this is a new file. And is included.
webapp/tools/confirm-withdraw-item.jsp - there are lots of changes here. I
have included the fie.
webapp/tombstone.jsp - this file has changed completely, so I'm just going to
include it.
CHNAGES TO=== EditItemServlet.java
267,268c267,271
+ // Withdraw the item
+ processWithdrawItem(context, request, response, item);
- // Withdraw the item
- item.withdraw();
- JSPManager.showJSP(request, response, /tools/get-item-id.jsp);
- context.complete();
774,791d776 ( new )
+ private void processWithdrawItem(Context context, HttpServletRequest
request,
+ HttpServletResponse response, Item item) throws
ServletException,
+ IOException, SQLException,
AuthorizeException
+ {
+ // Need to update the provenance information with the user selected
option.
+ String reason = request.getParameter(reason);
+
+ log.info(LogManager.getHeader(context, processWithdrawItem reason :
,reason ));
+
+ item.addDC(description, withdrawalreason, en, reason);
+ item.update();
+
+ item.withdraw();
+ JSPManager.showJSP(request, response, /tools/get-item-id.jsp);
+ context.complete();
+ }
CJHANGES TO == dspace-tags.tld
213,217d212
tag
nameitem/name
tagclassorg.dspace.app.webui.jsptag.ItemTag/tagclass
info
Tag for displaying an item. item must always be an
org.dspace.content.Item. style should be default or full, or can
be omitted to use default. collections should be the array of
collections the item is in, worked out beforehand to avoid the chance
of an error occurring during display. If collections is null, the
collections the item is in aren't listed.
/info
attribute
nameitem/name
requiredtrue/required
rtexprvaluetrue/rtexprvalue
/attribute
attribute
namestyle/name
requiredfalse/required
rtexprvaluetrue/rtexprvalue
/attribute
attribute
namecollections/name
requiredfalse/required
rtexprvaluetrue/rtexprvalue
+/attribute
+attribute
+ nametombstone/name
+ requiredfalse/required
+ rtexprvaluetrue/rtexprvalue
+/attribute
/tag
CHANGES == Add this to the end of the file dspace-tag.tld
+ #Tombstone
+ jsp.tombstone.reason1 = Removed from view by legal order.
+ jsp.tombstone.reason2 = Removed from view by DSpace.
+ jsp.tombstone.reason3 = Removed from view at request of the author.
+ jsp.tombstone.reasonforwithdraw = Select the reason for withdrawing item.
This will display in place of the files.
+ jsp.tombstone.accessdenied = The item you are trying
[Dspace-devel] [DuraSpace JIRA] Updated: (DS-615) Ability to perform maintenance on SOLR with solr.optimize
[ https://jira.duraspace.org/browse/DS-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Dietz updated DS-615: --- Attachment: screenshot_079.png Attaching a quick-and-dirty solution for performing solr.optimize on the search/discovery core. Not recommending this method as the way we do it, but it gets the job done. Also, I'd like to note that the initial update-discovery-index should switch to using autocommit, I've done this to my local install, and got a 10x speedup. So instead of needing a few hours, the indexing could be done in about 20 minutes. Other things like updating an existing record, or deleting a record should still use a hard solr.commit. Ability to perform maintenance on SOLR with solr.optimize - Key: DS-615 URL: https://jira.duraspace.org/browse/DS-615 Project: DSpace Issue Type: Improvement Components: Solr Environment: solr Reporter: Peter Dietz Assignee: Ben Bosman Fix For: 1.7.0 Attachments: screenshot_079.png, solr-optimize.patch, SolrOptimizeAndAutoCommit.patch By adding all historical log data to a SOLR index, as well as the constant addition of new records, it might be good for the efficient performance of a SOLR index to periodically run solr.optimize. Therefore I've created a patch which allows you to run solr.optimize on your solr instance from the command line. This could allow you to add a cron task that runs this periodic maintenance. It also spits out the amount of time taken to run the optimize task. Once you patch your instance. You can execute this with /dspace/bin/dspace stats-util --optimize or /dspace/bin/dspace stats-util -o Output looks like: SOLR Optimize -- Process Started:1277486321673 SOLR Optimize -- Process Finished:1277486321738 SOLR Optimize -- Total time taken:65 (ms). It would be useful to profile the amount of time taken for solr operations add/update/query before running this optimize task, and then afterwards. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- The Next 800 Companies to Lead America's Growth: New Video Whitepaper David G. Thomson, author of the best-selling book Blueprint to a Billion shares his insights and actions to help propel your business during the next growth cycle. Listen Now! http://p.sf.net/sfu/SAP-dev2dev ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Updated: (DS-615) Ability to perform maintenance on SOLR with solr.optimize
[ https://jira.duraspace.org/browse/DS-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Dietz updated DS-615: --- Attachment: screenshot_079.png Ability to perform maintenance on SOLR with solr.optimize - Key: DS-615 URL: https://jira.duraspace.org/browse/DS-615 Project: DSpace Issue Type: Improvement Components: Solr Environment: solr Reporter: Peter Dietz Assignee: Ben Bosman Fix For: 1.7.0 Attachments: screenshot_079.png, screenshot_079.png, solr-optimize.patch, SolrOptimizeAndAutoCommit.patch By adding all historical log data to a SOLR index, as well as the constant addition of new records, it might be good for the efficient performance of a SOLR index to periodically run solr.optimize. Therefore I've created a patch which allows you to run solr.optimize on your solr instance from the command line. This could allow you to add a cron task that runs this periodic maintenance. It also spits out the amount of time taken to run the optimize task. Once you patch your instance. You can execute this with /dspace/bin/dspace stats-util --optimize or /dspace/bin/dspace stats-util -o Output looks like: SOLR Optimize -- Process Started:1277486321673 SOLR Optimize -- Process Finished:1277486321738 SOLR Optimize -- Total time taken:65 (ms). It would be useful to profile the amount of time taken for solr operations add/update/query before running this optimize task, and then afterwards. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- The Next 800 Companies to Lead America's Growth: New Video Whitepaper David G. Thomson, author of the best-selling book Blueprint to a Billion shares his insights and actions to help propel your business during the next growth cycle. Listen Now! http://p.sf.net/sfu/SAP-dev2dev ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Issue Comment Edited: (DS-615) Ability to perform maintenance on SOLR with solr.optimize
[ https://jira.duraspace.org/browse/DS-615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=17982#action_17982 ] Peter Dietz edited comment on DS-615 at 11/3/10 9:59 PM: - Attaching a quick-and-dirty solution for performing solr.optimize on the search/discovery core. Not recommending this method as the way we do it, but it gets the job done. The patch can be run by running ./dspace update-discovery-index -o Also, I'd like to note that the initial update-discovery-index should switch to using autocommit, I've done this to my local install, and got a 10x speedup. So instead of needing a few hours, the indexing could be done in about 20 minutes. Other things like updating an existing record, or deleting a record should still use a hard solr.commit. was (Author: peterdietz): Attaching a quick-and-dirty solution for performing solr.optimize on the search/discovery core. Not recommending this method as the way we do it, but it gets the job done. Also, I'd like to note that the initial update-discovery-index should switch to using autocommit, I've done this to my local install, and got a 10x speedup. So instead of needing a few hours, the indexing could be done in about 20 minutes. Other things like updating an existing record, or deleting a record should still use a hard solr.commit. Ability to perform maintenance on SOLR with solr.optimize - Key: DS-615 URL: https://jira.duraspace.org/browse/DS-615 Project: DSpace Issue Type: Improvement Components: Solr Environment: solr Reporter: Peter Dietz Assignee: Ben Bosman Fix For: 1.7.0 Attachments: screenshot_079.png, screenshot_079.png, solr-optimize.patch, SolrOptimizeAndAutoCommit.patch By adding all historical log data to a SOLR index, as well as the constant addition of new records, it might be good for the efficient performance of a SOLR index to periodically run solr.optimize. Therefore I've created a patch which allows you to run solr.optimize on your solr instance from the command line. This could allow you to add a cron task that runs this periodic maintenance. It also spits out the amount of time taken to run the optimize task. Once you patch your instance. You can execute this with /dspace/bin/dspace stats-util --optimize or /dspace/bin/dspace stats-util -o Output looks like: SOLR Optimize -- Process Started:1277486321673 SOLR Optimize -- Process Finished:1277486321738 SOLR Optimize -- Total time taken:65 (ms). It would be useful to profile the amount of time taken for solr operations add/update/query before running this optimize task, and then afterwards. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- The Next 800 Companies to Lead America's Growth: New Video Whitepaper David G. Thomson, author of the best-selling book Blueprint to a Billion shares his insights and actions to help propel your business during the next growth cycle. Listen Now! http://p.sf.net/sfu/SAP-dev2dev ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Updated: (DS-615) Ability to perform maintenance on SOLR with solr.optimize
[ https://jira.duraspace.org/browse/DS-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Dietz updated DS-615: --- Attachment: (was: screenshot_079.png) Ability to perform maintenance on SOLR with solr.optimize - Key: DS-615 URL: https://jira.duraspace.org/browse/DS-615 Project: DSpace Issue Type: Improvement Components: Solr Environment: solr Reporter: Peter Dietz Assignee: Ben Bosman Fix For: 1.7.0 Attachments: search-solr-optimize-quick-and-dirty.patch, solr-optimize.patch, SolrOptimizeAndAutoCommit.patch By adding all historical log data to a SOLR index, as well as the constant addition of new records, it might be good for the efficient performance of a SOLR index to periodically run solr.optimize. Therefore I've created a patch which allows you to run solr.optimize on your solr instance from the command line. This could allow you to add a cron task that runs this periodic maintenance. It also spits out the amount of time taken to run the optimize task. Once you patch your instance. You can execute this with /dspace/bin/dspace stats-util --optimize or /dspace/bin/dspace stats-util -o Output looks like: SOLR Optimize -- Process Started:1277486321673 SOLR Optimize -- Process Finished:1277486321738 SOLR Optimize -- Total time taken:65 (ms). It would be useful to profile the amount of time taken for solr operations add/update/query before running this optimize task, and then afterwards. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- The Next 800 Companies to Lead America's Growth: New Video Whitepaper David G. Thomson, author of the best-selling book Blueprint to a Billion shares his insights and actions to help propel your business during the next growth cycle. Listen Now! http://p.sf.net/sfu/SAP-dev2dev ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Updated: (DS-615) Ability to perform maintenance on SOLR with solr.optimize
[ https://jira.duraspace.org/browse/DS-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Dietz updated DS-615: --- Attachment: search-solr-optimize-quick-and-dirty.patch Ability to perform maintenance on SOLR with solr.optimize - Key: DS-615 URL: https://jira.duraspace.org/browse/DS-615 Project: DSpace Issue Type: Improvement Components: Solr Environment: solr Reporter: Peter Dietz Assignee: Ben Bosman Fix For: 1.7.0 Attachments: search-solr-optimize-quick-and-dirty.patch, solr-optimize.patch, SolrOptimizeAndAutoCommit.patch By adding all historical log data to a SOLR index, as well as the constant addition of new records, it might be good for the efficient performance of a SOLR index to periodically run solr.optimize. Therefore I've created a patch which allows you to run solr.optimize on your solr instance from the command line. This could allow you to add a cron task that runs this periodic maintenance. It also spits out the amount of time taken to run the optimize task. Once you patch your instance. You can execute this with /dspace/bin/dspace stats-util --optimize or /dspace/bin/dspace stats-util -o Output looks like: SOLR Optimize -- Process Started:1277486321673 SOLR Optimize -- Process Finished:1277486321738 SOLR Optimize -- Total time taken:65 (ms). It would be useful to profile the amount of time taken for solr operations add/update/query before running this optimize task, and then afterwards. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- The Next 800 Companies to Lead America's Growth: New Video Whitepaper David G. Thomson, author of the best-selling book Blueprint to a Billion shares his insights and actions to help propel your business during the next growth cycle. Listen Now! http://p.sf.net/sfu/SAP-dev2dev ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Updated: (DS-615) Ability to perform maintenance on SOLR with solr.optimize
[ https://jira.duraspace.org/browse/DS-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Dietz updated DS-615: --- Attachment: (was: screenshot_079.png) Ability to perform maintenance on SOLR with solr.optimize - Key: DS-615 URL: https://jira.duraspace.org/browse/DS-615 Project: DSpace Issue Type: Improvement Components: Solr Environment: solr Reporter: Peter Dietz Assignee: Ben Bosman Fix For: 1.7.0 Attachments: search-solr-optimize-quick-and-dirty.patch, solr-optimize.patch, SolrOptimizeAndAutoCommit.patch By adding all historical log data to a SOLR index, as well as the constant addition of new records, it might be good for the efficient performance of a SOLR index to periodically run solr.optimize. Therefore I've created a patch which allows you to run solr.optimize on your solr instance from the command line. This could allow you to add a cron task that runs this periodic maintenance. It also spits out the amount of time taken to run the optimize task. Once you patch your instance. You can execute this with /dspace/bin/dspace stats-util --optimize or /dspace/bin/dspace stats-util -o Output looks like: SOLR Optimize -- Process Started:1277486321673 SOLR Optimize -- Process Finished:1277486321738 SOLR Optimize -- Total time taken:65 (ms). It would be useful to profile the amount of time taken for solr operations add/update/query before running this optimize task, and then afterwards. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- The Next 800 Companies to Lead America's Growth: New Video Whitepaper David G. Thomson, author of the best-selling book Blueprint to a Billion shares his insights and actions to help propel your business during the next growth cycle. Listen Now! http://p.sf.net/sfu/SAP-dev2dev ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Updated: (DS-268) XMLUI Item Mapper cannot handle multiple words in search box
[ https://jira.duraspace.org/browse/DS-268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrea Schweer updated DS-268: -- Attachment: administrative.js DS-268.patch Patch for administrative.js that removes escaping of query parameters etc; patched version of the file. XMLUI Item Mapper cannot handle multiple words in search box Key: DS-268 URL: https://jira.duraspace.org/browse/DS-268 Project: DSpace Issue Type: Bug Components: XMLUI Affects Versions: 1.5.1, 1.5.2 Reporter: Tim Donohue Attachments: administrative.js, DS-268.patch When searching in the XMLUI Item Mapper, multiple words (with a space in between) always returns no results. It seems like it could be an encoding problem, as inserting a plus symbol (+) in place of a space will return results. So, for example: A search on my topic (no quotes) will always return no results (even if there are matches). But, a search on my+topic (no quotes) will return the correct number of results. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- The Next 800 Companies to Lead America's Growth: New Video Whitepaper David G. Thomson, author of the best-selling book Blueprint to a Billion shares his insights and actions to help propel your business during the next growth cycle. Listen Now! http://p.sf.net/sfu/SAP-dev2dev ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Commented: (DS-268) XMLUI Item Mapper cannot handle multiple words in search box
[ https://jira.duraspace.org/browse/DS-268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=17986#action_17986 ] Andrea Schweer commented on DS-268: --- This also prevents searches that include special characters. The problem also affects searches for e-people and groups when performing administrative tasks, and possibly others. The reason appears to be that administrative.js URL-escapes query strings etc. This doesn't happen in, for example, submission.js and removing the escape method calls appears to fix the problem. I'm attaching a patch for administrative.js as well as a patched version of that file. XMLUI Item Mapper cannot handle multiple words in search box Key: DS-268 URL: https://jira.duraspace.org/browse/DS-268 Project: DSpace Issue Type: Bug Components: XMLUI Affects Versions: 1.5.1, 1.5.2 Reporter: Tim Donohue Attachments: administrative.js, DS-268.patch When searching in the XMLUI Item Mapper, multiple words (with a space in between) always returns no results. It seems like it could be an encoding problem, as inserting a plus symbol (+) in place of a space will return results. So, for example: A search on my topic (no quotes) will always return no results (even if there are matches). But, a search on my+topic (no quotes) will return the correct number of results. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- The Next 800 Companies to Lead America's Growth: New Video Whitepaper David G. Thomson, author of the best-selling book Blueprint to a Billion shares his insights and actions to help propel your business during the next growth cycle. Listen Now! http://p.sf.net/sfu/SAP-dev2dev ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Updated: (DS-268) XMLUI Item Mapper cannot handle multiple words in search box
[ https://jira.duraspace.org/browse/DS-268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrea Schweer updated DS-268: -- Comment: was deleted (was: Patch for administrative.js that removes escaping of query parameters etc; patched version of the file.) XMLUI Item Mapper cannot handle multiple words in search box Key: DS-268 URL: https://jira.duraspace.org/browse/DS-268 Project: DSpace Issue Type: Bug Components: XMLUI Affects Versions: 1.5.1, 1.5.2 Reporter: Tim Donohue Attachments: administrative.js, DS-268.patch When searching in the XMLUI Item Mapper, multiple words (with a space in between) always returns no results. It seems like it could be an encoding problem, as inserting a plus symbol (+) in place of a space will return results. So, for example: A search on my topic (no quotes) will always return no results (even if there are matches). But, a search on my+topic (no quotes) will return the correct number of results. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- The Next 800 Companies to Lead America's Growth: New Video Whitepaper David G. Thomson, author of the best-selling book Blueprint to a Billion shares his insights and actions to help propel your business during the next growth cycle. Listen Now! http://p.sf.net/sfu/SAP-dev2dev ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
