Re: [Dspace-tech] 1.6 Memory problems
I think that networking/name service stuff on the host is configured so that a call in Java to new InetAddress.getLocalHost().getHostAddress() will return 127.0.0.1, instead of the ethernet IP. Could this be conflicting with anything in the LocalHostRestrictionFilter that ships with the DSpace solr module? - Original Message - From: "Mark Diggory" To: "bill anderson" Cc: "dspace-tech" Sent: Wednesday, April 7, 2010 1:21:19 PM GMT -05:00 US/Canada Eastern Subject: Re: [Dspace-tech] 1.6 Memory problems You can temporarily move the /dspace/solr/statistics/data directory and restart solr, this should allow you to run without the imported statistics. In 1.6.1 we should probably be catching this error and emailing the administrator with it rather than letting it break the UI. Lets continue the thread to figure out why it is failing to load. Maybe you can give me some details on the data directoy. Mark On Wed, Apr 7, 2010 at 7:26 AM, wrote: > We upgraded to 1.6 yesterday. We tested for a couple of hours with no > problem, and then I converted the statistics for solr, set off the statistics > import script, and went home. When I got in this morning, our DSpace > installation was crashing constantly due to memory errors, and the final > output from the stat script looked like this: > > About to commit data to solr...Exception in thread "main" > org.apache.solr.common.SolrException: Bad Gateway > > Bad Gateway > > request: http://smartech.gatech.edu/solr/statistics/update > at > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:343) > at > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:183) > at > org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:217) > at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:85) > at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:74) > at > org.dspace.statistics.util.StatisticsImporter.load(StatisticsImporter.java:387) > at > org.dspace.statistics.util.StatisticsImporter.main(StatisticsImporter.java:493) > > We've restarted several times; but every time we do, solr begins madly > throwing 403s (see below), and eventually java runs out of memory, and we > crash. I've checked the lists and forums, but haven't found any reference to > this problem -- has anyone else seen it? > > Alternatively, can anyone suggest a quick and painless way to shut off solr > statistics until we track this down? > > Thanks, > > Bill > > > > 2010-04-07 10:20:38,009 ERROR org.dspace.statistics.SolrLogger @ Forbidden > > Forbidden > > request: > http://smartech.gatech.edu/solr/statistics/update?wt=javabin&version=2.2 > org.apache.solr.common.SolrException: Forbidden > > Forbidden > > request: > http://smartech.gatech.edu/solr/statistics/update?wt=javabin&version=2.2 > at > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:343) > at > org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:183) > at > org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:217) > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:63) > at org.dspace.statistics.SolrLogger.post(SolrLogger.java:244) > at > org.dspace.statistics.SolrLoggerUsageEventListener.receiveEvent(SolrLoggerUsageEventListener.java:41) > at > org.dspace.services.events.SystemEventService.fireLocalEvent(SystemEventService.java:154) > at > org.dspace.services.events.SystemEventService.fireEvent(SystemEventService.java:97) > at > org.dspace.app.webui.servlet.BitstreamServlet.doDSGet(BitstreamServlet.java:218) > at > org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:151) > at > org.dspace.app.webui.servlet.DSpaceServlet.doGet(DSpaceServlet.java:99) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.dspace.utils.servlet.DSpaceWebappServletFilter.doFilter(DSpaceWebappServletFilter.java:112) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) >
Re: [Dspace-tech] harvesting non-DSpace repositories - dc.identifier.uri issue
Good morning, Tim. I think that you're right about populating the appropriate fields on the repository side. It looks like the OAI-PMH generator for your repository is placing the identifier (URL) in the unqualified "identifier" field, instead of "identifier.uri". This is common, as many OAI-PMH gateways will only support the OAI-DC metadata format, which does not include the qualifiers, by default. If you can configure your repository side to generate another metadata format that will place your identifier field in identifier.uri, that would allow DSpace to identify your URL when it harvests. An example is the qdc format that the DSpace 1.6 harvester supports. The DSpace harvester also has a couple of parameters in the configuration file, harvester.acceptedHandleServer and harvester.rejectedHandlePrefix, that can prevent DSpace from generating that extra hdl.handle.net URL. They are documented in the configuration file. If you're unable to place your original URL from the repository in identifier.uri, you may still be able to work around this by changing the interface in DSpace to display the unqualified "identifier" field next to the "URI:" label on the item view pages. You'd probably want to limit this so that it only does so if the identifier starts with "http://scaa.suask.ca"; in case your repository also maps citations, issn, doi to that field. There are some interface HOW-TOs on the wiki that may help, and people on this list with experience in changing the display for the jspui and the xmlui, if you need to go that route. --keith - Mensaje original - De: "Tim Hutchinson" Para: dspace-tech@lists.sourceforge.net Enviados: Viernes, 7 de Mayo 2010 15:58:55 GMT -05:00 Región oriental EE. UU./Canadá Asunto: [Dspace-tech] harvesting non-DSpace repositories - dc.identifier.uri issue Hi, I have been involved in testing DSpace to harvest a collection not hosted in DSpace, but which is OAI compliant. We have been successful in doing the harvesting, but not successful in having the DSpace record point to the correct external record. The issue is that DSpace assigns a URL of the form http://hdl.handle.net/123456789/x to dc.identifier.uri although dc.identifier appears correctly, e.g. http://scaa.usask.ca/gallery/northern/image.php?ID=10857 (the main problem is that the correct identifier is not hyperlinked, and doesn’t appear in the simple view) As you can see, we are running a test implementation of DSpace and have not implemented the handle server, hence the “123456789” prefix. But I’m not sure that’s the only issue. What I’m wondering is: - Is there a way, by populating appropriate fields on the repository side or by configuring DSpace, or otherwise, to have the correct identifier (URL) show up in dc.identifier.uri, especially for the purpose of the test implementation? - Will implementing the handle server actually solve the problem? It’s not clear to be how this will work, since wouldn’t we need to register a prefix (or at least set up a mapping) for each external collection? As may be obvious, I'm not too familiar with how handles work. As additional background, I did also test with a repository using the OAI identifier format, i.e. oai:domain:id. The element dc.identifier resolves correctly, but dc.identifier.uri still uses the non-functional handle. On the other hand, harvesting from a DSpace collection works properly, with dc.identifier.uri being populated with the handle assigned by the other DSpace instance. In case it makes a difference, we are running DSpace version 1.6.0. Thanks for any guidance on this. Tim -- Tim Hutchinson University of Saskatchewan Archives 301 Main Library, 3 Campus Drive Saskatoon, SK S7N 5A4 tel: (306) 966-6028 fax: (306) 966-6040 e-mail: tim.hutchin...@usask.ca web: http://www.usask.ca/archives/ -- ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Solr Service Unavailable - Statistics
A colleague in charge of administering a larger repository mentioned seeing these errors on Friday. I looked through the log files on a smaller set of test systems that I've been using, and I also have them. They occur infrequently, maybe 2-3 times in a 24 hour period. The stats updates work most of the time, but it looks like every once in a while the update fails. I haven't been able to cause them to happen on my systems at will - they just happen occasionally. On his system, which receives substantial traffic, he can cause the errors by visiting an item page and rapidly clicking and re-clicking the refresh button on the browser. I think that he was going to adjust some solr settings to see if that fixed the error. If I hear more about this or have a chance to investigate some more I'll let you know. Please do the same if you find a solution! --keith - Mensaje original - De: "Antoanne Christopher" Para: dspace-tech@lists.sourceforge.net Enviados: Viernes, 7 de Mayo 2010 16:53:59 GMT -05:00 Región oriental EE. UU./Canadá Asunto: [Dspace-tech] Solr Service Unavailable - Statistics After update my DSpace instance [ http://virtualbib.fgv.br/dspace ] from 1.5.2 to 1.6.0 and start to user solr statistics, I have received several erros on dspace.log with message "Service Unavailable" about solr exception, . Someone has seen this error? Know how to configure the solr transacions log to see exactly waht is happening during the solr transactions? 2010-05-07 17:28:15,500 ERROR org.dspace.statistics.SolrLogger @ Service Unavailable Service Unavailable request: http://virtualbib.fgv.br/solr/statistics/update org.apache.solr.common.SolrException: Service Unavailable Service Unavailable request: http://virtualbib.fgv.br/solr/statistics/update at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:343) at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:183) at org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:217) at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:85) at org.dspace.statistics.SolrLogger.post(SolrLogger.java:246) at org.dspace.statistics.SolrLoggerUsageEventListener.receiveEvent(SolrLoggerUsageEventListener.java:41) at org.dspace.services.events.SystemEventService.fireLocalEvent(SystemEventService.java:154) at org.dspace.services.events.SystemEventService.fireEvent(SystemEventService.java:97) at org.dspace.app.xmlui.cocoon.BitstreamReader.setup(BitstreamReader.java:372) at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71) at $Proxy29.setup(Unknown Source) at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.setupReader(AbstractProcessingPipeline.java:560) at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.preparePipeline(AbstractProcessingPipeline.java:464) at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.process(AbstractProcessingPipeline.java:411) at sun.reflect.GeneratedMethodAccessor99.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71) at $Proxy28.process(Unknown Source) at org.apache.cocoon.components.treeprocessor.sitemap.ReadNode.invoke(ReadNode.java:94) at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78) at org.apache.cocoon.components.treeprocessor.sitemap.SelectNode.invoke(SelectNode.java:87) at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:55) at org.apache.cocoon.components.treeprocessor.sitemap.MatchNode.invoke(MatchNode.java:87) at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:55) at org.apache.cocoon.components.treeprocessor.sitemap.MatchNode.invoke(MatchNode.java:87) at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78) at org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode.invoke(PipelineNode.java:143) at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78) at org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode.invoke(PipelinesNode.java:81) at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTre
Re: [Dspace-tech] Handle server problems after update to 1.6.1 from 1.5.2
Try editing the last line of your bin/start-handle-server script so that net.handle.server.Main appears directly after dsrun (instead of where it is now). That should get rid of the "Invalid class name:" message. If there are other error messages after making this change, please post them to the list so that experts on the "dspace" launcher command can research more. --keith - Original Message - From: "Tonny Hjelmberg Laursen" To: dspace-tech@lists.sourceforge.net Sent: Monday, May 24, 2010 7:55:29 AM GMT -05:00 US/Canada Eastern Subject: [Dspace-tech] Handle server problems after update to 1.6.1 from 1.5.2 Hey, Im in the middle of updating a DSpace installation and everything seems fine. But im having a problem with starting the handle server. When I execute bin/start-handle-server im getting this error in the handle-server logfile: Error in launcher.xml: Invalid class name: -Ddspace.log.init.disable=true Im running RedHat 5.5 Any ideas? Thanks, Tonny -- ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Handle server problems after update to 1.6.1 from 1.5.2
Goodness! Enjoy the rest of the day! We'll open a bug report and someone will find a better fix for you. - Original Message - From: "Tonny Hjelmberg Laursen" To: "keith gilbertson" Cc: dspace-tech@lists.sourceforge.net Sent: Monday, May 24, 2010 10:38:06 AM GMT -05:00 US/Canada Eastern Subject: Re: [Dspace-tech] Handle server problems after update to 1.6.1 from 1.5.2 One of the things that I didn't test on the test server (1.6.0) was the handle-server. I have now replaced dsrun with the one from the 1.6.0 source package and that seems to have fixed it. Thanks alot for your help Keith ;) I won't do anymore tests today, since it's a day off and I really need som fresh air. But I will be willing to do more testing tomorrow if it's needed, but it seemslike theres a bug here :( Tonny On 24-05-2010 16:23, keith.gilbert...@library.gatech.edu wrote: > There are log4j configuration files in your config directory, > including a separate one for the handle server. However, I'm not sure > that the handle server is being started in your case. > > You can change the word INFO to DEBUG in these files. This will fill > up disk space rapidly so remember to change it back when finished. > > I'm wondering if your handle server works under 1.6.0? Did you have a chance > to try everything with 1.6.0? You could try using the dsrun file from the > 1.6.0 release, but I'm very much guessing blindly now, so use at your own > risk: > > http://scm.dspace.org/svn/repo/dspace/tags/dspace-1.6.0/dspace/bin/dsrun > > > Also, are you starting the handle server from within the "bin" directory? (I > think that's the way it should be done). > > > > - Original Message - > From: "Tonny Hjelmberg Laursen" > To: "Keith R Gilbertson" > Cc: dspace-tech@lists.sourceforge.net > Sent: Monday, May 24, 2010 9:38:51 AM GMT -05:00 US/Canada Eastern > Subject: Re: [Dspace-tech] Handle server problems after update to 1.6.1 from > 1.5.2 > > Thanks, > > The error is gone now, but now I got another error in the log file (The > handle server is still not running.) > > Usage: > > The bin/start-handle-server line looks like this now: nohup > $BINDIR/dsrun net.handle.server.Main $handledir > -Ddspace.log.init.disable=true > -Dlog4j.configuration=log4j-handle-plugin.properties> > $logdir/handle-server.log 2>&1& > > I can't find any other error messages. How can I enable somekind of > debug logging? > > > On 24-05-2010 15:18, Gilbertson, Keith R wrote: > >> Try editing the last line of your bin/start-handle-server script so that >> net.handle.server.Main >> appears directly after dsrun (instead of where it is now). >> >> That should get rid of the "Invalid class name:" message. If there are >> other error messages after making this change, please post them to the list >> so that experts on the "dspace" launcher command can research more. >> >> --keith >> >> >> - Original Message - >> From: "Tonny Hjelmberg Laursen" >> To: dspace-tech@lists.sourceforge.net >> Sent: Monday, May 24, 2010 7:55:29 AM GMT -05:00 US/Canada Eastern >> Subject: [Dspace-tech] Handle server problems after update to 1.6.1 from >> 1.5.2 >> >> Hey, >> >> Im in the middle of updating a DSpace installation and everything seems >> fine. But im having a problem with starting the handle server. When I >> execute bin/start-handle-server im getting this error in the >> handle-server logfile: >> >> Error in launcher.xml: Invalid class name: -Ddspace.log.init.disable=true >> >> Im running RedHat 5.5 >> >> Any ideas? >> >> Thanks, >> Tonny >> >> >> >> >> >> -- >> >> ___ >> DSpace-tech mailing list >> DSpace-tech@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/dspace-tech >> >> >> >> > > > > > -- ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Handle server problems after update to 1.6.1 from 1.5.2
The first suggestion that I made to Tonny, to edit the start-handle-server script and change the order of the parameters, was a bad one and didn't work. So, if you made changes to the start-handle-server script, undo the changes and put the start-handle-server script back the way it was originally. - Mensaje original - De: "Ricardo Borillo" Para: "Keith R Gilbertson" CC: "Tonny Hjelmberg Laursen" , dspace-tech@lists.sourceforge.net Enviados: Martes, 25 de Mayo 2010 8:43:13 GMT -05:00 Región oriental EE. UU./Canadá Asunto: Re: [Dspace-tech] Handle server problems after update to 1.6.1 from 1.5.2 Hi all, I have followed your indications, but i still get this error message and my handle-server didn't start ... I have replaced my 1.6.1 dsrun file with the one in the 1.6.0 tag and i get the same error: Usage: Thanks a lot --- Salut, Ricardo Borillo Domenech http://xml-utils.com / http://twitter.com/borillo -- ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Collection item count in 1.6.2
Hello Mika. If you are using xmlui, make sure that webui.strengths.cache = true in your dspace.cfg file after the upgrade. - Original Message - From: "mikan.d.dspace listmail" To: "Dspace Tech" Sent: Wednesday, June 16, 2010 7:54:09 AM GMT -05:00 US/Canada Eastern Subject: [Dspace-tech] Collection item count in 1.6.2 I upgraded my DSpace 1.6.0 to 1.6.2 and replaced the default themes with my own. Everything else seems to be working fine, but the collection / community strengths are not shown. I had this configured in my themes which work fine in DSpace 1.6.0. Any hints where the problem might be ? thanks, Mika -- ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] XMLUI/Manakin theme with timeout warning?
Has anyone implemented a "Warning - your session is about to timeout - Click here to stay logged in" type of pop-up message for the xmlui, via a theme or otherwise? I think this may be useful for administrators to have and was hoping someone has already done this. --keith -- This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
