Re: [Dspace-tech] Nested Metadata

[Graham Triggs]

Hi Peter,

I think you may not be too far off with that approach.

However, one key thing that gets missed - you provide an example where
dc.contributor.author that captures rich metadata. Which we shouldn't be
doing - dublin core is meant to be simple, and simple data is what we
should be capturing in the "dc" schema.

Now, we can register any other schemas we like, and the fields will store
strings opaquely. We can throw any format we want to define into there. For
instance, it's feasible to have a "mods" schema, and in it:

mods.titleinfo =
At Gettysburg, or, What a Girl Saw and Heard of the
Battle: A True Narrative**

**
The problem then is that there isn't anything built in for handling the
display or data entry of that rich field definition, but then that would
basically be true of any arbitrary rich structure.

G


On 4 August 2015 at 14:53, Peter Dietz  wrote:

> Thanks for all the responses, and insight thus far.
>
> My thinking about the problem space of nested metadata, is that that is
> trying to solve the problem of metadata about metadata about an item.
> Item/1234 => Author => [Peter Dietz, Developer, Longsight, Brown eyes, ...]
>
> What we've been doing is that all metadata objects are really of one type,
> and thats text. We don't care what it is, we don't validate it, we just
> store the text value of whatever you've given us. Some of the problems that
> we run into is for date fields. They try to parse the text, and its not a
> valid/parseable date. Date => Unknown, Date => 2015/04, Date => 1960's,
> Date => 08/04/2015. We could/should validate things stored in the date-type
> of metadata fields as ISO8601.
>
> So, for storing other types of information. In the case of tying a
> metadata field to be backed by some authority control system, we store a
> foreign key / reference, and then SOLR stores an encoding of the metadata
> we fetch from the metadata service provider. In the case of the ORCID
> integration it can grab:
>
> givenNames,  familyName,  creditName,  otherNames, country, keyword,
> external_identifier, researcher_url, biography
>
> So we have a form of a schema for storing this "object" inside of a
> metadata value. Our current metadata system is basically a key/value store.
> key = metadata field (i.e. dc.title), and value is unspecified, but usually
> just text. Could we validate that we have a type called
> _nested_orcid_author, which has to be json, and only contain the above
> fields? That looks like an object, an OrcidAuthor object. We'd need a
> schema to enforce that. But then we're building tables and classes for that
> field. Maybe some type of key/key/value store would be appropriate?
> dc.author => {{ _nested_metadata_object }}
>
> Then a NestedMetadataObject can have keys (metadata_field_id) , and values
> (unspecified text).
>
> So.
> NestedMetadataValues nmValues = item.getMetadata('dc.author');
> nmValue0 = nmValues[0];
> nmValue0.getMetadata('dc.author.firstname') ==> "Peter Dietz"
>
>
> That approach. Or is it best to stick with the authority framework. Build
> some type of MetadataAuthorityProvider for each "rich" / "nested" metadata
> object? But, if I need to have 10 fields that each need a metadata
> authority backing store... And there is no Library of Congress metadata
> service provider for each, do you need to construct your own metadata
> silos? Could you build a single external metadata service provider system,
> that could be integrated with DSpace, and be mapped to 10 different fields?
> Author (firstname, lastname, institution), Review(# of stars, title,
> description), Link(link name, url), ScientificClassification(Kingdom,
> Phylum, Class, Order, Suborder, Family, Genus, Species), ...
>
> For reference, I've stored some items with the value for author serialized
> as JSON.
> https://trydspace.longsight.com/handle/123456789/175
> https://trydspace.longsight.com/rest/handle/123456789/175?expand=all
>   "metadata": [
> {
>   "key": "dc.contributor.author",
>   "value": "{firstname:Mary Davis, lastname:MacNaughton, role:Editor}",
>   "language": ""
> },
> {
>   "key": "dc.contributor.author",
>   "value": "{firstname:Michael, lastname:Duncan, role:Contributor}",
>   "language": ""
> },
>
> http://dspace-rest-client-play.herokuapp.com/item/202
> http://dspace-rails.herokuapp.com/item/202
>
> Or, is flat metadata really best? Do you really need DSpace to store
> metadata about metadata (i.e. Author.eye-color), or is storing "Dietz,
> Peter" sufficient, or just our current limitation.
>
>
> 
> Peter Dietz
> Longsight
> www.longsight.com
> pe...@longsight.com
> p: 740-599-5005 x809
>
> On Thu, Jul 30, 2015 at 9:06 AM, Mark H. Wood  wrote:
>
>> On Wed, Jul 29, 2015 at 04:06:19PM -0400, Peter Dietz wrote:
>> > Has anyone stored nested / rich metadata in DSpace?
>> >
>> > An example I'm thinking of is for storing richer amounts of metadata
>> for an
>> > object. For example:
>> >
>> >- Aut

Re: [Dspace-tech] tomcat bound to port 80

[Graham Triggs]

Hi,

The error that you have received is because the main DSpace application
can't communicate with the secondary Solr application - which by default it
expects to be in the same Tomcat, and on the default port of 8080.

Note that the circumstances of this error point to a very good reason why
you should NOT relocate the Tomcat port.

By putting the main Tomcat connector on port 80, you are - by default -
exposing(*) all of the applications hosted in that Tomcat to the internet.
This includes the "internal" support applications like Solr, which may be a
security risk.

(* this may very depending on other firewalls that you may be running).

If you instead use something like Apache HTTPd with mod_proxy / mod_jk, you
can be more explicit about what applications you mount through HTTPd,
limiting what is exposed to potential attacks. Also, it gives you a place
to more easily add other security modules / features.

Plus, if it is ever necessary, it gives you a means to disable general
access to the underlying application, whilst you still have direct access
to the Tomcat application.

Regards,
Graham

On 3 August 2015 at 16:09, Yongming Wang  wrote:

> Hi,
>
> I'm new to DSpace. My situation is: DSpace 5.2 on RH Enterprise Linux 7,
> using jspui.
>
> I'm trying to change the default tomcat port 8080 to 80. I searched the
> Internet and dspace mailing-list and decided to use the following method:
>
> Step one: modify [tomcat]/conf/server.xml, change connector port from 8080
> to 80
> Step two: install the AUTOBIND package. But instead my school IT
> department installed something called "jsvc" which they told me is newer
> and better than AUTHBIND.
>
> Now I got some internal error message when trying to access the
> application homepage at port 80. I'll paste it below.
>
> Does anyone have any suggestions and insight to this issue?
>
> Or in general, how do you change the 8080 port to 80 so user won't need to
> type 8080 to access the application? What is the simplest way?
>
>
>
> Thanks very much in advance!
>
>
>
> Yongming
>
> -
> Yongming Wang
> Systems Librarian
> The College of New Jersey Library
> phone: 609-771-3337
> email: wan...@tcnj.edu
> -
>
> exception stack:
>
> -- URL Was: http://xx.tcnj.edu/jspui/internal-error
> -- Method: GET
> -- Parameters were:
>
>
> Exception:
> org.apache.jasper.JasperException: javax.servlet.ServletException:
> org.dspace.plugin.PluginException:
> org.dspace.app.webui.components.RecentSubmissionsException:
> org.dspace.browse.BrowseException:
> org.dspace.discovery.SearchServiceException: Server refused connection at:
> http://localhost:8080/solr/search
> at
> org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:549)
> at
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:455)
> at
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390)
> at
> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.dspace.utils.servlet.DSpaceWebappServletFilter.doFilter(DSpaceWebappServletFilter.java:78)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
> at
> org.

Re: [Dspace-tech] dspace import throws NullPointerException

[Graham Triggs]

Hi,

I think the point is that the value is returning null - e.g. it is not set
in your dspace.cfg - and so a null is passed into the new File(), and
throwing a NullPointerException.

Ugh.

That code needs updating - the static initializer in ItemImport ought to
have try {} catch around it, and there should be explicit handling of the
configuration property not being present (null).

G

On 31 July 2015 at 11:31, Layale Bassil  wrote:

> Hello Stefan,
>
>
>
> I am also facing exception when I try to import. I am using the same
> import command as the one you used, and I am getting the following
> exception at Line 108 which is (File tempWorkDirFile = new
> File(tempWorkDir);
>
>
>
> Exception in thread "main" java.lang.ExceptionInInitializerError
>
> at java.lang.Class.forName0(Native Method)
>
> at java.lang.Class.forName(Class.java:274)
>
> at
> org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:141)
>
> at
> org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:78)
>
> Caused by: java.lang.NullPointerException
>
> at java.io.File.(File.java:277)
>
> at
> org.dspace.app.itemimport.ItemImport.(ItemImport.java:108)
>
> ... 4 more
>
>
>
> The error is not telling me anything, but it might be related to creating
> the temp working directory. Where can I find the value of this property:
> ConfigurationManager.getProperty("org.dspace.app.batchitemimport.work.dir");
> ?
>
>
>
> Your help is very much appreciated to know how to solve this issue.
>
>
>
> Thanks.
>
>
>
> -Original Message-
> From: Stefan Fritzsche [mailto:stefan.fritzs...@tu-dresden.de]
> Sent: Thursday, July 30, 2015 1:50 PM
> To: dspace-tech@lists.sourceforge.net
> Subject: [Dspace-tech] dspace import throws NullPointerException
>
>
>
> Dear Members,
>
>
>
> i try to ingest some content with the commandline tool "dspace import".
>
>
>
> I want to ingest the content via the Simple Archive Format (
> https://wiki.duraspace.org/display/DSDOC5x/Importing+and+Exporting+Items+via+Simple+Archive+Format
> ).
>
>
>
>
>
> The ingest is started with
>
>
>
> dspace@server> dspace import -a -e f...@bar.de -s . -m mapfile.
>
> No collections given. Assuming 'collections' file inside item directory
> Adding items from directory: .
>
> Generating mapfile: mapfile
>
> Processing collections file: collections ...
>
>
>
> The content of the collection file is "123456789/9".
>
>
>
> When i start the ingest, the process throws a NullPointerException but if
> i use the -c option with the collection_id from the file it works.
>
>
>
> I use the debugger to find the cause of the exception. The method
> addItem(Context c, Collection[] mycollections, String path, String
> itemname, PrintWriter mapOut, boolean template) throws Exception) in the
> package org.dspace.app.itemimport throws the exception.
>
> I determine that the method addItem is called with the parameter
> mycollections = null but if i use the -c option the parameter holds the
> collection_id.
>
>
>
> I add at line 736 the statement mycollections = clist. After this change
> the process behave like -c option is set.
>
>
>
> My questions are:
>
> Is this a bug and should i report it?
>
> Has my fix some side effects?
>
>
>
> Thanks in advance,
>
> Stefan
>
>
>
>
> --
>
> ___
>
> DSpace-tech mailing list
>
> DSpace-tech@lists.sourceforge.net
>
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
> List Etiquette:
> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>
>
> --
>
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> List Etiquette:
> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>
--
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette

Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)

[Graham Triggs]

Is it at all possible that you have two separate processes / applications 
attempting to access the same Solr index - so one locks it, and the other is 
unable to?

For example, maybe a context file under /config/Catalina/localhost and 
an application in /webapps?

Regards,
G

On 08/04/2015 17:17:08, George Stanley Kozak  wrote:
Hi, Tim:

Yes, this has me stumped.
Here is the permissions on the writelock:

-rw-rw-r-- 1 dspace dspace 0 Apr 8 09:59 write.lock

Tomcat is owned by the dspace user:

drwxr-xr-x 3 dspace dspace 4096 Apr 2 2014 tomcat

Solr is owned by the dspace user:

drwxrwxr-x 6 dspace dspace 4096 Apr 8 09:58 solr

I did as you suggested and stopped tomcat, recursively changed the permissions 
on the solr indexes:

sudo chown -R dspace:dspace /cul/app/dspace/solr

I deleted the write.lock and restarted tomcat.

I then did /cul/app/dspace/bin/dspace index-discovery -b

Immediately, I received the write lock error and the re-index did not work.

I restored the postgres database and solr indexes to what they were before and 
did a mvn build and ant update of my DSpace 4.3 source and everything works 
fine(?!?!)
There is obviously some subtle difference on how DSpace 5.1 and DSpace 4.3 (and 
lower) are handling the solr indexes and my server set up is just right (or 
wrong) enough to cause this problem.

George Kozak
Digital Library Specialist
Cornell University Library Information Technologies (CUL-IT)
218 Olin Library
Cornell University
Ithaca, NY 14853
607-255-8924



-Original Message-
From: Tim Donohue [mailto:tdono...@duraspace.org]
Sent: Wednesday, April 08, 2015 10:16 AM
To: George Stanley Kozak; dspace-tech@lists.sourceforge.net
Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional 
information)

Hi George,

This really has me stumped. It almost starts to sound like a permissions issue 
in your index directory (though I know from looking back at this thread, you 
said you looked at that).

Since you essentially started with a "fresh" index (no content), and it still 
doesn't work, this doesn't sound like an issue with the index files themselves. 
While DSpace 5.1 uses a newer version of Solr, it doesn't act that much 
different from DSpace 4.3. So, it's confusing to me that this works for you on 
DSpace 4.3 and not on DSpace 5.1.

When you encounter that "Index locked for write for core search", there should 
be a "write.lock" file sitting in your /cul/app/dspace/solr/search/data/index/ 
folder. What are the permissions on that "write.lock" file? Is it owned by your 
Tomcat user?
Also is *everything* under /cul/app/dspace/solr/ owned by the Tomcat user (all 
files and subdirectories)?

You may even want to try and do the following:

1. Stop Tomcat
2. Recursively change the ownership (just in case), e.g. chown -R 
[tomcat-user]:[tomcat-group] /cul/app/dspace/solr/ 3. Delete that "write.lock"
4. Start Tomcat again.
5. (You may also need to manually reindex: 'dspace index-discovery -b')

Yea, I know you checked these permissions before, but I'm rather stumped as to 
why a fresh, empty index won't even work on your system. I've never seen that 
before, and it implies that there may be something lower level (like a 
permissions problem) causing problems with Solr.

- Tim



On 4/7/2015 2:28 PM, George Stanley Kozak wrote:
> Hi, Tim:
>
> To answer one of your questions: my original DSpace 1.8.2 solr statistics 
> core did have data in it. Attached is a "snapshot" of the index directory.
>
> Now, I did as you asked. I deleted the all of the solr cores
> (actually everything under /cul/app/dspace/solr)
>
> I did the mvn and ant update and everything went well. I copied the webapps 
> over to tomcat and restarted tomcat. I watched the DSpace log and the 
> catalina.out log. Everything seemed to move along smoothly. When everything 
> was done, I waited a few minutes before I went to my DSpace home page. When 
> my home page came up, I began to see errors in my DSpace log:
> ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @
> anonymous:session_id=B3B12E105B95C8FB8E63E9FB94B5971F:ip_addr=128.84.1
> 17.195:Error in Discovery while setting up date facet range:date
> facet\colon; dateIssued.year SolrCore 'search' is not available due to
> init failure: Index locked for write for core
> search,trace=org.apache.solr.common.SolrException: SolrCore 'search'
> is not available due to init failure: Index locked for write for core
> search
>
> Any attempt to do any browse or search results with errors. By the way, the 
> errors on the web appear with " 
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: 
> Expected mime type application/octet-stream but got text/html." And then raw 
> HTML after that.
>
> What has me stumped is that outside of the problem with the solr statistics, 
> none of these problems appear in my DSpace 4.3 upgrade. I have no write lock 
> problems in DSpace 4.3 install.
>
> George Kozak
> Digital Library Specialist
> 

Re: [Dspace-tech] Build Errors: DSpace 5.1 + Maven 2

[Graham Triggs]

Hi David,

The installation document states that Maven 3.x is required.

Regards,
Graham

On 06/04/2015 14:36:58, Baker, David A.  wrote:
I'm attempting a fresh install of 5.1 using the Maven 2 theme on Debian Wheezy. 
I'm getting errors on the build. I've tried both:

mvn -U clean package -Dmirage.on=true
mvn package -Dmirage.on=true

I am not building as root, and I have also changed git to use port 443 with 
command:

git config --global url."https://github.com/".insteadOf git://github.com/

I've attached the output from Maven

Thanks

--
David Baker
Digital Resources & Web Development Specialist
P.H. Welshimer Memorial Library, Milligan College

--
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette--
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette

Re: [Dspace-tech] Must I Build/Install as Tomcat service user?

[Graham Triggs]

On 19 December 2014 at 21:47, Chris Gray  wrote:

>  You will run into many problems along the way.  I started that way and
> had to revert.
>
> DSpace seems to be programmed under the assumption that $CATALINA_BASE is
> the same as $CATALINA_HOME (which it isn't under the Ubuntu/Debian package)
> and that Tomcat is run by the dspace user.  You'll have problems with
> DSpace looking in the wrong place for files and with file permission
> conflicts between the tomcat7 and dspace user.
>

Not exactly.

There may be something in the deployment scripts (I've not looked at them
in a long time) that makes some assumption about _BASE / _HOME, but the
code itself doesn't.

Similarly, it doesn't really make any assumptions about the user that
Tomcat is running as - but there is a complication that DSpace consists of
both applications that sit within the container (Tomcat), and code /
scripts to be executed at the command line (administrative tasks, batch
ingestion, etc.).

So the one assumption is that all the code (be it in Tomcat, or executed
from a shell), has access to the DSpace home directory, the configuration,
the asset store, etc. - not just for reading, but for writing in both
environments.

And that's really where the issues arise. Anything created by the web
application will assume the identity that the container is running as, and
that needs to be accessible to the shell scripts, and vice versa - anything
created by running a shell script will take on the identity that the script
was executed under, but that needs to be accessible to the web application.

On top of that, if you are doing a file system level deployment to the
container (Tomcat) of the web application (e.g. copying it to the webapps
directory), then the credentials used to execute the copy need to be
harmonized with the credentials of the application server.


There are other potential ways round it, but if you've got a dedicated
server, the simplest approach is to harmonize the credentials used for
Tomcat and for executing the shell scripts (and by turn, the owners of the
files) - it avoids a lot of complication later on.

FWIW, I ran DSpace on Debian, using the packaged Tomcat server, without
running into any _BASE / _HOME issues. I did ensure that Tomcat was running
as my 'dspace' user though.

G
--
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette

Re: [Dspace-tech] umlauts in Search results

[Graham Triggs]

Hi Kevin,

Please see the comments in this class:

https://github.com/DSpace/DSpace/blob/03276502d60b2644c0c2e6e1ace8846eb6cd5bf5/dspace-api/src/main/java/org/dspace/sort/OrderFormat.java

Note that for authors, the standard normalization used is:

https://github.com/DSpace/DSpace/blob/03276502d60b2644c0c2e6e1ace8846eb6cd5bf5/dspace-api/src/main/java/org/dspace/sort/OrderFormatAuthor.java

which only calls on DecomposeDiacritics and LowerCaseAndTrim filters. You
will need to create your own delegate (e.g. a copy of OrderFormatAuthor),
and add the StripDiacritics filter after the DecomposeDiacritics:

public class MyOrderFormatAuthor extends AbstractTextFilterOFD
{
{
filters = new TextFilter[] { new DecomposeDiactritics(),
 new StripDiacritics(),
 new LowerCaseAndTrim() };
}
}

and add it to the dspace.cfg:

plugin.named.org.dspace.sort.OrderFormatDelegate=

org.dspace.sort.MyOrderFormatAuthor=author


Regards,
G

On 6 December 2013 12:38, Evans, Kevin  wrote:

>  Hi All,
>
>
>
> Our Library staff have identified an issue when using the Search in
> Dspace; specifically, browsing for "Muller-Wille" doesn't find the
> researcher "Müller-Wille" because the umlaut wasn’t included in the search
> term.
>
>
>
> I assume this is the same for academics who have accents etc. in their
> names.
>
>
>
> Is it possible to change the browse so that it ignores accents, umlauts
> etc. or treats "Müller-Wille" the same as "Muller-Wille? Can I update
> synonyms to do this?
>
>
>
> Any advice much appreciated.
>
>
>
> Cheers
>
>
>
> Kevin
>
>
>
> Kevin Evans
> Systems Developer,
>
> Research & Collaborative Systems,
> Enterprise Applications,
> Room 919, 9th Floor, Laver Building,
> University of Exeter,
> Exeter,
> Devon,
> EX4 4QE
> 01392 725573
>
>
>
> Embrace. Extend. Extinguish.
>
>
>
>
> --
> Sponsored by Intel(R) XDK
> Develop, test and display web and hybrid apps with a single code base.
> Download it for free now!
>
> http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> List Etiquette:
> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>
--
Sponsored by Intel(R) XDK 
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette

Re: [Dspace-tech] Browser

[Graham Triggs]

Try right-clicking on the element, and selecting 'Inspect element'
from the menu.

If you look at the CSS for #ds-header-logo, you'll see that you are
using a relative path for the header image. That isn't going to work
as the URL changes.

G

On 22 August 2012 12:41, Webshet, Sisay (ILRI)  wrote:
> Hi All,
>
>
>
> Yellow & green colours on when viewd in Chrome... and header is missing [but
> colour combination is normal - burgundy & black - when viewed in IE and
> Mozilla]... Amazing! Iam using dspace 1.72 using xmlui Version.
>
>
>
> http://cgspace.cgiar.org/handle/10568/1
>
>
>
>
>
> can someone suggest
>
>
>
> Thanks
>
> sisay
>
>
> --
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>

--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Aside: PostgreSQL 9 working well with DSpace

[Graham Triggs]

Actually, 8.2 went EOL at the end of last year, so it wouldn't hurt to
make the official 'requirements' be at least 8.3, even if 8.2 happens
to work.

For the next release, I would even be tempted to say that the official
minimum is 8.4 (it's already highly recommended). It may work with 8.3
/ 8.2, but 8.3 will be EOL before the following major DSpace release,
so it would be prudent for people to be on at least 8.4 anyway, with a
recommendation for 9 and above.

G

On 14 June 2012 23:13, helix84  wrote:
> On Thu, Jun 14, 2012 at 3:12 PM, Mark H. Wood  wrote:
>> We've been using 9.0 in production with DSpace 1.7.2 for some months,
>> without incident.  Time to update requirements?
>
> Glad to hear that, we should mention that in docs. But I don't think
> we should bump up _requirements_.
>
> Regards,
> ~~helix84
>
> --
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] dspace java upload limit

[Graham Triggs]

Hi,

Using Maven 3 is not currently supported. Please use version 2.2.1.

Regards,
G

On 24 May 2012 12:52, Ajay Parikh  wrote:

> Dear All
> I am new to D-space as a developer
> I have install d-sapce. i am using Eclips to customize d-space  . i have
> set up environment when i build (Maven) project it show following error
>
> Two or more projects in the reactor have the same identifier
>
> Ajay Parikh
>
>
> Please do not print this email unless it is absolutely necessary. Spread
> environmental awareness.
>
>
>
>
>
>
> On Thu, May 24, 2012 at 3:05 PM, Bram Luyten  wrote:
>
>> Hi Jimmy,
>>
>> assuming you are using XMLUI:
>>
>> in DSpace 1.8.2, the limit was increased to Apache Cocoon's maximum value
>> of 2GB
>> https://jira.duraspace.org/browse/DS-1124
>>
>> If you are using a pre 1.8.2 version you could change this yourself in
>> the core.properties file.
>>
>> hope this helps,
>>
>> Bram Luyten
>>
>> --
>> [image: logo]
>> *Bram Luyten* *@mire*
>> *2888 Loker Avenue East, Suite 305, Carlsbad, CA. 92010*
>> *Esperantolaan 4, Heverlee 3001, Belgium*
>>   
>> www.atmire.com
>>
>>
>> On Wed, May 23, 2012 at 7:44 PM, Jimmy Ghaphery  wrote:
>>
>>> I'm running into the same problems described below, I don't see an
>>> answer to this question. Any ideas or guidance appreciated.
>>>
>>> Is there a separate java upload limit in addition to the limit expressed
>>> in dspace.cfg?
>>>
>>> thanks
>>>
>>> Jimmy
>>>
>>> >
>>> > [Dspace-tech] dspace java upload limit
>>> > From: Mark Ludwig  -2011-01-04 21:25
>>> > I am trying to archive some filesystems
>>> > as zip files in DSpace.
>>> > It's unable to upload files of one Gbyte or more.
>>> > I set the upload.max parm in dspace.cfg to 4 Gbytes
>>> > but it has no effect, so I think this is some Java
>>> > restriction.
>>> > I'm getting:
>>> > java.io.IOException: Multipart element is too large and was discarded.
>>> > This is Tomcat 5.5.26
>>> > dspace 1.5.1
>>> > and  jre-1.6.0-openjdk.x86_64
>>> > on Fedora.
>>> > Is there some obvious java setting I need to make?
>>> > If so, how high can/should it go?
>>> > I have multi-gig files but wonder how
>>> > much to chop it up.
>>> > Thanks,
>>> >
>>> > Mark Ludwig
>>> > University Libraries
>>> > SUNY at Buffalo
>>> > Buffalo, NY 14260
>>> > 716 645 5952
>>>
>>>
>>> --
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> ___
>>> DSpace-tech mailing list
>>> DSpace-tech@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>>
>>
>>
>>
>> --
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> ___
>> DSpace-tech mailing list
>> DSpace-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>
>>
>
>
> --
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] dspace kernel java segfault error 6

[Graham Triggs]

Hi Evelio,

It's very rare to see segfaults from Java in normal circumstances. And it's
highly unlikely that the Apache proxy has any effect on it.

There are really only two likely culprits for a segfault in this case:

a) A bug in the Java virtual machine - this could be more likely if you are
running a JVM other than Oracle / Sun (e.g. OpenJDK, particularly before 7).

b) Any native code executed from Java via JNI.

Given that you are using Postgres, the only likely route for native code
being used with your Tomcat instance is if it is using the Apache Portable
Runtime (APR). Whilst this can make the incoming connections more
efficient, I've personally had nothing but grief from attempting to use APR
with Tomcat.

Regards,
G

On 15 April 2012 21:19, Evelio Martínez  wrote:

> Hi all!
>
> ** **
>
> We have a DSpace instance with nearly 17000 items
>
> ** **
>
> Components/version:
>
> - RHEL  5.5  (6 Gb RAM)
>
> - Postgresql 8.4.2max_connections=500
>
> - DSpace 1.6.2   db.maxconnections = 100
>
> - Tomcat 6.0.33   with mod_jk 1.2.32
> JAVA_OPTS="$JAVA_OPTS -Xmx2048m -Xms512m -XX:MaxPermSize=384m
>
> -* *Apache MPM Worker 2.2.21
>
> - JDK 1.6.0_30
>
> ** **
>
> We are having Tomcat stopped suddenly with these messages:
>
> ** **
>
> Apr 15 13:24:47 dspace kernel: java[24630]: segfault at 43962788
> rip 2aaab2750360 rsp 43962780 error 6
>
> Apr 15 16:37:53 dspace kernel: java[26841]: segfault at 41763508
> rip 2aaab2750360 rsp 41763500 error 6
>
> Apr 15 20:04:35 dspace kernel: java[4214]: segfault at 41139938
> rip 2aaab2750360 rsp 41139930 error 6
>
> ** **
>
> Any suggestions to debug the problem?
>
> ** **
>
> Thanks in advance
>
> ** **
>
> [image: cid:f0154016-8e66-4c6b-9255-4e7524133341]
>
> *Evelio Martínez*
>
> Director proyectos software.
>
> CISA, CCNA
>
> Dept. de Tecnologías y Sistemas de Información
>
> C/ Garcilaso, 15-bajo
> 46003 Valencia
> Telf: 96 369 41 23 / 677 405 909
> Fax: 96 369 34 39 
>
> www.masmedios.com
>
> [image: cid:f0154016-8e66-4c6b-9255-4e7524133341]
>
> ** **
>
> [image: cid:65691c43-2689-4f20-9b5d-5425927026a4]
>
> Este correo electrónico y, en su caso, cualquier fichero anexo al mismo,
> puede contener información de carácter confidencial exclusivamente dirigida
> a su(s) destinatario(s). De acuerdo con la LOPD, MASmedios informa de que
> los datos personales contenidos quedarán sometidos a un tratamiento del que
> aquélla es responsable. Los interesados consienten el tratamiento de sus
> datos para estos fines, pudiendo ejercitar sus derechos de acceso,
> rectificación, cancelación y oposición dirigiéndose por escrito a *MASmedios
> para la gestión de la información*, calle Garcilaso 15-B 46003 Valencia o
> en proteccionda...@masmedios.com
>
> ** **
>
>
> --
> For Developers, A Lot Can Happen In A Second.
> Boundary is the first to Know...and Tell You.
> Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
> http://p.sf.net/sfu/Boundary-d2dvs2
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
<><>--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [Dspace-devel] DSpace & IDEs - Features, Observations, and Recommendations

[Graham Triggs]

On 13 January 2012 19:25, Peter Dietz  wrote:
> You can't debug XSLT development, but that doesn't bother me.

As of IntelliJ IDEA 11 - well, the commercial 'Ultimate' edition -
that isn't the case any more. If you create an XML file for the input
that you expect to pass to the stylesheet, you can set up a 'run'
profile for a stylesheet and XML file, and either simply run the
transformation, or set a breakpoint in the XSLT file and debug it. The
operation of the debugger is a little bit idiosyncratic compared to
the Java debugging, but it does work.

G

--
RSA(R) Conference 2012
Mar 27 - Feb 2
Save $400 by Jan. 27
Register now!
http://p.sf.net/sfu/rsa-sfdev2dev2
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] java.sql.SQLException: org.postgresql.util.PSQLException: ERROR: relation "bi_5_dmap" does not exist

[Graham Triggs]

Hi Ian,

Sounds like the browse configuration that your web app is running with is
out of sync with the database and/or the dspace.cfg.

I would restart your web application, and run a 'bin/dspace index-init -r'
to complete recreate the browse tables.

Regards,
G

On 22 August 2011 12:18, Wellaway, Ian  wrote:

> Hi all,
>
> ** **
>
> I try to delete items under ‘Unfinished Submissions’ and get the following
> message: java.sql.SQLException: org.postgresql.util.PSQLException: ERROR:
> relation "bi_5_dmap" does not exist.
>
> ** **
>
> Any idea what is going on here?
>
> ** **
>
> Thanks
>
> ** **
>
> Ian
>
> ** **
>
> PS. Dspace 1.6.2 with xmlui
>
> ** **
>
> https://eric.exeter.ac.uk
>
> ** **
>
> ** **
>
> ** **
>
> 
>
> Ian Wellaway
> Senior System Support & Development Officer
> Academic Systems
> Exeter IT
>
> Room 909, Laver Building
> University of Exeter
> EX4 4QE
> UK
>
> tel +44 1392 722852
>
> ** **
>
>
> --
> uberSVN's rich system and user administration capabilities and model
> configuration take the hassle out of deploying and managing Subversion and
> the tools developers use with it. Learn more about uberSVN and get a free
> download at:  http://p.sf.net/sfu/wandisco-dev2dev
>
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
uberSVN's rich system and user administration capabilities and model 
configuration take the hassle out of deploying and managing Subversion and 
the tools developers use with it. Learn more about uberSVN and get a free 
download at:  http://p.sf.net/sfu/wandisco-dev2dev
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] The file "update-discovery-index" not found...

[Graham Triggs]

Hi Sergiu,

The dspace launcher does not use other scripts within the bin directory, so
there presence or lack is not the issue here.

Commands for the launcher are configured in the config/launcher.xml - the
current version is here:

http://scm.dspace.org/svn/repo/dspace/trunk/dspace/config/launcher.xml

As you can see, it has an 'update-discovery-index' command (configured to
use the org.dspace.discovery.IndexClient class)

Regards,
G

On 9 June 2011 14:33, Sergiu Ioan Irimia  wrote:

> Mr. Graham Triggs,
>
> I know how to use the dspace launcher:
>
> -server:Windows 2003
>
> - run cmd
>
> - cd /d D:\DSpace\bin
>
> - D:\DSpace\bin>dspace update-discovery-index
>
> - error: command not found: update-discovery-index
>
> because the file "update-discovery-index" is not in D:\DSpace\bin and I
> can't seem to find it anywhere...
>
> So I need the file update-discovery-index so that the dspace launcher shall
> have what to launch...
>
> I've uploaded a jpg... maybe I failed to explain the problem...
>
> Or maybe I failed to understand your solution...
>
> Thank you for your understanding and for your time...
>
> --- On *Thu, 6/9/11, Graham Triggs * wrote:
>
>
> From: Graham Triggs 
> Subject: Re: [Dspace-tech] The file "update-discovery-index" not found...
> To: "Sergiu Ioan Irimia" 
> Cc: dspace-tech@lists.sourceforge.net
> Date: Thursday, June 9, 2011, 12:18 PM
>
>
> Hi,
>
> All of the batch scripts have been replaced by a launcher. Typically, you
> would replace:
>
> bin/

Re: [Dspace-tech] The file "update-discovery-index" not found...

[Graham Triggs]

Hi,

All of the batch scripts have been replaced by a launcher. Typically, you
would replace:

bin/

Re: [Dspace-tech] Problem with ordering in browsing

[Graham Triggs]

Please take a look at a previous post of mine on this subject:

http://dspace.2283337.n4.nabble.com/Browse-UTF-8-and-sorting-in-1-5-tp3281449p3281450.html

Regards,
G

On 19 May 2011 15:18, Peter Dietz  wrote:

> Hi Ladislav,
>
> I've noticed that our librarians here are happier with sorting when we use
> the collate of C as opposed to utf8/en_US.
>
> postgres=# create database "dspace" with owner = dspace encoding='utf8' 
> tablespace=pg_default lc_collate = 'C' lc_ctype='en_US.UTF-8' template 
> template0;
>
>
> I've add these three authors to a test collection that had some sample data
> in it, and it has the results you were expecting:
> == Author Name ==
> Cabanová, Zuzana
> Cablová, Barbora
> creatorlast, creatorfirst
> Čabla, Michael
>
>
>
>
> Peter Dietz
>
>
>
>
> On Thu, May 19, 2011 at 4:41 AM, Ladislav Kulhanek <
> ladislav.kulha...@vsb.cz> wrote:
>
>> Hello everybody.
>>
>> We have data in our DSpace in czech language (code "cs" in accordance
>> with ISO 639-1) and we have a problem with order in browsing by
>> author, titles and subjects (order in search results is correct).
>> There are letters with diacritic in czech alphabet, for example "Č"
>> (0x010C code in unicode). This letter should be ordered between "C"
>> and "D", but in DSpace it is ordered to the same place as "C". For
>> example we have ordered list as
>>
>> Cabanová, Zuzana
>> Čabla, Michael
>> Cablová, Barbora
>>
>> and this list should be
>>
>> Cabanová, Zuzana
>> Cablová, Barbora
>> Čabla, Michael
>>
>> And czech alphabet contains letter "Ch" (it consists from two
>> characters). This letter should be ordered between "h" and "i". This
>> letter is ordered in DSpace correctly. So it looks like DSpace order
>> in accordance with czech alphabet, but ignore diacritics.
>> We have DSpace 1.7.1, Manakin, db PostgreSQL 8.4 (database has
>> Collation and Ctype set as cs_CZ.UTF-8), and in tomcat connector is
>> URIEncoding="UTF-8". Any idea how to solve it? Thanks.
>>
>> Ladislav Kulhanek
>>
>>
>> --
>> What Every C/C++ and Fortran developer Should Know!
>> Read this article and learn how Intel has extended the reach of its
>> next-generation tools to help Windows* and Linux* C/C++ and Fortran
>> developers boost performance applications - including clusters.
>> http://p.sf.net/sfu/intel-dev2devmay
>> ___
>> DSpace-tech mailing list
>> DSpace-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>
>
>
>
> --
> What Every C/C++ and Fortran developer Should Know!
> Read this article and learn how Intel has extended the reach of its
> next-generation tools to help Windows* and Linux* C/C++ and Fortran
> developers boost performance applications - including clusters.
> http://p.sf.net/sfu/intel-dev2devmay
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its 
next-generation tools to help Windows* and Linux* C/C++ and Fortran 
developers boost performance applications - including clusters. 
http://p.sf.net/sfu/intel-dev2devmay___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] NPE error in searching after an upgrade to 1.7.1 (plus patches)

[Graham Triggs]

Hi,

There is an error in the 1.7.1 source. Lines 170 & 171 of

org.dspace.app.xmlui.aspect.artifactbrowser.AbstractSearch

need to be moved so that they come after the call to performSearch().
These two lines are expecting the queryResults object to be set, but
that won't happen until after the search has been executed.


Regards,

G


On 15 May 2011 20:50, Pottinger, Hardy J.  wrote:

> Hi, I'm in a pickle, our upgrade to 1.7.1 in production is not working out
> so well, we can browse OK, but searching is throwing NPE errors, I'm looking
> for advice to troubleshoot.
>
> Here's a link that is currently throwing an NPE:
> https://mospace.umsystem.edu/xmlui/search?query=test&submit=Go
>
> And of course, we didn't see anything like this in our testing. Both
> production and dev boxes are running the same toolchain on RHEL5. Very
> similar hardware.
>
> Have done the normal upgrade steps (have rebuilt the index a few times,
> restarted everything a few times, cleared the cocoon cache).
>
> Thanks for any pointers you can provide, I really appreciate it.
>
> --Hardy
>
> --
> Achieve unprecedented app performance and reliability
> What every C/C++ and Fortran developer should know.
> Learn how Intel has extended the reach of its next-generation tools
> to help boost performance applications - inlcuding clusters.
> http://p.sf.net/sfu/intel-dev2devmay
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
--
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Conflict with mixing metadata schemas and batch ingest

[Graham Triggs]

On 10 May 2011 16:44, Mark H. Wood  wrote:

>
> http://www.dspace.org/1_7_0Documentation/System%20Administration.html#SystemAdministration-Configuringmetadata\prefix\.xmlforDifferentSchema
>
> DS-534 addressed the documentation of this feature.  How could we
> improve it further?
>
>
I said IIRC!! I didn't actually look at the current docs :)
--
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Conflict with mixing metadata schemas and batch ingest

[Graham Triggs]

On 9 May 2011 09:31, Robin Taylor  wrote:

> > 2) While you can mix metadata elements for an item from multiple
> > schemas in manual submission, for the batch ingest you can only
> > specify a single schema for the batch.
>
> I haven't used this in recent times but I thought you could mix schema.
> I know there are lots of references to the 'Dublin Core metadata' in the
> code but I thought that was just a hangover from the days when only
> Dublin Core was used, and in fact you could mix metadata schema. have
> you tried this out ?
>
>
IIRC, this isn't particularly well documented. But it is possible to create
import packages using multiple metadata schemas.

The trick is to have separate metadata files for each schema. 'dc' goes into
the standard dublin_core.xml, But for any other schema, you need to create a
file called:

metadata_.xml

The contents of which is structured just like dublin_core.xml, except on the
 element, you need to include a schema="" attribute.

G
--
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network 
management toolset available today.  Delivers lowest initial 
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Re bugs in the Oracle implementation

[Graham Triggs]

Tim,

There is absolutely no requirement to have an Oracle license for a
testathon. The Oracle XE database is easy to install, works exactly the same
as the full licensed copy, and we'll get nowhere near the limits in data
sizes for the purposes of a testathon.

I'm quite happy to assist with implementing this on Duraspace provided
servers.

G

On 2 May 2011 23:05, Tim Donohue  wrote:

> Hi Hardy & Richard,
>
> I'll readily admit that our Oracle support has been more difficult
> recently. The primary reason for this is that the Committer who was
> leading DSpace's Oracle support has moved on to a different role and no
> longer works with DSpace. So, at this point in time, we have no (zero)
> Committers who actively use Oracle in their workplace (at least none
> that I'm aware of, unless someone has switched recently).
>
> Obviously, it is a concern to us that we no longer have anyone who is
> actively using Oracle. It also means we become much more dependent on
> the community members who use Oracle to report issues or problems
> (especially during Testathons & beta testing).
>
> The Committers do still attempt to do some basic Oracle testing before
> releases. But, sometimes this testing may not be as thorough as that
> provided by community members (as we may accidentally overlook testing
> of a particular feature, e.g. editing or adding e-people via XMLUI as
> Hardy reported in DS-841). Again, a large part of this is that none of
> our Committers run Oracle on a day-to-day basis, and we're all
> volunteers with a limited amount of time to devote to testing.
>
> So, to help ensure we are supporting Oracle fully, it'd be wonderful if
> those who use Oracle (and want to ensure it remains supported) can help
> out. Here's a few ways you can help:
>
> (1) Offer to install a Testathon DSpace instance backed by Oracle
> during our next Testathon. Currently, 'http://demo.dspace.org' (the
> primary Testathon server) is running on PostgreSQL as DuraSpace doesn't
> have an Oracle license, and we don't have an Oracle DB expert on staff.
>
> (2) Offer to download & test an Oracle-based DSpace install on your
> local servers during next Testathon/beta-testing & report back any
> issues you find before the final release
>
> (3) If you want, you can join the 'dspace-changelog' listserv to keep
> track of whether changes look to be Oracle specific or not.
> Alternatively, you could also just help test the latest Trunk code
> against Oracle every once in a while (and report back what you find).
>
> During development, the Committers are trying to ensure all
> patches/changes will work with Oracle (but obviously sometimes things
> slip through). So, in most cases, just testing the latest SVN Trunk code
> every once in a while should be good enough to locate any potential
> problems with the upcoming release.
>
> RE: 1.7.2 Release
>
> I'll talk to the Committers around whether we can push out a very minor
> 1.7.2 update just to fix DS-841. Are there any other known Oracle issues
> in DSpace 1.7.0/1.7.1 we should be resolving at the same time? Or is the
> only remaining Oracle issue the one described in DS-841?
>
> NOTE: At this point in time, the DSpace Committers actually were not
> planning a full 1.7.2 release to resolve any other issues.  We had
> discussed a 1.7.2. But, we had tentatively decided against it, as we
> have 1.8.0 coming up in October and most of us are actively working hard
> on features to release in 1.8.0.
>
> *However*, we should be making sure that Oracle is *fully* supported in
> a 1.7.x release series (I know there were other Oracle issues we already
> fixed between 1.7.0 and 1.7.1). So, I would be willing to recommend a
> very small 1.7.2 release *just* to fix the remaining Oracle issue(s).
>
> So, Hardy & Richard, if you have found any other issues with 1.7.1, it'd
> be good to know so that we can release a working Oracle 1.7.x version in
> the near future.
>
> - Tim
>
>
> On 5/2/2011 12:33 PM, Pottinger, Hardy J. wrote:
> > Hi, Richard, I think what needs to happen is someone from our community
> of
> > Oracle installations needs to commit an Oracle-backed installation to the
> > next Testathon, to catch items like this. I am also going to try to do
> > more testing along the way. I think what we need to do as a community is
> > to be especially vigilant for patches that touch the classes that
> interact
> > with the database in any way, as this area of DSpace is particularly
> > important to our use of DSpace.
> >
> > I'd like to reach out to the more-experienced developers on this list: is
> > there a good way, besides "paying more attention" to follow the commits
> > and watch out for changes to the database classes? Do I need to just join
> > the commit mail list and set up a a filter in my mail client? Or is there
> > some better way to keep on top of these things?
> >
> > Also, Richard, this a very minor detail, but I am the reporter for
> DS-841,
> > Peter was kind enough to add a comment

Re: [Dspace-tech] Lost Indexes

[Graham Triggs]

Herbert,

You need to work back through the stack trace. First thing you'll notice is
that the BrowseException that is thrown in updateCommunityMappings is a
wrapping of a caught SQLException.

Following the stack trace of the SQLException ('caused by' in your output),
you'll go through the Oracle libraries, through the DatabaseManager and out
into


org.dspace.browse.BrowseCreateDAOOracle.getAllCommunityIDs(BrowseCreateDAOOracle.java:962)

The only SQL directly issued within the getAllCommunityIDs method is against
Community2Item - a view that is comprised of a join between
Community2Collection and Collection2Item.

If you look at the update script for the database in version 1.6, you'll see
that both of these tables (Com2Col and Col2Item) have been ALTERed. This
will have invalidated the Community2Item view.

Go to the database and recompile any invalidated objects - eg. ALTER VIEW
Community2Item COMPILE; - and it will fix the problem.

Regards,
G

On 1 February 2011 17:04, Herbert Lensch wrote:

> DSpace-Tech:
>
> One of the users of our DSpace 1.6.0 site noticed that the indexes are
> no longer searchable from the homepage, and worse there are no items listed
> in the entire site.
>
> When I try to run a reindex on the site, I get the following error:
>
> [root@dspaceprod run]# /dspace/run/bin/dsrun org.dspace.browse.IndexBrowse
>> -f -r -v
>
> Creating browse indexes for DSpace
>
> init complete (0 ms)
>
> Preparing browse tables
>
> Deleting old indices
>
> Checking for bi_1
>
> Checking for bi_1_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_2
>
> Checking for bi_2_dis
>
> ...found
>
> Deleting old index and associated resources: bi_2_dis
>
> Checking for bi_3
>
> Checking for bi_3_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_4
>
> Checking for bi_4_dis
>
> ...found
>
> Deleting old index and associated resources: bi_4_dis
>
> Checking for bi_5
>
> Checking for bi_5_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_6
>
> Checking for bi_6_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_7
>
> Checking for bi_7_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_8
>
> Checking for bi_8_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_9
>
> Checking for bi_9_dis
>
> ... doesn't exist; but will carry on as there may be something that
>> conflicts
>
> Checking for bi_10
>
> Checking for bi_10_dis
>
> ... doesn't exist; no more tables to delete
>
> Creating browse index dateissued: index by null sortable by:  dc.title
>>  dc.date.accessioned  dc.date.issued
>
> Creating browse index author: index by dc.contributor.*,dc.creator sortable
>> by:  dc.title  dc.date.accessioned  dc.date.issued
>
>  Creating browse index title: index by null sortable by:  dc.title
>>  dc.date.accessioned  dc.date.issued
>
> Creating browse index subject: index by dc.subject.* sortable by:  dc.title
>>  dc.date.accessioned  dc.date.issued
>
> Browse tables prepared
>
> tables prepped (1089 ms, 1089 ms)
>
> Exception: java.sql.SQLException: ORA-00942: table or view does not exist
>
>
>> org.dspace.browse.BrowseException: java.sql.SQLException: ORA-00942: table
>> or view does not exist
>
>
>>  at
> org.dspace.browse.BrowseCreateDAOOracle.updateCommunityMappings(BrowseCreateDAOOracle.java:649)
>
>>  at org.dspace.browse.IndexBrowse.indexItem(IndexBrowse.java:402)
>
>>  at org.dspace.browse.IndexBrowse.createIndex(IndexBrowse.java:1148)
>
>>  at org.dspace.browse.IndexBrowse.initBrowse(IndexBrowse.java:1098)
>
>>  at org.dspace.browse.IndexBrowse.main(IndexBrowse.java:749)
>
>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>>  at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
>>  at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
>>  at java.lang.reflect.Method.invoke(Method.java:616)
>
>>  at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:212)
>
>> Caused by: java.sql.SQLException: ORA-00942: table or view does not exist
>
>
>>  at
> oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
>
>>  at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:331)
>
>>  at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:288)
>
>>  at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:745)
>
>>  at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:219)
>
>>  at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:813)
>
>>  at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1049)
>
>>  at
> oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDesc

Re: [Dspace-tech] Lots of browse errors

[Graham Triggs]

Jeffrey, I agree with Claudia that the request itself looks odd. I would
want to look at / find out what was generating that link in the first place.

But at the same time, the code ought to be able to handle with this more
gracefully (ie. know that it's an item index and ignore the value
parameter).

Please log as a bug - you can assign it to me if you like.

G

On 27 January 2011 17:19, Claudia Juergen  wrote:

> Hello Jeffrey,
>
> did you experiment with the title before and google got old stuff indexed?
> Your title index atm seems ok.
>
> Claudia
>
> > Claudia,
> >
> > The default is what came with dspace.cfg :
> >
> > webui.browse.index.1 = dateissued:item:dateissued
> > webui.browse.index.2 = author:metadata:dc.contributor.*,dc.creator:text
> > webui.browse.index.3 = title:item:title
> > webui.browse.index.4 = subject:metadata:dc.subject.*:text
> >
> > Something is just wacked if you ask me.
> >
> >
> > On Jan 27, 2011, at 10:46 AM, Claudia Jürgen wrote:
> >
> >> Hi Jeffrey,
> >>
> >> should have taken a close look,
> >>
> >>
> http://digital.maag.ysu.edu:8080/jspui/browse?type=title&order=ASC&rpp=20&value=Sigma+Gamma+clean-up+project
> .
> >>
> >> value is only used for metadata browse indices not item indices, they
> >> got an offset for paging. Title is usually an item index, seems as if
> >> you changed it to metadata, so it is not a default browse setting?
> >>
> >> Hope that helps
> >>
> >> Claudia Jürgen
> >>
> >> Am 27.01.2011 16:30, schrieb Jeffrey Trimble:
> >>> I did run it...and actually did it a second time--this morning.
> >>> Nothing, zilch.
> >>>
> >>> Quite yucky if you ask me.
> >>>
> >>> Thanks,
> >>>
> >>> On Jan 27, 2011, at 10:21 AM, Claudia Jürgen wrote:
> >>>
>  Hi Jeffrey,
> 
>  did you run index-init, when setting up this instance?
> 
>  Hope that helps
> 
>  Claudia Jürgen
> 
> 
>  Am 27.01.2011 16:07, schrieb Jeffrey Trimble:
> > I'm getting lots of browse errors.  (Like 200 within 14 minutes)  I
> > don't see but two index tables showing up
> > in my postgresql.  Also, the default browse config has been
> > implemented (nothing fancy!)  (Yeah, I know
> > its a google-bot).
> >
> >
> > Here's a sample:
> >
> > Date:   1/27/11 10:04 AM
> > Session ID: 9FB39CC6A53C23EC1A80EDCA2E76E847
> > User:   Anonymous
> > IP address: 66.249.71.110
> >
> > -- URL Was:
> >
> http://digital.maag.ysu.edu:8080/jspui/browse?type=title&order=ASC&rpp=20&value=Sigma+Gamma+clean-up+project
> .
> > -- Method: GET
> > -- Parameters were:
> > -- value: "Sigma Gamma clean-up project."
> > -- rpp: "20"
> > -- type: "title"
> > -- order: "ASC"
> >
> >
> > Exception:
> > javax.servlet.ServletException: org.dspace.browse.BrowseException:
> > org.postgresql.util.PSQLException: ERROR: relation "bi_3_dmap" does
> > not exist
> >  Position: 79
> >   at
> >
> org.dspace.app.webui.servlet.AbstractBrowserServlet.processBrowse(AbstractBrowserServlet.java:371)
> >   at
> >
> org.dspace.app.webui.servlet.BrowserServlet.doDSGet(BrowserServlet.java:112)
> >   at
> >
> org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:151)
> >   at
> >
> org.dspace.app.webui.servlet.DSpaceServlet.doGet(DSpaceServlet.java:99)
> >   at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> >   at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> >   at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> >   at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> >   at
> >
> org.dspace.utils.servlet.DSpaceWebappServletFilter.doFilter(DSpaceWebappServletFilter.java:112)
> >   at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> >   at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> >   at
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> >   at
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> >   at
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
> >   at
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> >   at
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> >   at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> >   at
> >
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
> >   at
> >
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
> >   at
> >
> org.apache.tomcat.util.n

Re: [Dspace-tech] browse index config changes not reflecting in nav panel (1.6.2)

[Graham Triggs]

Which UI are you using - JSP or XML?

If it's XMLUI, then it sounds like it might be caching elements of the
Cocoon pipeline, in which case you should stop the application and remove
the cache from within the web application directory, and/or disable Cocoon
caching.


On 19 January 2011 15:03, Fred Edwards  wrote:

>
> as a test, I've recently added a browse index to our dspace and initially
> it correctly added the items to the nav. panel...
>
> however, after removing the browse index in the config file the nav panel
> is now doing weird things.
>
> at the top level it is retaining menu items for browse that is no longer
> there and clicking leads to an error page obviously...
>
> go down a level or two into communities and collections you start finding
> the nav panel in some areas do not show these links, others do...
>
> I've recompiled countless times... re-indexed countless times...
> re-initialized the index countless times...
>
> the last go I re-added the browse indexes and still the same situation...
> in some places on... in others off...
>
> we have a quasi live collection or two in the server, and no test server
> currently, so we'd rather not have to completely rebuild...
>
> I'm also loath to:  ant fresh_install   without getting some feedback from
> the community on how safe this would be...
>
>
>
>
>
>
> Fred Edwards
>
> Systems Technician
> Patrick Power Library
> Saint Mary's University
> Halifax, Nova ScotiaB3H 3C3
>
> Phone: (902) 420-5096
> Fax:   (902) 420-5561
> E-mail:fred.edwa...@smu.ca
> Website:   http://www.smu.ca/library/
>
> --
>
>
> --
> Protect Your Site and Customers from Malware Attacks
> Learn about various malware tactics and how to avoid them. Understand
> malware threats, the impact they can have on your business, and how you
> can protect your company and customers by using code signing.
> http://p.sf.net/sfu/oracle-sfdevnl
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand 
malware threats, the impact they can have on your business, and how you 
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Search

[Graham Triggs]

That sounds plausible. Can you find the number if you phrase search the two
parts - ie, "45 858"?

On 4 January 2011 15:14, Marcelo Henrique Gomes wrote:

> Hi Graham,
>
> I run filter-media and index-update every day and I find others terms in
> the same document.
>
> I guess the problem is in the mask of number ##.### because the "." can be
> a delimiter line.
>
> What do you thing?
>
> If anyone wants to know my DSpace is
> http://www.iof.mg.gov.br/ultima-edicao.html and send comments and
> suggestions. I'll be very happy.
>
> Regards,
> Marcelo
>
> On Tue, Jan 4, 2011 at 12:49 PM, Graham Triggs wrote:
>
>> Hi,
>>
>> There are all sorts of possibilities for this. Is the term in the metadata
>> (you need to ensure the metadata field is in the search configuration), or
>> in a document file?
>>
>> Have you run filter-media?
>>
>> There is a limit to the number of terms in a single field, could this be
>> an issue? (see documentation for configuration option:
>> search.maxfieldlength).
>>
>> Finally, can you find other terms in the same document?
>>
>> If you can find other terms from the same document, but not the numbers,
>> then this would indicate it's the behaviour of BrazilianAnalyzer that is
>> causing the issue, for which you may get better responses directly from the
>> Lucene community.
>>
>> Regards,
>> G
>>
>> On 4 January 2011 11:44, Marcelo Henrique Gomes wrote:
>>
>>> Hello,
>>>
>>> In my files, I have numbers (45.858) and when I try to search for this
>>> number I don't find.
>>>
>>> I used BrazilianAnalyzer.
>>>
>>> search.analyzer = org.apache.lucene.analysis.br.BrazilianAnalyzer
>>>
>>> What can I do?
>>>
>>> Regards,
>>> Marcelo
>>>
>>>
>>> --
>>> Learn how Oracle Real Application Clusters (RAC) One Node allows
>>> customers
>>> to consolidate database storage, standardize their database environment,
>>> and,
>>> should the need arise, upgrade to a full multi-node Oracle RAC database
>>> without downtime or disruption
>>> http://p.sf.net/sfu/oracle-sfdevnl
>>> ___
>>> DSpace-tech mailing list
>>> DSpace-tech@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>>
>>>
>>
>
--
Learn how Oracle Real Application Clusters (RAC) One Node allows customers
to consolidate database storage, standardize their database environment, and, 
should the need arise, upgrade to a full multi-node Oracle RAC database 
without downtime or disruption
http://p.sf.net/sfu/oracle-sfdevnl___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] strange effect with dspace-1.7.0

[Graham Triggs]

That doesn't entirely surprise me - I've seen odd things with 2.0.x, and it
all gets tied up with the overlay war capabilities that were introduced to
the war plugin during that release cycle.

As Maven 2.2.0 was released back in July(?) 2009, I don't think it's
entirely unreasonable to just deprecate the use of 2.0.x and focus on 2.2.0
and above.

G

On 21 December 2010 17:07, Tim Donohue  wrote:

> Hi Jochen,
>
> It looks like we have a small bug somewhere.  I've verified that
> building DSpace using Maven 2.0.x doesn't seem to work 100% properly.
>
> Essentially, everything doesn't get copied to
> [dspace-src]/dspace/target/dspace-1.7.0-build/webapps/ properly, which
> causes the fresh_install issues you've seen.
>
> The temporary fix is one of the following:
>
> (1) Either install Maven 2.2.1 (which works fine)
>
> OR
>
> (2) Manually copy the webapps from their
> [dspace-src]/dspace/modules/[webapp-name]/target/ directory.  So, for
> example, the XMLUI webapp is built properly to the path:
>
> [dspace-src]/dspace/modules/xmlui/target/xmlui-1.7.0/
>
> However, it doesn't get copied over to the "webapps" directory properly
> (which is something Maven is supposed to do for you, but it doesn't seem
> to be working).  So, if you manually perform the copy, everything should
> work fine.
>
> I'm digging around to see if I can figure out a better fix to this issue.
>
> - Tim
>
> On 12/21/2010 5:25 AM, Jochen Lienhard wrote:
> > Hi,
> >
> > I installed dspace-1.7.0 at two different Linux distribution:
> >
> > 1) openSuSE 11.3 (Apache Maven 2.2.1)
> > 2) openSuSE 11.1 (Maven version: 2.0.9)
> >
> > in case 1) all is perfect.
> > in case 2) in the webapps directory all files are in one directory ...
> > even a strange effect in the numbers of the files:
> >
> > 1)
> > fresh_install:
> > [copy] Copying 8 files to /opt/dspace-1.7.0/bin
> > [copy] Copying 71 files to /opt/dspace-1.7.0/lib
> > [copy] Copying 21 files to /opt/dspace-1.7.0/etc
> > [copy] Copying 27 files to /opt/dspace-1.7.0/solr
> >
> > copy_webapps:
> > [copy] Copying 967 files to /opt/dspace-1.7.0/webapps
> > [copy] Copied 130 empty directories to 6 empty directories under
> > /opt/dspace-1.7.0/webapps
> > [copy] Copying 6 files to /opt/dspace-1.7.0/webapps
> >
> > 2)
> > fresh_install:
> > [copy] Copying 8 files to /opt/dspace-1.7.0/bin
> > [copy] Copying 70 files to /opt/dspace-1.7.0/lib
> > [copy] Copying 21 files to /opt/dspace-1.7.0/etc
> > [copy] Copying 27 files to /opt/dspace-1.7.0/solr
> >
> > copy_webapps:
> > [copy] Copying 668 files to /opt/dspace-1.7.0/webapps
> > [copy] Copied 90 empty directories to 1 empty directory under
> > /opt/dspace-1.7.0/webapps
> > [copy] Copying 1 file to /opt/dspace-1.7.0/webapps
> >
> > Does anyone know, which could be the problem?
> >
> > Greetings
> >
> > Jochen
> >
> > P.S.
> > If I copy the xmlui folder from 1) to the webapps folder of 2) ... it
> > works . very strange
> >
> >
> >
> >
> --
> > Lotusphere 2011
> > Register now for Lotusphere 2011 and learn how
> > to connect the dots, take your collaborative environment
> > to the next level, and enter the era of Social Business.
> > http://p.sf.net/sfu/lotusphere-d2d
> >
> >
> >
> > ___
> > DSpace-tech mailing list
> > DSpace-tech@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
> --
> Forrester recently released a report on the Return on Investment (ROI) of
> Google Apps. They found a 300% ROI, 38%-56% cost savings, and break-even
> within 7 months.  Over 3 million businesses have gone Google with Google
> Apps:
> an online email calendar, and document program that's accessible from your
> browser. Read the Forrester report: http://p.sf.net/sfu/googleapps-sfnew
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
--
Forrester recently released a report on the Return on Investment (ROI) of
Google Apps. They found a 300% ROI, 38%-56% cost savings, and break-even
within 7 months.  Over 3 million businesses have gone Google with Google Apps:
an online email calendar, and document program that's accessible from your 
browser. Read the Forrester report: http://p.sf.net/sfu/googleapps-sfnew___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Prism Metadata for DSpace

[Graham Triggs]

That's probably because a lot of the examples of Prism in practice relate to
online publications - which don't have the same pagination requirements.

If you have a look at Nature - for example:
http://www.nature.com/nri/journal/v10/n11/full/nri2858.html - and view the
source of that document, you can see that they have embedded Prism metadata
with an ending page.

G

On 14 November 2010 20:25, Leonie Hayes  wrote:

> We would welcome the addition of a metadata schema that deals with full
> elements of Journal Article and Conference items. There are a number of us
> in the community who found working our way around the missing elements in
> Dublin Core a frustration.
>
> Some of the examples of Prism schema I looked at don't contain ending page,
> perhaps an oversight, so other examples would be helpful.
>
> We ended up adding our own mini-schema, but an out of the box DSpace
> containing all the elements based on an existing standard - very attractive!
>
> Other benefits -  take advantage of Open URL functionality, create a
> citation, export citations, and Zotero capture.
>
> When you do not store these elements separately, export to EndNote, BibTeX
> etc, and creating the various citation styles (APA, HARVARD etc), are much
> more difficult to implement.
>
>
> Leonie Hayes
> Research Repositories Manager
> http://www.library.auckland.ac.nz/contacts/?stid=124
> http://researchspace.auckland.ac.nz
>
>
>
>
>
>
> --
> Centralized Desktop Delivery: Dell and VMware Reference Architecture
> Simplifying enterprise desktop deployment and management using
> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
> client virtualization framework. Read more!
> http://p.sf.net/sfu/dell-eql-dev2dev
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
--
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] postgres/postmaster 99%

[Graham Triggs]

Steve,

I've been trying again on my machine. Postgres 8.4 seemed to be quite slow /
expensive (can't recall the numbers precisely - it was the other day). I
have dumped the data and loaded it into Postgres 9.0, and the results are
interesting.

Just after restoring the database, the same query (for the first 10 records)
took about 6 seconds to execute (with a lot of CPU usage). After vacuuming
bi_4_dmap and bi_item (pgadmin recommended both be vacuumed), the query took
3 seconds.

This is with 379,900 records in the item / bi_item tables, and 4,775,227
records in bi_4_dmap.

I would recommend you vacuum and analyze the tables. And possibly consider
upgrading the version of Postgres.

G

On 11 November 2010 01:30, Steve Swinsburg wrote:

> Hi Graham,
>
> We have ~48000 records in the item table, and ~263,000 in the bi_4_dmap
> table. We've recreated the indexes (only via index-update though, should we
> run index-init?) and things went ok for a while, but now back up to 99% and
> the same query appears in pg_stat_activity.
>
> We are running Postgres 8.1.18.
>
> cheers,
> Steve
>
>
>
>
>
> On 11/11/2010, at 11:10 AM, Graham Triggs wrote:
>
> It's a second level browse - ie. if your 4th browse index is 'subject',
> then it's someone looking at all the items that have a particular subject
> entry.
>
> This shouldn't be that expensive a query (given there is no offset
> involved), but you may have an issue with an index missing (although the
> browse code should create all the ones that it needs when it creates the
> table), or more likely you have too low a value for your shared_buffers
> (required to load the indexes), or work_mem (used for the join between the
> tables). Additionally, you may need to analyze and/or reindex the tables.
>
> How many items are in the repository, and how many values do you have in
> the 4th browse option (subject?). And what version of Postgres are you
> running?
>
> G
>
> On 10 November 2010 22:42, Steve Swinsburg wrote:
>
>> A followup:
>>
>> I restarted postgres and within minutes the same query has appeared in the
>> stats and CPU is back up to 99%. Could we be missing some indexes or
>> something? We only recently ran the filter-media script and generated
>> thousands of thumbnails and branded previews but have since performed a
>> vacuum.
>>
>> Any information would be much appreciated.
>>
>> cheers,
>> Steve
>>
>> On 11/11/2010, at 9:12 AM, Steve Swinsburg wrote:
>>
>> Hi all,
>>
>> We are experiencing an issue on both of our dspace instances where
>> postmaster spins up 99% of the CPU. Sometimes it's just one process at 99%,
>> othertimes its a dozen or more processes around 7-9% each. I ran some stats
>> on postgres via:
>>
>> select * from pg_stat_activity
>>
>> I found this query in the output about 15 times:
>>
>> SELECT bi_item.* FROM bi_item, (SELECT bi_4_dmap.item_id FROM bi_4_dmap,
>> bi_4_dis WHERE bi_4_dmap.distinct_id=bi_4_dis.id AND
>> bi_4_dis.sort_value=$1 ) mappings  WHERE  bi_item.item_id=mappings.item_id
>> ORDER BY sort_3 ASC  LIMIT $2
>>
>> The earliest has a start time of about 5 hours ago.
>>
>> Anyone know whats up?
>>
>> cheers,
>> Steve
>>
>>
>>
>>
>> --
>> Centralized Desktop Delivery: Dell and VMware Reference Architecture
>> Simplifying enterprise desktop deployment and management using
>> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
>> client virtualization framework. Read more!
>> http://p.sf.net/sfu/dell-eql-dev2dev
>> ___
>> DSpace-tech mailing list
>> DSpace-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>
>>
>
>
--
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] postgres/postmaster 99%

[Graham Triggs]

Index update will only make corrections to the values written to the browse
tables and Lucene indexes - index-init recreates the table and index
definitions in Postgres.

Can you run the query with EXPLAIN ANALYZE in front of it (substitute an
actual value from sort_value in the bi_4_dis table in place of the bind
variable)? That will tell you what Postgres is actually doing.

G

On 11 November 2010 01:30, Steve Swinsburg wrote:

> Hi Graham,
>
> We have ~48000 records in the item table, and ~263,000 in the bi_4_dmap
> table. We've recreated the indexes (only via index-update though, should we
> run index-init?) and things went ok for a while, but now back up to 99% and
> the same query appears in pg_stat_activity.
>
> We are running Postgres 8.1.18.
>
> cheers,
> Steve
>
>
>
>
>
> On 11/11/2010, at 11:10 AM, Graham Triggs wrote:
>
> It's a second level browse - ie. if your 4th browse index is 'subject',
> then it's someone looking at all the items that have a particular subject
> entry.
>
> This shouldn't be that expensive a query (given there is no offset
> involved), but you may have an issue with an index missing (although the
> browse code should create all the ones that it needs when it creates the
> table), or more likely you have too low a value for your shared_buffers
> (required to load the indexes), or work_mem (used for the join between the
> tables). Additionally, you may need to analyze and/or reindex the tables.
>
> How many items are in the repository, and how many values do you have in
> the 4th browse option (subject?). And what version of Postgres are you
> running?
>
> G
>
> On 10 November 2010 22:42, Steve Swinsburg wrote:
>
>> A followup:
>>
>> I restarted postgres and within minutes the same query has appeared in the
>> stats and CPU is back up to 99%. Could we be missing some indexes or
>> something? We only recently ran the filter-media script and generated
>> thousands of thumbnails and branded previews but have since performed a
>> vacuum.
>>
>> Any information would be much appreciated.
>>
>> cheers,
>> Steve
>>
>> On 11/11/2010, at 9:12 AM, Steve Swinsburg wrote:
>>
>> Hi all,
>>
>> We are experiencing an issue on both of our dspace instances where
>> postmaster spins up 99% of the CPU. Sometimes it's just one process at 99%,
>> othertimes its a dozen or more processes around 7-9% each. I ran some stats
>> on postgres via:
>>
>> select * from pg_stat_activity
>>
>> I found this query in the output about 15 times:
>>
>> SELECT bi_item.* FROM bi_item, (SELECT bi_4_dmap.item_id FROM bi_4_dmap,
>> bi_4_dis WHERE bi_4_dmap.distinct_id=bi_4_dis.id AND
>> bi_4_dis.sort_value=$1 ) mappings  WHERE  bi_item.item_id=mappings.item_id
>> ORDER BY sort_3 ASC  LIMIT $2
>>
>> The earliest has a start time of about 5 hours ago.
>>
>> Anyone know whats up?
>>
>> cheers,
>> Steve
>>
>>
>>
>>
>> --
>> Centralized Desktop Delivery: Dell and VMware Reference Architecture
>> Simplifying enterprise desktop deployment and management using
>> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
>> client virtualization framework. Read more!
>> http://p.sf.net/sfu/dell-eql-dev2dev
>> ___
>> DSpace-tech mailing list
>> DSpace-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>
>>
>
>
--
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] A metadata schema for journals - Prism ?

[Graham Triggs]

Hi Robin,

Theo is quite right - I think PRISM is a good standard for repositories to
support for published material. In fact, it's probably more important for
SWORD to be able to accept PRISM metadata than it is to support SWAP - given
the majority of sources that might supply data via SWORD already support
PRISM (and it contains more useful discrete metadata - like volume / issue -
than is described by SWAP).

G

On 11 November 2010 11:05, TAYLOR Robin  wrote:

> Hi all,
>
> Periodically the subject of how to store journal volume and issue numbers
> as Dublin Core comes up on this list. Our repo admin Theo Andrew came up
> with this schema as a possible solution so I just thought I would pass it
> on. The attached diagram shows how it can be used.
>
> http://www.prismstandard.org
>
> Cheers, Robin.
>
> Robin Taylor
> Main Library
> University of Edinburgh
> Tel. 0131 6513808
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
>
> --
> Centralized Desktop Delivery: Dell and VMware Reference Architecture
> Simplifying enterprise desktop deployment and management using
> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
> client virtualization framework. Read more!
> http://p.sf.net/sfu/dell-eql-dev2dev
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] postgres/postmaster 99%

[Graham Triggs]

It's a second level browse - ie. if your 4th browse index is 'subject', then
it's someone looking at all the items that have a particular subject entry.

This shouldn't be that expensive a query (given there is no offset
involved), but you may have an issue with an index missing (although the
browse code should create all the ones that it needs when it creates the
table), or more likely you have too low a value for your shared_buffers
(required to load the indexes), or work_mem (used for the join between the
tables). Additionally, you may need to analyze and/or reindex the tables.

How many items are in the repository, and how many values do you have in the
4th browse option (subject?). And what version of Postgres are you running?

G

On 10 November 2010 22:42, Steve Swinsburg wrote:

> A followup:
>
> I restarted postgres and within minutes the same query has appeared in the
> stats and CPU is back up to 99%. Could we be missing some indexes or
> something? We only recently ran the filter-media script and generated
> thousands of thumbnails and branded previews but have since performed a
> vacuum.
>
> Any information would be much appreciated.
>
> cheers,
> Steve
>
> On 11/11/2010, at 9:12 AM, Steve Swinsburg wrote:
>
> Hi all,
>
> We are experiencing an issue on both of our dspace instances where
> postmaster spins up 99% of the CPU. Sometimes it's just one process at 99%,
> othertimes its a dozen or more processes around 7-9% each. I ran some stats
> on postgres via:
>
> select * from pg_stat_activity
>
> I found this query in the output about 15 times:
>
> SELECT bi_item.* FROM bi_item, (SELECT bi_4_dmap.item_id FROM bi_4_dmap,
> bi_4_dis WHERE bi_4_dmap.distinct_id=bi_4_dis.id AND
> bi_4_dis.sort_value=$1 ) mappings  WHERE  bi_item.item_id=mappings.item_id
> ORDER BY sort_3 ASC  LIMIT $2
>
> The earliest has a start time of about 5 hours ago.
>
> Anyone know whats up?
>
> cheers,
> Steve
>
>
>
>
> --
> Centralized Desktop Delivery: Dell and VMware Reference Architecture
> Simplifying enterprise desktop deployment and management using
> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
> client virtualization framework. Read more!
> http://p.sf.net/sfu/dell-eql-dev2dev
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] manikin question

[Graham Triggs]

On 8 October 2010 19:06, Walker, David  wrote:

> > breaking up the xslt into separate files for each page
> > actually defeats the design goals of the XMLUI theming tier.
> > I.E. having one file within which all customizations reside to
> > manage your theme.
>
> I think there are advantages and disadvantages to both approaches.
>

I agree there are pros and cons to both approaches. And yes, if we focus on
some specific areas of the implementation, there may be too much
presentation being encoded in the DRI.

But in general, I would agree with Mark. The design of Cocoon / Manakin is
based around pipeline processing of the model, and for you to be able to
chain in components/aspects, you neither want to do it after a page based
transformation (and have to put in multiple special cases for each of the
pages), or have page based transformations after the aspect (requiring that
you modify each of those pages as well, and not just add the new component).


But the thrust of my argument here is that we should have *templates* that
> correspond to pages.  Whether each of those templates lives in it own file,
> or whether they all live together in one big file, is a minor detail, as far
> as I'm concerned.
>


I definitely agree with your point that the approach taken by Cocoon /
Manakin is a learning curve for some people, and that a number of people
would benefit from having page based templates. But if you are going to do
that, then you might as well set Cocoon aside - there isn't any point in
taking the added dependencies, xml serialization and transformation, if you
can't make [effective] use of the transformation pipeline.

Imho, if you want to use a page based approach, then you are better off
using Spring WebMVC, and writing the templates in Freemarker [disclaimer: I
did start implementing such a framework, but it doesn't have any functional
components as yet].

G
--
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] A simplified version of dspace-services and Tomcat unloading

[Graham Triggs]

On 7 October 2010 06:55, Mark Diggory  wrote:

> You know I'm for (1) simplified spring only service manager. (2) jdbc
> data source delivered by servlet container (3) dropping reflectutils.
>

Well, the JDBC datasource is a change to dspace-api rather than -services,
but there is no reason why it can't go into DSpace 1.7.


> I'm not so sure about dropping the MBean registration completely, the
> whole idea was that we could have a container level service manager
> and services deployed across webapplications.  But I assume this is
> just too crazy an approach to maintain and educate others on its
> usage.  I'm just not convinced we would use a servlet container in
> this manner and simplification is no doubt more important...
>

My default position is to say that if we aren't actively using something, we
shouldn't have it. Whilst the registration technically works, we haven't got
close to properly testing and qualifying sharing resources across contexts
(/classloaders), and people will run into a lot of problems attempting to
use it that way.

Besides which, MBean isn't really the way to share resources in that way -
JNDI is. Although MBeans are available outside a container, whereas JNDI
isn't necessarily. I wouldn't class that as a good reason to use MBeans -
it's a good reason to have that part of the functionality switchable -
rather than a static manager that combines singleton and MBean registration
as conditional logic, it should be a static facade to either a singleton or
Mbean (or JNDI) implementation.

That's slightly beside the point though, as in general, I agree with you -
cross context sharing is a headache, a problem I don't think 'we' want to
try and solve, and difficult for people to understand. If you need that kind
of dynamic/component environment (and I don't think it's that big a deal to
either assemble a web application combining multiple artifacts, or deploying
it as a whole), then there are better ways of dealing with it (either
SpringDM or EJB).


>
> My only concern is that the ability I put inplace for addons to
> deliver their own spring config and have the manager load it (in a
> SpringDM style manner) be maintained, and I see its still there... :-)
>

It's possibly an area that could yet be simplified though (I've taken a
somewhat pragmatic approach so far to not rip up any existing public APIs
unless they are problematic in the current form). One question to ask is
would it only need the SpringDM style approach when you would be using
SpringDM (on the assumption that you are using that for a dynamic
environment)?


What if we put it in the dspace-services trunk, adjust the dspace
> trunk to use it and use this to get some community testing/feedback?
> If it fairs well over the next month... then we can do an official
> release of dspace-services just prior to the dspace 1.7.0 release
> candidates...
>

Yes, that sounds like a plan. Although it passes it's unit tests (with only
minor test modifications necessitated by the contract tightening), and I've
used it in a DSpace 1.6.2 application without any direct modifications
(related to the services at least), it does need more extensive
in-application testing.

G
--
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] DSpace/Maven help request - update dependency version

[Graham Triggs]

That version of tm-extractors is quite old.

There is a newer version on the Google site -
http://code.google.com/p/text-mining/ - but it will take a bit of work
wrapping things up for general use.

It has dependencies on newer versions of POI than 0.4, and some distinct
improvements to it's robustness.

G

On 6 October 2010 16:39, Tim Donohue  wrote:

> Ugh -- sounds like you've entered dependency hell.
>
> Though, I think the one shred of good news here is that it seems to only
> have a dependency conflict in one place in our codebase.
>
> It looks like (at a glance) if our WordFilter can be re-written to no
> longer need the org.textmining project, you *might* be OK (i.e.
> hopefully it wouldn't snowball on you). But, that would require finding
> a Word document text extractor that is as good as (or better than) that
> 'org.textmining' one, and then hoping it doesn't cause another
> dependency conflict.  Not sure of any alternative Word text extractors,
> off the top of my head, but maybe others know of one?
>
> - Tim
>
>
> On 10/6/2010 5:51 AM, Keith Gilbertson wrote:
> > Thanks, Tim.  That helped me to understand.  I put the version numbers of
> the dependency in the parent pom.xml ('dspace-src/pom.xml') and left the
> version numbers out of 'dspace-src/dspace-api/pom.xml'.
> >
> > So then I found another thing I didn't look at closely enough.  The
> WordFilter doesn't use poi directly, but the org.textmining project that it
> uses depends on that old version of POI.  To confuse things more, the old
> versions of poi had groupId 'poi', and the new versions have groupId
> 'org.apache.poi'.
> > I can convince Maven to forget about the old version of the POIi library
> by making this exclusion change in the parent pom:
> >   
> >  org.textmining
> >  tm-extractors
> >  0.4
> >  
> > 
> >poi
> >poi
> > 
> >  
> >   
> >   
> >
> > Then only the new version, org.apache.poi/poi/3.6 is included in the
> project.  Unfortunately, the org.textmining extractors really do need that
> version of POI.  The PowerPointFilter works, but I've broken the WordFilter:
> >
> > Exception:
> org.apache.poi.poifs.filesystem.POIFSFileSystem.getRoot()Lorg/apache/poi/poifs/filesystem/DirectoryEntry;
> > java.lang.NoSuchMethodError:
> org.apache.poi.poifs.filesystem.POIFSFileSystem.getRoot()Lorg/apache/poi/poifs/filesystem/DirectoryEntry;
> >   at
> org.textmining.text.extraction.WordExtractor.extractText(WordExtractor.java:51)
> >   at
> org.dspace.app.mediafilter.WordFilter.getDestinationStream(WordFilter.java:95)
> >
> > I have two programs that share the same classpath, but need different
> versions of the same library.
> >
> > I could rewrite the WordFilter so that it no longer uses the
> org.textmining package which needs the old library, but I keep thinking that
> the more I try to "fix" stuff, the more I'm likely to break:
> >
> >
> http://www.nypost.com/p/news/local/brooklyn/rat_bastards_f5onjzgcqxm0fu3RFz3ySL
> >
> >
> >
> > On Oct 5, 2010, at 4:09 PM, Tim Donohue wrote:
> >
> >> Hi Keith,
> >>
> >> Simply put, it's because you were accidentally looking in the wrong
> pom.xml :)  There's many of them sprinkled through the DSpace codebase, and
> they all inherit many of their settings from one main pom.xml.
> >>
> >> So, you noticed that the 'dspace-api/pom.xml' file included a dependency
> for "poi".   But, if you look closely, that dependency doesn't list
> a.  This is because, for DSpace, we manage all the versions of
> dependencies in one parent pom.xml (which is loaded via the  tag
> within the dspace-aip/pom.xml).
> >>
> >> Now, take a look at the [dspace-src]/pom.xml. This is the main Parent
> pom.xml for dspace (with an artifactid of 'dspace-parent')
> >>
> >> http://scm.dspace.org/svn/repo/dspace/trunk/pom.xml
> >>
> >> This is the pom.xml which actually lists the versions of every
> dependency used by the various APIs of DSpace.  If you search in this
> pom.xml, you'll find this entry:
> >>
> >> 
> >> poi
> >> poi
> >> 2.5.1-final-20040804
> >> 
> >>
> >> That's where the 2.5.1 version is sneaking in.  If you make your
> necessary changes to this pom.xml, everything should act as you expect it
> to. So, just undo your changes in 'dspace-src/dspace-api/pom.xml', and
> instead make those changes to 'dspace-src/pom.xml'
> >>
> >> I hope that helps!
> >>
> >> - Tim
> >>
> >> On 10/5/2010 2:36 PM, Keith Gilbertson wrote:
> >>> Hi,
> >>>
> >>> I've been experimenting with a Media Filter for text extraction from
> PowerPoint files.  It's based on the Apache POI libraries, as was suggested
> by others in a previous thread.
> >>>
> >>> It uses the poi, poi-scratchpad, and poi-ooxml artifacts, in version
> 3.6, the latest release version from Apache.  I haven't done much with
> Maven, and am not sure how to tell it which libraries I need.
> >>>
> >

Re: [Dspace-tech] A simplified version of dspace-services and Tomcat unloading

[Graham Triggs]

On 5 October 2010 19:17, Sands Alden Fish  wrote:

> What resources were leaking here exactly?  Leaking resources across
> requests can have some serious consequences depending on the resources, and
> I'm curious to know exactly what type of errant behavior we could expect
> from a 1.6.x version running the flawed services framework.
>

I would say that the ability to leak resource across requests is more
theoretical than demonstrated in a real world scenario. By which I mean that
it's clearly demonstrated that the Caching service has the ability to store
data either at the wrong scope, or to have request scope caches that
actually get bound / re-bound to a later, different, request - however, I
haven't exercised a full DSpace application in such a way to observe this
happening.

But even if the application wouldn't normally run into an issue, that
doesn't mean we shouldn't tighten up it's behaviour to prevent it from
becoming an issue later.

G
--
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] tomcat reporting memory leak?

[Graham Triggs]

On 5 October 2010 16:33, Simon Brown  wrote:

> Which nobody has requested, making this a massive red herring. I fail
> to see how cutting back on unnecessary and redundant database access
> constitutes "overhead to cover up the problems of larger
> repositories".


One person's "unnecessary and redundant database access" is another's very
necessary database access - well, at least it can be.

I remember the patch for reducing the updating of browse / search indexes,
and I can see why it would be useful to not do those updates during a batch
import if you have an appropriate workflow.

That won't be the case for all of the repositories - quite a few will
welcome the ability to see those items as and when they are added. There is
also the issue of how long it takes to do the one very big update at the end
of the batch run vs. incremental changes as you go - it may be less work
overall, but having one big change can be more disruptive in some cases.


> Any repository, regardless of size, will see
> improvements with this kind of optimisation, at least one example of
> which I have already highlighted (and had my arguments shouted down -
> this is also, incidentally, why I haven't bothered to open any other
> JIRA tickets on other performance issues we've seen. What would be the
> point?)
>

No, you didn't get shouted down for raising a performance issue. Where the
argument came was because you assumed that this would clearly be of benefit
to "any repository", when you did nothing to address the underlying
performance issues (which could have been helped quite dramatically with
some small SQL tweaks and some configuration work in Postgres), and instead
just bypassed them for one very specific use case.

It doesn't matter how large or small a repository is, if they don't perform
batch uploads using the ItemImporter, your change will do *nothing* for
them. But an alteration to the underlying SQL, and guidelines for getting
the best out of Postgres would benefit everyone - regardless of how large or
small the repository is, or the means by which they populate it.


> The pertinent question for me is why, whenever the issue of
> performance comes up, is one of these "theoretical future of
> repositories" screeds pulled out and slammed down in front of the
> conversation? People are reporting problems with the systems they have
> *right now*.


It's not meant to be a barrier to conversation, but a question as to what
you want to resolve. Do you want to address the *scalability* of DSpace, or
do you just want to avoid an immediate performance bottleneck? If we
conflate these, conversations are going to stall, and we're not going to
make any progress.


> Or rather, they were. And yes, it is true that there is a
> finite limit to what the hardware is capable of, but the quality of
> the software plays a significant role in how quickly that limit is
> reached. But we've had this conversation before. I don't really expect
> it to end any better this time than it did then.
>

I completely agree - but a solution that breaks the encapsulation of the
components in the system, and leaves important indexes in an inconsistent
state for an extended period of time is not an automatic win for the
majority of the community.

I offered a lot of suggestions as to how that code could be better
structured, improvements both to the SQL and the configuration of Postgres
to handle the load more efficiently, and suggestions for further tweaks that
would reduce the amount of updates that the code would have needed to do
still further. All of which would have be more beneficial to the community
(not just improving batch uploads, but interactive / singular deposits and
edits) - and not only that, would have improved the performance of your
systems further than you had so far achieved.

Any method of increasing the processing capabilities of a system,
> either through more powerful hardware or improvements in the software,
> is "postponing the inevitable" for any repository with continued
> growth. The difference is in how much cost there is to any individual
> repository in each of those methods. Our system, with the changes
> we've made to it, struggles at around 300,000 items. People are
> reporting problems (presumably running stock 1.6.2) at around 50,000,
> from what I can gather.


This is where we need to be careful about what we are reporting. Quite a few
of the issues around 1.6.x appear to be around rampant memory usage, rather
than a clear function of how many records there are in the database. There
are also different issues involved if we are talking about adding / editing
lots of records, or simply highly accessed.

Even so, regardless of what we do to the code to make it efficient, it does
not and can not absolve the system administrator of correctly maintaining
both DSpace itself, and it's dependencies. I wouldn't want to get drawn on
where that point is without any evidence, but there is a lot of scope for
altering and im

[Dspace-tech] A simplified version of dspace-services and Tomcat unloading

[Graham Triggs]

Hello all,

So, we've been debating the performance / resource usage of DSpace 1.6. This
has just happened to coincide with work that I needed to do to qualify
DSpace 1.6 (and/or 1.7) for our own use. So, for the past week or so, I've
been hammering away at profiling DSpace. This isn't a comprehensive analysis
of the entire codebase, but focusing on the serious issue of stability and
resource leakage.

This immediately led to finding two issues with the dspace-services code and
it's dependencies:

1) The resources in the FinalizableReferenceQueue were leaking (this was
noted before in a Tomcat stack trace posted here) - this could have been
avoided if a newer version of reflectutils had been used, along with the
LifeCycleManager it contains

2) A ThreadLocal in the CachingServiceImpl was also leaking - the service
did attempt to clean up in the shutdown, but it would only ever have been
able to clear up the resources of the thread executing the shutdown, not all
the other threads that would have processed requests.

Investigating the second issue, it turned out that this was being used to
keep track of Request scope caches - as well as potentially leaking
resources (across requests), this highlighted a problem in the service
contract, and a dangerous coupling between the implementation of the
services.

The Caching service was written to return a Request scoped cache without any
valid Request to bind it to - this was explicitly used in the listeners /
filters to acquire a cache and assign request objects into it before the
request service was called to 'start' a request and bind to the scoped
cache. There were also tests that expected this behaviour. Whilst the
request service will clear a request scoped cache if it's been bound to it,
you can't guarantee that the cache gets bound to a request.

Worse though, the Request service was depending on the Caching service
tracking request scoped caches against the current thread in order to be
able to track the current request. So, the one responsibility that the
request service should have, it was passing on and hoping that another
service might do the job for it without it being explicitly clear in the
contract.

If that wasn't bad enough, getCache doesn't actually properly respect the
CacheScope when trying to retrieve a particular cache (it could retrieve a
non-Request scoped cache if one of the same name had already been created,
regardless of the scope parameter). The Caching service also registered an
MBean that was never removed. And, to wrap up my examination of the Caching
service, it didn't close an InputStream that it opened.

So, onwards - wanting to simplify this code a bit (as we don't need to
support non-Spring IoC containers, at least not yet - and we do need to be
able to understand and maintain this code) - I quickly noticed an issue in
the SpringServiceManager where it returned multiple instances of the same
service name, because it was under the mistaken belief that it needed to
track some of the service names itself, when it was actually getting them
returned from Spring.

And the ServiceMixinManager was entirely redundant - as long as we can rely
on Spring, we can rely on the container to do everything that the Mixin
manager does, without us having to worry about the complexity of it (and it
is both relatively complex, and a source of our problems earlier as it uses
the FinalizableReferenceQueue). I've actually got a real bugbear about us
using the term 'mixin'. A mixin means something specific - a class that
provides an actual implementation, and is combined with other mixins to
produce other classes [through multiple-inheritance]. When you look at what
is implemented in dspace-services under the term mixin, well we don't
actually have mixins... we have interfaces that are implemented by service
classes. Each service class completely and wholly provide implementations of
the methods defined in all the interfaces they implement. That's a concept
that Java has had for a long time, and is (should be) well understood by all
Java programmers. That doesn't make them mixins. We don't/shouldn't want
'mixins' (multiple inheritance - there is a reason Java doesn't support it).
And we shouldn't be using a term incorrectly just because it's cool.

While I'm on a terminology rant, I'm not particularly happy about the
'interceptors' (ie. RequestInterceptor). They are looking and feeling a lot
more like listeners than interceptors - allowing other services to react to
events taking place (start request / end request), rather than affecting
it's behaviour. I've not renamed them, for the sake of keeping as much of
the existing API intact as I can (whilst clearing out [currently] redundant
or problematic code), but actually it seems that it should really be pushing
a synchronous and immediately executing event notification, so that the
listener registration can occur in a single place rather than duplicating
that functionality anywhere similar concepts may 

Re: [Dspace-tech] tomcat reporting memory leak?

[Graham Triggs]

On 29 September 2010 14:17, Tom De Mulder  wrote:

> I know you like to talk down the problem, but that really isn't helping.
>

This isn't about talking down the problem - it's about finding where the
real problems are and not just patching the immediate concerns. And
considering the interests of nearly 1000 DSpace instances that are
registered on dspace.org - many of whom will probably be more worried about
rampant resource usage for small repositories from adding overhead to cover
up the problems of larger repositories.


> We run 5 DSpace instances, three of these are systems with hundreds of
> thousands of items, and it's dog slow and immensely resource-intensive. And
> yes, we want these to be single systems. Why shouldn't we?
>

Surely the more pertinent question is why wouldn't you want to be able to
run a multi-node solution? I'm sure I don't need to tell you that no matter
how good a job you do of making the system perform better with larger
datasets, there will always be a finite limit to how large the repository
can be, how many users you can service, and how quickly it will process
requests for any given hardware allocation.

Yes, DSpace can do a better job than it currently does, but it's just
postponing the inevitable. How much in technology relies on just making
things bigger/faster? Even our single system hardware is generally made of
multiple identical components - CPUs with multiple cores, memory consisting
of multiple 'sticks', each consisting of multiple storage chips, storage
combining multiple hard drives each having multiple platters.

And much of our dependencies are going the same way - Oracle database
clusters, Solr is designed to get scalability from running over multiple
shards, even Postgres has taken a major step towards clustering /
replication with it's 9.0 release.

Either way, you will always hit a hard limit with keeping things on a single
system - so at some point, something has to give, whether it's separating
out DSpace application, Solr and Postgres instances to separate machines, or
accepting this reality in the repository and building it to scale across
multiple nodes itself. This in turn would bring benefits to how easily you
can scale (in theory, a lot easier to scale at the repository level than
scaling each of it's individual components), as well as potentially better
preservation and federation capabilities.

G
--
Virtualization is moving to the mainstream and overtaking non-virtualized
environment for deploying applications. Does it make network security 
easier or more difficult to achieve? Read this whitepaper to separate the 
two and get a better understanding.
http://p.sf.net/sfu/hp-phase2-d2d___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] tomcat reporting memory leak?

[Graham Triggs]

That begs the question as do you think something else should be chosen /
recommended?

There really isn't anything preventing you using Jetty, etc. but Tomcat is
actually a pretty solid server that does a lot of things quite well - and
particularly in recent versions in being defensive against bad application
behaviour.

And when you look at the grand scheme of things, the smaller footprint of
Jetty doesn't really make a whole lot of difference.

G

On 29 September 2010 11:47, Mark Ehle  wrote:

> Why was tomcat chosen as a platform for DSpace?
>
>
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] tomcat reporting memory leak?

[Graham Triggs]

On 29 September 2010 11:48, Tom De Mulder  wrote:

> A lot of the back-end code of DSpace, the very core of it, is inherently
> inefficient


I don't entirely disagree with that statement - there are some things that
can definitely be improved, particularly where you have to deal with more
items in a single instance.

But take a look at my numbers - at it's core, it really isn't that bad for
the vast majority of DSpace users (how many have more than even 50,000 items
currently)? And some of it depends on correct system setup (Postgres
version/options, etc.)

It's adding xmlui, solr, etc. that is putting a lot more demands on the
system.


G
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] tomcat reporting memory leak?

[Graham Triggs]

On 29 September 2010 11:38, Hilton Gibson  wrote:

> Using the XMLUI.
> Does DSpace really need this and what happens when we go to one million
> items ??
>
>
Does DSpace really need that? No. As I have said, I'm running 30 separate
repositories - using JSPUI (circa 1.4.2 / 1.5 codebase) - all on a single
server / Tomcat instance.

Some of those repositories have 1000s of items, and get quite decent levels
of access.

The server has 8GB installed, 3GB heap turned over to Tomcat (plus 1GB for
non-heap).

The Tomcat instance has 2GB of *free* heap space, rarely runs above 5% cpu
usage, and has plenty of capacity to run more repositories (the rate at
which files are opened/closed is actually a bigger issue for Tomcat
startup).

Although, it's worth pointing out that the database is hosted on a separate
server - I can't say how many resources that is really using, as it's shared
with other services, but it is apparently 'tiny'.



What happens at one million items? Well, that's an interesting issue. But is
it really the right question to be asking? How far do you want/need to be
able to scale a 'monolithic' instance, before you spread it over multiple
servers?

As long as you can spread it over multiple servers, it gives you a much
higher ceiling than relying on a single box - and it is easier to scale for
increasing size/usage by adding more boxes (you don't have to migrate).

If you focus on scaling a single installation, then you end up increasing
the overall requirements (ie. memory for caching), and make it harder to
have scaling over multiple boxes at all.

G
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] tomcat reporting memory leak?

[Graham Triggs]

On 20 September 2010 15:59, Tom De Mulder  wrote:

> On Mon, 20 Sep 2010, Damian Marinaccio wrote:
>
> > I'm seeing the following log messages in catalina.out:
> > [...]
> > SEVERE: The web application [] appears to have started a thread named
> [FinalizableReferenceQueue] but has failed to stop it.
> > This is very likely to create a memory leak.
>
> There are quite a few memory leaks in DSpace. We have a cronjob to restart
> Tomcat nightly, because otherwise it'll break the next day.
>


Hi all,

Oh, welcome to my world!!

I'm going to start off by pointing out that the majority of DSpace code is
actually quite well behaved. Going back to the codebase circa 1.4.2 / 1.5,
and using the JSP user interface - I've got *thirty* spearate DSpace
repositories / applications running in a single Tomcat instance, which has
operated without a restart in over 90 days. And whilst be able to undeploy
and redeploy any of those applications at will - or just reload them so that
they pick up new configuration.

That does require a bit of careful setup / teardown in the context listeners
(that wasn't always part of the DSpace code), and you need to get certain
JARs - particularly the database/pooling drivers - out of the web
applications entirely and into the shared level of Tomcat. Most of that is
actually just good / recommended practise for systems administration of a
Java application server anyway.

I was careful to point out that I have achieved that with pre-1.6 code and
JSP only. Both 1.6 and XML ui (of any age) change the landscape. XML ui has
always taken a large chunk of resources, although whilst it was still based
on Cocoon 2.1, I managed to at least clean up it's startup / shutdown
behaviour by repairing it's logging handler. This behaviour has changed with
Cocoon 2.2, and I'll come back to that shortly.

So, 1.6 - I've been doing some work on the resource usage and clean
loading/unloading of both JSP and XML using 1.6.2 recently, and neither are
clean out of the box.

The first issue you run into is the FinalizableReferenceQueue noted in the
stack trace above. This is coming from a reference map in reflectutils - and
was found to be a cleanup problem in course of DSpace 2 development (the
kernel / services framework was backported from that work). I added a
LifecycleManager to reflectutils that was released as version 0.9.11 that
allows the internal structures to be shutdown cleanly, and implemented this
as part of DSpace 2, however this appears to have been ignored in the
backport.

So, with the reflectutils/Lifecycle changes, and careful placement of JARs,
etc. I did get the JSP ui to unload cleanly last week. I would note that I
didn't stress the application too heavily, so there may be some operations
that might trigger different code paths that are still a problem, but at the
baseline it was working correctly.

XML ui has proven to be a somewhat more challenging beast. I first ran into
two problems that are inside Cocoon 2.2 itself - 1) in the sitemap
processing, it's using a stack inside a ThreadLocal, but it never removes
the stack when it empties it, and 2) in one class relating to flowscript
handling, it does not clean up the Mozilla Rhino engine correctly when it's
finished using it (curiously, it's used in a number of places, and
everywhere else it appears to be structured correctly to clean up - just
this one class is screwed up).

With locally patched versions of the sitemap and flowscript JARs from Cocoon
(the ThreadLocal patch isn't really guaranteed to not leak in unexpected
circumstances - but it was sufficient to remove the problem in the scope of
this testing. Basically, ThreadLocal is really dangerous to use), I then ran
into another issue, this time with the CachingService that was backported.

With XML ui, it's using the RequestScope function of the caching service (it
didn't appear to be exercising this part with JSP - that may just be because
I only ran through limited code paths). For the RequestScope, it's tying the
cache not to the request object... but to a ThreadLocal. And that
ThreadLocal isn't being cleaned up at the end of the request. (The shutdown
code is also incapable of doing the job it's intended for, as it will only
ever execute on a single thread, and not see all the other threads that may
have processed requests).

There is a high probability of this leaking memory all over the place, and
there is also the nasty potential of leak information across requests that
is undesirable.

I made another hacked version that removes the ThreadLocal, but replicates a
lot of it's thread affinity behaviour (so, it still has the nasty side
effects of the implementation, but at least removed the hold the system had
over the application resources). XML ui was *still* not unloading correctly,
and at this point the profiler stopped giving me pointers to strong
references that were being held. So right now I'm not sure what else is up -
but there is at least one more troubling part of the c

Re: [Dspace-tech] Improper Install?

[Graham Triggs]

The 'work' directory is under the control of Tomcat. If the application
specific directories are not being created, then it is almost certainly
because the user that Tomcat is being run under does not have permission to
write to the work directory.

G

On 7 April 2010 19:56, bryce ray  wrote:

> I'm having some difficulties with my installation of dspace.  I've
> discovered that directories that should have been created were not.  When
> are these directories created? and what might have caused them not to be
> created?
>
> Directories in:
> /usr/share/tomcat6/work/Catalina
>
> When viewing this directory there are sub directories for other
> installations of dspace.  However, there is no directory for my most recent
> install of dspace.
>
> Let me know if you need any more information, and thank you for helping me
> diagnose this problem.
>
>
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] DSpace 1.6 Run With Tomcat Security Manager Enabled, Anybody?

[Graham Triggs]

Hi Christian,

It's good to see someone else working on this. I did look at it myself at
one stage, in a mad ambition to be all 'correct' about it. And after chasing
down the rabbit hole for hours creating a seemingly never-ending list of
permissions, I came to the conclusion that as I've got a firewall and httpd
in between my application server and the nefarious horde, it probably wasn't
worth torturing myself trying to get it completed.

But one thing in your email did strike me...

On 7 April 2010 00:07, Christian Voelker  wrote:

> I found a hint on the web, that it should be possible to put a policy file
> into the WEB-INF directory of a web application which I think would be a
> great solution, but it has to work before.
>
>
It seems really odd that an application server / security manager should
allow you to do that. The whole point is to limit the extent to which a
deployed application can interact with / harm the system, and it's quite a
large hole in the wall if an application can simply say "actually, do you
mind if I just mess around with the system a little".

One thing you might want to try to find out what is going on, is to enable
debugging of the security manager, with the CATALINA_OPTS:

export CATALINA_OPTS=-Djava.security.debug=all(Unix)
set CATALINA_OPTS=-Djava.security.debug=all   (Windows)

Good luck!
G
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] log4j causing slowness for DSpace

[Graham Triggs]

Jason,

It's highly unlikely that Log4J is causing the startup to be slow. Whilst
your startup takes in the region of 10 minutes, there is no indication in
the log as to when each of those log4j warnings appeared, or what was
occurring at the time.

The regular re-occurrence of org.apache.commons.digester.Digester.sax
appender warnings, and the org.apache.catalina.startup.Embedded warning,
would suggest that the application server itself (Tomcat?) is missing some
logging configuration.

Those Digester messages look like they relate to the initialization of each
webapp deployed to the server - ie. where it loads WEB-INF/web.xml (and
correlating with the 'using dspace.cfg' messages that would also appear with
each one).

Deploying multiple web applications can take some time - loading the JARs,
starting any listeners / servlets that are set to load on startup. In the
case of xmlui (and Solr?) at least that will trigger a fairly significant
initialization process.

And it appears that you are deploying the xmlui application twice (once as
xmlui, and once as root?)

Regards,
G


On 5 April 2010 21:50, Jason Fowler  wrote:

> I've been beating my brains in over this issue, and I have tried every
> possible solution I can find. I've recently migrated to 1.6.0, and since
> making the change, problems with log4j occur at startup. This makes my
> Dspace startup take ten minutes. I've narrowed down the problem (I think) to
> being an issue with log4j.
>
> I just appended a date stamp to my catalina.out file while I restarted. I
> have output it below. The first thing that shows up after the stamp is log4j
> errors. After churning for 10 minutes, everything works fine, even logging,
> solr, etc. Does anyone have any idea what might be causing this problem? If
> so, many, many thanks in advance.
>
> Mon Apr  5 15:29:51 CDT 2010
> log4j:WARN No appenders could be found for logger
> (org.apache.catalina.startup.Embedded).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN No appenders could be found for logger
> (org.apache.commons.digester.Digester.sax).
> log4j:WARN Please initialize the log4j system properly.
> INFO: Loading provided config file: /mnt/storage/dspace/config/dspace.cfg
> INFO: Using dspace provided log configuration (log.init.config)
> INFO: Loading: /mnt/storage/dspace/config/log4j.properties
> Loading catalog:
> file:/usr/local/apache-tomcat-5.5.26/work/Catalina/localhost/xmlui/cocoon_xml_resolver_entities/catalog
> log4j:WARN No appenders could be found for logger
> (org.apache.commons.digester.Digester.sax).
> log4j:WARN Please initialize the log4j system properly.
> INFO: Loading provided config file: /mnt/storage/dspace/config/dspace.cfg
> INFO: Using dspace provided log configuration (log.init.config)
> INFO: Loading: /mnt/storage/dspace/config/log4j.properties
> - Loading provided config file: /mnt/storage/dspace/config/dspace.cfg
> - Using dspace provided log configuration (log.init.config)
> - Loading: /mnt/storage/dspace/config/log4j.properties
> log4j:WARN No appenders could be found for logger
> (org.apache.commons.digester.Digester.sax).
> log4j:WARN Please initialize the log4j system properly.
> INFO: Loading provided config file: /mnt/storage/dspace/config/dspace.cfg
> INFO: Using dspace provided log configuration (log.init.config)
> INFO: Loading: /mnt/storage/dspace/config/log4j.properties
> log4j:WARN No appenders could be found for logger
> (org.apache.commons.digester.Digester.sax).
> log4j:WARN Please initialize the log4j system properly.
> INFO: Loading provided config file: /mnt/storage/dspace/config/dspace.cfg
> INFO: Using dspace provided log configuration (log.init.config)
> INFO: Loading: /mnt/storage/dspace/config/log4j.properties
> Loading catalog:
> file:/usr/local/apache-tomcat-5.5.26/work/Catalina/localhost/_/cocoon_xml_resolver_entities/catalog
> log4j:WARN No appenders could be found for logger
> (org.apache.commons.digester.Digester.sax).
> log4j:WARN Please initialize the log4j system properly.
> INFO: Loading provided config file: /mnt/storage/dspace/config/dspace.cfg
> INFO: Using dspace provided log configuration (log.init.config)
> INFO: Loading: /mnt/storage/dspace/config/log4j.properties
> Apr 5, 2010 3:39:31 PM org.apache.solr.servlet.SolrDispatchFilter init
> INFO: SolrDispatchFilter.init()
> Apr 5, 2010 3:39:31 PM org.apache.solr.core.SolrResourceLoader
> locateInstanceDir
> INFO: Using JNDI solr.home: /mnt/storage/dspace/solr
>
> Jason Fowler, CA, MSLS
> Archives and Special Collections Librarian
> The Southern Baptist Theological Seminary
> Vice President, ALABI
> jfow...@sbts.edu
>
>
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> __

Re: [Dspace-tech] Handle Server Problems

[Graham Triggs]

Sounds like you need to remove the lock file from the txns directory.

Regards,
G

On 11 March 2010 22:32, Cameron, Jacob  wrote:

>  Our handle server has stopped responding.
>
>
>
> I’ve killed and restarted the process several times and had it working
> again this morning.  But it has died again this afternoon and I’m unable to
> resurrect it.  We just renewed our handle server so I know it’s not
> shutdown.  I have even tried restarting the server.
>
>
>
> Does anyone have any suggestions?
>
>
>
> --
>
> Jake Cameron, BCS(UNB)
>
> Technical Specialist III
>
> Information Systems and Technical Services
>
> University of Lethbridge Library
>
> Phone:(403)329-2756
>
>
>
>
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Problem with authorization created during Submission process in DSpace 1.5.1

[Graham Triggs]

Hi Sue,

Don't worry, the READ policy on the COLLECTION has no effect on the access
of the items.

In fact, it doesn't really have any effect on the access of the Collection
home page/browse itself, either :(

For accessing items, what you need to be concerned with is the
DEFAULT_ITEM_READ and DEFAULT_BITSTREAM_READ - those are the policies that
Items and Bitstreams inherit when they are installed (accepted) into the
repository - and it is only the policies defined on each Item or Bitstream
individually (which are created from the default) that determine access when
someone tries to view or download the item / bitstream.

Regards,
G

On 11 March 2010 18:48, Thornton, Susan M. (LARC-B702)[RAYTHEON TECHNICAL
SERVICES COMPANY]  wrote:

>  Hi,
>
>  We are in the process of implementing a new secure DSpace instance
> where the default access has to be *“no-access”* for anonymous users.
> During the “Define Collection” process, despite the fact that I am NOT
> checking the box that says *“New items should be publicly readable”*, the
> following authorization is being added at the Collection level for a new
> Collection:
>
>
>
>
>
> Maybe this is ok and maybe I’m unclear on the difference between READ
> access to a Collection and DEFAULT-ITEM-READ access to a Collection.
>
>
>
> If the only access I have to a Collection is READ access, exactly what can
> I do and see in that Collection?
>
>
>
> Thanks and Best regards,
>
> Sue
>
>
>
>
>
> *Sue Walker-Thornton*
>
> *ConITS Contract**
> NASA Langley Research Center
> Integrated Library Systems ~ Developer/DBA*
>
> *130 Research Drive*
>
> *Hampton, VA  23666*
>
> *Office: (757) 224-4074**
> Fax:(757) 224-4001
> Mobile:  (757) 506-9903*
>
> *Email:** ** **susan.m.thorn...@nasa.gov* 
>
>
>
>
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
<>--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Sloooow submission process in DSpace 1.5.1

[Graham Triggs]

Hi Simon,

On 23 February 2010 17:47, Simon Brown  wrote:

> Sorry to stick my oar in here, but...
>

Oars are always welcome.


> I don't think this is the case. I'm sure it was the intention, but
> from what we've been able to determine, each DescribeStep for in-
> progress items calls InProgressSubmission.update() which for both
> workflow and non-workflow items calls Item.update(), which will fire a
> MODIFY_METADATA event for that item. The BrowseConsumer will process
> that event whether the item is installed or not. We determined this
> after an increasing number of user complaints about the submission
> process "slowing down" and added an isArchived() check to our
> BrowseConsumer, which made the submissions process noticeably snappier.
>
>
Yes, that probably is happening at a BrowseConsumer level (the event
mechanism / BrowseConsumer was added after this browse code was committed,
I'm not 100% sure of the circumstances of it's use).

However, the BrowseConsumer calls indexItem(), which has the explicit check
in it:

if (item.isArchived() || item.isWithdrawn())
{
indexItem(new ItemMetadataProxy(item));

// Ensure that we remove any invalid entries
pruneIndexes();
}

So, the indexing / pruneIndexing won't happen if the item is not in either
'archive' or 'withdrawn' state - and it shouldn't be in either whilst it is
still in the workspace / workflow. Whilst it passes through the browse
indexer, it shouldn't be doing anything that is expensive (or gets more
expensive with repository size), before installItem() is called.

AFAIK, the BrowseConsumer shouldn't have just an isArchived() check, as that
would prevent indexes being updated correctly when an item is withdrawn. But
it could replicate the if (isArchived() || isWithdrawn()) check, and doing
it in the BrowseConsumer would avoid some overhead that is incurred when
IndexBrowse is created.

G
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Sloooow submission process in DSpace 1.5.1

[Graham Triggs]

Hi Joel,

On 12 February 2010 17:04, Richard, Joel M  wrote:

> I'll put my two cents in here...
>
> I haven't dug deep into ways of making these queries work better than they
> already do, but my experience tells me that there has to be a faster way of
> doing these operations. Sequence scans (i.e., "Seq Scan on bi_2_dmap") are
> extremely costly on large tables and these are getting on to being large
> (186k rows.)


> And I don't mean to second-guess any of the DSpace developers, but I've
> worked on databases with hundreds of thousands (even millions) of records
> and never experienced anything this bad for regularly-used queries. I know
> we're in a heterogeneous environment and we don't know what kind of server
> Susan has or how it's tuned, but any query with a cost of 300,000 is
> insanely high. There's got to be a better way. :)
>

It's a bit of an exaggeration to say that this is a regularly used query. It
will only be used when:

a) Installing a new item
b) Withdrawing an item
c) Reinstating an item
d) Updating the metadata of an item that has at some point been installed

It's not called during the submission / workflow process. It's not called on
any general user access. In fact, the very reason for it's existence is to
remove the need to use a costly DISTINCT when displaying browse pages to
general users - which is a much more common operation.

Personally, I have never used the EXCEPT clause and so I am not familiar
> with it, but it does seem, Susan, that you found a faster query, and from a
> first glance it looks like it would achieve the same results. It's faster
> because you've found a query that does an "Index Scan" which is always tons
> faster than a "Sequence Scan". But you need to satisfy yourself as to
> whether or not the same rows are being deleted.
>

The way I'm reading it, that's a bit of a jump ahead. In Sue's query, yes
there is an index scan being used, but as part of a NOT filter subplan.
There is still a sequence scan on the bi_2_dis table, and so it's doing a
separate index scan for each row returned by bi_2_dis - that's a loop of
around 7 index scans.

In the EXCEPT version, it is only doing 3 sequence scans - none of the
operations are looped. However, it is the sort that it uses to implement the
EXCEPT in this case that sucks the performance / scalability out of the
query.

If you look at the query in a Postgres 8.4 environment that has been given
sufficient work_mem, then it still only does 3 scans. But the results are
hash joined, not sorted - and the resulting execution is better than the
EXISTS query in 8.2 (with it's 7 index scans).

Now take a look at the same EXISTS query running in 8.4 - it doesn't use
index scans at all. In this case, it's gone to using just two sequence
scans, and hash joining the results. Making it the most efficient execution
of all (but not by much as two of the sequence scans in the hash joined
EXCEPTS are on the same table and can benefit from caching). But
importantly, the EXISTS degrades more gracefully when the work_mem is
insufficient.

Regards,
G
--
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Bad robot! Googlebot and Internal Server Errors

[Graham Triggs]

On 11 February 2010 14:37, Tom De Mulder  wrote:

> You should add "/dspace" to the start of those disallowed patterns,
> because your DSpace URLs start with "/dspace" after the hostname.
>
>
You should also ensure that the robots.txt is available at the root of the
server... ie.

https://dspace.stir.ac.uk
/robots.txt

and not just

https://dspace.stir.ac.uk
/dspace/robots.txt

G
--
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Sloooow submission process in DSpace 1.5.1

[Graham Triggs]

Hi Sue,

(You don't say explicitly, so I'll assume that you are using Postgres)

I can't explain why the addition of metadata rows is taking so long - there
are a number of factors that could come into play... server getting many /
expensive requests, resource contention, postgres needs maintaining (ie.
vacuum) or tuning or upgrading, or all three.

However, the issue of slow installing (license granting) of items, is quite
likely answered in this post:

http://sourceforge.net/mailarchive/message.php?msg_name=9efc26fb-749c-4a67-8481-f1a6ce09b...@gmail.com

to put it briefly - the item installation takes on some extra work in browse
indexing to ensure that general browse lookups are efficient (you only add
an item once, users might issue 100s of browse requests a day). The SQL that
makes that calculation is written in it's most efficient form.

But you still need the database to be configured properly in order for that
operation to be executed in an efficient way. If you are using the default
work_mem setting, you can easily save 60% of your installation time - and
generally improving the efficiency of the database - by increasing it
appropriately.

Regards,
G

On 4 February 2010 23:29, Thornton, Susan M. (LARC-B702)[RAYTHEON TECHNICAL
SERVICES COMPANY]  wrote:

>  Hello,
>
>
>
>  We have been noticing recently that, as our repository grows in size
> (we currently have approx. 125,000 Items and adding more each week), our
> online submission process is getting slower and slower and slower.  It’s
> gotten so bad that, even if I’m only trying to add more authors to an Item
> by clicking the “Add more” button:
>
>
>
> It is taking a really long time (just now it took approximately 20 seconds)
> for the application to respond with the subsequent screen:
>
>
>
>
>
> To make things worse, once the customer clicks on the “I grant the license”
> button, the application hangs not just for seconds, but for minutes (I saw
> one case of 11 minutes recently).  Our customers are getting very frustrated
> with this.
>
>
>
> Has anyone else had this problem or does anyone have an idea of where
> DSpace might be getting bogged-down.  I looked at the INFO messages in the
> dspace.log file and there seems to be a whole lot of “update_item” lines in
> there while not much seems to be happening on the screen, but I didn’t look
> at the programs in detail.
>
>
>
> Any help would certainly be appreciated.
>
>
>
> Best regards,
>
> Sue
>
>
>
> *Sue Walker-Thornton*
>
> *ConITS Contract**
> **NASA Langley Research Center**
> **Integrated Library Systems Application & Database Administrator*
>
> *130 Research Drive*
>
> *Hampton, VA  23666*
>
> *Office: (757) 224-4074**
> **Fax:(757) 224-4001**
> **Mobile:  (757) 506-9903*
>
> *Email:** ** **susan.m.thorn...@nasa.gov* 
>
>
>
>
> --
> The Planet: dedicated and managed hosting, cloud storage, colocation
> Stay online with enterprise data centers and the best network in the
> business
> Choose flexible plans and management services without long-term contracts
> Personal 24x7 support from experience hosting pros just a phone call away.
> http://p.sf.net/sfu/theplanet-com
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
<><>--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [Dspace-devel] JSPUI language detection - request for comment

[Graham Triggs]

BTW, 1 + 2 are technically the same thing... the EPerson's default locale is 
stored in the current session when they log in - overriding what is currently 
there.

The UI selection will update the Locale in the current session - overriding 
whatever is there currently (either a previous choice, or the EPerson's default 
that has been inserted in the session).

So priority of choosing a message to display is:

1. Session locale setting
2. Browser default locale
3. DSpace default locale

G

On 24 Jan 2010, at 20:07, Kim Shepherd wrote:

> Hi all,
> 
> JSPUI's i18n settings are currently a bit broken in trunk (future 1.6). When 
> an EPerson is logged in and has configured a language in their profile which 
> doesn't match the locale requested by their web browser, they will end up 
> seeing both languages used on some pages.
> This is basically due to inconsistent i18n key lookup methods used throughout 
> JSPUI.
> 
> Now's a good time to go through and make everything consistent, so right now 
> the idea is to go back to the order that locales were applied before 1.5:
> 
> (1 is top priority, 5 is bottom, so the first successful match will be 
> selected)
> 
> 1. UI selection (eg. manually selecting 'English' or 'German' from the JSPUI 
> or XMLUI frontend while browsing)
> 2. Eperson default locale (configured via Edit Profile in JSPUI or XMLUI)
> 3. Browser default locale (first locale requested by end user's web browser)
> 4. Dspace default locale (from dspace.cfg)
> 
> Any feedback, concerns or ideas are welcomed: email replies (preferably 
> onlist) or JIRA comments.
> 
> The full JIRA issue is here: http://jira.dspace.org/jira/browse/DS-418 
> 
> If no objections are raised in the next few days, I'll just go ahead with a 
> patch based on the order given above.
> 
> Cheers,
> 
> Kim
> --
> Kim Shepherd
> IRR Technical Specialist
> ITS Systems & Development
> The University of Waikato
> New Zealand
> 
> DDI +64 7 838 4025
> 
> 
> --
> Throughout its 18-year history, RSA Conference consistently attracts the
> world's best and brightest in the field, creating opportunities for Conference
> attendees to learn about information security's most important issues through
> interactions with peers, luminaries and emerging and established companies.
> http://p.sf.net/sfu/rsaconf-dev2dev
> ___
> Dspace-devel mailing list
> dspace-de...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-devel


--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] DSpace & secure passwords

[Graham Triggs]

I've not tried doing this, but after looking around, I would attempt the 
following as a non-invasive way of making this work.

http://www.coderanch.com/t/413965/Servlets/java/Force-URL-redirect-from-http

Create a security constraint in your web.xml, that will force certain URLs to 
use a secure transport.



SSL Pages
/ldap-login*
GET
PUT
POST


CONFIDENTIAL



If this doesn't work, the next best option would be to write a servlet filter 
that checks for secure transport, and redirects if it isn't - then map it to 
the login pages in the web.xml.

Graham Triggs
Technical Architect
Open Repository
http://www.openrepository.com/




-Original Message-
From: Wynne, David [mailto:d.wy...@ljmu.ac.uk]
Sent: Fri 16-Oct-09 14:52
To: dspace-tech@lists.sourceforge.net
Subject: [Dspace-tech] DSpace & secure passwords
 
Apologies for posting ag a in so quickly.
 
I've now got DSpace working with LDAP ( Active Direstory   )
 
But the login page just uses HTTP, which I am concerned about as my main
Uni password  will be passed unencrypted.
 
1. I enabled HTTPS in th e  tomcat server.xml file.
 
2. I generated a certificate using keytool
 
3. I now have HTTPS enabled in Tomcat 6 on Port 8443
 
My questyion is how to force the login page to use:
 
https://dspace.cms.livjm.ac.uk:8443/jspui/ldap-login
 
Not bothered about the whole of DSpace using HTTPS just the login.
 
Tried:
 
xmlui.force.ssl = true
 
In dspace.cfg, but that doesn't do it.
 

Also since setting LDAP to work I can't get the local administrator
username / password that I setup when I initially setup DSpace (
d.s.will...@ljmu.ac.uk ).
 
2009-10-16 14:43:20,855 INFO
org.dspace.authenticate.LDAPHierarchicalAuthentication @
anonymous:session_id=E841C1A380011133295B5204F4A74449:ip_addr=150.204.48
.5:auth:attempting trivial auth of user=d.s.will...@ljmu.ac.uk
2009-10-16 14:43:20,936 INFO
org.dspace.authenticate.LDAPHierarchicalAuthentication @
anonymous:session_id=E841C1A380011133295B5204F4A74449:ip_addr=150.204.48
.5:failed_login:no DN found for user d.s.will...@ljmu.ac.uk
2009-10-16 14:43:20,936 INFO  org.dspace.app.webui.servlet.LDAPServlet @
anonymous:session_id=E841C1A380011133295B5204F4A74449:ip_addr=150.204.48
.5:failed_login:netid=d.s.will...@ljmu.ac.uk, result=2
 
I read about the Stackable Authenication in dspace.cfg & I've got both:
 
plugin.sequence.org.dspace.authenticate.AuthenticationMethod = \
org.dspace.authenticate.PasswordAuthentication
 
plugin.sequence.org.dspace.authenticate.AuthenticationMethod = \
org.dspace.authenticate.LDAPHierarchicalAuthentication
 

But is doesn't seem to use the first one which I believe is for "local"
accounts.
 
Thanks
 

Dave Wynne
Senior Technical Officer
School of Computing and Maths
Liverpool John Moores University
Byrom Street
Liverpool L2 2AF
E-Mail: d.wy...@ljmu.ac.uk
Tel:0151-231-2108 

 
 

--
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Google Analytics and DSpace 1.5.2 part 2

[Graham Triggs]

Hi,

It won't be the upgrade to 1.5.2. Google Analytics tracking works via 
Javascript in the browser - so as long as the code appears correctly in the 
HTML, and can be executed, then it will track.

And in this case, the code does appear correctly. But it isn't being executed.

Hacking a saved file of the html, it looks like this line:

Re: [Dspace-tech] Creative Commons problem

[Graham Triggs]

FYI, the problem that I reported to Creative Commons has been fixed.

G

-Original Message-
From: Graham Triggs [mailto:gra...@biomedcentral.com] 
Sent: 23 July 2009 10:38
To: Robin Taylor; Tech General
Subject: Re: [Dspace-tech] Creative Commons problem


Hmmm... Not sure about that one. I've run into this issue:

http://code.creativecommons.org/issues/issue142

G 

> -Original Message-
> From: Robin Taylor [mailto:robin.tay...@ed.ac.uk] 
> Sent: 23 July 2009 10:31
> To: Tech General
> Subject: [Dspace-tech] Creative Commons problem
> 
> A speculative query - Anyone experiencing any problems with 
> CC licences over the last couple of days ? A problem has been 
> reported on the spanish users mailing list and we find 
> ourselves now unable to anonymously view items that were 
> submitted yesterday and had CC licences attached. 
> 
> Thanks, Robin.
> 
> Robin Taylor
> Main Library
> University of Edinburgh
> Tel. 0131 6513808
> 
> 
> --
> The University of Edinburgh is a charitable body, registered 
> in Scotland, with registration number SC005336.
> 
> 
> --
> 
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> 

--
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

--
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Creative Commons problem

[Graham Triggs]

Hmmm... Not sure about that one. I've run into this issue:

http://code.creativecommons.org/issues/issue142

G 

> -Original Message-
> From: Robin Taylor [mailto:robin.tay...@ed.ac.uk] 
> Sent: 23 July 2009 10:31
> To: Tech General
> Subject: [Dspace-tech] Creative Commons problem
> 
> A speculative query - Anyone experiencing any problems with 
> CC licences over the last couple of days ? A problem has been 
> reported on the spanish users mailing list and we find 
> ourselves now unable to anonymously view items that were 
> submitted yesterday and had CC licences attached. 
> 
> Thanks, Robin.
> 
> Robin Taylor
> Main Library
> University of Edinburgh
> Tel. 0131 6513808
> 
> 
> --
> The University of Edinburgh is a charitable body, registered 
> in Scotland, with registration number SC005336.
> 
> 
> --
> 
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> 

--
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] AJP Errors

[Graham Triggs]

Looks like a known (now fixed) bug in Cocoon 2.2
 
http://www.mail-archive.com/d...@cocoon.apache.org/msg57582.html
 
G



From: Sean Carte [mailto:sean.ca...@gmail.com]
Sent: Fri 12/06/2009 07:01
To: Van Ly
Cc: dspace-tech
Subject: Re: [Dspace-tech] AJP Errors



2009/6/12 Van Ly :
> # this looks suspect in the stacktrace
> $Proxy10.service(Unknown Source)

I thought so too. But searching for that on Google doesn't turn up
anything useful.

I changed the proxy from ajp to proxy_http, but the same errors still
occur. I'm starting to wonder if this isn't related to using tomcat5.5
with DSpace 1.5.2.

Is nobody else seeing anything like this?

I wouldn't be too worried about it, but, if the load is high enough,
it does eventually cause the server to stop responding. And this is on
3 production servers.

Sean
--
Sean Carte
esAL Library Systems Manager
+27 72 898 8775
+27 31 373 2490
fax: 0866741254
http://esal.dut.ac.za/

--
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech


--
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables unlimited
royalty-free distribution of the report engine for externally facing 
server and web deployment.
http://p.sf.net/sfu/businessobjects___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [Dspace-devel] TAMU OAI Harvester

[Graham Triggs]

From: Mark H. Wood [mailto:mw...@iupui.edu]
>3) "How can addons interject into the administrative interface?"
>
>   Well, in DSpace 2.0 we'll have Spring underneath it all (as I
>   understand it), and may be able to do dependency-injection tricks
>   to cram new administrative UI into the existing structure.  But how
>   do we accomplish this in 1.x?  We need a way to configure
>   additional choices into the admin. pages.  And if we build that.
>   now, we may not have to work DI magic later.
>
>   Come to think of it, maybe we ought to look at decomposing the
>   existing admin. UI into features which *all* worm their way into a
>   blank framework.

This is a very tightly focused question, which I think avoids the bigger 
issues. I'm not an expert on xmlui, so there may be some issues in the current 
configuration that would need to be addressed. But I don't see why it should be 
that difficult to add new pages, or tag on features to pages with aspects. 
Maybe there is an issue with modular addons contributing to the sitemap?

But there is also the question of how will multiple modules / addons co-exist? 
Isn't it possible for one aspect to alter the DRI in a way that means a 
subsequent aspect can't function correctly - so there would be a specific order 
that those aspects need to be applied, or maybe they simply can't co-exist at 
all?

If we expect to be able to simply drop in dependencies / modules / web 
applications and expect everything to fit together magically, well, this is 
going to be a concern...

>4) "And what do we do about the JSPUI?"
>
>   'tis a puzzlement.  There are folk who know a lot more about JSP
>   than I do, but I can't see any easy way forward to the modular,
>   decentralized future of DSpace for JSPUI.  It's just a page
>   template language with lots of hair.  Any ideas?

Quite right, JSP as technology fundamentally limits the possibility of modular 
contributions of interface elements. In DSpace 2.0, there is a replacement that 
uses Freemarker templates - these look very similar to JSP templates (although 
minus all the embedded code), and can even utilise JSP tag libraries if that's 
necessary.

However, these templates can be retrieved from a variety of locations - from 
the web application directory, the classpath, JARs, filesystem, etc. - and in 
fact, I already have examples of themes (CSS, images and some template 
overrides) being supplied entirely from a JAR module that can be included as a 
dependency to the web application.

So there is a distinct possibility for a new feature to supply it's own 
separate pages from within it's own module. It's not going to give you 
aspect-style weaving of features into pages, but then you can't always rely on 
things happening magically ;)

G
--
Register Now & Save for Velocity, the Web Performance & Operations 
Conference from O'Reilly Media. Velocity features a full day of 
expert-led, hands-on workshops and two days of sessions from industry 
leaders in dedicated Performance & Operations tracks. Use code vel09scf 
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Java Heap dumps during Filter-Media

[Graham Triggs]

Nice work Larry,

I've replaced our PDF text extraction and thumbnail generation with this code.

Thankfully, running on Debian, adding the third party tools was as hard as 
"apt-get install xpdf" ;)

I actually ran into a few more difficulties with the ImageIO libraries - it's a 
pity that you don't get a simple ClassNotFoundException to be able to report 
this more clearly.

But aside from that, my limited tests seem to work quite well.

G 

-Original Message-
From: Larry Stone [mailto:l...@mit.edu] 
Sent: 08 April 2009 22:21
To: Tim Donohue
Cc: DSpace Tech; Jeffrey Trimble
Subject: Re: [Dspace-tech] Java Heap dumps during Filter-Media

The PDFBox library is _always_ going to be a problem because of its 
architecture.  It insists on reading the entire PDF document, images included, 
into memory.  This is not necessary, PDF was explicitly designed to let 
renderers process a page at a time in limited memory.
Perhaps it could gain a lot by adding a "mode" where it ignores images (e.g. 
for text extraction, it is a complete waste of time to even read them into 
memory since it won't be getting any text out of them).

I took a different approach that may be helpful to sites with a lot of PDF 
content that is pathological to PDFBox.  I wrote a couple of filters that 
invoke the XPDF utilities as external OS-level command processes to do the 
dirty work.  They are a bit more complicated to maintain since they rely on 
outside programs that have to be installed, but I've found the xpdf tools to be 
simple to install and maintain.
The XPDF-based text extractor is about three times as fast as PDFBox and the 
only inputs it failed on PDFs were corrupt.  There were also no issues with 
heap space since it runs outside of the JVM.

See patch #2745393 for the code:
https://sourceforge.net/tracker/?func=detail&aid=2745393&group_id=19984&atid=319984

-- Larry


--
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

--
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] download citation in EndNote format

[Graham Triggs]

Morning,

Yes, I do know more about it ;)

It was developed just prior to the 1.5 release, and I had been 
targetting getting it into that release. However, we were entering a 
phase of stabilising the code at that time, and there was one 
significant problem with it...

The detail of metadata captured by DSpace in a standard configuration is 
not sufficient to reliably produce the export data. In order to make it 
work, I had to implement some rather horrible manipulation of the 
metadata. And even then, it isn't always reliable, as the metadata in 
typical repositories isn't consistent enough [to support the manipulation].

So as it stands, this won't be incorporated into DSpace before 2.0. We 
need to develop better working practises for using metadata first.

However, this isn't the only option for people that really want to do 
this. If you install the Firefox extension Zotero, then you can save 
entries to it from the embedded RDF data that comes with 1.5.x 
installations.

Once you have the data in Zotero, it's then possible to manipulate it if 
necessary, and export from there to a number of different formats, 
including RIS, Refer and BibTex.

G

On 05/03/2009 09:19, Andrew Marlow wrote:
> Hello,
>
> I was browsing to see what plans (if any) there are to add a citation
> export facility to Dspace, when I came across this:
> http://blogs.openaccesscentral.com/blogs/orblog/category/Open+Repository+Development.
> It looks like it has already been done for OpenRepository, which is
> based on DSpace. Does anyone know more about this please? If there any
> chance it will be incorporated into DSpace before version 2.0? Assuming
> the answer is "yes" I hope people will consider the Refer format as
> well. This is a subset of the EndNote format and there is a module in
> BibTeX that supports Refer. This would make it easier to support BibTeX
> users until there is a direct export to BibTeX format.
> --
> Regards,
>
> Andrew M.
> http://www.andrewpetermarlow.co.uk
>
>
> 
>
> --
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
>
>
> 
>
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech


--
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Does anyone use SWORD with DSpace yet?

[Graham Triggs]

Hi,

Yes, we offer the SWORD interface on all Open Repository hosted 
repositories - and we have clients that are working with it to develop 
systems to populate the their repositories with content.

Which brings us to an interesting limitation of the reference 
implementation, that I would like to see some thought and discussion on.

Currently, the SWORD interface accepts metadata in an EPrints DC Xml 
document, conforming to the SWAP profile. By only implementing the SWAP 
profile, then there is only a limited amount of metadata that is 
recognised and ingested.

For our deployment (and client's needs), this was insufficient. We 
needed a way of ingesting metadata for any field that is registered in 
the repository.

What I came up with was a way of representing the available metadata 
fields as propertyURIs in the EPDCX document. This would allow extended 
data to be passed to the repository, and still have a graceful fallback 
where the extra metadata isn't understood - or at least allow for the 
package to be easily retargetted.

The URI encoding is a very simple strategy of using 
'http://openrepository.com/metadata///'

ie. http://openrepository.com/metadata/dc/description/abstract

This was a simple way of letting our clients target *any* metadata 
schema/field that has been registered, and can be handled in a single 
small XSL transformation in the ingest:


 http://openrepository.com/metadata/')">
 http://openrepository.com/metadata/')"
 
/>
 
 
 
 




 


 
 

 



 


A quick and very simple solution to a problem that was biting us in the 
short term. Question is, would it be useful to the community to 
crystalise a strategy for passing information beyond that defined by the 
SWAP profile? For example, take the above strategy, but declare a 
dspace.org URI namesspace for the propertyURI encoding?

And there potentially is a wider question of an automated discovery of 
the available metadata fields understood by a repository.

G

On 22/01/2009 09:50, Timo Aalto wrote:
> Hi all,
>
> I'd like to know if the SWORD implementation for DSpace 1.5.x is being
> used anywhere in a production environment. Are there any known bugs or
> other issues with it? I'm asking this because I'm researching the
> possiblility of having our university's forthcoming Current Research
> Information System to implement SWORD for interacting with our DSpace
> repository. This would seem to be a better approach than further
> developing an existing DSpace-specific connector that the CRIS vendor
> now offers.
>
> Thanks in advance for any input,
>
> T
>
> --
> Timo Aalto
> Planning Officer
> University of Helsinki Library Coordination Unit
> timo dot j dot aalto at helsinki dot fi
>
>
> 
>
> --
> This SF.net email is sponsored by:
> SourcForge Community
> SourceForge wants to tell your story.
> http://p.sf.net/sfu/sf-spreadtheword
>
>
> 
>
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech


--
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] SVN 1_5_X branch fails to compile

[Graham Triggs]

The latest revision in SVN looks fine to me:

http://dspace.svn.sourceforge.net/viewvc/dspace/branches/dspace-1_5_x/dspace-api/src/main/java/org/dspace/app/statistics/StatisticsLoader.java?revision=3321&view=markup


Looks like you have a merge with your own local changes.

G

On 08/12/2008 09:00, mikan.d.dspace listmail wrote:
> I just checked out the latest revision from SVN branch 1_5_x. It fails
> to compile.
>
> [INFO] Compilation failure
>
> /dspace-api/src/main/java/org/dspace/app/statistics/StatisticsLoader.java:[355,0]
> illegal start of expression
> /dspace-api/src/main/java/org/dspace/app/statistics/StatisticsLoader.java:[359,0]
> illegal start of expression
> /dspace-api/src/main/java/org/dspace/app/statistics/StatisticsLoader.java:[361,0]
> illegal start of expression
>
> As I looked into these, the files have strange lines, such as :
> <<<  .mine
>
> -Mika
>
> --
> SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
> The future of the web can't happen without you.  Join us at MIX09 to help
> pave the way to the Next Web now. Learn more and register at
> http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech


--
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
The future of the web can't happen without you.  Join us at MIX09 to help
pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Can you upgrade directly from DSpace 1.4.2 to 1.5.1 without first installing 1.5?

[Graham Triggs]

Glenn,

I would advise to do these as separate steps. Overall, you'll take a bit 
more down time, but doing them together increases the risk of having a 
problem / makes it harder to diagnose.

Not that either step is particularly risky, but if you had a problem 
with - say - the handle server, would it be because of the server 
migration, or the upgrade?

G

On 21/11/2008 16:42, Bunton, Glenn A. wrote:
> A somewhat related question I hope...
>
> If I want to move from a server that currently is running 1.4.2 to a new 
> server would the best approach be to install a clean 1.5.1 on the new server 
> and then move the data over? If so, are there instructions for how to migrate 
> content from a 1.4.2 instance to a 1.5.1 instance?
>
>
>
> -Original Message-
> From: Tim Donohue [mailto:[EMAIL PROTECTED]
> Sent: Friday, November 21, 2008 11:25 AM
> To: Thornton, Susan M. (LARC-B702)[NCI INFORMATION SYSTEMS]
> Cc: dspace-tech@lists.sourceforge.net
> Subject: Re: [Dspace-tech] Can you upgrade directly from DSpace 1.4.2 to 
> 1.5.1 without first installing 1.5?
>
> Sue,
>
> You should be able to upgrade from 1.4.2 to 1.5.1, assuming you make
> sure to perform all of the 1.5.x database upgrades.   So, even though
> you are upgrading to 1.5.1, make sure to *first* perform the upgrade
> steps laid out for 1.4.2->1.5.0.
>
> For DSpace, we make it a point that minor upgrades (1.5.0->1.5.1) only
> involve code changes (i.e. the database structure will never change
> during these upgrades).   Major upgrades (e.g. 1.4.2->1.5.0) will often
> involve changes to the database structure.
>
> Hope that helps,
>
> - Tim
>
> Thornton, Susan M. (LARC-B702)[NCI INFORMATION SYSTEMS] wrote:
>> Hi,
>>
>>   We are working on the upgrade from DSpace 1.4.2 to 1.5.x and I'd
>> like to know if we can upgrade from 1.4.2 directly to 1.5.1, or must we
>> install 1.5 first?
>>
>> Thanks,
>>
>> Sue
>>
>>
>>
>>
>>
>> **Sue Walker-Thornton**
>>
>> **ConITS Contract***
>> ***NASA Langley Research Center***
>> ***Integrated Library Systems Application&  Database Administrator**
>>
>> **130 Research Drive**
>>
>> **Hampton, VA  23666**
>>
>> **Office: (757) 224-4074***
>> ***Fax:(757) 224-4001***
>> ***Pager: (757) 988-2547 **
>> **Email: **/*/ /*//*/[EMAIL PROTECTED]/*/
>> 
>>
>>
>>
>>
>> 
>>
>> -
>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>> Build the coolest Linux based applications with Moblin SDK&  win great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>
>>
>> 
>>
>> ___
>> DSpace-tech mailing list
>> DSpace-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Importing problem -- items not going into collection

[Graham Triggs]

Hi,

The browse configuration in your dspace.cfg does not match the tables 
that have been created in the database.

You need to run index-init first - then subsequent index-updates will be 
successful. If you change the browse configuration again, you'll have to 
run index-init.

G

Thomas A McGee wrote:
> This is what I get when I run the index update --->
>
> Creating browse index
> Exception in thread "main" org.dspace.browse.BrowseException: 
> java.sql.SQLException: ORA-00942: table or view does not exist
>
>  at 
> org.dspace.browse.BrowseCreateDAOOracle.deleteByItemID(BrowseCreateDAOOracle.java:402)
>  at org.dspace.browse.IndexBrowse.removeIndex(IndexBrowse.java:302)
>  at org.dspace.browse.IndexBrowse.indexItem(IndexBrowse.java:456)
>  at org.dspace.browse.IndexBrowse.createIndex(IndexBrowse.java:1106)
>  at org.dspace.browse.IndexBrowse.main(IndexBrowse.java:661)
> Caused by: java.sql.SQLException: ORA-00942: table or view does not exist
>
>  at 
> oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
>  at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:331)
>  at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:288)
>  at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:745)
>  at 
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:219)
>  at 
> oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:970)
>  at 
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1190)
>  at 
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3370)
>  at 
> oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3454)
>  at 
> org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
>  at 
> org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
>  at 
> org.dspace.storage.rdbms.DatabaseManager.updateQuery(DatabaseManager.java:354)
>  at 
> org.dspace.browse.BrowseCreateDAOOracle.deleteByItemID(BrowseCreateDAOOracle.java:397)
>  ... 4 more
> Creating search index
>
> < the browse build is crashing, but the search build works.
>
> 
> Tom McGee
> Seton Hall University
> 
> 
>
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK&  win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech


 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Install many instance of Dspace in 1 machine

[Graham Triggs]

Cachiusa,

PermGen is a seperate area of memory from the heap, set aside by the JVM 
to store 'permanent' information like class information and string literals.

(Such information is always resident for the duration of the ClassLoader 
that created/loaded it - in a container environment like Tomact, those 
ClassLoaders may not actually live for the entire duration of the 
application server [you can unload a web application], in which case 
that 'permanent' information should be garbage collected).

The default size for the permanent generation is only 64M. You can 
increase this by using the JVM parameter (add to JAVA_OPTS):

-XX:MaxPermSize=256m

Note that it is not sufficient to just increase the space so that the 
web applications load initially. Due to the on demand loading of 
classes, etc. permanent generation space will continue to be used up as 
you exercise more of the application.

(You may have been able to open 12 instances in your current 
configuration - I can pretty much guarantee that you won't be able to 
fully utilise all of them)

IIRC, you'll need to set aside at least 20MB for each instance with the 
JSP interface, and 50MB for each with the XML/Manakin interface. This is 
assuming that you don't remove the commons-dbcp / commons-pool / 
postgres jars from each instance, and only have them at the common 
layer. Doing that would save in the region of 8MB per instance.

If you try to split across multiple Tomcat's then you will have the 
added overhead of each JVM and Tomcat information. If you split across 
multiple virtual machines, then you have the overhead of each VM, each 
OS, each JVM and each Tomcat!

Increasing the PermGen to the point where all your instances can run is 
your best chance of getting all 24 instances to run on a single physical 
machine. Although ideally you would want it to be a 64-bit machine, a 
64-bit JVM, and at least 8Gb installed in the machine.

G

cachiusa wrote:
> Dear all,
> I am looking for solution for 24 Dspace for 24 communities run on 1 server.
> Which solution I should choose:
> 
> 1) 24 instances of Tomcat, one for each Dspace instance, or
> 2) One Tomcat for 24 Dspace instances.
> 
> I choose second solution by following the link
> http://www.nabble.com/Changing-DB-Name-%C2%BFhow--td19053520.html 
> and already finished installation. But my server is broken if I  try to
> access 24 DSpace at the same time (it is ok to open 12 Dspace).
> 
> Some sites show error:
> java.lang.OutOfMemoryError: PermGen space
> 
> The others return errors:
>  java.io.FileNotFoundException:
> /usr/local/gmseenet/partners/gmseenet/dspace/webapps/jspui/WEB-INF/web.xml
> (Too many open files)
>  
> or:
> org.apache.jasper.JasperException: Unable to compile class for JSP
> 
> I already set heap memory for Tomcat to 2048 by
> JAVA_OPTS="-Xmx2048M -Xms64M -Dfile.encoding=UTF-8", but I still face the
> same problem.
> 
> Quick checking on internet tells me the problem is "garbage collection" of
> Java class:
> http://blogs.sun.com/fkieviet/entry/classloader_leaks_the_dreaded_java.
> 
> I have to run 24 DSpace in 1 machine, so what is solution for me?
> Please help me. Thanks in advance.
> Best,
> Cachiusa

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Problem Searching

[Graham Triggs]

Hi,

Are there any errors reported in your dspace.log?

The DSIndexer class has some static initialisation code (that reads from 
dspace.cfg). If you have any errors in your configuration, then the 
class won't initialise, giving you this exception.

G

Zaya Kh wrote:
> Hi all expert, I installed Dspace 1.5.1 Jspui, I'm searching by
> unicode and english character, but I hava a error page show me,
> 
> This error messages:
> 
> type Exception report
> 
> message
> 
> description The server encountered an internal error () that prevented
> it from fulfilling this request.
> 
> exception
> 
> java.lang.NoClassDefFoundError: Could not initialize class
> org.dspace.search.DSIndexer
>   org.dspace.search.DSQuery.doQuery(DSQuery.java:150)
>   
> org.dspace.app.webui.servlet.SimpleSearchServlet.doDSGet(SimpleSearchServlet.java:242)
>   
> org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:151)
>   org.dspace.app.webui.servlet.DSpaceServlet.doGet(DSpaceServlet.java:99)
>   javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>   javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
> 
> note The full stack trace of the root cause is available in the Apache
> Tomcat/6.0.16 logs.
> 
> What can I do? Help me.
> Best Regards, Zoloo
> 
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] filter-media problem - question on size limit

[Graham Triggs]

If anyone has example PDFs that cause the text extraction to fail 
(smaller PDFs preferably!) that they are able to share, please send them 
- or a link to retrieve them - to me.

Thanks,
G

Mark H. Wood wrote:
> I found this:
>
>http://java-source.net/open-source/pdf-libraries
>
> PJX and PDF Jester look, at first glance, as though they might be
> worth considering.
>
> OTOH it looks like PDFBox might be getting more attention in its new
> home, and if so, then it makes sense to stick with it and help to
> improve it.
>
>
>
> 
>
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK&  win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>
>
> 
>
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] filter-media problem - question on size limit

[Graham Triggs]

Tim Donohue wrote:
> (2) If you look closely at the PDFBox site, you'll notice that software
> has *not* had an updated release since Oct 2006.

And if you keep looking at that site, you aren't likely to see much 
activity in the future.

You should try looking here:

http://incubator.apache.org/pdfbox/

> Personally, because of the lack of activity around the PDFBox software,
> I feel DSpace should start to investigate *other means* to filter PDFs.
>If anyone out there is aware of software that could *replace* PDFBox,
> I think that would be worth closer investigation.  I'd love to help
> DSpace get around these filtering issues...we just need to come up with
> some potential solutions.

There are a few options, but all require licensing - although one has a 
'dubious' dual licensing scheme (not that I'm saying there license is 
dubious, but it's implications for us are).

Apache incubation should be taken as a positive step though, and the 
best route of progress is to get hands on in attempting to fix the bugs 
that we do find.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Sub-community policies

[Graham Triggs]

Richard M. Davis wrote:
> We have a Sub-Community (called HOUP) and we want access to the 
> bit-streams restricted to one Group of users (apart from Administrator). 
> (It's not important whether or not the abstract/metadata view gets 
> restricted too.)
> 
> So I've done what you'd expect: removed the Anonymous "READ" action from 
> the Sub-Community; created a group called HOUP-Read; added that group to 
> the Sub-Community policy with "READ" action.

Assuming that you want this behaviour for any new items added to 
collections within this community, then you will need to:

1) Edit the collection(s)
2) Edit the collection authorizations
3) Remove the DEFAULT_BITSTREAM_READ / Anonymous policy
4) Add a DEFAULT_BITSTREAM_READ / HOUP-Read policy

(You could go through the Admin / Authorization / Manage a collection's 
policies interface to the same effect).

This will give any new items / bitstreams the correct policies, but 
won't affect any of the existing content.

> But when I add ordinary e-people to this HOUP-Read group, they still 
> can't access abstracts or bitstreams in the Sub-Community when logged in.

For existing bitstreams, the quickest resolution would be through Admin 
/ Authorization / Advanced item wildcard policy admin tool.

 From there, choose the collection(s) that contain the items, change 
content type to bitstream, choose the 'Anonymous' group and the action 
READ, then 'clear policies'

Then, once more choose the collection(s), content type 'bitstream', 
choose the 'HOUP-Read' group, and action 'READ', then 'Add policy'.

G

 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] DSPACE browse error

[Graham Triggs]

No, you've copied over an old jsp/layout/navbar-default.jsp.

You need to look at how it's done in 1.5, as it dynamically outputs the 
links based on the indexes that you have configured (with the correct 
urls ;)

G

Charl Roberts wrote:
> Hi,
> 
> Thanks for the feedback. How do I update the URLs? Did a miss a step on the 
> re-indexing during the upgrade, as the JSPs are still following an older link 
> path?
> 
> Thanks again,
> 
> Charl Roberts
> Manager - Library Systems &  Digital Initiatives
> University of the Witwatersrand
> +27 11 717 1950
> 
> 
> -Original Message-
> From: Graham Triggs [mailto:[EMAIL PROTECTED] 
> Sent: 17 October 2008 16:12 PM
> To: Charl Roberts
> Cc: dspace-tech@lists.sourceforge.net
> Subject: Re: [Dspace-tech] DSPACE browse error
> 
> If you are getting a document not found error, then your URLs are wrong.
> 
> Are you still using links to /[dspace]/browse-title, 
> /[dspace]/browse-author, etc?
> 
> The new urls are /[dspace]/browse?type=title, 
> 
> Charl Roberts wrote:
>> Hi,
>>
>>  
>>
>> We’ve recently upgraded to 1.5. All seems to work fine except for the 
>> browse functions.
>>
>> Communities and collections are fine and can be browsed, but title, 
>> author, date and subject cannot be browsed. I simply get an document not 
>> found error.
>>
>> Is this a case of a missing JSP, path problem or an indexing issue?
>>
>>  
>>
>> Any help or suggestions would be most welcome.
>>
>>  
>>
>> Regards,
>>
>>  
>>
>> Charl Roberts
>>
>> Manager - Library Systems &  Digital Initiatives
>>
>> University of the Witwatersrand
>>
>> +27 11 717 1950
>>
>>  
>>
>> This communication is intended for the addressee only. It is 
>> confidential. If you have received this communication in error, please 
>> notify us immediately and destroy the original message. You may not copy 
>> or disseminate this communication without the permission of the 
>> University. Only authorized signatories are competent to enter into 
>> agreements on behalf of the University and recipients are thus advised 
>> that the content of this message may not be legally binding on the 
>> University and may contain the personal views and opinions of the 
>> author, which are not necessarily the views and opinions of The 
>> University of the Witwatersrand, Johannesburg. All agreements between 
>> the University and outsiders are subject to South African Law unless the 
>> University agrees in writing to the contrary.
>>
>>
>> 
>>
>> -
>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>
>>
>> 
>>
>> ___
>> DSpace-tech mailing list
>> DSpace-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> 
> 
>  
>  
> This e-mail is confidential and should not be used by anyone who is not the 
> original intended recipient. BioMed Central Limited does not accept liability 
> for any statements made which are clearly the sender's own and not expressly 
> made on behalf of BioMed Central Limited. No contracts may be concluded on 
> behalf of BioMed Central Limited by means of e-mail communication. BioMed 
> Central Limited Registered in England and Wales with registered number 
> 3680030 Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 
> 4LB
> 
> This 
> communication is intended for the addressee only. It is confidential. If you 
> have received this communication in error, please notify us immediately and 
> destroy the original message. You may not copy or disseminate this 
> communication without the permission of the University. Only authorized 
> signatories are competent to enter into agreements on behalf of the 
> University and recipients are thus advised that the content of this message 
> may not be legally binding on the University and may contain the personal 
> views and opinions of the author, which are not necessarily the views and 
> opinions of The University of the Witwatersra

Re: [Dspace-tech] Handle Server died

[Graham Triggs]

Hi George,

So, you opted to use encryption in the simple setup?

Looks like the encrypted file that was generated is/has become corrupt. 
But this is all entirely with the CNRI provided code (and the JVM).

I would contact CNRI for support.

G

George Kozak wrote:
> Hi...
>
> Two weeks ago I moved my DSpace instance from one server to
> another.  I set up my new sitebndl.zip and contacted CNRI and got my
> new handle-server running.  I had one day where it seemed to be
> giving me troubles, but I restarted everything and then all was
> well.  This morning, it died.  I have tried to restart it several
> times, but it keeps shutting down.  This is a snippet of the errors that I 
> see:
> "javax.crypto.BadPaddingException: Given final block not properly padded
>   at com.sun.crypto.provider.SunJCE_h.b(DashoA12275)
>   at com.sun.crypto.provider.SunJCE_h.b(DashoA12275)
>   at com.sun.crypto.provider.DESCipher.engineDoFinal(DashoA12275)
>   at javax.crypto.Cipher.doFinal(DashoA12275)
>   at
> net.handle.security.provider.GenericProvider.decrypt_DES_ECB_PKCS5(GenericProvider.java:146)
>   at net.handle.hdllib.Util.decrypt(Util.java:1078)
>   at net.handle.server.HandleServer.(HandleServer.java:306)
>   at
> net.handle.server.AbstractServer.getInstance(AbstractServer.java:72)
>   at net.handle.server.Main.initialize(Main.java:152)
>   at net.handle.server.Main.main(Main.java:75)
> Unable to initialize server signature object: java.lang.Exception:
> Incorrect passphrase
> java.lang.Exception: Incorrect passphrase
>  at net.handle.hdllib.Util.decrypt(Util.java:1083)
>   at net.handle.server.HandleServer.(HandleServer.java:306)
>   at
> net.handle.server.AbstractServer.getInstance(AbstractServer.java:72)
>   at net.handle.server.Main.initialize(Main.java:152)
>   at net.handle.server.Main.main(Main.java:75)
> Shutting down...
>
> Does anyone have any clues as to what may be wrong?  I didn't change
> anything since I started things a couple of weeks ago.
>
>
> ***
> George Kozak
> Coordinator
> Web Development and Management
> Digital Media Group
> 501 Olin Library
> Cornell University
> 607-255-8924
> ***
> [EMAIL PROTECTED]
>
>
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK&  win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech


 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] DSPACE browse error

[Graham Triggs]

If you are getting a document not found error, then your URLs are wrong.

Are you still using links to /[dspace]/browse-title, 
/[dspace]/browse-author, etc?

The new urls are /[dspace]/browse?type=title, 

Charl Roberts wrote:
> Hi,
> 
>  
> 
> We’ve recently upgraded to 1.5. All seems to work fine except for the 
> browse functions.
> 
> Communities and collections are fine and can be browsed, but title, 
> author, date and subject cannot be browsed. I simply get an document not 
> found error.
> 
> Is this a case of a missing JSP, path problem or an indexing issue?
> 
>  
> 
> Any help or suggestions would be most welcome.
> 
>  
> 
> Regards,
> 
>  
> 
> Charl Roberts
> 
> Manager - Library Systems &  Digital Initiatives
> 
> University of the Witwatersrand
> 
> +27 11 717 1950
> 
>  
> 
> This communication is intended for the addressee only. It is 
> confidential. If you have received this communication in error, please 
> notify us immediately and destroy the original message. You may not copy 
> or disseminate this communication without the permission of the 
> University. Only authorized signatories are competent to enter into 
> agreements on behalf of the University and recipients are thus advised 
> that the content of this message may not be legally binding on the 
> University and may contain the personal views and opinions of the 
> author, which are not necessarily the views and opinions of The 
> University of the Witwatersrand, Johannesburg. All agreements between 
> the University and outsiders are subject to South African Law unless the 
> University agrees in writing to the contrary.
> 
> 
> 
> 
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> 
> 
> 
> 
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech


 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB

-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Index-init

[Graham Triggs]

You can run index-init with the website up, but bear in mind:

1) There may be a few exceptions thrown whilst the tables are being 
created (prior to population)

2) The browse lists won't be fully populated until the index-init script 
has finished running (may take some time on a large repository).

3) If you have changed the browse configuration in dspace.cfg, you will 
need to restart the web application to reload the values (otherwise you 
will get errors trying to access the browse tables).

G

Blanco, Jose wrote:
> Can I be running index-init with the website up, or does the website
> need to be down.
>
> I just upgraded to 151 and was getting some out of memory error.  I'm
> thinking I needed to run index-init after the upgrade.
>
> Thanks!
> Jose
>
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK&  win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Problem with filter-media

[Graham Triggs]

Hi Susan,

These are long known issues with PDF text extraction. And they are  
both due to bugs in the underlying libraries that are used, and not  
necessarily an issue with the PDF content or size.


For the heap space issue, a new configuration option was added to  
DSpace 1.5 - if you add to your dspace.cfg:


pdffilter.skiponmemoryexception=true

then it will skip the PDF when an out of memory exception occurs,  
rather than failing the process.


But there isn't anything that we can do to extract data from PDFs  
where the errors are occurring.


Note that if you aren't running DSpace 1.5, you might want to make  
changes to your local PDFFilter class, in line with the diff here:


http://fisheye3.atlassian.com/browse/dspace/branches/dspace-1_5_x/dspace-api/src/main/java/org/dspace/app/mediafilter/PDFFilter.java?r1=2260&r2=2581

G

On 9 Oct 2008, at 21:20, Thornton, Susan M. (LARC-B702)[NCI  
INFORMATION SYSTEMS] wrote:


 We’ve been having a problem with filter-media for as long as I  
can remember, with DSpace 1.3.1 and now with DSpace 1.4.2.  I’ve  
emailed the list and discussed this problem with some of the  
developers before, but we’ve never had a resolution.  I’ve been  
doing some more research on it myself for the past day or so and  
here are some interesting things that I’ve found:


99% of our documents are .pdf files.  filter-media seems to fail  
with two different types of errors:

Java heap space – memory error
Possibly unreadable character(s) error or problem with the actual  
format and/or scanning of the document


filter-media does not actually fail with error type (b.) above, but  
it does fail with error type (a.).  This error has resulted in  
hundreds, maybe thousands of our documents not being filtered and,  
consequently, not being full-text searchable.


I used to think that perhaps the memory error was caused by our  
repository being fairly large (right now we have a total of 101,633  
Items and are in the process of loading thousands more) – that  
perhaps the memory problem resulted *after* filtering lots of  
documents – maybe it had eaten up all the memory in the process.   
Today I figured out that is absolutely not the problem.  What I did  
in an attempt to get all the unfiltered documents filtered, is I  
wrote a sql query that created a filter-media execution line  
(“$BINDIR/dsrun org.dspace.app.mediafilter.MediaFilterManager -n -i  
2121/68481 [EMAIL PROTECTED]) for each individual Item in DSpace that did NOT have  
a $$$.pdf.txt document in the Bitstream table, then I copied all  
these lines into one script and ran it.  So basically what happens  
is that filter-media executes over and over again, with the –i  
option (where you specify a handle you want filtered), once for each  
document that hadn’t been previously filtered.  What I found is that  
the errors were occurring on the filtering of a *single* document  
and were not caused by an “memory accumulation” effect.


In looking at some of the documents that were causing the errors, it  
appears that perhaps it is the larger documents that are getting the  
Java heap space error, although I’m not quite sure of this.  Here is  
one of the errors that occurred:



Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.nio.CharBuffer.wrap(CharBuffer.java:350)
at java.nio.CharBuffer.wrap(CharBuffer.java:373)
at java.lang.StringCoding 
$StringDecoder.decode(StringCoding.java:138)

at java.lang.StringCoding.decode(StringCoding.java:173)
at java.lang.String.(String.java:444)
at java.lang.String.(String.java:516)
at  
org.fontbox.cmap.CMapParser.createStringFromBytes(CMapParser.java:418)

at org.fontbox.cmap.CMapParser.parse(CMapParser.java:152)
at org.pdfbox.pdmodel.font.PDFont.parseCmap(PDFont.java:535)
at org.pdfbox.pdmodel.font.PDFont.encode(PDFont.java:387)
at  
org.pdfbox.util.PDFStreamEngine.showString(PDFStreamEngine.java:325)

at org.pdfbox.util.operator.ShowText.process(ShowText.java:64)
at  
org.pdfbox.util.PDFStreamEngine.processOperator(PDFStreamEngine.java: 
452)
at  
org 
.pdfbox.util.PDFStreamEngine.processSubStream(PDFStreamEngine.java: 
215)
at  
org.pdfbox.util.PDFStreamEngine.processStream(PDFStreamEngine.java: 
174)
at  
org.pdfbox.util.PDFTextStripper.processPage(PDFTextStripper.java:336)
at  
org.pdfbox.util.PDFTextStripper.processPages(PDFTextStripper.java:259)
at  
org.pdfbox.util.PDFTextStripper.writeText(PDFTextStripper.java:216)
at  
org 
.dspace 
.app.mediafilter.PDFFilter.getDestinationStream(PDFFilter.java:142)
at  
org 
.dspace 
.app.mediafilter.MediaFilter.processBitstream(MediaFilter.java:169)
at  
org 
.dspace 
.app 
.mediafilter 
.MediaFilterManager.filterBitstream(MediaFilterManager.java:344)
at  
org 
.dspace 
.app 
.mediafilter.MediaFilterManager.filterItem(MediaFil

Re: [Dspace-tech] plugins and custom classes in 1.5.x

[Graham Triggs]

I'll concur with Mark's comments about the use of Maven modules.

A small note about Java package spaces though - org.dspace.authenticate 
is the correct place for it to be if you are intending to submit it as a 
patch.

If it's just a local customisation, it's better to use a package 
appropriate to your institution - ie. edu.kumc.authenticate.

G

Diggory Mark wrote:
> Yes, this is a safe place to put such a customization if you want it
> to be part of more than one of your webapplications (if it is just one
> webapplication that will be using it,  just place it in that
> webapplications src/main/java directory).  You will do the following:
>
> 1.) Create an "api" directory
> 2.) create a "api/pom.xml" that has a artifactId, groupId and version
> that is the same as your current dspace version.
> 3.) make sure it is also dependent on org.dspace:dspace-api:
>
> 4.) put you code in api/src/main/java
> 5.) add the api directory to the modules/pom.xml as both a module and
> a dependency
>
> when you now build, the api will be included into the modules list for
> modules being built and the dependency will make sure it shows up in
> you WEB-INF/lib of your compiled webapplications.
>
> Finally, if you want your project to show up in the [dspace.dir]/lib,
> you'll want to also add the dependency to the dspace/pom.xml
>
> Don't hesitate to ask any questions if you get hung up on any of this.
>
> Cheers,
> Mark
>
> On Oct 1, 2008, at 2:44 PM, Jason Stirnaman wrote:
>
>> Follow up to my question:
>> Is
>> [dspace-source]/dspace/modules/api/src/main/java/org/dspace/
>> authenticate
>> the correct place to put a custom authentication method?
>>
>> ***
>> Can anyone point to good documentation about where to place plugins
>> and
>> custom class files in 1.5.x?  It's not clear from the 1.5.1 manual.
>> Specifically, in what directory would I place a custom authentication
>> method like the MIT-Special-Groups in 1.4.x?
>> Thanks,
>> Jason
>> --
>>
>> Jason Stirnaman
>> Digital Projects Librarian/School of Medicine Support
>> A.R. Dykes Library, University of Kansas Medical Center
>> [EMAIL PROTECTED]
>> 913-588-7319
>>
>>
>> -
>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>> challenge
>> Build the coolest Linux based applications with Moblin SDK&  win
>> great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in
>> the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> ___
>> DSpace-tech mailing list
>> DSpace-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK&  win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech


 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] FW: Java heap space error when trying to update a Collection

[Graham Triggs]

Elliot Metsger wrote:
> We've noticed issues with heap size as well, specifically when using the
> wildcard policy tool, the media filter manager, and some in-house tools that
> iterate over dspace objects using a single Context.
>
> When dspace objects like Item or Bitstream are instantiated, they place a
> reference to itself in a hash map attached to the context.  So over time
> that map will get large.  We modified the tools to clear the context "cache"
> when they were done operating on the dspace object.

As you say above, single Context. The HashMap is eligible for garbage 
collection once you no longer use that Context. So you will only build 
up a large cache if you are using a SINGLE Context over a long period of 
time. (As a rough guide, for requests to the web interface there will be 
a single Context used for each request, for the duration of that request).

But Context's that are held open for a very long time have the potential 
to run into all sorts of other issues - dropping of idle database 
connections, lock contentions/waits, excessive database resource usage.

And any error in that long run could cause the entire process to be 
rolled back - which may not be the best course of action.

If you are running anything that takes an excessive amount of time, it's 
better to really consider if there are better ways you can be using 
(/releasing) contexts, before jumping to clear the cache.

G

 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Range query

[Graham Triggs]

Yes, in theory (at least for 1.5+ - haven't checked earlier)

For example, a query like

author:[aa TO ab]

Will find all items with authors that start with 'aa' (it will also find 
any authors that exactly match 'ab', but not 'aba', etc.)

G

Jochen Lienhard wrote:
> Hi,
>
> does DSpace support range queries?
>
> like author{Aa TO Am}
>
> Greetings
>
> Jochen
>


 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Question regarding handle.jar

[Graham Triggs]

Mark Diggory wrote:
> I will ask why are you trying to alter the existing handle.jar? The
> current one we publish into the maven repository (which your build
> has downloaded) should be adequate for DSpace 1.5.1 and the current
> Handle service. Are you encountering problems running the handle
> service?

I will note that there is a known problem with some handle server 
versions - which I believe includes the one that 'ships' with DSpace - 
that the handle server will take 100% CPU utilization on a day when the 
logs are configured to rollover.

This is fixed in the latest handle versions. Although we know that we 
can't simply ship the latest version for everyone due to the terms of 
use changes.

I think we need to ensure (if possible) that the latest versions of the 
handle.jar are in the Maven repository, along with the 'standard' 
version, and make it easier - possibly even through a profile - for 
people to simply switch to the newer versions, without having to go 
through the Maven artifact installation routine.

G

 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Dspace and Open JDK

[Graham Triggs]

Hi,

I don't believe that anyone has tried this, however I don't see any 
reason why this shouldn't work.

G

mahesh mahesh wrote:
> Hi,
>
> I am new to this .
> Can i compile dspace with open jdk package ?
>
> Thanks
>
> Mahesh
>
>
> 
>
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK&  win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>
>
> 
>
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech


 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] FW: Java heap space error when trying to update aCollection

[Graham Triggs]

Sue,

I'm not denying that there is an efficiency problem / bug in DSpace to 
be bombing out on a minor Collection change just because of the number 
of Items.

But 256 / 512Mb really is virtually nothing these days. I can easily 
exceed that editing the DSpace source in an IDE. I'll regularly see my 
web browser running higher than that after I've been using it for a few 
hours.

Admittedly, I'm running 24(?) DSpace instances in my Tomcat server - but 
it's on an 8Gb machine, running a 64-bit JVM, currently with a max heap 
of 4Gb.

And these servers - with multi-cores and using commodity RAM - are 
amongst the cheapest you can buy these days.

Thornton, Susan M. (LARC-B702)[NCI INFORMATION SYSTEMS] wrote:
> Very interesting.  I just changed our web server JVM options from:
>
>  -Xms128m –Xmx*256*m
>
> To:
>
>  -Xms128m -Xmx*512*m
>
> and it corrected the heap space error.  I’m worried though, because we
> are getting ready to load thousands more documents into DSpace and I’m
> wondering at what point we will start getting this error again.  I agree
> with François that it appears DSpace is cycling through every record in
> the Collection doing something (I’m not sure what since I was just
> updating the Collection description) and because there are 44,938
> records in this Collection and the Java heap space was filled up.
>
> Sue

 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Getting other fields in browse

[Graham Triggs]

Hi,

Take a look at the dspace.cfg file. There is an entry

webui.itemlist.columns

that you can use to specify which metadata fields you want displayed in 
the list.

Note that you will also need entries in your messages.properties file(s) 
for the column headings - they start 'itemlist.' followed by the schema 
/ element / qualifier of the metadata field - for example:

itemlist.dc.title

Regards,
G


Satya Ranjan Sahu wrote:
> Hi All,
> 
>   By default - Issue Date, Title & Author(s) fields get listed 
> for browsing of title, date, subject or author in Dspace 1.4.2. I wish 
> to replace Author(s) field with another metadata field in the display list?
> 
>  
> 
> I checked the JSP files under browse directory but unable to find the 
> solution. Kindly guide me to solve it.
> 
>  
> 
> With regards
> 
> Satya
> 
> 
> Satya Ranjan Sahu
> National Information Centre for Marine Sciences
> National Institute of Oceanography (CSIR)
> Dona Paula, Goa - 403 004
> Phone: 0832-2450370 (O)
> --
> 
>  
> 
> 
> 
> 
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> 
> 
> 
> 
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Configuring the Browse List (JSPUI)

[Graham Triggs]

Vezina Marie-Helene wrote:
> Thank you Graham,
> 
> Very clear answer.  
> 
> I guess the final goal I was pursuing while playing around with the browse 
> configuration was to be able, for my Browse lists, to have one Date column 
> that would display dc.date.submitted for ETDs and dc.date.issued for non-ETDs 
> (and assuming there would be no issue date for ETDs), so I was hoping to be 
> able to mix the content of various fields for the browse lists... In light of 
> your explanations I don't see how this is possible. The closest I got was to 
> have two columns (one for dc.date.submitted and the other one for 
> dc.date.issued), or to have the submitted date in place of the issue date 
> whenever the datesubmitted browse index or sort option was selected Or 
> did I miss something?


Correct, you can't mix the data from two separate metadata fields into a 
single sort column / order. Can you imagine the complexity of a 
configuration syntax that would allow such selective behaviour?

For the future, I can imagine a way of configuring a class to control 
the extraction of metadata, in a way that would allow you to replace it 
with your own code that provides a more complex, context-sensitive form 
of extraction as you wish.

But we aren't there yet. Right now, you could have two columns, one for 
each metadata field.

Or, you could 'invent' an additional metadata field into which you want 
the value that you want that particular record to be sorted on. It's 
more work in maintaining each item, but it would allow you to use/copy 
the date that you want for each item.

G

 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB

-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] 2 questions about large files

[Graham Triggs]

Larry Stone wrote:
> The LNI *might* be able to do this for you.  It streams directly from
> the incoming PUT body through a Zip decoder into a Bitstream, so theoretically
> there is no artificial length limit.  However, the _implementation_ of any of
> those layers of software might not be able to handle files larger
> than 2Gb -- the HTTP client, the HTTP servlet container (i.e. Tomcat),
> Zip decoder in JVM... I'm not even sure if the Zip format itself accomodates
> such large files.

It's presumably not a coincidence that 2Gb is the largest file size 
(count of bytes) that can be represented in a signed 32-bit integer.

For Zips, as far as I know, the standard format is based around unsigned 
32-bit values - the largest a Zip file (or any member file) is 4Gb.

There is an extended 64-bit format that eliminates those restrictions 
(and the limit of 65,535 member files per zip), but I'm not sure how 
widespread the support for that is.

G

 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Configuring the Browse List (JSPUI)

[Graham Triggs]

Hi,

>2008-09-19 16:21:01,748 WARN  org.dspace.app.webui.jsptag.LayoutTag @ 
> Exception
>javax.servlet.ServletException: javax.servlet.jsp.JspException:
>org.apache.jasper.JasperException: javax.servlet.ServletException:
>org.dspace.browse.BrowseException: Browse Index configuration is not 
> valid: webui.browse.index.1 = dateissued:dc.date.issued:date:full

The above is the old pre-1.5 syntax.

The 1.5+ syntax is:

> 1. webui.browse.index.  =  : metadata : prefix>.[.|.*] : (date | title | text) : (asc | desc)
>
> OR
>
> 2. webui.browse.index.  =  : item :  : (asc 
> | desc)

> If limited to use the "old" syntax for defining browse index, I don't 
> understand how I can define new browse indices of the "item" type (#2 above). 
> I'm trying to create a browse index based on the content of dc.date.submitted 
> for our ETDs.
> I can't see where the content of the index can be defined, for example, 
> where, in the following default definition:
>
> "webui.browse.index.1 = dateissued:item:dateissued"
>
> is the index name related to the content of the dc.date.issued metadata 
> field? Is it hardcoded somewhere?

Your syntax is essentially correct. The index name (the first part after 
the =) can be anything you like - it's used to tie the selection of an 
index in the interface to something less fragile than the index numbers 
(which you'll change if you want to re-order how the links appear) - so, 
when you select the 'browse by date of issue' option, it will pass 
?type=dateissued as a parameter to the servlet, which refers back to the 
index name in the above configuration.

The key part that you are missing is in what an 'item' browse 
represents. If you browse by date issued, or title (in the standard 
configuration) - then what you have is always a list of exactly the same 
length. It's one entry, for each item in your repository.

The only thing that differentiates those two browse lists isn't what 
items are represented, but the order in which they are presented.

That was conflated in DSpace before 1.5, and has been separated in the 
new config. So, what is important to actually making your configuration 
work is the  - the dateissued at the END of your 
configuration line.

This refers to a 'webui.itemlist.sort-option.X' entry, and determines 
what metadata field is used to define the ordering. In the standard 
configuration, this is the second sort-option that is defined.

Note that for 'item' browse (and search) lists, all of the configured 
sort options are available to select in the controls provided with the 
list. So, if you want to have your items browsable by the accession 
date, the key thing is to have a sort option configuration such as:

webui.itemlist.sort-option.3 = dateaccessioned:dc.date.accessioned:date

you should only include a webui.browse.index - eg.

webui.browse.index.5 = dateaccessioned:item:dateaccessioned

if you want a link for it to appear in the browse navigation. Without 
that webui.browse.index, you will still be able to re-order an 
item-based browse list into date order from the controls (providing the 
sort-option configuration has been specified).

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Can the workflow step "I Grant the License" be bypassed in DSpace 1.4.2???

[Graham Triggs]

Yes... and no.

There is no way of changing this behaviour without altering the DSpace 
code (which you have access to).

To be strictly accurate, you would still have to write code to 
accomplish this with 1.5 - the key difference is that you aren't forced 
to alter the code that is shipped as DSpace (it can be written as 
entirely new code), and that it should be easier to migrate that code to 
a later release of DSpace in the future.

This step / code does create a license file/bundle on the item, and I'm 
not sure how strictly that is enforced/assumed in the rest of the 
system. But regardless, you would be strongly advised if making this 
change to retain the code that attaches the license to the item - just 
have it run without displaying the JSP form asking users to agree to it.

G

Thornton, Susan M. (LARC-B702)[NCI INFORMATION SYSTEMS] wrote:
> We would like to bypass the step in the online repository submission 
> process that displays a screen where the User clicks on, “I Grant the 
> License” (or “I Do Not Grant the License”).  Is this possible in DSpace 
> 1.4.2?  I’ve noticed in the online help documentation where it says,
> 
> **/“SUBMIT: License/**
> 
>   
> 
> */top of submit 
> /*
> 
>   
> 
> */top 
> /*
> 
> */DSpace requires agreement to this non-exclusive distribution license 
> before your item can appear on DSpace. Please read the license 
> carefully. If you have any questions, please contact your DSpace 
> Administrator 
> . “/*
> 
> Does this mean that you cannot bypass this step due to the internal 
> workings of DSpace?
> 
> Thanks in advance,
> 
> Sue


 
 
This e-mail is confidential and should not be used by anyone who is not the 
original intended recipient. BioMed Central Limited does not accept liability 
for any statements made which are clearly the sender's own and not expressly 
made on behalf of BioMed Central Limited. No contracts may be concluded on 
behalf of BioMed Central Limited by means of e-mail communication. BioMed 
Central Limited Registered in England and Wales with registered number 3680030 
Registered Office Middlesex House, 34-42 Cleveland Street, London W1T 4LB
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] css bitstreams

[Graham Triggs]

Dorothea Salo wrote:
> Good point. Would I be right in hazarding a guess that this was
> originally designed as a gesture toward relatively simple updating of
> this set of configuration options (without direct database-mucking)?
> And that this is therefore a special case of the general issues with
> how difficult DSpace (and its underlying tech stack) is to configure?

Ahh... but it *is* relatively simple to update the registered bitstream 
formats without direct database-mucking. There is an admin panel to do it!

And there is the rub that makes handling updates harder - even if you 
can distinguish precisely which items have been added / modified since 
the version that a user is upgrading from (and we have to account for 
skipping versions, which complicates that a little), you have to be able 
to cope with the fact that the user may have altered their registry - 
adding conflicting entries, removing others, etc.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Reusing bitstream sequence number

[Graham Triggs]

Richard Rodgers wrote:
> I do worry about opening door #1 [content rejection],
> since taking assets as found seems pretty close to the bedrock
> use-case for digital repositories - at least preservation-minded ones.
>
Well, that is an interesting argument! Now, if we look at assets 'as 
found' then they will [probably] be located in a users file system. That 
file system will already be enforcing a unique constraint on the names 
of files within a directory.

Now, in your example you had a user with two files that had the same 
name but located in different directories. Presumably there is implicit 
knowledge in the particular organisation of the file structure. And we 
are not taking it 'as found' because DSpace is forcing the user to throw 
away that organisation (and therefore any knowledge/information it 
implies) when attaching all those files to a single item.

In terms of the sequence number, we assign a 'genuine unique id' to 
every bitstream that is ingested, and there is no reason why that id 
can't be used in place of the sequence number in the url in the case 
where disambiguation is necessary.

There is nothing wrong with presenting a disambiguation page if a url is 
provided without that unique id, and where the filename can't uniquely 
resolve.

There just isn't any need to use a sequence number in this way, and 
include it as part of the URL. What there is a need for is a way to 
define the order in which the bitstreams are presented for an item - 
which should be the job of a sequence number, but it isn't used for that.

(Note that the above is true for the majority using the 80/20 rule. 
There may be some exceptional cases that doesn't fit into the above 
statements, but then they may not be serviced sufficiently by the 
existing use of sequence numbers either).

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Configurable browse error

[Graham Triggs]

You have to run [dspace-installation]/bin/index-init after you have 
changed/installed the configuration.

G

Anderson, Charles W wrote:
> We've been playing around with the configurable browse, and decided to 
> configure a new browse index based on the metadata element "dc.type".  I 
> added the following line to our dspace.config:
>
> webui.browse.index.5 = type:metadata:dc.type:text
>
> This did, indeed, create a navigation link for "Type"; but clicking on the 
> link produces a server error.  Checking the logs reveals a whole bunch of 
> errors, each with the following line at the top of the stack:
>
> org.dspace.browse.BrowseException: org.postgresql.util.PSQLException: ERROR: 
> relation "bi_5_dis" does not exist
>
> Anybody else experienced this?
>
> Thanks,
>
> Bill
>
>

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] DSpace 1.4.2 Browse performance issues

[Graham Triggs]

Sue,

You are right both about the performance of browse in 1.4.2, and the 
problems that you will have trying to limit the number of rows 
considered by the query.

The real problem with the query is the use of the DISTINCT view (note 
that the 'where sort_author' part outside of the view in your rewritten 
query shouldn't make any difference - the improvement is all coming from 
restricting the number of rows that are considered in the DISTINCT query).

Now, the good news as far as DSpace is concerned is that all of this has 
been completely replaced in 1.5, and retrieving the author list does not 
use similar DISTINCT queries.

I don't know what amount of customisations you have, but it's likely 
that upgrading to 1.5 would be the path of least resistance, and you'll 
gain a number of additional new features.

If you must stick with 1.4.2, then I would suggest that you create a 
'materialized view' in postgres that just contains the distinct rows 
from ItemsByAuthor, and replace the '(SELECT DISTINCT * from 
ItemsByAuthor) distinct_view' with the materialized view.

This page:

http://jonathangardner.net/tech/w/PostgreSQL/Materialized_Views

contains information on how you can create a materialized view / table 
using triggers (probably shouldn't use the entirely general purpose 
triggers presented there - make them specific to handling the 
ItemsByAuthor table, and ensure that you have an index on sort_author on 
the created table).

G

Thornton, Susan M. (LARC-B702)[NCI INFORMATION SYSTEMS] wrote:
>
> We are experiencing very poor performance with our DSpace Browses 
> (DSpace 1.4.2/postgreSQL 8.2/Solaris 10). I’ve spent the past day 
> looking at code and experimenting with queries and explains and I’ve 
> got a good idea of a sql query modification I could make to 
> drastically improve performance – I’m not just quite sure how to 
> accomplish it without disrupting other DSpace code and functionality. 
> Here’s an example:
>
> I’ve been looking at the */itemsbyauthor /*table. Ours contains 94,401 
> rows. Here is the DSpace query that clicking on the 
> browse-items-by-author link generates:
>
> *select distinct author, sort_author*
>
> * from (SELECT DISTINCT * from ItemsByAuthor ) distinct_view*
>
> * order by sort_author LIMIT 21*
>
> * *
>
> and here is the Explain from executing the query against our database:
>
> "Limit (cost=28209.73..28211.30 rows=21 width=64) (actual 
> time=8382.920..8384.061 rows=21 loops=1)"
>
> " -> Unique (cost=28209.73..28917.73 rows=9441 width=64) (actual 
> time=8382.915..8384.045 rows=21 loops=1)"
>
> " -> Sort (cost=28209.73..28445.73 rows=94401 width=64) (actual 
> time=8382.910..8383.992 rows=56 loops=1)"
>
> " Sort Key: sort_author, author"
>
> " -> Subquery Scan distinct_view (cost=14410.61..16534.63 rows=94401 
> width=64) (actual time=385.620..671.460 rows=94401 loops=1)"
>
> " -> Unique (cost=14410.61..15590.62 rows=94401 width=42) (actual 
> time=385.611..603.050 rows=94401 loops=1)"
>
> " -> Sort (cost=14410.61..14646.61 rows=94401 width=42) (actual 
> time=385.605..504.765 rows=94401 loops=1)"
>
> " Sort Key: items_by_author_id, item_id, author, sort_author"
>
> " -> Seq Scan on itemsbyauthor (cost=0.00..3383.01 rows=94401 
> width=42) (actual time=19.232..113.700 rows=94401 loops=1)"
>
> "Total runtime: 8384.581 ms"
>
> Basically, a Sequential Scan is being done against the itemsbyauthor 
> table and the index is not being used. (By the way, the response time 
> was around 25 seconds last night prior to running a vacuum and 
> reindex! – Terrible!!)
>
> Here’s my idea: if I could somehow capture an alphabetic range to use 
> in a where clause, based on what the User clicks on, I could add a 
> “where” clause to the query which would force postgreSQL to use the 
> index, “sort_author_idx”, and the elapsed time that it would take to 
> execute the query would be a fraction of what it’s taking now. For 
> instance, if I knew the User was starting the browse from the letter 
> “B” (I know this is oversimplifying, but you’ll get the idea), I could 
> structure the query as follows:
>
> *select distinct author,sort_author from (SELECT DISTINCT * from 
> ItemsByAuthor where sort_author > 'a' and sort_author < 'c' ) 
> distinct_view *
>
> *where sort_author > 'a' and sort_author < 'c'*
>
> *order by sort_author LIMIT 21*
>
> Look at the explain from this query, below and just look at the 
> difference in performance! From 8.4 seconds to 1.3 seconds!!
>
> "Limit (cost=6544.60..6546.17 rows=21 width=64) (actual 
> time=1253.393..1253.710 rows=21 loops=1)"
>
> " -> Unique (cost=6544.60..6650.94 rows=1418 width=64) (actual 
> time=1253.388..1253.696 rows=21 loops=1)"
>
> " -> Sort (cost=6544.60..6580.04 rows=14179 width=64) (actual 
> time=1253.382..1253.641 rows=56 loops=1)"
>
> " Sort Key: sort_author, author"
>
> " -> Subquery Scan distinct_view (cost=4663.32..4982.35 rows=14179 
> width=64) (actual time=113.859..145.529 rows=11463 loops=1)"
>
> " -> Unique (co

Re: [Dspace-tech] [Fwd: How to configure dspace to use a mail server running on ssl ports]

[Graham Triggs]

The options in dspace.cfg only allow you to connect to a mail server 
running on a non-standard port.

They do not tell DSpace to make a secure connection to that port.

See the link below for more information on using SSL with JavaMail.

http://www.javaworld.com/javatips/jw-javatip115.html

G

Nitin Bhadauria wrote:
>> Please send a full stack trace and the relevant lines from your
>> dspace.cfg.
> 
> Here is the configuration ..
> 
> # SMTP mail server
> mail.server=mail.sarai.net
> 
> # SMTP mail server authentication username and password (if required)
> mail.server.username = [EMAIL PROTECTED]
> mail.server.password = mypassword
> 
> # SMTP mail server alternate port (defaults to 25)
> mail.server.port = 465
> 
> # From address for mail
> mail.from.address = [EMAIL PROTECTED]
> 
> # Currently limited to one recipient!
> feedback.recipient = [EMAIL PROTECTED]
> 
> # General site administration (Webmaster) e-mail
> mail.admin = [EMAIL PROTECTED]
> 
> 
> 
> 
> then this is the error log from dspace.log ..
> 
> 2008-08-02 07:07:50,127 INFO  org.dspace.app.webui.servlet.RegisterServlet
> @
> anonymous:session_id=EB37E08A3F822F94E6F992CD3596F26C:ip_addr=192.168.130.1:sendtoken_register:[EMAIL
>  PROTECTED]
> 2008-08-02 07:12:54,805 INFO  org.dspace.app.webui.servlet.RegisterServlet
> @
> anonymous:session_id=EB37E08A3F822F94E6F992CD3596F26C:ip_addr=192.168.130.1:error_emailing:[EMAIL
>  PROTECTED]
> javax.mail.MessagingException: Could not connect to SMTP host:
> mail.sarai.net, port: 465, response: -1
> at
> com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1270)
> at
> com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:370)
> at javax.mail.Service.connect(Service.java:297)
> at javax.mail.Service.connect(Service.java:156)
> at javax.mail.Service.connect(Service.java:105)
> at javax.mail.Transport.send0(Transport.java:168)
> at javax.mail.Transport.send(Transport.java:98)
> at org.dspace.core.Email.send(Email.java:362)
> at
> org.dspace.eperson.AccountManager.sendEmail(AccountManager.java:296)
> at
> org.dspace.eperson.AccountManager.sendInfo(AccountManager.java:256)
> at
> org.dspace.eperson.AccountManager.sendRegistrationInfo(AccountManager.java:101)
> at
> org.dspace.app.webui.servlet.RegisterServlet.processEnterEmail(RegisterServlet.java:287)
> at
> org.dspace.app.webui.servlet.RegisterServlet.doDSPost(RegisterServlet.java:202)
> at
> org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:147)
> at
> org.dspace.app.webui.servlet.DSpaceServlet.doPost(DSpaceServlet.java:105)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874)
> at
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
> at
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
> at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
> at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
> at java.lang.Thread.run(Thread.java:595)
> 
> 
> 
> 
> 
>> Jim
>>
>> On Fri, Aug 01, 2008 at 10:02:35AM +, Nitin Bhadauria wrote:
>>> please help me ...
>>>
>>>
>>>
>>>  Original Message
>>> 
>>> Subject: [Dspace-tech] How to configure dspace to using a mail server
>>> running on ssl ports
>>> From:"Nitin Bhadauria" <[EMAIL PROTECTED]>
>>> Date:Thu, July 31, 2008 7:15 pm
>>> To:  dspace-tech@lists.sourceforge.net
>>> --
>>>
>>> Dear Frnds
>>>
>>>
>>> my mail server is running on ssl ports so when i try to configure
>>> dspace.cfg with the new port 465 error says.
>>>
>>> javax.mail.MessagingException: Could not connect to SMTP host:
>>> mail.sarai.net, port: 465, resp

Re: [Dspace-tech] 1.5 - sort results bug

[Graham Triggs]

Hi,

In dspace-jspui-webapp/src/main/webapp/search/results.jsp, replace the  
line:



with

" />


If you are using the overlay war [against the war distributed in the  
Maven repository], you can simply copy the entire search/results.jsp  
from the standard war, place it at the appropriate location in your  
overlay war source directory, and make the change there.

G

On 31 Jul 2008, at 21:32, Susan Parham wrote:

> Hi,
>
> I didn't get a response to my message last week, so am hoping someone
> will spot it if I re-post.  Has anyone else experienced this problem?
>
> Susan
>
> -- Forwarded message --
> From: Susan Parham <[EMAIL PROTECTED]>
> Date: Wed, Jul 16, 2008 at 9:57 AM
> Subject: Sort results in 1.5
> To: dspace-tech@lists.sourceforge.net
>
>
> We recently upgraded to 1.5 --
>
> After conducting a quoted phrase search (e.g., "computer science"), I
> am unable to re-sort the results.  Results are listed by Relevance
> (the default). When I try to resort the results list by any other
> value, such as Issue Date or Title, I get the message that the search
> produced no results.
>
> Is this a 1.5 bug which others experience?
>
> Susan
>
>
>
>
>
> --
> Susan Wells Parham
> Head, Digital Library Development
> Georgia Institute of Technology
> Library & Information Center
> 404-894-4522
> [EMAIL PROTECTED]
>
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] [Dspace-devel] Batch Import document

[Graham Triggs]

Bradley McLean wrote:
> Unfortunately, I can't think of a good way for us to collectively edit
> either a .doc or an .odt, so that presents a challenge.

Well, we can collectively edit a document in Google Docs, which can then 
be downloaded as a PDF...

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Another Ant fresh_install error: Bad Packet Type

[Graham Triggs]

Sounds like you might have a wonky ojdbc jar.

What version is it?

Can you try replacing it with the latest version downloaded from Oracle?

G

Rebekah Burke wrote:
> Hi all,
> 
> I am on step 7, the ant fresh_install step, of installing Dspace 1.5 
> from the PDF manual, using Oracle 11g on Debian Linux.
> 
> I am receiving this Bad Packet Type java error when I run ant 
> fresh_install..
> 
> 
> 
> 
> [EMAIL 
> PROTECTED]:/home/rburke/dspace-1.5.0-release/dspace/target/dspace-1.5.0-build.dir$
>  
> ant fr
> esh_install
> Buildfile: build.xml
> 
> init_installation:
> 
> init_configs:
>  [copy] Copying 1 file to 
> /usr/local/apache-tomcat-6.0.16/webapps/dspace/config
> 
> setup_database:
>  [java] 2008-07-21 15:21:06,909 INFO  
> org.dspace.core.ConfigurationManager @ Loading system provided config 
> property (-Ddspace.configuration): config/dspace.cfg
>  [java] 2008-07-21 15:21:06,915 INFO  
> org.dspace.core.ConfigurationManager @ Using default log4j provided log 
> configuration,if uninitended, check your dspace.cfg for (log.init.config)
>  [java] 2008-07-21 15:21:06,915 INFO  
> org.dspace.storage.rdbms.InitializeDatabase @ Initializing Database
>  [java] 2008-07-21 15:21:07,098 FATAL 
> org.dspace.storage.rdbms.InitializeDatabase @ Caught exception:
>  [java] java.sql.SQLException: Io exception: Bad packet type
>  [java] at 
> oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
>  [java] at 
> oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146)
>  [java] at 
> oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:255)
>  [java] at 
> oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:387)
>  [java] at 
> oracle.jdbc.driver.PhysicalConnection.(PhysicalConnection.java:420)
>  [java] at 
> oracle.jdbc.driver.T4CConnection.(T4CConnection.java:165)
>  [java] at 
> oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:35)
>  [java] at 
> oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:801)
>  [java] at 
> java.sql.DriverManager.getConnection(DriverManager.java:582)
>  [java] at 
> java.sql.DriverManager.getConnection(DriverManager.java:185)
>  [java] at 
> org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:48)
>  [java] at 
> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:290)
>  [java] at 
> org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:840)
>  [java] at 
> org.apache.commons.dbcp.PoolingDriver.connect(PoolingDriver.java:175)
>  [java] at 
> java.sql.DriverManager.getConnection(DriverManager.java:582)
>  [java] at 
> java.sql.DriverManager.getConnection(DriverManager.java:207)
>  [java] at 
> org.dspace.storage.rdbms.DatabaseManager.getConnection(DatabaseManager.java:514)
>  [java] at 
> org.dspace.storage.rdbms.DatabaseManager.loadSql(DatabaseManager.java:869)
>  [java] at 
> org.dspace.storage.rdbms.InitializeDatabase.main(InitializeDatabase.java:100)
> 
> BUILD FAILED
> /home/rburke/dspace-1.5.0-release/dspace/target/dspace-1.5.0-build.dir/build.xml:380:
>  
> Java returned: 1
> 
> 
> 
> I can't find anything about this in relation to Dspace online. It seems 
> to be something to do with the OJDBC drivers. Can anybody help?
> 
> Regards,
> Rebekah
> 
> 
> 
> 
> 
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> 
> 
> 
> 
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Filter-media in 15

[Graham Triggs]

Larry Stone wrote:
> It _should_ never get an NPE over a missing configuration key.  Given
> our sparse testing resources it behooves us to make the code as
> resilient as possible.

I'll disagree slightly - it isn't necessarily wrong for code to blow up. 
In cases like this, it's probably better for the code to blow up, than 
to quietly pretend to work, with only a vigilant pair of eyes on the log 
file realising if it isn't.

But it should blow up with something more useful than an NPE - testing 
the returned value for null, or catching the NPE and rethrowing a more 
descriptive exception.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Achieving security by obscurity

[Graham Triggs]

Scott Phillips wrote:
> We wanted to expose dspace's metadata in a way that can be used by  
> other applications. They are nice restfull urls that are actionable  
> and easily predictable.

In most cases, that should be seen as a good thing.

I can see in certain situations that you wouldn't want to expose that 
publicly though. So, in the interests of furthering our collective 
knowledge, I'll pose the question for tackling it another way -

is it possible to restrict what has access to the /metadata urls? (ie. 
by IP address).

If you can restrict part of the url space to only being accessible 
either from an internal Cocoon resolution and/or specific IP addresses 
(ie. 127.0.0.1), then you can prevent leakage of [sensitive] 
information, whilst still allowing the internal processes, your own 
debugging, and possibly even 'trusted partners' access to the data they 
need.

G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Not able to display item

[Graham Triggs]

Hi,

Yes.

Obtain the latest XHTMLHeadCrosswalk class from the 1.5 branch in 
Subversion.

(known bug where it doesn't check if the metadata value contains null, 
and tries to insert it into the JDOM tree anyway)

G

Blanco, Jose wrote:
> I'm getting an internal error when accessing an item.  The error is
> coming from the following line in HandleServlet:
> 
> List l = xHTMLHeadCrosswalk.disseminateList(item);
> 
> 
> I ran index-init and it completed without failures.  Here is a bit more
> of the error:
> 
> Exception:
> java.lang.NullPointerException
>   at
> org.dspace.app.webui.servlet.HandleServlet.displayItem(HandleServlet.jav
> a:397)
>   at
> org.dspace.app.webui.servlet.HandleServlet.doDSGet(HandleServlet.java:21
> 1)
>   at
> org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.
> java:174)
>   at
> org.dspace.app.webui.servlet.DSpaceServlet.doGet(DSpaceServlet.java:102)
> 
> 
> Any ideas?
> 
> Thanks!
> 
> Jose
> 
> -
> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
> Studies have shown that voting for your favorite open source project,
> along with a healthy diet, reduces your potential for chronic lameness
> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Borked sequence in author browse

[Graham Triggs]

Steve Thomas wrote:
> Hi.
> 
> We're running 1.4.2.
> 
> Our Author browse is producing an odd sequence of authors, e.g.
> 
> Brown, Alfred Leonard
> Brown, Alice
> Brown, A. M.
> Brown, A. R.
> Brown-Augsburger, P.
> 
> -- it looks very much like the sequence is ignoring punctuation (good) 
> AND ignoring spaces (b. BAD).
> 
> Has anyone else experienced this?
> 
> Is there a fix?

Hi,

I'm not particularly familiar with this situation. But if you want to 
alter the way the sorting occurs, the best approach would be to apply 
this patch:

http://sourceforge.net/tracker/index.php?func=detail&aid=1672065&group_id=19984&atid=319984

it is [almost] the same code that is part of the 1.5 release, and allows 
you to define your own normalisation characteristics that will be used 
for the sorting.

Regards,
G
This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Browse, UTF-8 and sorting in 1.5

[Graham Triggs]

Urban,

It is done this way, because simply writing UTF-8 to the database tables 
results in very random sorting (for diacritics). And by random, I mean 
things like Z with a particular diacritic appearing in the middle of the 
'A' entries - that's just broken in any language / locale.

By decomposing to the base and combining character, we guarantee that 
the ordering is at least predictable, and sensible for those of us of an 
English persuasion.

The normalisation that is applied to the strings that will be used for 
sorting is configurable though - you simply need to specify the plugins 
used for a particular order format, by defining the OrderFormatDelegate 
in dspace.cfg - ie:

plugin.named.org.dspace.sort.OrderFormatDelegate= \
org.dspace.sort.OrderFormatTitleMarc21=title

In this case, wherever I specify that my sort column is of type 'title', 
then it will use the OrderFormatTitleMarc21 class (an implementation of 
OrderFormatDelegate) to normalise the string before writing it to the 
database.

You can name a type anything you like - ie:

plugin.named.org.dspace.sort.OrderFormatDelegate= \
org.dspace.sort.OrderFormatTitleMarc21=marc21title

webui.itemlist.sort-option.1 = title:dc.title:marc21title

would work correctly - you just need to tie the 'name' of the plugin 
(after the equals), with the last component of the sort-option 
configuration.

In your case, you should look at the LocaleOrderingFilter (in 
org.dspace.browse), that will normalise a String so that the resulting 
value can be sorted in a way that is correct for a specified Locale.

You will need to create an OrderFormatDelegate implementation, but that 
is as simple as:

public class OrderFormatLocale extends AbstractTextFilterOFD
{
   {
 filters = new TextFilter[] {   
  new LowerCaseAndTrim(),
   new LocaleOrderingFilter() };
   }
}

Note that the LocaleOrderingFilter produces a String that is NOT 
readable, and is inappropriate to be passed to other filters like 
LowerCaseAndTrim, hence the particular ordering presented above.

Also note, that this will always produce sort strings based on the 
Locale that you define in 'webui.browse.sort.locale' - if your interface 
supports being displayed in multiple locales, the ordering can't be 
changed based on the locale being displayed. This is a limitation of 
making the system performant - if we used a Java collator to do all the 
sorting during display, the browse / search function would not scale to 
large numbers of items.

G

Urban Andersson wrote:
> Hello all,
> 
> One obstacle that I see in 1.5 is that terms containing diacritics are 
> sorted by base character in the browse indexes - i.e. the letter "Ö" 
> (oumlaut) is treated like "O" etc., which result in records not being 
> sorted according to (national) standards (or the postgres locale).
> 
> I notice that these characters are decomposed (diacritic stored after 
> the base character) before writing the sorting terms to the database, 
> and this result in the above.
> I know that this has been discussed  in the past, at the DSUG meeting in 
> Rome for instance, but I cannot remember the exact reason why it was 
> done this way (but I remember that there was an explanation and I would 
> be happy if someone could please repeat it).
> 
> Modifying the utf-8 characters manually in bi_2_dis.sort_value et al 
> will result in records being sorted correctly, but then the links from 
> the indexes don't seem to work. 
> 
> Has anyone else looked into this and is there perhaps a known, quick 
> (and possibly dirty) solution to get the most rudimentary sorting to 
> work properly?
> Our 1.5 runs on Debian linux, Postgres and Tomcat 6.
> 
> 
> / Urban Andersson
> 

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] No browse index for date

[Graham Triggs]

Gary,

The browse indexes are generated in tables within the database.

All the 'metadata' indexes (see dspace.cfg) generate bi_1, bi_2, etc.
tables (again, see the comments in dspace.cfg)

For the non-metadata indexes, they are just references to the bi_item 
table, that specify which sort column to use - the bi_item table is 
generated with a number of sort columns, based on the 
webui.itemlist.sort-option.? configuration (one column for each entry).

G

Gary Browne wrote:
> Never mind, it seems to have started working. Perhaps there was a 
> problem with the cron job which has resolved itself. Still, I would be 
> interested to know if I can browse the indexes.
>  
> Regards
> Gary
>  
>  
> //
> Gary Browne
> Development Programmer
> Library IT Services
> University of Sydney
> ph: 9351-5946
>  
> 
> 
> *From:* [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] *On Behalf Of
> *Gary Browne
> *Sent:* Tuesday, 24 June 2008 9:16 AM
> *To:* dspace-tech@lists.sourceforge.net
> *Subject:* [Dspace-tech] No browse index for date
> 
> Hi all
>  
> For some reason in 1.5, the indexer is not creating a date index
> (browse) - at least, that's what the exception says (see below). Is
> there any way to actually check whether an index is created or not
> eg: some way of using Luke or such?
>  
> Or is there some configuration that I'm missing for the browse by
> date index to be created?
>  
> Thanks
> 
> Gary
>  
>  
> 
> -
> 
> An internal server error occurred on
> _http://ses.library.usyd.edu.au_ :
> 
> Date: 6/21/08 6:19 PM
> 
> Session ID: 120B8DBFAA3A4EA1E223CFD0DB9D7A23
> 
> -- URL Was: _http://ses.library.usyd.edu.au/browse?type=date_
> 
> -- Method: GET
> 
> -- Parameters were:
> 
> -- type: "date"
> 
>  
> 
> Exception:
> 
> javax.servlet.ServletException: There is no browse index for the request
> 
> at
> 
> org.dspace.app.webui.servlet.BrowserServlet.doDSGet(BrowserServlet.java:92)
> 
> 
> -
> 
>  
> //
> Gary Browne
> Development Programmer
> Library IT Services
> University of Sydney
> ph: 9351-5946
>  
> 
> 
> 
> 
> -
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> 
> 
> 
> 
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech


This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] index init

[Graham Triggs]

Jeffrey,

Page 52 is correct. Page 64 is a copy of the old comments from dspace.cfg.

The comments and example in dspace.cfg are also correct, and should have 
guided you away from the erroneous information in p64 of the manual ;)

Your configuration should be:

webui.browse.index.1 = dateissued:item:dateissued:date
webui.browse.index.2 = author:metadata:dc.contributor.*:text
webui.browse.index.3 = title:item:title
webui.browse.index.4 = subject:metadata:dc.subject.*:text

G

Jeffrey Trimble wrote:
> Okay, I've followed the conflicting examples for the browse index  (see 
> p. 52 vs. p. 64 of the May 2008 manual)
> and here's the error I've run into the browse configuration:
> 
> Creating browse index
> Exception in thread "main" org.dspace.browse.BrowseException: Browse 
> Index confi
> guration is not valid: webui.browse.index.1 = 
> dateissued:dc.date.issued:date:ful
> l
> at org.dspace.browse.BrowseIndex.(BrowseIndex.java:220)
> at 
> org.dspace.browse.BrowseIndex.getBrowseIndices(BrowseIndex.java:676)
> at org.dspace.browse.IndexBrowse.(IndexBrowse.java:150)
> at org.dspace.browse.IndexBrowse.main(IndexBrowse.java:618)
> Creating search index
> 
> 
> Any suggestions??
> 
> Here's the configuration as I have it:
> 
> webui.browse.index.1 = dateissued:dc.date.issued:date:full
> webui.browse.index.2 = author:dc.contributor.*:text:single
> webui.browse.index.3 = title:dc.title:title:full
> webui.browse.index.4 = subject:dc.subject.*:text:single
> 
> 
> Jeffrey Trimble
> Systems Librarian
> Maag Library
> Youngstown State University
> 330-941-2483 (Office)
> [EMAIL PROTECTED] 
> http://www.maag.ysu.edu
> http://digital.maag.ysu.edu
> 
> 
> 
> 
> 
> 
> -
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> 
> 
> 
> 
> ___
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

This email has been scanned by Postini.
For more information please visit http://www.postini.com


-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

Re: [Dspace-tech] Erratic Browse functionality

[Graham Triggs]

Flemion Shafeeq wrote:

>> Checked code to find a table named *Community2Item, *which is absent 
>> in the dspace database tables,(there is a table named 
>> communities2item) but comfortably used in the code.
>> The interesting part is the same script when run on the development 
>> machine runs fine without any hiccups.Are we running on any issue with 
>> our machine setup for the production or??

Flemion,

Community2Item is a view that is created over the Community2Collection 
and Collection2Item tables. You need to look at the VIEWs in the 
database, not the tables.

That said, the code in updateCommunityMappings should be referring to 
the Communities2Item table, NOT the Community2Item view.

G

-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech