Re: [dspace-tech] Integrate dSpace with another application (play,akka)
Hi Cosmin, I built a Play! java app several years ago that uses the DSpace REST API with some basic functionality for a proof of concept, that allows you to browse communities, collections, items, and bitstreams. https://github.com/peterdietz/dspace-rest-play http://dspace-rest-client-play.herokuapp.com/communities The workflow is not available in the REST API, so you would have to do something creative to be able to work with it. Such as either building/implementing workflow endpoints in DSpace REST, or building JVM integration in play to use DSpace classes. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Wed, May 31, 2017 at 10:25 AM, Cosmin Oarga wrote: > Hi there, > > Is there a way to access the dSpace workflow from an external source? > I want to write an application (for my dissertation thesis) in java with > play and akka and I want to know if there is a possibility to access the > workflow? For example a user added a new content. My applications checks > the content and then if everything is ok the content it gets accepted? > > Any opinions are welcomed :) > > Thanks, > Cosmin > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] AWS S3 Connection Pool Error
The TransferManager code just made the implementation simpler. Otherwise you have to write code that determines how large the bitstream is, and if its small, transfer the file to somewhere in memory, and then stream that, or if its larger, you could store it to temp file, and then stream that. You also have to ensure that you delete the temp file when complete. If its very large, larger than 5GB, you can't transfer that in a single GET. The TransferManager code handles the multipart uploads and download. Also, it can give a performance benefit, where GET and PUT can be done multi-thread multi-part, instead of a single long running GET or PUT to S3. The transfermanager also seemed to do a good job of closing resources when the transfer was complete. http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/TransferManager.html High level utility for managing transfers to Amazon S3. TransferManager provides a simple API for uploading content to Amazon S3, and makes extensive use of Amazon S3 multipart uploads to achieve enhanced throughput, performance and reliability. When possible, TransferManager attempts to use multiple threads to upload multiple parts of a single upload at once. When dealing with large content sizes and high bandwidth, this can have a significant increase on throughput. TransferManager is responsible for managing resources such as connections and threads; share a single instance of TransferManager whenever possible. TransferManager, like all the client classes in the AWS SDK for Java, is thread safe. Call TransferManager.shutdownNow() to release the resources once the transfer is complete. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Fri, May 26, 2017 at 9:18 AM, wrote: > Thanks Peter, > > I have implemented most of your changes on the DSpace 6 code and now I'm > not running out of connections. I haven't updated to use the Transfer > Manager yet and I was wondering what advantage you found using the Transfer > Manager over the original implementation. > > Thanks, > Justin > > On Thursday, May 18, 2017 at 11:02:22 AM UTC-4, Peter Dietz wrote: >> >> Hi Justin, >> >> With production use, and encountering similar issues to you, I have made >> fixes to our code to protect against http connection leaking with AWS S3 >> assetstore, and also to handle larger uploads/downloads. Here is some >> commit history to the (5.x) S3BitStore: https://github.com >> /LongsightGroup/DSpace/commits/longsight-5.6/dspace-api/src/ >> main/java/org/dspace/storage/bitstore/impl/S3BitStore.java >> >> >> >> Peter Dietz >> Longsight >> www.longsight.com >> pe...@longsight.com >> p: 740-599-5005 x809 <(740)%20599-5005> >> >> On Thu, May 18, 2017 at 10:57 AM, Claudia Jürgen < >> claudia...@tu-dortmund.de> wrote: >> >>> Hello jcdalton, >>> >>> which UI are you using and are you using google analytics? >>> >>> For the time being you may set the config parameter db.maxidle to >>> something else than -1 >>> see >>> https://jira.duraspace.org/browse/DS-3564?jql=text%20~%20%22maxidle%22 >>> >>> Hope this helps >>> >>> Claudia Jürgen >>> >>> >>> Am 18.05.2017 um 16:25 schrieb jcda...@email.wm.edu: >>> >>>> I'm using S3 as a bitstore on DSpace 6 and I'm coming up with a lot of >>>> errors like the following: >>>> >>>> com.amazonaws.http.AmazonHttpClient @ Unable to execute HTTP request: >>>> Timeout waiting for connection from pool >>>> org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting >>>> for >>>> connection from pool >>>> >>>> It looks to me like either DSpace isn't closing the connections as it >>>> should. Anyone else had this issue? >>>> >>>> Thanks >>>> >>>> >>> -- >>> Claudia Juergen >>> Eldorado >>> >>> Technische Universität Dortmund >>> Universitätsbibliothek >>> Vogelpothsweg 76 >>> 44227 Dortmund >>> >>> Tel.: +49 231-755 40 43 >>> Fax: +49 231-755 40 32 >>> claudia...@tu-dortmund.de >>> www.ub.tu-dortmund.de >>> >>> Wichtiger Hinweis: Die Information in dieser E-Mail ist vertraulich. Sie >>> ist ausschließlich für den Adressaten bestimmt. Sollten Sie nicht der für >>> diese E-Mail bestimmte Adressat sein, unterrichten Sie bitte den Absender >>> und vernichten Sie diese Mail. Vielen Dank. >>> Unbescha
Re: [dspace-tech] AWS S3 Connection Pool Error
Hi Justin, With production use, and encountering similar issues to you, I have made fixes to our code to protect against http connection leaking with AWS S3 assetstore, and also to handle larger uploads/downloads. Here is some commit history to the (5.x) S3BitStore: https://github.com/LongsightGroup/DSpace/commits/longsight-5.6/dspace-api/src/main/java/org/dspace/storage/bitstore/impl/S3BitStore.java Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Thu, May 18, 2017 at 10:57 AM, Claudia Jürgen < claudia.juer...@tu-dortmund.de> wrote: > Hello jcdalton, > > which UI are you using and are you using google analytics? > > For the time being you may set the config parameter db.maxidle to > something else than -1 > see > https://jira.duraspace.org/browse/DS-3564?jql=text%20~%20%22maxidle%22 > > Hope this helps > > Claudia Jürgen > > > Am 18.05.2017 um 16:25 schrieb jcdal...@email.wm.edu: > >> I'm using S3 as a bitstore on DSpace 6 and I'm coming up with a lot of >> errors like the following: >> >> com.amazonaws.http.AmazonHttpClient @ Unable to execute HTTP request: >> Timeout waiting for connection from pool >> org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for >> connection from pool >> >> It looks to me like either DSpace isn't closing the connections as it >> should. Anyone else had this issue? >> >> Thanks >> >> > -- > Claudia Juergen > Eldorado > > Technische Universität Dortmund > Universitätsbibliothek > Vogelpothsweg 76 > 44227 Dortmund > > Tel.: +49 231-755 40 43 > Fax: +49 231-755 40 32 > claudia.juer...@tu-dortmund.de > www.ub.tu-dortmund.de > > Wichtiger Hinweis: Die Information in dieser E-Mail ist vertraulich. Sie > ist ausschließlich für den Adressaten bestimmt. Sollten Sie nicht der für > diese E-Mail bestimmte Adressat sein, unterrichten Sie bitte den Absender > und vernichten Sie diese Mail. Vielen Dank. > Unbeschadet der Korrespondenz per E-Mail, sind unsere Erklärungen > ausschließlich final rechtsverbindlich, wenn sie in herkömmlicher > Schriftform (mit eigenhändiger Unterschrift) oder durch Übermittlung eines > solchen Schriftstücks per Telefax erfolgen. > > Important note: The information included in this e-mail is confidential. > It is solely intended for the recipient. If you are not the intended > recipient of this e-mail please contact the sender and delete this message. > Thank you. Without prejudice of e-mail correspondence, our statements are > only legally binding when they are made in the conventional written form > (with personal signature) or when such documents are sent by fax. > > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Re: Integrating a new BitStore for Google Cloud Storage
Hi Pedro, Thanks for working your way through your GCS implementation of the bitstore. Contributions here are very welcome, especially to make DSpace able to fit natively into the cloud. The cleanup/delete logic is pretty dense, and I don't have any specific advice until I can dive back into that chunk of code. I was pretty sure that checksum and deletes worked when that code was checked in. I would also add, that if anyone is able to cleanup the cleanup process, then feel free. I'm not sure which branch is best to target. It would probably make sense to make a pull request against 6.x, however, I'm not sure what stage the branches are in, such that new features can't go into 6.x, but would have to be targeted against 7.x. I suppose there could be a case to be made that GCS implementation isn't a new feature, but just filling in a missing piece to an existing 6.x feature. In which case you'd have to have a PR to fit it into 6.x and 7.x. I do know that an alternative to a delete would be to migrate the assetstore. bitstore-migrate -a 0 -b 1. Would move/copy all the assets that DSpace still knows about from bitstore[0] to bitstore[1]. You could then delete the bucket/folder for bitstore[0], not graceful. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Tue, Jan 17, 2017 at 1:58 PM, Pedro Amorim wrote: > Hello Peter, > > First of all, I'm sorry for not replying sooner, I had some rather long > (and deserved) vacations and also because I didn't want to get back to this > before starting to implement it. > Secondly, thank you so much for replying to this as your input as helped > me a lot in my work. > Now, I've had many difficulties trying to implement this partly due to > being very inexperienced in java, never worked with GCS client library > before nor have I ever programmed in the DSpace core. > > I believe I've made some good progress, I can already write and read files > from a bucket in Google Cloud Storage. Everything works as expected in the > application and everything is surprisingly fast and consistent. > However, I'm struggling to delete objects from the bucket and the main > problem is that the remove() (in my GCSBitStoreService.java class) function > is never called. Every other method is called and working as expected (put, > get and about) and I also implemented the updateTime, hash and Etag. Note > that I can remove the object in the application, it gets removed from the > record, from the database and the index, just not from the bitstore (Google > Cloud Storage). > > I was expecting the object to remain in the bucket until cleanup is run - > either as cron or manually, because when you delete an object from the > application it actually doesn't remove it from the storage. > After that, I run *cleanup -v* and everything runs smoothly, no error no > exceptions nothing, but the object still remains in the bucket. It can't be > related to bucket permissions because the application never actually runs > the remove() method (which contains the Storage call). > > After that, I started digging in the cleanup method: > https://github.com/DSpace/DSpace/blob/master/dspace-api/ > src/main/java/org/dspace/storage/bitstore/BitstreamStorageServiceImpl. > java#L220 > > Noticed the 'recent' validation and made sure the object was deleted more > than an hour ago. Still no success. Even commented that out and nothing. > > Looking at the cleanup code, I can't really tell where the object is > supposed to be removed. > It seems to me it only calls the remove() method if versioning is enabled > and more than one object if found: > https://github.com/DSpace/DSpace/blob/master/dspace-api/ > src/main/java/org/dspace/storage/bitstore/BitstreamStorageServiceImpl. > java#L289 > Any insight on this? > > Also, one last question: > If I am to contribute my implementation to the DSpace project how should I > go about it? Make a pull request in dspace6.0 branch on github? > > Sorry for the wall of text. > > Thank you very much, > > Pedro Amorim > > > segunda-feira, 19 de Dezembro de 2016 às 13:50:49 UTC-1, Peter Dietz > escreveu: >> >> Hi Pedro, >> >> Yes, these steps look like what you have to do to add a new >> storage/BitStore implementation. And yes, add the Google SDK >> to dspace-api/pom.xml instead of dspace/pom.xml. >> >> One implementation pointer I'd like to suggest is that it would be nice >> to support the use case of simple/small objects ~5MB, and also multipart >> upload/download so that it can work with large 5GB - 5TB sized objects. >> https://cloud.google.com/storage/docs/json_api/v1/how-tos/ >> multipart-upload &
Re: [dspace-tech] Cleanup in S3 Storage
Hello Jaime, I was under the impression that everything works as well regardless of what implementation of assetstore is being used. Checking the code, when cleanup eventually calls the remove() method, it would delete an object in S3. https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/storage/bitstore/BitstreamStorageServiceImpl.java#L291 stores.get(bitstream.getStoreNumber()).remove(bitstream); Is there anything in your logs? Perhaps your IAM role does not grant you the ability to delete objects from that bucket? Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Wed, Jan 11, 2017 at 11:59 AM, Jaime Solorzano wrote: > Hello > > I see that cleanup does not delete the s3 files, which method should be > used? > > Someone done this? > > > Jaime Solorzano > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Re: Integrating a new BitStore for Google Cloud Storage
Hi Pedro, Yes, these steps look like what you have to do to add a new storage/BitStore implementation. And yes, add the Google SDK to dspace-api/pom.xml instead of dspace/pom.xml. One implementation pointer I'd like to suggest is that it would be nice to support the use case of simple/small objects ~5MB, and also multipart upload/download so that it can work with large 5GB - 5TB sized objects. https://cloud.google.com/storage/docs/json_api/v1/how-tos/multipart-upload Also, be mindful as to what you want to use to track the checksums. https://cloud.google.com/storage/docs/hashes-etags Lastly, ensure that you close any resources opened when putting or getting objects to the store. You can run out of resources if you open an HTTP connection each time, but never close it. Good luck. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Mon, Dec 19, 2016 at 7:52 AM, Pedro Amorim wrote: > Just noticed, the GCS Java client in 1) should be added to > [dspace-source]/dspace-api/pom.xml instead of [dspace-source]/dspace/pom.xml > as originally mentioned. > > segunda-feira, 19 de Dezembro de 2016 às 11:46:10 UTC-1, Pedro Amorim > escreveu: >> >> Hello everyone, >> >> I'd like to store DSpace bitstreams in a GCS bucket, much like the way >> the S3BitStore is implemented. >> Before I start tinkering and testing, I'd much appreciate some input on >> the matter from more experienced Java programmers, namely DSpace >> programmers. >> >> So, from what I've gathered so far, I need to: >> >> 1) Add Google Cloud Storage Java libraries in >> [dspace-source]/dspace/pom.xml and perform a rebuild. This will be needed >> for step 2). These libraries are as provided here: >> https://cloud.google.com/storage/docs/reference/libraries# >> client-libraries-install-java >> >> 2) Create and implement a new GCSBitStoreService.java based on the one >> created for S3: >> https://github.com/DSpace/DSpace/blob/master/dspace-api/src/ >> main/java/org/dspace/storage/bitstore/S3BitStoreService.java >> >> 3) Add new BitStore in bitstore.xml: >> https://github.com/DSpace/DSpace/blob/master/dspace/config/ >> spring/api/bitstore.xml >> >> 4) Activate the new BitStore in bitstore.xml as documented here: >> https://wiki.duraspace.org/display/DSDOC6x/Storage+Layer#Sto >> rageLayer-ConfiguringAmazonS3Storage >> >> And that's about it? >> >> Thanks as always, >> >> Pedro Amorim >> > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
[dspace-tech] How to reindex solr authority
Hi All, We have a Solr Authority index that needs to be reindexed. Some of the data is out of date. And also, for unknown reasons, we are missing the external authority ID for some records (the equivalent of orcid_id is missing). I've found org.dspace.authority.UpdateAuthorities as some sort of stub for how to find records in Solr authority, process them, but it doesn't appear to do anything. I've used that as a start, I find records, and then since I'm missing data, search from that data present against the external authority system to find matches. My question, is how to actually reindex a record in solr. I can find an authority value from solr, but I don't see how to write it with updates back to solr. Here's what I'm thinking for reindex. String selectedID = "1f8e4a15-4631-4b82-8289-5e92463b776b"; AuthorityValue byUID = authorityValueFinder.findByUID(context, selectedID); MyAuthorityValue searchValue = myAuthoritySource.queryAuthorities(byUID.getValue(), 1); byUID.setRemoteID(searchValue.getRemoteID()); byUID.update(); //Doesn't do anything. I wonder if instead of trying to write to solr, I need to update the item metadata to trigger a reindex... It might be easier to declare dspace solr authority bankruptcy, and write a third-party script that gets the solr authority index in a state I need it to be... Anyways, the Solr documentation lists using SOLR as a data source <https://wiki.apache.org/solr/HowToReindex> as a huge no no. Using Solr as a Data Source Don't do this unless you have no other option. Solr is not really designed for this role. Every attempt is made to ensure that Solr is stable, but indexes do get corrupted by unanticipated situations, and by things completely outside developer control. Storing a text_value="Dietz, Peter" and authority=orcid:0012-1234-4567, in DSpace database metadatavalue would have been better than setting authority=abcdef-12345-ghijk-6789, and then having to trust that nothing has happened to SOLR, and that that id has the data you are looking for. Well, we don't have anything inside that id, for many entries. DSpace database should be the rock-solid trustable data source, and solr should just have a reindex function, where it updates itself from the database values. And, perhaps some even better solution could be found than just this, such as a table just for authority keys. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] DSpace 6: REST API return codes
Hi Ari,
Lets walk through:
GET /items/{itemID}
That will try to find an item with an ID of "something_invalid".
The code for that lookup is:
item = itemService.findByIdOrLegacyId(context, id);
if (item == null)
{
context.abort();
log.warn("Item(id=" + id + ") was not found!");
throw new WebApplicationException(Response.Status.NOT_FOUND);
}
So, it does have some logic that if the item isn't found you should get the
404.
However, it looks like some of the logic inside of the findByIdOrLegacyId
is:
UUID.fromString(id)
Parsing UUID.fromString("something_invalid") is probably what causes the
exception that gets thrown, which gets back to REST, and passed along as a
500 server exception, as opposed to 404 not found. I'm guessing that before
it tries to parse the UUID, it should ensure that it is a UUID. Otherwise
that method should return the null. So, I think this issue resides in
ItemServiceImpl, as opposed to REST.
Thanks for the discovery.
Peter Dietz
Longsight
www.longsight.com
pe...@longsight.com
p: 740-599-5005 x809
On Fri, Aug 19, 2016 at 6:08 AM, Ari wrote:
> Hi,
> I'm writing a tool for metadata editing using DSpace's REST API. I wonder
> if there will be change in REST API return codes in final release.
>
>
> For example this
> https://demo.dspace.org/rest/items/something_invalid
> returns 500 (internal server error) but AFAIK the correct code would be
> 404 (not found)
>
> Ari
>
> --
> You received this message because you are subscribed to the Google Groups
> "DSpace Technical Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dspace-tech+unsubscr...@googlegroups.com.
> To post to this group, send email to dspace-tech@googlegroups.com.
> Visit this group at https://groups.google.com/group/dspace-tech.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Batch update of preferred names for ORCIDs?
Hi Alan, Does: /bin/dspace dsrun org.dspace.authority.UpdateAuthorities work for you? This will iterate over every solr record currently in use (unless the -i argument is provided), query the ORCID web service for the latest data and update the information in the cache. If configured, the script will also update the metadata of the items in the repository where applicable. https://wiki.duraspace.org/display/DSDOC5x/ORCID+Integration#ORCIDIntegration-Storageofrelatedmetadata Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Thu, Aug 18, 2016 at 9:32 AM, Alan Orth wrote: > Hi, > > During the item submission process you can look up authors in the > ORCID service and DSpace will store the version of the author's name > that was found at the time of the lookup — I believe this has been > called the "preferred" name before on this list. We're slowly getting > our scientists to register for ORCIDs and as this evolves I'm sure > users will change their name format from time to time. > > Has there been any thought put into a process for updating stored > author names from ORCID? It could be a manual job you kick off > periodically (via the web interface or a cron job). > > Thanks, > > -- > Alan Orth > alan.o...@gmail.com > https://englishbulgaria.net > https://alaninkenya.org > https://mjanja.ch > "In heaven all the interesting people are missing." ―Friedrich Nietzsche > GPG public key ID: 0x8cb0d0acb5cd81ec209c6cdfbd1a0e09c2f836c0 > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Author name alternate spellings
Hi Alan,
Automated way of fixing mis-matched authorities would be good. I'm not sure
that I'm aware of this, but does DSpace have a way of fixing authority
records?
In your case, maybe you could have a script that says when there are
multiple's for distinct(text_value, authority, confidence), then what's the
best course of action? If one of those has a high confidence score, then
have that value overwrite the others. Is the one with the 600 confidence
the correct one? Does confidence mean that it came from the authority
backstore, or does confidence mean that a human touched a button?
Or, perhaps another script that looks at all the text_value records that
are missing authority. Have it search authority (likely Orcid) for a match
on text-value, and if there is a very close match, than have it make a
recommended replacement? Then present to the user a whole bunch of
corrections.
Orcid and text_value match = 100%
a
b
c
80% match
50% match
5% match...
Locally we've just gotten an issue where user's are doing the first half of
a submission in xmlui using authority, then switching to jspui to upload
the files (using html5 multi-upload, which they love), and we're just now
discovering that sometimes the authority information from xmlui got lost in
translation, (but author Lastname, First is available).
Peter Dietz
Longsight
www.longsight.com
pe...@longsight.com
p: 740-599-5005 x809
On Thu, Aug 4, 2016 at 9:50 AM, Alan Orth wrote:
> Hi,
>
> Thanks for the responses. Here is one author who has duplicates due to
> mismatched authority and confidences:
>
> dspace=# select count(text_value) from metadatavalue where
> metadata_field_id=3 and text_value='Grace, D.';
> count
> ---
>516
>
> dspace=# select distinct text_value, authority, confidence from
> metadatavalue where metadata_field_id=3 and text_value='Grace, D.';
> text_value | authority | confidence
> +--+
> Grace, D. | 0b4fcbc1-d930-4319-9b4d-ea1553cca70b | 0
> Grace, D. | 83a8848e-1651-40df-8453-831eabdee9e0 | 0
> Grace, D. | 0b4fcbc1-d930-4319-9b4d-ea1553cca70b |600
> Grace, D. | | -1
> Grace, D. | 0b4fcbc1-d930-4319-9b4d-ea1553cca70b | -1
>
> @Andrea, you can see this is actually the top author in our
> repository, with 516 records (as in the first SQL command). She has
> something like 462 and 40 items if you look carefully in the authors
> of this particular community:
>
> https://cgspace.cgiar.org/handle/10568/1/search-filter?field=author
> https://cgspace.cgiar.org/handle/10568/1/search-filter?
> field=author&offset=130
>
> @Peter, yes it totally helps, and I've done it for a few high-profile
> authors, but I was hoping to just make Discovery use text_value for
> now, as there's no way to do an automated/batch cleanup for our tens
> of thousands of authors (not to mention, in the future the problem
> will still be there). Unless I suppose this is just growing pains from
> moving an existing DSpace installation into the DSpace 5+ era!
>
> Ciao,
>
> On Thu, Aug 4, 2016 at 3:46 PM, Chris Gray wrote:
> > This could be a problem with the underlying id stored for each name. By
> > default, the authority index doesn't assume two names are for the same
> > person unless they are explicitly associated with the same underlying id.
> >
> > An authority record looks like this:
> >
> > {
> > "id": "badd7401-12aa-4c5c-8d62-515e5746b9d3",
> > "field": "dc_contributor_author",
> > "value": "Author, Random",
> > "deleted": false,
> > "creation_date": "2015-11-27T05:46:05.066Z",
> > "last_modified_date": "2015-11-27T05:46:05.066Z",
> > "authority_type": "person",
> > "first_name": "Random",
> > "last_name": "Author"
> > },
> >
> > That first "id" element determines what can or can't be considered the
> same
> > person for indexing.
> >
> > We discovered this problem with ORCIDs. Just because two entries are
> given
> > with the same ORCID, still the entries were given different ids and show
> up
> > in browse lists and facets as multiple authors. We have to force a
> certain
> > id when adding a new dc.contributor field.
> >
> > On Thursday, August 4, 2016 at 8:08:05 AM UTC-4, Alan Orth wrote:
> >>
> >> Hi
Re: [dspace-tech] Author name alternate spellings
Does batch metadata editing and merging the author's to have the same
authority keys help? I feel that that feature might be under documented.
On Aug 4, 2016 8:08 AM, "Alan Orth" wrote:
Hi,
We have hundreds or thousands of duplicate authors that have the same
exact text_value, but show as separate authors in Discovery's author
sidebar facet. I have tried a handful of these configuration keys
(with a full discovery reindex after) on DSpace 5.1 but I never see
any change.
First I tried:
index.authority.ignore-prefered.dc.contributor.author=true
index.authority.ignore-variants.dc.contributor.author=false
Then:
index.authority.ignore=true
index.authority.ignore-prefered=true
index.authority.ignore-variants=true
Then:
discovery.index.authority.ignore-prefered.dc.contributor.author=true
discovery.index.authority.ignore-variants=true
What is the trick to getting Discovery to use author text values for
its indexes? Is this a bug that upgrading to 5.{2,3,4,5} will fix? I'm
going slightly crazy. :)
Thanks,
On Mon, Jan 4, 2016 at 11:19 PM, Andrea Bollini wrote:
> Hi all,
> the relevant parameters are
> discovery.browse.authority.ignore-prefered
> discovery.index.authority.ignore-prefered
>
> I was probably partially wrong about the browse behavious as looking to
the code (sorry I have had no chances to make an actual test) the
metadatavalue is never recorded in the browse index when it is authority
controlled
> see
> https://github.com/DSpace/DSpace/blob/master/dspace-api/
src/main/java/org/dspace/browse/SolrBrowseCreateDAO.java#L186
>
> instead the search (facets) only include the prefered form if you use
default configuration, see
> https://github.com/DSpace/DSpace/blob/master/dspace-api/
src/main/java/org/dspace/discovery/SolrServiceImpl.java#L1091
>
> it looks also as the browse system is buggy when the ignore-prefered is
set to true as probably nothing is indexed in the browse in this case.
Probably we have fixed this bug on our dspace-cris fork and we have forget
to back port to the basic dspace
> see
> https://github.com/Cineca/DSpace/blob/dspace-5_x_x-cris/
dspace-api/src/main/java/org/dspace/browse/SolrBrowseCreateDAO.java#L224
>
> about your second question, which is the authority for authors in a
dspace-cris instance it is the internal researcher pages database and the
ORCID registry, in a standard DSpace istance you can configure the ORCID
registry as first lookup for the authors name, when the item is archived
the orcid record is recorded in a local cache solr based that is what is
actually used as authority
> https://github.com/DSpace/DSpace/blob/master/dspace/
config/dspace.cfg#L1563
>
> this mean that, at least if you no edit the metadata value directly in
the database or using the admin edit, you cannot have an authority with a
corresponding value different than the "prefered one". With DSpace-CRIS
where also variants are managed out-of-box this can happen more easily.
>
> Best,
> Andrea
>
> - Messaggio originale -
> Da: "Hilton Gibson"
> A: "Peter Dietz"
> Cc: "DSpace Technical Support"
> Inviato: Lunedì, 4 gennaio 2016 16:40:31
> Oggetto: Re: [dspace-tech] Author name alternate spellings
>
>
>
> Hi All,
>
>
> " You can also configure the system (see the discovery.cfg options) to
ignore the metadatavalue and put in the index only the prefered form of a
name as provided by the authority."
>
>
> 1. Where in "discovery.cfg" is this configured? A github link would help.
> 2. Who is the "authority" for authors? Excuse the pun!
>
>
> Regards
>
>
> hg
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hilton Gibson
> Stellenbosch University Library
>
> http://orcid.org/-0002-2992-208X
>
>
>
>
> On 4 January 2016 at 17:13, Peter Dietz < pe...@longsight.com > wrote:
>
>
>
> Hi All,
>
>
> Lets say you have an author record that might different values, but all
representing the same person.
> Peter Dietz
> Peter M Dietz
> PM Dietz
> Dietz, Peter M
> Dietz, PM
>
>
> Does DSpace have any facility to map these distinct values to the same
author, such that a search for "Peter Dietz", will match when the value is
stored as "Dietz, PM". Or, is the recommendation just to edit the metadata,
and consolidate the distinct values into one single acceptable form.
(Probably the longest version of a name).
>
>
> My read of "Authority" is that it will give you a backstore, but I don't
see it as being a mapping of different ways of spelling the same thing.
>
>
> Perhaps this is another need for richer metadata objects? name.givenname,
name.surname, ...
>
>
>
>
> Peter Diet
Re: [dspace-tech] HA Clustering Dspace
>From my investigations into DSpace, the key element that I would like to de-couple from DSpace is SOLR. Say you were going to build a new frontend to DSpace that heavily used the DSpace REST API. You could have multiple servers, each running tomcat and the DSpace REST API deployed. With nginx outside of that proxying / load balancing. No problems. Especially as you have postgres as an external service (rds), the assetstore is located outside of DSpace (s3). However, I don't see how you can run multiple instances of DSpace SOLR. SOLR stores data, and it wouldn't be as simple as just adding another server running the webapp. But you would need to coordinate the SOLR cluster, using SolrCloud / ZooKeeper. Maybe its not as complicated as I think. But, I thought that I read at one point that DSpace had some custom solr code present, or the solr configs would have to be managed, and I'm not sure how much work it would be to build up a solr cluster with that config. It could be possible to ensure that DSpace can use stock SOLR, or to write another implementation for storage/search/index/engine that might be more cloud friendly than Solr, such as DynamoDB/CloudSearch. A normal use-case of SOLR is to use it only as an index, that your important data lives in a persistant data store, such as the database, and you could wipe out your search index, and reindex your source data to repopulate it. However, DSpace's use of solr relies on it as being a source of data for some elements (authority, statistics). ____ Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Thu, Jul 21, 2016 at 2:34 PM, Luiz dos Santos wrote: > Hi Tim, > > It seems interesting , but I have a point, relay the high availability in > hardware and in a big monolithic software seems more like mitigate the > problem but not solve it, you could have Solr and PostgresSQL in clusters, > they have their own cluster possibilities, but you will end up with a one > DSpace in one Tomcat that can fall and put your repository down, right? > > Maybe to a high availability DSpace need something more, something with > microservices, something in agreement with the Reactive Manifesto ( > http://www.reactivemanifesto.org/). In Dspace 7, the new GUI model will > bring the possibility of run the GUI in another server, that is great, but > DSpace will be relay in a one DSpace backend, right? Do you see a way to > have two or more DSpace back end running simultaneous. > > One last point, as a volunteer, I would like to take part in the > clustering group. > > Best regards > Luiz > > On Thu, Jul 21, 2016 at 1:11 PM, Hernán Lagos > wrote: > >> Hi Tim >> >> Thanks, your feedback has been very useful. >> If we set up a cluster of Dspace, it will be announced. >> >> Regards >> >> El jueves, 21 de julio de 2016, 12:27:17 (UTC-4), Tim Donohue escribió: >>> >>> Hi Hernán, >>> >>> The simple answer here is that, currently, there is no "standard" high >>> availability setup for DSpace, and DSpace has no inherent ability to do >>> load balancing or clustering on its own. >>> >>> That said, DSpace is essentially just a web application that runs on >>> Tomcat (or similar), uses a PostgreSQL database (or similar) to store >>> metadata/relationships, and uses Apache Solr for searching/browsing. Each >>> of these three tools (Tomcat, PostgreSQL and Solr) *do* provide clustering >>> options. So, it may be plausible to rely on the clustering options at >>> those levels to create a DSpace cluster. >>> >>> However, I'll admit that I'm not aware of anyone who has done that >>> before. If someone has, I'm hoping they will speak up here to provide us >>> all a bit more clues/hints. There is an older (outdated now) wiki page >>> where such discussions started a long time ago, but they never came to any >>> final decision/proposal: >>> >>> https://wiki.duraspace.org/display/DSPACE/Clustering >>> >>> All that said, I suspect there are others who would be of interested in >>> more easily enabling clustering within DSpace itself. That seems like >>> it'd make a wonderful addition to the software platform, but it'd take one >>> (or more) institutions who could help us to better define the gaps, what is >>> missing/needed, and then start to figure out a way forward. DSpace has no >>> centralized development team (developers are volunteers or allowed to work >>> on the project by their institutions). So we are entirely reliant on the >>> institutions using DSpace to help us make such improvements (see h
Re: [dspace-tech] Re: OAI driver and openaire sets not populated
Hi Francis, One potential thing that I can think of. We recently had an issue with OAI config, and changed a typo that was in config/crosswalks/oai/xoai.xml, in which the filter has to change from "red" to "ref". https://github.com/DSpace/DSpace/commit/563d90f7c4b4b738fc422a222ef997f6ca039367 Check your xoai.xml, to ensure that you have that change. You would need DSpace 5.4+ for this to work, as there were two bugs (one in xoai library, and a change to config to take effect). ____ Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Tue, Jul 19, 2016 at 9:54 AM, Francis Brouns wrote: > Hi Rafa, > > did you manage to solve this problem? We have the same problem. > > best wishes, Francis > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Using private s3 object store instead of aws
Hello Malte,
The redesign for the asset store assumes that if you wanted to use a
different implementation for storage, then you would write a new java class
that implements the interface.
Given that some non-Amazon service could implement the same S3 API, to use
the non-Amazon service, then the only change you would need to make, would
be to set a different endpoint. You will need to make a small customization
to the S3BitSoreService.java file inside the init() method, and do
s3Service.setEndpoint("http://my-ceph-cluster.url";);
________
Peter Dietz
Longsight
www.longsight.com
pe...@longsight.com
p: 740-599-5005 x809
On Fri, Jun 17, 2016 at 7:37 AM, wrote:
> Hello,
>
> is there a way in Dspace 6.x to use s3 storage which is NOT Amazon but
> instead a local private object storage? For example a Ceph Cluster?
> The configuration file does not provide a parameter to configure the
> server, it assumes it will be amazon
> The protocol should be the same...
>
> Kind regards
> Malte Willert
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "DSpace Technical Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dspace-tech+unsubscr...@googlegroups.com.
> To post to this group, send email to dspace-tech@googlegroups.com.
> Visit this group at https://groups.google.com/group/dspace-tech.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Re: trouble with a patch
Jose, Not sure what the issue is. But this PR was against master at some point, which might be different than your current branch, so there could be some other semi-relevant changes that you are missing. Also, this PR's commit activity shows something weird of commits reverting commits. You might need to investigate that. One doesn't usually use the word patch when using GitHub. Though, you can obtain the "patch", by appending ".patch" to the pull request. https://github.com/DSpace/DSpace/pull/999.patch There is also cherry-picking, and there is no way to cherry-pick a PR, but instead you have to cherry-pick commits. This PR has some weird history of it, as there are three commits, which one reverts things... Maybe you need all three commits ? Something like: git cherry-pick b7727a0 git cherry-pick 5010d7f git cherry-pick 88ed31d I would also compare your current file with: https://github.com/christian-scheible/DSpace/blob/88ed31d1470049cfd325b9fb7298f3d9aa1802fb/dspace-api/src/main/java/org/dspace/app/util/GoogleMetadata.java (diff, diffmerge, meld, ...) And/or open your GoogleMetadata.java in an IDE to help determine missing imports. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Thu, Jun 9, 2016 at 2:43 PM, Jose Blanco wrote: > I'm still stuck on this one. I wonder what happens with the code in > dspace-api? GoogleMetadata.java is there. I keep thinking that something > needs to be cleared. > > -Jose > > On Wed, Jun 8, 2016 at 2:21 PM, Jose Blanco wrote: > >> I wonder if I have to clear the java memory? It seems like it's looking >> for Set when it should be looking for Collection? >> >> On Wed, Jun 8, 2016 at 2:00 PM, Jose Blanco wrote: >> >>> I'm working on incorporating this patch: >>> >>> https://github.com/DSpace/DSpace/pull/999 >>> >>> I changed the file indicated in the patch in my development environment, >>> then I did a build and update, and I was getting this error: >>> >>> 2016-06-07 10:31:12,110 ERROR >>> org.dspace.app.xmlui.cocoon.DSpaceCocoonServletFilter @ Serious Error >>> Occurred Processing Request! >>> >>> org.springframework.web.util.NestedServletException: Handler processing >>> failed; nested exception is java.lang.NoSuchMethodError: >>> org.dspace.app.util.GoogleMetadata.getMappings()Ljava/util/Set; >>> >>> >>> which I think originated from: >>> >>> Caused by: java.lang.NoSuchMethodError: >>> org.dspace.app.util.GoogleMetadata.getMappings()Ljava/util/Set; >>> >>> at >>> org.dspace.app.xmlui.aspect.artifactbrowser.ItemViewer.addPageMeta(ItemViewer.java:329) >>> >>> >>> I was uncertain whether I had to change anything else, other than the >>> file indicated on the PR, but Tim assured me that was it, so I did a >>> rebuild just to try it again and then the error went away and it's working >>> great. Today I went to release the prod with the same code and I'm getting >>> the same error I was initially getting. I'm confused why it did not work. >>> I guess if I knew why it did not work the after the 1st built, and then it >>> worked in dev, I would have a better handle on the problem. Any guesses? >>> >>> >>> Thank you! Jose >>> >> >> > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Usage Statistics and the REST API
Hi Evgeni,
If your concern is that using REST increased the usage count, then you can
disable UsageEvents from being counted by REST through a config.
config/modules/rest.cfg
https://wiki.duraspace.org/display/DSDOC5x/REST+API#RESTAPI-ConfigurationforDSpaceREST
There's no parameter you can add to a rest request that says "this is an
insignificant request, please don't count this as usage". All or nothing.
____
Peter Dietz
Longsight
www.longsight.com
pe...@longsight.com
p: 740-599-5005 x809
On Wed, Jun 8, 2016 at 9:50 AM, Evgeni Dimitrov
wrote:
> I am getting the usage statistics (top 100 titles) for a community with
>
> // http://localhost:8080/solr/statistics/select
> // ?q=type:2&fq=owningComm:12
> // &fq=time:[2016-05-01T00:00:00Z TO 2016-06-01T00:00:00Z]
> // &rows=0&facet=true&facet.field=id
> // &facet.limit=100&indent=on
>
> as a list of pairs (count, id).
>
> To give to the librarian a nice result, I am replacing the id with the
> handle and the title of the book.
> Using
> REST GET /items/{item id}
>
> Works fine, but every time I run the report, the count (the usage) of this
> 100 books is increased by one :(
>
> Will it be the same, if instead of
> GET /items/{item id}
>
> I use
> GET /items/{item id}/metadata
>
> ??
>
> If the answer is "yes", is there any unintrusive way to decode the item id
> into handle + title?
>
> Best regards
>
> --
> You received this message because you are subscribed to the Google Groups
> "DSpace Technical Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dspace-tech+unsubscr...@googlegroups.com.
> To post to this group, send email to dspace-tech@googlegroups.com.
> Visit this group at https://groups.google.com/group/dspace-tech.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.
[dspace-tech] OAI, restrict specific collection?
Hi All, I was wondering if there is a way to have OAI not disseminate a few specific collections? If I change the collection permission to not be anonymous=read, will that hide those sets from OAI's list of sets? And items in those collections, I'm not sure that setting a collection level permission would affect items. (As newly set dspace permissions don't propagate down). ________ Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Re: configure browse index -
It will be something more like: webui.browse.index.2 = author:metadata:dc.contributor,dc.contributor.author,dc.creator:text Also not tested. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Wed, May 4, 2016 at 8:32 AM, wrote: > Hello Monika, > > smth like > > webui.browse.index.2 = > author:metadata:dc.contributor,author:metadata:dc.contributor.author,dc.creator:text > > in dspace.cfg > > This is not tested ... > > > > Am Donnerstag, 21. April 2016 20:27:14 UTC+2 schrieb momeven: >> >> I would like to configure the author browse index to include >> >> dc.contributor, >> dc.contributor.author, >> dc.creator:text >> >> but according to the documentation I need to either specify ‘*’ or a >> proper qualifier >> >> the reason I do not use dc.contributor.* is that we have >> dc.contributor.advisor field, >> that goes into our advisor index but should not be part of the author >> index >> >> is there a way to this in version 5 ? >> >> >> Monika >> >> - >> Monika Mevenkamp >> Digital Repository Infrastructure Developer >> Princeton University >> Phone: 609-258-4161 >> Skype: mo-meven >> >> >> >> -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Installing DSPACE in a Google Cloud Server with Data Repository stored on my local network
Hi Bruno, There is no DSpace BitStore implementation for using Google Cloud Storage, but the next release of DSpace has refactored the asset store plumbing to allow for different implementations to be used. This advice might not be very helpful for a current DSpace 5 instance, but would be helpful for a development instance on DSpace 6, which adds under-the-hood improvements that would help you better make use of Google Cloud Storage. The Interface for BitStore https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/storage/bitstore/BitStoreService.java Implementation to use AWS S3 storage. https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/storage/bitstore/S3BitStoreService.java If you are interested in adding Google Cloud Storage support just implement a GoogleCloudStorageBitStoreService class and implement a few methods: init, get, put, about, remove. Once you create a GCS implementation, then you can wire it in with some spring bean xml. https://github.com/DSpace/DSpace/blob/master/dspace/config/spring/api/bitstore.xml We allow for a hybrid storage approach. Wire localStore to store#0, the localStore would be seen as "local storage" even though you will be mounting some remote NAS storage. Then wire the GCSStore to store#1, and make that the incoming. Old assets on the NAS, new assets on GCS. ________ Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Tue, Mar 1, 2016 at 8:57 AM, Bruno Cabral wrote: > Hi Monika, > > Thank you! > > The idea is not having the databases inside the docker images. > > But is it possible to have the digital objects on a local netowrk attached > storage while the dcoker is in Google Cloud, with dspace linking from the > cloud to our NAS? > > And a mixed solution, having part of the objects on our NAS and other on > Google Cloud Storage? > > Thanks and Regards, > > BC > > 2016-03-01 12:38 GMT-01:00 Monika Mevenkamp : > >> I am not exactly sure either what the details are. But if you can make >> your NAS storage visible inside the docker image such that you can link >> /dspace/assetstore -> /NAS_assetstore >> you should be fine. >> >> You need to think carefully about what you do with your database. >> How do you plan to keep backups ? >> >> Keep in mind that, if you manage the database inside docker >> you will not be able to upgrade software by building a new docker >> image. Building a new image means loosing all data created/touched >> inside the old image after that image was built and started. >> >> Monika >> >> Monika Mevenkamp >> mo.me...@gmail.com >> >> http://mo-meven.tumblr.com/ >> http://mcmprogramming.com/mo.meven/ >> >> >> >> On Feb 29, 2016, at 1:24 PM, Bruno Cabral >> wrote: >> >> Hi list, >> >> I'm member of a team that is developing and implementing a solution using >> DSPACE. >> >> We intend that the apps including DSPACE stays on the Google Cloud >> Infrastructure, in docker containers, and that the digital objects of the >> DSPACE repository stay stored and linked on our local network Storage, on a >> Network Attached Storage. >> >> Is this possible? How? >> >> Thanks, >> >> BC >> >> -- >> You received this message because you are subscribed to the Google Groups >> "DSpace Technical Support" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to dspace-tech+unsubscr...@googlegroups.com. >> To post to this group, send email to dspace-tech@googlegroups.com. >> Visit this group at https://groups.google.com/group/dspace-tech. >> For more options, visit https://groups.google.com/d/optout. >> >> >> > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] A bug in REST API introduced in 5.3 or 5.4
Look at Git Blame to find out why things changed:
https://github.com/DSpace/DSpace/blame/dspace-5_x/dspace-rest/src/main/java/org/dspace/rest/HandleResource.java
And that line leads to:
https://github.com/DSpace/DSpace/pull/1007
So, adding that was intending to prevent stale connections / contexts from
being held open for a very long time, and behave like other endpoints. The
problem was that you could view /rest/handle, and get data, then change
that object, and the next time you view that /rest/handle, the data would
not have changed.
If you have a stack trace, please consider investigating the root of the
bug for 5x. In the upcoming DSpace 6x, contexts are completely different
with the services refactor, and changes here would have limited utility.
Peter Dietz
Longsight
www.longsight.com
pe...@longsight.com
p: 740-599-5005 x809
On Tue, Feb 23, 2016 at 3:19 PM, Evgeni Dimitrov
wrote:
> Comparing 5.2 and 5.4 - org.dspace.rest.HandleResource - I see that after
> the line:
>
> return new Community((org.dspace.content.Community) dso, expand, context);
>
> some lines are added:
>
> } finally{
> processFinally(context);
> }
>
> Because context is not null and valid, processFinally generates 500
> "INTERNAL_SERVER_ERROR"
>
> What was the intention for adding "finally" there? What should be the fix?
> I can just delete these lines . . .
>
> --
> You received this message because you are subscribed to the Google Groups
> "DSpace Technical Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dspace-tech+unsubscr...@googlegroups.com.
> To post to this group, send email to dspace-tech@googlegroups.com.
> Visit this group at https://groups.google.com/group/dspace-tech.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] /rest issue using master branch
Doing some testing, the Jersey 2 update to REST fixes rest on master, so I have merged that PR. https://github.com/DSpace/DSpace/pull/1102 Onwards and upwards! Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Tue, Feb 2, 2016 at 11:26 AM, Peter Dietz wrote: > We have a PR to update master/6x REST API to Jersey 2. Honestly I thought > we were going to merge that in months ago. > https://github.com/DSpace/DSpace/pull/1102 > > I've just verified that REST is broken since the PubMed jersey2 reference > got added to API. (Mismatch of jersey1 and jersey2 in different places). > I'll test updating REST API to JERSEY 2, and see what the results of that > are. > > > Peter Dietz > Longsight > www.longsight.com > pe...@longsight.com > p: 740-599-5005 x809 > > On Tue, Feb 2, 2016 at 10:39 AM, Tim Donohue > wrote: > >> Digging more deeply, it looks like we do have BOTH Jersey 1 and Jersey 2 >> now in our POMs. >> >> >> https://github.com/DSpace/DSpace/search?l=maven-pom&q=jersey&utf8=%E2%9C%93 >> >> 1) Jersey-Server v1 is used by REST >> 2) A reference to Jersey-Client v2 came into the 'dspace-api' via DS-2880 >> (PubMed integration), which was merged in the last week or so: >> https://jira.duraspace.org/browse/DS-2880 >> Here's the commit: >> https://github.com/DSpace/DSpace/commit/002b8783bffd15024f9827d5166f1e0cb52a95ce >> >> So, I think Bram's guess may be correct. We may have a Jersey version >> conflict on "master". >> >> - Tim >> >> >> On 2/2/2016 7:53 AM, Bram Luyten wrote: >> >> Hi Terry, >> >> not sure what it is, but the following stackoverflow thread seems to >> indicate this means that you're trying to load Jersey 1 and 2 at the same >> time: >> http://stackoverflow.com/questions/23277429/exception-in-rest-jersey >> >> The error only matches on >> >> *java.lang.AbstractMethodError: >> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;* >> *at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)* >> >> So I'm not 100% confident the issues are related. >> >> Bram >> >> -- >> [image: logo] >> *Bram Luyten* >> *250 Lucius Gordon Drive, Suite B-3A, West Henrietta, NY 14586* >> *Esperantolaan 4, Heverlee 3001, Belgium* >> >> <http://atmire.com/website/?q=services&utm_source=emailfooter&utm_medium=email&utm_campaign=braml> >> www.atmire.com >> >> On 2 February 2016 at 00:32, Terry Brady >> wrote: >> >>> I would like to do some testing with the Enhanced Configurations code. >>> >>> I deployed the latest code on the master branch to my test server and I >>> am seeing the following error when I tried to access /rest. I can access >>> /xmlui, /solr, and /oai. >>> >>> I believe that I have created a proper local.cfg, >>> >>> Can you offer any advice? >>> >>> Feb 01, 2016 6:27:38 PM org.apache.catalina.core.StandardWrapperValve >>> invoke >>> SEVERE: Servlet.service() for servlet [DSpace REST API] in context with >>> path [/rest] threw exception [Servlet execution threw an exception] with >>> root cause >>> java.lang.AbstractMethodError: >>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder; >>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119) >>> at >>> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:649) >>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) >>> at >>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) >>> at >>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) >>> at >>> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) >>> at >>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) >>> at >>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) >>> at >>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) >>> at >>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) >>> at >>> org.apache.catalin
Re: [dspace-tech] /rest issue using master branch
We have a PR to update master/6x REST API to Jersey 2. Honestly I thought we were going to merge that in months ago. https://github.com/DSpace/DSpace/pull/1102 I've just verified that REST is broken since the PubMed jersey2 reference got added to API. (Mismatch of jersey1 and jersey2 in different places). I'll test updating REST API to JERSEY 2, and see what the results of that are. ________ Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Tue, Feb 2, 2016 at 10:39 AM, Tim Donohue wrote: > Digging more deeply, it looks like we do have BOTH Jersey 1 and Jersey 2 > now in our POMs. > > https://github.com/DSpace/DSpace/search?l=maven-pom&q=jersey&utf8=%E2%9C%93 > > 1) Jersey-Server v1 is used by REST > 2) A reference to Jersey-Client v2 came into the 'dspace-api' via DS-2880 > (PubMed integration), which was merged in the last week or so: > https://jira.duraspace.org/browse/DS-2880 > Here's the commit: > https://github.com/DSpace/DSpace/commit/002b8783bffd15024f9827d5166f1e0cb52a95ce > > So, I think Bram's guess may be correct. We may have a Jersey version > conflict on "master". > > - Tim > > > On 2/2/2016 7:53 AM, Bram Luyten wrote: > > Hi Terry, > > not sure what it is, but the following stackoverflow thread seems to > indicate this means that you're trying to load Jersey 1 and 2 at the same > time: > http://stackoverflow.com/questions/23277429/exception-in-rest-jersey > > The error only matches on > > *java.lang.AbstractMethodError: > javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;* > *at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)* > > So I'm not 100% confident the issues are related. > > Bram > > -- > [image: logo] > *Bram Luyten* > *250 Lucius Gordon Drive, Suite B-3A, West Henrietta, NY 14586* > *Esperantolaan 4, Heverlee 3001, Belgium* > > <http://atmire.com/website/?q=services&utm_source=emailfooter&utm_medium=email&utm_campaign=braml> > www.atmire.com > > On 2 February 2016 at 00:32, Terry Brady > wrote: > >> I would like to do some testing with the Enhanced Configurations code. >> >> I deployed the latest code on the master branch to my test server and I >> am seeing the following error when I tried to access /rest. I can access >> /xmlui, /solr, and /oai. >> >> I believe that I have created a proper local.cfg, >> >> Can you offer any advice? >> >> Feb 01, 2016 6:27:38 PM org.apache.catalina.core.StandardWrapperValve >> invoke >> SEVERE: Servlet.service() for servlet [DSpace REST API] in context with >> path [/rest] threw exception [Servlet execution threw an exception] with >> root cause >> java.lang.AbstractMethodError: >> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder; >> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119) >> at >> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:649) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) >> at >> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) >> at >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) >> at >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) >> at >> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610) >> at >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) >> at >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) >> at >> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) >> at >> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) >> at >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) >> at >> org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190) >> at >> org.apache.coyote.AbstractProtocol$Ab
[dspace-tech] Examples of extending/reusing Authority Control / ORCID
Hi All, The DSpace Authority framework currently has one implementation, ORCID. And, the documentation for the Authority Framework, offers some guidance on extending this, to build your own implementation to wire up other Authority sources. I was wondering if anyone has done this yet, and had some lessons learned? https://github.com/DSpace/DSpace/tree/master/dspace-api/src/main/java/org/dspace/authority Perhaps, this is loosely a follow up to my previous "nested metadata" discussion. Though we don't need to support nested-metadata for this, just a trustworthy source of data. https://groups.google.com/d/topic/dspace-tech/z7Nibc-3rJw/discussion At this moment, we don't plan on integrating with an existing data store, but would be creating a data store from scratch, and then populating that store from a local person provider (database, ldap, csv, ...). Then, building an authority-service to use that new data store. It seems like a lot of work, just for the purpose of having auto-complete for a given metadata field. ____ Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] DSpace UI Prototype Challenge
Hi Luiz, The prototypes are being presented each week in a webinar. There have been 3 sessions thus far. They are every Monday / Thursday at 10am (New York time). https://wiki.duraspace.org/display/DSPACE/DSpace+UI+Prototype+Challenge#DSpaceUIPrototypeChallenge-PrototypePresentationSchedule The entries also have code and documentation available on that same wiki page. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Thu, Jan 21, 2016 at 12:53 PM, Luiz dos Santos wrote: > Anybody knows something about the results of this challenge? > > Best Regards > Luiz > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
[dspace-tech] Author name alternate spellings
Hi All, Lets say you have an author record that might different values, but all representing the same person. Peter Dietz Peter M Dietz PM Dietz Dietz, Peter M Dietz, PM Does DSpace have any facility to map these distinct values to the same author, such that a search for "Peter Dietz", will match when the value is stored as "Dietz, PM". Or, is the recommendation just to edit the metadata, and consolidate the distinct values into one single acceptable form. (Probably the longest version of a name). My read of "Authority" is that it will give you a backstore, but I don't see it as being a mapping of different ways of spelling the same thing. Perhaps this is another need for richer metadata objects? name.givenname, name.surname, ... Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] SSL with REST
Hi Keith, DSpace REST requires SSL, because the client could send the auth-token to any request, and that is considered sensitive information. The auth-token allows the request to authorize the Dspace context as a certain DSpace eperson based on that token. So, if certain metadata fields, items, bitstreams, collections, communities are hidden, then an authorized person could access them. Also, the token enables that request to Create, Edit, Delete. Tokens are UUID's and are generated once a request successfully posts username/password to /login. I don't think we document this, but, you could disable REST's requirement for SSL. Caveat emptor. https://github.com/DSpace/DSpace/blob/master/dspace-rest/src/main/webapp/WEB-INF/web.xml#L52 Change: CONFIDENTIAL To: NONE You don't even have to rebuild DSpace to do that. Just edit /dspace/webapps/rest/WEB-INF/web.xml and restart tomcat. I understand adding this bypass for a localhost development environment. But I would advise against doing this in a deployed / production instance. I assume the likelihood of someone snooping is very low, but someone posting their token in a coffee shop wifi over port 80 is not encrypted, and they could take authorize as you. Sites like facebook in the past would have been vulnerable to a firesheep-style problem with having unencrypted session/auth information. ____ Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Tue, Dec 8, 2015 at 11:16 AM, Keith Jones wrote: > > We want to run REST and the documentation says it should run with SSL > enabled. > > What did you do to configure just REST running as SSL? > > Thanks > Keith > > On Tue, Dec 8, 2015 at 9:09 AM, helix84 wrote: > >> On Tue, Dec 8, 2015 at 2:32 PM, Keith Jones wrote: >> > I have another REST question? Really about SSL, has anyone set up their >> site >> > to run REST with SSL but do not run the repository with SSL? >> >> Yes, why? >> >> http://demo.dspace.org/xmlui/ >> https://demo.dspace.org/rest/ >> >> >> Regards, >> ~~helix84 >> >> Compulsory reading: DSpace Mailing List Etiquette >> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >> > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Announcement: Release 2.0 of DSpace REST Client for Ruby
Hi Bruno, Much appreciated. This appears to make easy work of some of the messier details. I'm having an odd issue getting this to work in a rails app. uninitialized constant CommunitiesController::Dspace class CommunitiesController < ApplicationController def index client = Dspace::Client.new(dspace_api: ' https://trydspace.longsight.com/rest') This is through editing Gemfile, and adding gem 'dspace_rest_client' bundle rails s I can, however, get this to work if I completely cheat. clone the dspace-rest-client source code. Copy dspace-rest-client/lib/dspace/ to myproject/lib/dspace/, and lib/dspace.rb And add: gem 'resource_kit' gem 'faraday' gem 'net-http-persistent' So, perhaps a certain version of rails, ruby, gem, something is required? Perhaps, I'm doing something wrong. I am able to also run dspace-rest-client via bin/console which is helpful for testing. Also, I'm having trouble cloning from your gitlab site, and my browser is upset about the self-signed ssl certs. Any chance C3SL would be interesting in distributing/forking/cloning this on github as well? Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Wed, Dec 2, 2015 at 2:20 PM, Bruno Zanette wrote: > Hello all, > > I'm proud to announce, in name of C3SL (http://www.c3sl.ufpr.br/) and its > developers, the release of version 2.0 of the Dspace Rest Client Ruby Gem. > > The code has been completely revised to improve it's design and to > simplify the usage, and tests were added. Everything from getting an item > to uploading and downloading a bitstream is perfectly working, and it has > been used and tested on our projects. > > Unfortunately we didn't have time yet to create a proper Wiki nor a sample > app, but we're working on it. As soon as it is done i'll announce it here. > > Please, don't think twice about contacting us if you have any doubt about > how to use it or if you found any bug or had any trouble. Any feedback will > be of great importance and we'll be at standby to answer it! > > The code can be pulled from its official branch: > https://gitlab.c3sl.ufpr.br/c3sl/dspace-rest-client/ > > And also from its page on RubyGems website: > https://rubygems.org/gems/dspace_rest_client/ > > Thanks!! > -- > Bruno Nocera Zanette > +55 41 9992-2508 > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] google-stats file downloads
Hi Susan, I have discovered this issue, today, as well. It appears that the usage event listener, that writes data to GA isn't adding in any information of the hit hierarchy. i.e. This bitstream has this owning item, this owning collection, and these owning communities. So, when it comes time for reporting, the data doesn't have the connection. I've heard that Robin has overnight processing of the data to assist with reporting. I was looking at this code since then, and was trying to think of how best to store this data in Google Analytics. My first thought is to add new "Custom Dimensions" to Google analytics, and then write owningItem:123, owning:Coll:345, owningComm: ... There can be multiple owning communities, so, store the communityID, store an array of id's, separate with commas. Or add many custom dimensions owningComm1, owningComm2, owningComm3, ... Another thought I had would be to overwrite the Document Path = /comm:123/comm:234/comm:345/coll:221/item:456/bundle:ORIGINAL/bitstream:789 Then a report could pull out results where the document path either started a certain way, or contained a certain substring. I like the document path route because it doesn't involve creating custom variables "Custom Dimensions", that you would have to make each user of this feature also use. I just created a JIRA ticket about this: https://jira.duraspace.org/browse/DS-2899 Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Wed, Nov 18, 2015 at 5:36 PM, Borda, Susan wrote: > Hi- > We have enabled google-stats in Dspace but are only seeing 0 for file > downloads. File download counts are visible in Google Analytics itself so > it seems we are having an issue displaying these counts correctly (or at > all), we are displaying “Page Views” just fine: > http://scholarworks.montana.edu/xmlui/handle/1/2999/google-stats > > Is anyone else having this issue? > > Thanks, > susan > > — > Susan Borda > Digital Technologies Development Librarian > Montana State University Library > 406-994-1873 > > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Re: Captcha in XMLUI
Thanks Olivier, Would you have any interest in contributing that to DSpace as a Pull Request for DSpace 6? The pull request time window for features to this version is closing soon, so it would have to be submitted quickly. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Thu, Nov 12, 2015 at 4:25 AM, Olivier Nicole wrote: > Hi, > > The following patch can be used to force the reCaptcha through a proxy. > > This applies on to of what I submitted on DSpace community list earlier > today. > > Note that the proxy hostname and port are hard coded. > > Best regards, > > Olivier > > > -- > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] cache in tomcat
Hi Jose, ehcache will write to your java temp directory. https://github.com/DSpace/DSpace/blob/dspace-5_x/dspace-services/src/main/resources/caching/ehcache-config.xml XMLUI / cocoon will write to a cache dir. tomcat/work/Catalina/localhost/xmlui/cache-dir Batch Uploads (in jspui??) can write to a temp location in [dspace]/imports/batchuploads PDF Citation Coverpage will generate temp files. Media filter will generate temp files. Some of these java temp files will only clean themselves automatically when the jvm shuts down, which would be tomcat restarts, or from command line operations when the command line operation completes. Your trading disk usage for caching and performance. And in general, at least to me, its not that much disk space for the temp overhead. But, it might be wise to explicitly put bounds on things. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Wed, Nov 11, 2015 at 11:56 AM, Jose Blanco wrote: > I noticed some files being created in in my tomcat temp directory. Is this > where files uploaded are initially stored before moving to the assetstore? > > Also, What are the bounds for growths for the tomcat cache-dir? Should I > clear the java cache periodically using the UI? > > Thank you! > > > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Re: Captcha in XMLUI
For the recaptcha, this setup doesn't require the user to be logged in. The typical use case we are thinking is anonymous user who stumbles upon an article they want. They don't have to be logged in, but have to pass Google's captcha challenge, either clicking the button if you are non-suspicious, or if Google has concerns, having to solve a sequence of challenges (choose the street signs, pick images containing chocolate, pick an image that doesn't contain salad, ...). (All handled by Google) If they pass the captcha, then a form gets submitted to the item-request-receiver, and contains the form information, such as form value for person name, person email, a message, and some identifiers for which document they wanted. The item request workflow has its own options, but typically the DSpace admin, or article author will click a button that grants the requester access to the document, using a hashed value in a link, without logging in. ____ Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Wed, Nov 11, 2015 at 11:27 AM, Hilton Gibson wrote: > Hi Pieter, > > Will this require an external account again? > External a counts can be very problematic with institutions that have very > strict campus firewalls. > > Cheers > > Hg > > On Wed, 11 Nov 2015 18:22 Peter Dietz wrote: > >> Hi All, >> >> I don't have a finished solution, but we have a work-in-progress for >> recaptcha for the item-request form. >> https://github.com/LongsightGroup/DSpace/commits/longsight-4_x-recaptcha >> >> The easy part was just adding the recaptcha javascript widget. Getting >> the server-side to validate the data was not that difficult. But, some >> todo's left on that would be to make the Google key configurable, and to >> make it so that the g-recaptcha was only present for the item-request form, >> where as the XSL only let you apply it to all forms, so search form, login >> form, edit item form, all had a captcha button. >> >> It would be really great to get this finished, because we hear a lot >> about users that are essentially getting spammed through this... >> >> >> >> Peter Dietz >> Longsight >> www.longsight.com >> pe...@longsight.com >> p: 740-599-5005 x809 >> >> On Tue, Oct 27, 2015 at 11:00 PM, Olivier Nicole < >> olivier.nic...@cs.ait.ac.th> wrote: >> >>> Hi Leandro, >>> >>> > you can configure the option to only allow registered people to send >>> the >>> > request form. >>> > in dspace.cfg you have to erase the following line >>> > >>> > request.item.type = all >>> >>> That would not work for me as registered users have direct access to the >>> documents. Only external users need to fill the request form. >>> >>> > I have the same problem in this days. I figured out how to put >>> reCaptcha in >>> > my site, but its a lot of code and it was too messy. >>> > >>> > https://www.google.com/recaptcha/admin >>> > >>> > this is an example of the site: >>> > >>> > >>> http://ri.conicet.gov.ar/handle/11336/2568/restricted-resource?bitstreamId=9552 >>> >>> That's exactly what I am looking for. Is there a possibility that you >>> share the code? >>> >>> Best regards, >>> >>> Olivier >>> >>> > >>> > >>> > >>> > >>> > >>> > El martes, 27 de octubre de 2015, 3:55:32 (UTC-3), Olivier Nicole >>> escribió: >>> >> >>> >> Hi, >>> >> >>> >> My users complain about the increasing number of spam sent through the >>> >> page for requesting the copy of a document. >>> >> >>> >> So far I could not find any implementation of captcha for that page. >>> >> >>> >> Does anything exists at all? >>> >> >>> >> TIA, >>> >> >>> >> Olivier >>> >> -- >>> >> >>> >>> -- >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "DSpace Technical Support" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to dspace-tech+unsubscr...@googlegroups.com. >>> To post to this group, send email to dspace-tech@googlegroups.com. >>> Visit this group at http://groups.goo
Re: [dspace-tech] Re: Captcha in XMLUI
Hi All, I don't have a finished solution, but we have a work-in-progress for recaptcha for the item-request form. https://github.com/LongsightGroup/DSpace/commits/longsight-4_x-recaptcha The easy part was just adding the recaptcha javascript widget. Getting the server-side to validate the data was not that difficult. But, some todo's left on that would be to make the Google key configurable, and to make it so that the g-recaptcha was only present for the item-request form, where as the XSL only let you apply it to all forms, so search form, login form, edit item form, all had a captcha button. It would be really great to get this finished, because we hear a lot about users that are essentially getting spammed through this... ________ Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Tue, Oct 27, 2015 at 11:00 PM, Olivier Nicole < olivier.nic...@cs.ait.ac.th> wrote: > Hi Leandro, > > > you can configure the option to only allow registered people to send the > > request form. > > in dspace.cfg you have to erase the following line > > > > request.item.type = all > > That would not work for me as registered users have direct access to the > documents. Only external users need to fill the request form. > > > I have the same problem in this days. I figured out how to put reCaptcha > in > > my site, but its a lot of code and it was too messy. > > > > https://www.google.com/recaptcha/admin > > > > this is an example of the site: > > > > > http://ri.conicet.gov.ar/handle/11336/2568/restricted-resource?bitstreamId=9552 > > That's exactly what I am looking for. Is there a possibility that you > share the code? > > Best regards, > > Olivier > > > > > > > > > > > > > El martes, 27 de octubre de 2015, 3:55:32 (UTC-3), Olivier Nicole > escribió: > >> > >> Hi, > >> > >> My users complain about the increasing number of spam sent through the > >> page for requesting the copy of a document. > >> > >> So far I could not find any implementation of captcha for that page. > >> > >> Does anything exists at all? > >> > >> TIA, > >> > >> Olivier > >> -- > >> > > -- > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Has anyone migrated from Digital Commons to DSpace?
Hi Ryan, I don't have any specific advice, since I have never seen any published information about what format BePress will export your contents to. But, DSpace has several importing routes. One challenge might be in recreating your site structure, of hierarchy. DSpace uses the terms Community and Collection as sort of a folder, sub-folder concept of organization, and things that hold metadata and files are called Items. This could either be generated manually, through command-line structure-builder <https://wiki.duraspace.org/display/DSDOC18/Importing+Community+and+Collection+Hierarchy>, or programmatically via REST API <https://wiki.duraspace.org/display/DSDOC5x/REST+API>. And then importing all of your digital objects. Depending on what your data will look like, a process for batch importing is to get all of your files (PDF's) into a folder, and have a spreadsheet with all of the filenames in one column, and then other columns holding metadata. You can then run that through a tool like SAFBuilder <https://github.com/DSpace-Labs/SAFBuilder> to generate the packaging for DSpace. You could then either import this command line through "import", or through your browser via Batch Import ZIP. I would imagine that someone else has accomplished this before. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Thu, Nov 5, 2015 at 11:40 AM, Ryan Steans wrote: > Hi all, > > We're being asked to consider migrating a Digital Commons/ bepress site to > DSpace. I was wondering if anyone else had successfully completed this > task, or if you'd tried and couldn't get it to work. > > best regards, > > Ryan Steans > TDL.org > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] migrated content now can't create new communities
Hi Chris, Glad that worked out for you. Personally, I wouldn't use the AIP route. Something will be lost in translation. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Thu, Nov 5, 2015 at 10:45 AM, Chris Gray wrote: > Thanks, Peter, > > I recreated the database with pg_dump and pg_restore and that seems to > have solved the problem. > > On to the next gotcha! > > On Thursday, November 5, 2015 at 12:12:32 AM UTC-5, Peter Dietz wrote: >> >> Hi Chris, >> >> By migrate to a new server, I'm wondering what your migration process >> was. Perhaps there was some existing data on the database you restored on >> top of??? >> >> You should have success with a pg_dump on the old server, and then a >> pg_restore on the new server, in a fresh/clean database. The state of >> things would be almost unchanged according to the DB. I like the custom / >> binary format -Fc. You can use pgadmin3, if the ports are open, to make it >> easier. That way you ensure you don't miss any sequences, or indexes. >> >> I don't know if it would be possible, or wise, to somehow drop indexes, >> and re-add them >> >> Another possible thing to look into is: >> https://wiki.postgresql.org/wiki/Fixing_Sequences >> (Caveat, at own risk, backup before trying, ..., untested) >> >> >> >> Peter Dietz >> Longsight >> www.longsight.com >> pe...@longsight.com >> p: 740-599-5005 x809 >> >> On Wed, Nov 4, 2015 at 11:05 PM, Andrea Schweer >> wrote: >> >>> On 05/11/15 16:25, Chris Gray wrote: >>> >>> Yes. I updated the sequences when I finished the ingest and then again >>> when this error first appeared, but it hasn't helped. >>> >>> >>> Fair enough. Is there anything useful in the DSpace log? Or a longer >>> stack trace shown to you on the page? >>> >>> cheers, >>> Andrea >>> >>> >>> On Wednesday, November 4, 2015 at 9:11:50 PM UTC-5, Andrea Schweer >>> wrote: >>>> >>>> Hi, >>>> >>>> On 05/11/15 14:53, Chris Gray wrote: >>>> >>>> We've migrated our content to a new server. Now we can't create new >>>> communities. >>>> >>>> The error is: >>>> >>>> ERROR: duplicate key value violates unique constraint "handle_pkey" >>>> Detail: Key (handle_id)=(9863) already exists. >>>> >>>> >>>> The number goes up by one with each attempt. >>>> >>>> >>>> Have you tried the updating the db sequences? >>>> >>>> >>>> https://wiki.duraspace.org/display/DSDOC5x/Storage+Layer#StorageLayer-MaintenanceandBackup >>>> (see the final bit in the "Maintenance and Backup" section) >>>> >>>> cheers, >>>> Andrea >>>> >>>> -- >>>> Dr Andrea Schweer >>>> IRR Technical Specialist, ITS Information Systems >>>> The University of Waikato, Hamilton, New Zealand+64-7-837 9120 >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "DSpace Technical Support" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to dspace-tech...@googlegroups.com. >>> To post to this group, send email to dspac...@googlegroups.com. >>> Visit this group at http://groups.google.com/group/dspace-tech. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >>> Dr Andrea Schweer >>> IRR Technical Specialist, ITS Information Systems >>> The University of Waikato, Hamilton, New Zealand+64-7-837 9120 >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "DSpace Technical Support" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to dspace-tech...@googlegroups.com. >>> To post to this group, send email to dspac...@googlegroups.com. >>> Visit this group at http://groups.google.com/group/dspace-tech. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] migrated content now can't create new communities
Hi Chris, By migrate to a new server, I'm wondering what your migration process was. Perhaps there was some existing data on the database you restored on top of??? You should have success with a pg_dump on the old server, and then a pg_restore on the new server, in a fresh/clean database. The state of things would be almost unchanged according to the DB. I like the custom / binary format -Fc. You can use pgadmin3, if the ports are open, to make it easier. That way you ensure you don't miss any sequences, or indexes. I don't know if it would be possible, or wise, to somehow drop indexes, and re-add them Another possible thing to look into is: https://wiki.postgresql.org/wiki/Fixing_Sequences (Caveat, at own risk, backup before trying, ..., untested) ________ Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Wed, Nov 4, 2015 at 11:05 PM, Andrea Schweer wrote: > On 05/11/15 16:25, Chris Gray wrote: > > Yes. I updated the sequences when I finished the ingest and then again > when this error first appeared, but it hasn't helped. > > > Fair enough. Is there anything useful in the DSpace log? Or a longer stack > trace shown to you on the page? > > cheers, > Andrea > > > On Wednesday, November 4, 2015 at 9:11:50 PM UTC-5, Andrea Schweer wrote: >> >> Hi, >> >> On 05/11/15 14:53, Chris Gray wrote: >> >> We've migrated our content to a new server. Now we can't create new >> communities. >> >> The error is: >> >> ERROR: duplicate key value violates unique constraint "handle_pkey" >> Detail: Key (handle_id)=(9863) already exists. >> >> >> The number goes up by one with each attempt. >> >> >> Have you tried the updating the db sequences? >> >> >> https://wiki.duraspace.org/display/DSDOC5x/Storage+Layer#StorageLayer-MaintenanceandBackup >> (see the final bit in the "Maintenance and Backup" section) >> >> cheers, >> Andrea >> >> -- >> Dr Andrea Schweer >> IRR Technical Specialist, ITS Information Systems >> The University of Waikato, Hamilton, New Zealand+64-7-837 9120 >> >> -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > > > -- > Dr Andrea Schweer > IRR Technical Specialist, ITS Information Systems > The University of Waikato, Hamilton, New Zealand+64-7-837 9120 > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] CAS login - how to prohibit changing password
Hi Graham, Don't let me scare you away by saying that "DSpace doesn't support CAS authentication", i.e. this JIRA ticket that provides CAS Authentication provider to DSpace has not been accepted into DSpace. https://jira.duraspace.org/browse/DS-1028, it looks like there have been 5 pull requests trying to bring that feature in... I'm not familiar with those contributions, but we would love DSpace to support all the relevant authentication stacks. (Sidenote, I've been working on Spring Security SAML, and would like to provide the ability for DSpace itself to authenticate directly with SAML-based auth mechanisms, i.e. shibboleth, and ADFS, without a mod-shibb) Check to see what your CAS code is doing. i.e. dspace-api/src/main/java/org/dspace/authenticate/CASAuthentication.java (its not a part of stock DSpace). method allowSetPassword. It should "return false;" if you don't want to allow them to set the password. For example, ShibAuthentication doesn't allow changing the password. https://github.com/DSpace/DSpace/blob/387ee1c1af4ce26656e8ca585224390a358bb760/dspace-api/src/main/java/org/dspace/authenticate/ShibAuthentication.java#L407-L410 It looks like XMLUI shouldn't show the form to change their password if that is that way. https://github.com/DSpace/DSpace/blob/387ee1c1af4ce26656e8ca585224390a358bb760/dspace-xmlui/src/main/java/org/dspace/app/xmlui/aspect/eperson/EditProfile.java#L401 However If you have multiple authentication methods enabled, such as CASAuthentication and PasswordAuthentication, if passwordauthentication allows the user to change their password, then it will show the user as being allowed to set their password. (But, each time they log in, CAS with have already approved them with their campus credentials). In this event, you'll probably want to edit xmlui / aspect/eperson / EditProfile.java, and just yank out all the allowSetPassword section. Or perhaps CSS to hide it? Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Thu, Oct 29, 2015 at 8:12 PM, Graham Faulkner < graham.faulk...@uwaterloo.ca> wrote: > Hi all, > > > > On our newest staging server (DSpace 5.3 with Mirage 2) we have CAS > authentication implemented (with a CAS login URL oddity that I just posted > to the list). > > > > Once the CAS user logins successfully and goes to their Profile page, it > shows the Security section with the form for changing the password. I am > not sure why it is showing at all since in dspace/config/dspace.cfg I have > set the following values: > > > > xmlui.user.registration = false > > xmlui.user.editmetadata = false > > > > Thankfully it does not in fact allow the user to change their password as > passwords are handled by the CAS system. However, I would like the option > removed from the interface so as to avoid confusion. > > > > As pointed out in a previous email, this newest staging server is using > Apache as a reverse proxy alongside Tomcat. On our other development > server that only uses Tomcat I have the above two lines set to false as > well in dspace.cfg and it does not show the Security section, as expected. > > > > Any pointers in the right direction are greatly appreciated. > > > > Cheers, > > > > Graham > > > > - > > Graham Faulkner > > Web Developer / Programmer > > Digital Initiatives, Library > > University of Waterloo > > Waterloo, Ontario N2L 3G1 CANADA > > 519-888-4567 x32461 > > graham.faulk...@uwaterloo.ca > > > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] REST API search operation
Anja LeBlanc built search functionality for REST API, but it never got integrated. It only needed minor cleanup at the time (it had direct DB query inside REST), but functionally, it fit the bill. https://github.com/DSpace/DSpace/pull/556 I'll take a look at this. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Fri, Oct 23, 2015 at 9:57 AM, Germán Biozzoli wrote: > Thank you very much Iván > Please correct me, but as far as I understand the OpenSearch for XMLUI is > not broken, it changes the URL for > > /open-search/discovery > > The old open-search is based on lucene > Isn't it? > > Regards & thanks a lot > Germán > > > 2015-10-23 10:49 GMT-03:00 helix84 : > >> On Fri, Oct 23, 2015 at 3:24 PM, Germán Biozzoli < >> germanbiozz...@gmail.com> wrote: >> >>> I'm asking if is there a plan to implement search service over REST API >>> and if is there any roadmap. In the meantime, could I suppose that >>> OpenSearch could offer some of this service? >>> >> >> As of 5.x, your options are: >> * OpenSearch (broken in XMUI) >> >> http://dspace.2283337.n4.nabble.com/How-to-create-query-URL-that-will-dynamically-compose-RSS-feed-URL-td4679543.html >> * Solr (doesn't have a read-only interface; fine for server-to-server, >> but needs additional security solution for client-to-server) >> * REST find-by-metadata - not fulltext search >> >> Pick your poison. >> >> There are no formal plans for the development of REST API, but someone >> submits a feature, it will be considered. A search endpoint in REST API >> would be a wrapper on top of Discovery, which is implemented by Solr. >> Essentially the same as using Solr directly with a frontend to make it >> read-only. >> >> >> Regards, >> ~~helix84 >> >> Compulsory reading: DSpace Mailing List Etiquette >> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >> >> > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] item view on mirage 2
Hi Massimiliano, If you are using Longsight's modification of Mirage2 (static, non-bower / non-grunt / non-npm) version, we have also customized metadata display, and have it reading from https://github.com/LongsightGroup/DSpace/blob/longsight-5_x/dspace-xmlui/src/main/webapp/i18n/messages.xml#L2261 Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Tue, Oct 13, 2015 at 6:01 AM, Massimiliano CILURZO wrote: > Hi all, > I have a little question about simple/full item view. > I'm using DSPACE 5.3 with mirage 2 as theme. > In the "simple view" I see for example > > xmlui.metadata.dc.contributor.author john doe, > xmului.metadata.dc > > I'd like to change it to have: > > authorjohn doe > title.. > > What I can do to customize simple/full item view? > > Thanks > Best regards > Massimiliano Cilurzo > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] dspace rest 5.x configuration
Hi Monika,
You'll have to edit all of the *Resource.java in REST.
@Produces({"application/json;qs=0.5", "application/xml;qs=1"})
That will show XML as being the API's preferred response format, if the
client doesn't send a header specifying their preferred format.
It looks like lots of RESTafarians, and perhaps just Jersey defaults are to
make the client specify a header of preferred response format. Imagine
you've got smart systems that can interchange between various formats:
JSON, XML, JSONB (binary json), Google Protobuf, ... Where some binary
transmission is the most efficient, and would be preferred if everyone
speaks that.
But, unfortunately, it means that some languages stumble on setting
headers. (I think a simple javascript AJAX call gets much more complicated
when having to set a header).
I think it would be best to figure this out, how to set a format header, or
get the PHP Drupal module to be equally capable of parsing either format,
especially if the plan is to distribute this widely with many users. Users
want the plugin to be compatible with out-of-the-box DSpace.
Peter Dietz
Longsight
www.longsight.com
pe...@longsight.com
p: 740-599-5005 x809
On Fri, Oct 16, 2015 at 11:10 AM, Monika Mevenkamp
wrote:
> The Drupal developer group here at Princeton would like to use the REST
> interface to integrate with their sites. The Drupal module we have deals
> with XML. It can of cause send a header along telling the REST interface
> to produce XML. If I can configure the REST API to return XML instead of
> Json by default - we’d like to do that.
>
> So can I configured the default return format to be XML instead JSON ?
>
> Monika
>
>
> Monika Mevenkamp
> mo.me...@gmail.com
>
> http://mo-meven.tumblr.com/
> http://mcmprogramming.com/mo.meven/
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "DSpace Technical Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dspace-tech+unsubscr...@googlegroups.com.
> To post to this group, send email to dspace-tech@googlegroups.com.
> Visit this group at http://groups.google.com/group/dspace-tech.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at http://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] How to Add a Language Switch Over in UI
Dear Mansoor Ali, In dspace.cfg set: webui.supported.locales = en, ar And then restart tomcat. This feature is available since DSpace 4. It will give you the ability to change the selected language. You might need to clear XMLUI's cache after making this change. Login -> Control Panel -> Java Information -> Clear Cache This is this feature: https://jira.duraspace.org/browse/DS-842 ________ Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Wed, Oct 7, 2015 at 6:27 AM, Mansoor Ali wrote: > Hai all, > > Iam willing to switch between the Languages English and Arabic by UI and i > have tried to resolve but i didn't get the perfect result because it was > displaying the languages on UI so as i click on it then it was not > responding,i have attached that file along with this mail and the red arrow > in it is indicating about the language switch over. > Please if any body have an idea about this then kindly suggest me. > > Any help is appreciated here > > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
[dspace-tech] SOLVED: Can't create EPerson after potentially botched 4 to 5 upgrade
Hi All, Just writing a note in case anyone runs into this issue. I don't know the root cause of what happened, the logs were 90 days ago, and we don't keep that far back. Symptom, can't create a new eperson. java.lang.NullPointerException at org.dspace.eperson.EPerson.searchResultCount(EPerson.java:438) at org.dspace.app.xmlui.aspect.administrative.eperson.ManageEPeopleMain.addBody(ManageEPeopleMain.java:118) Further digging shows that there is no metadata schema registry for the new registries in DSpace 5. dcterms, eperson, workflow. Also, I noticed that select * from metadatavalue where metadata_field_id is null; Had results (32 rows, I feel lucky it wasn't in the thousands, only a few users in this repository) and resource_type_id was all 7, which is EPERSON. So the new metadata registries didn't get created, and the eperson metadata did get moved from the eperson table to metadata4all's metadatavalue. Checking Flyway, showed that the database migrations were all successful. I fixed this by manually running a job to create the metadata registries. bin/dspace registry-loader -metadata config/registries/dcterms-types.xml bin/dspace registry-loader -metadata config/registries/eperson-types.xml bin/dspace registry-loader -metadata config/registries/sword-metadata.xml bin/dspace registry-loader -metadata config/registries/workflow-types.xml I then used navicat postgres to query all of these null metadata values, and then navicat is REALLY nice in that you can just click a value, change it, and it will automatically update. (Maybe phpmyadmin has some nice thing like that too, but different language, different tool). select * from metadatavalue where metadata_field_id is null order by resource_id asc; SELECT metadata_field_id, element FROM public.metadataschemaregistry, public.metadatafieldregistry WHERE metadataschemaregistry.metadata_schema_id = metadatafieldregistry.metadata_schema_id AND metadataschemaregistry.short_id = 'eperson'; 127;"firstname" 128;"lastname" 129;"phone" 130;"language" Then you'll have to set things that look like "Peter" to have metadata_field_id = 127 "Dietz" to 128, "en" to 130, and phone numbers to 129. I don't have the old upgrade logs to see what caused this. If there perhaps was an error thrown during the upgrade. There's a small chance this instance had some custom changes to its data model, I can't recall. But, just posting this on dspace-tech, so that someone can eventually find it in their search results. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Mirage2: sidebar on the left
Hi Marie-Hélène, You should be able to just swap those two, and then the sidebar appears on the left. We have done that for: http://dc.statelibrary.sc.gov/ At Longsight, we use a customized variant of mirage2 that doesn't require recompiling: https://github.com/LongsightGroup/DSpace/blob/longsight-5_x/dspace-xmlui/src/main/webapp/themes/scgov-mirage2/xsl/core/page-structure.xsl#L98-L103 In our process I did not run into your issue of intermediate widths rescaling. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Mon, Sep 28, 2015 at 12:45 PM, wrote: > Dear all, > > I'm trying to figure out how to properly swap the sidebar (cointaining > dri:options) from right to left starting from the original Mirage2 layout > (v. 5.3). I slightly changed the page-structure.xsl layout in order to have > something like: > > > > > > > * role="navigation">* > > instead of > > > > > > role="navigation"> > > and haven't touched the *.scss files yet for this specific change. > > but that's not quite it it's not rescaling/floating properly for > intermediate widths. I'm trying to understand what > http://getbootstrap.com/css/#grid-responsive-resets says about that but > must say I would gladly appreciate help on this one as I'm not familiar > with bootstrap. Anyone has done that on their DSpace site? > > Thanks > > Marie-Hélène Vézina > Université de Montréal > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Dspace Packager Command
Hmm. There is always the REST API. If you have some sort of existing structure information, you could built a tool that sends lots of requests that create this structure in your DSpace via REST. Some bash scripts, that for example create communities and collections, using REST. https://github.com/DSpace-Labs/dspace-rest-requests I would probably build this in a friendly language, such as python/php/java, rather than pure bash, as you would likely need to add programming logic. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Mon, Sep 28, 2015 at 10:50 AM, Arthur Sady Cordeiro Rossetti < arthurs...@gmail.com> wrote: > The problem is, Im not only importing the items, I need to catalogue them > into collections named acording to their theme I have more than 19.000 > items to import and between those I have no idea how many themes and > therefore how many collections I will need to put under specific > structures, I am going to use the Item import command in a later step, but > first I need to create the structure of collections. If it was for me I > would do it diferently, but its a request from the university library so I > need to find a way to insert those collections. I am trying to find a way > where I don't need to directly insert data in the sql database, because > after seeing the map it seems quite easy to forget some relation between > tables and make things not work properly. > > Thats why Im trying the packager command, at first I thought the > StructBuilder would solve my problem, and if it wern't for the fact it > doesn't recognize already existing structures, it really would. > > 2015-09-28 11:21 GMT-03:00 Peter Dietz : > >> Hi Arthur, >> >> If all you are doing is batch importing lots of items into an existing >> collection, or collections, then batch import is probably the easier tool. >> No mets needed. All you need is a folder of files, and a spreadsheet, that >> you add metadata into columns and rows. >> >> https://github.com/DSpace-Labs/SAFBuilder >> >> And another option is from Terry at Georgetown: >> http://georgetown-university-libraries.github.io/batch-tools/ >> >> SAFBuilder is a command line tool. I've just started working on building >> a web frontend for it, to eventually make it easier for users to get >> started with. >> >> >> Peter Dietz >> Longsight >> www.longsight.com >> pe...@longsight.com >> p: 740-599-5005 x809 >> >> On Mon, Sep 28, 2015 at 10:05 AM, Arthur Sady Cordeiro Rossetti < >> arthurs...@gmail.com> wrote: >> >>> Hi guys, I want to create an empty collection in the my Dspace instance >>> through the command line to be able to import a huge amount of data under >>> an existing structure instead of having to do it manually (I tried the >>> StructBuilder command but it doesn't recognize an existing structure, it >>> creates a new one entirely) >>> >>> The problem for me is, I don't know how to create a METS.xml file that >>> would indicate an empty collection, anyone could give me a hand? >>> >>> I tried using an empty file, but it (as it should) failed. >>> >>> Thanks in advance for the attention >>> Best Regards >>> -- >>> Arthur Sady C. Rossetti >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "DSpace Technical Support" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to dspace-tech+unsubscr...@googlegroups.com. >>> To post to this group, send email to dspace-tech@googlegroups.com. >>> Visit this group at http://groups.google.com/group/dspace-tech. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> > > > -- > Arthur Sady C. Rossetti > > > > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Dspace Packager Command
Hi Arthur, If all you are doing is batch importing lots of items into an existing collection, or collections, then batch import is probably the easier tool. No mets needed. All you need is a folder of files, and a spreadsheet, that you add metadata into columns and rows. https://github.com/DSpace-Labs/SAFBuilder And another option is from Terry at Georgetown: http://georgetown-university-libraries.github.io/batch-tools/ SAFBuilder is a command line tool. I've just started working on building a web frontend for it, to eventually make it easier for users to get started with. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Mon, Sep 28, 2015 at 10:05 AM, Arthur Sady Cordeiro Rossetti < arthurs...@gmail.com> wrote: > Hi guys, I want to create an empty collection in the my Dspace instance > through the command line to be able to import a huge amount of data under > an existing structure instead of having to do it manually (I tried the > StructBuilder command but it doesn't recognize an existing structure, it > creates a new one entirely) > > The problem for me is, I don't know how to create a METS.xml file that > would indicate an empty collection, anyone could give me a hand? > > I tried using an empty file, but it (as it should) failed. > > Thanks in advance for the attention > Best Regards > -- > Arthur Sady C. Rossetti > > > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
Re: [dspace-tech] Difference between jspui and xmlui
Hi Natalia, For somewhat recent information on JSPUI, be sure to look at the slides Cineca presented at OR14 in Helsinki. http://www.doria.fi/bitstream/handle/10024/97721/JSPUI-reborn.pdf?sequence=1 It is probably best to just look at all the slides, but they provide their opinion on page 92/93. Basically, both UI's are supported by a faction of committers and the community. They don't have full parity of features (one might have something the other doesn't), but are pretty close. To an end user or administrator they have similar feature sets. JSPUI has made some very impressive improvements in the past year or so. It has a lot of administrative buttons, and its Batch Import from the UI works better as it runs as a background thread. But some statistics viewers aren't available in JSPUI. XMLUI, for developers, relies on cocoon, pipelines, and xslt transformations, which has a pretty steep learning curve. To change a piece of content on a page will involve checking a dozen possible generators, sitemaps, xsl stylesheets, and then you might still be limited, unless you want to tackle a big project. XMLUI seems to have the most currently active user base at the moment. To an end user or administrator it has a lot of features. JSPUI, for developers, uses JSPs, which mix html and java into the view. It could be better arranged such that less Java logic happened in the views. However, JSPUI is much much simpler to work with than XMLUI. Not relevant at the current moment, but the DSpace developer community is looking at plans for a "next UI" that picks a best new technology framework to build a new user interface from. One that can hopefully replace both xmlui and jspui, and have users adopt that new UI in the future, and not have the decision to make about which UI, or splitting the user community. There are some prototype UI's built on top of the REST API. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Thu, Sep 17, 2015 at 5:50 PM, Natalia Queiroz wrote: > Hello Goup, > > i'm new on Dspace and I could not understand the difference between jspui > and xmlui web applicaton. > > Are they the samething? > > Regards, > Natália > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dspace-tech+unsubscr...@googlegroups.com. > To post to this group, send email to dspace-tech@googlegroups.com. > Visit this group at http://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
