Re: [Dspace-tech] Problems with JPEG media filter and Branded
David, I'm able to generate thumbnails for images that are hundreds of MB and my max memory is also set to 2048, so your images should be able to go without a problem. What version of DSpace and Java are you using? Is it Oracle or OpenJDK? Try disabling thumbnail.blurring and thumbnail.hqscaling in dspace.cfg one at a time to get a start on where the issue is. On Mon, May 19, 2014 at 3:43 PM, Schuster, David dschus...@mail.twu.eduwrote: Are there limits to the size of the JPEG that the filter will create thumbnails of? Our Marketing department provided me some files, but they are pretty big – 10 MB and when I run media filter on it I get ./dspace filter-media -i 11274/446 -p JPEG Thumbnail -v The following MediaFilters are enabled: Full Filter Name: org.dspace.app.mediafilter.JPEGFilter org.dspace.app.mediafilter.JPEGFilter original size: 6016.0,4016.0 x scale factor: 0.013297873 new size: 80.0,53.404255 created thumbnail size: 80.0, 53.404255 Exception: Java heap space java.lang.OutOfMemoryError: Java heap space at java.awt.image.DataBufferInt.init(DataBufferInt.java:75) at java.awt.image.SinglePixelPackedSampleModel.createDataBuffer(SinglePixelPackedSampleModel.java:242) at java.awt.image.Raster.createWritableRaster(Raster.java:941) at java.awt.image.BufferedImage.getData(BufferedImage.java:1427) at java.awt.image.ConvolveOp.createCompatibleDestImage(ConvolveOp.java:280) at java.awt.image.ConvolveOp.filter(ConvolveOp.java:177) at org.dspace.app.mediafilter.JPEGFilter.getBlurredInstance(JPEGFilter.java:226) at org.dspace.app.mediafilter.JPEGFilter.getDestinationStream(JPEGFilter.java:148) at org.dspace.app.mediafilter.MediaFilterManager.processBitstream(MediaFilterManager.java:737) at org.dspace.app.mediafilter.MediaFilterManager.filterBitstream(MediaFilterManager.java:561) at org.dspace.app.mediafilter.MediaFilterManager.filterItem(MediaFilterManager.java:511) at org.dspace.app.mediafilter.MediaFilterManager.applyFiltersItem(MediaFilterManager.java:479) at org.dspace.app.mediafilter.MediaFilterManager.main(MediaFilterManager.java:353) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:225) at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:77) If I resize the file to 69 KB it runs the filter without a problem. My memory is set to 2048 max do I need to increase that and what are the proportions? David Schuster Texas Woman's University Director of Library Information Technology Technical Support Phone: 940-898-3909 dschus...@twu.edu [image: Description: emailsignature] -- Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available Simple to use. Nothing to install. Get started now for free. http://p.sf.net/sfu/SauceLabs ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Jason Sherman Systems Librarian University of Science and Arts of Oklahoma 405.574.1340 -- Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available Simple to use. Nothing to install. Get started now for free. http://p.sf.net/sfu/SauceLabs___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] DSpace becomes unresponsive, and I'm out of ideas
Steve, I've previously dealt with performance penalties with particular combos of Java and Tomcat versions. Was your Java or Tomcat recently updated? What version of Java are you using? What exact version of Tomcat are you on? eg. 7.x.x What is your logging level set to in log4j.properties? On Wed, Apr 2, 2014 at 8:24 PM, Steve Thomas stephen.tho...@adelaide.edu.au wrote: We're having continuing performance problems with our DSpace server. We're running version 1.6.2 on RHEL 4, so, yeah, everything is old. But service has been reliable, mostly, until recently. The symptom is that DSpace will become unresponsive at the browser. Checking the server, we'll typically find that the java process is running at 170% cpu, so my guess is a runaway process, which explains the slow/no response. But why? Checking the logs, both dspace.log and catalina.out, gives no real clue. dspace.log is typically full of broken pipe errors, but these are,I guess, another symptom. We also get broken pipe errors through the day, even when DSpace is running normally. Possibly unrelated, we also periodically find that dspace.log has filled up all available disk space -- more than 100GB! With a file that large, it's difficult to scroll through, but I've not found any obvious answers there. So I'm out of ideas here as to where to even look for cause. (And yes, I've already had the upgrade idea. Not currently an option I'm afraid.) Can anyone help with suggestions? Stephen Thomas Library Systems Manager Barr Smith Library UNIVERSITY OF ADELAIDE SA 5005 AUSTRALIA Phone: +61 8 831 35190 / Mobile: 0402 069 087 / Fax: +61 8 831 34369 Email: stephen.tho...@adelaide.edu.au CRICOS Provider Number 00123M *Please consider the environment before printing.* IMPORTANT: This message may contain confidential or legally privileged information. If you think it was sent to you by mistake, please delete all copies and advise the sender. For the purposes of the SPAM Act 2003, this email is authorised by The University of Adelaide. -- ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Jason Sherman Systems Librarian University of Science and Arts of Oklahoma 405.574.1340 -- ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
[Dspace-tech] Item ID and handle mismatch issue
I recently moved to DSpace 4.0 from 3.x. I set up an empty Dspace 4.0 system, did an AIP backup of the whole 3.x site and then restored the site into the new system. It's been fine for the most part, but when adding an item recently, I got a error at the end of the submission process, saying there was a problem assigning handle x/1623, which isn't surprising since that handle is in use. Upon closer inspection, I see that the ID of my new item is 1623. It did end up getting assigned an available handle x/1631 (which is our highest assigned handle so far), but obviously there was a bump in the road on the way there. We did have a handful of deleted items on our old system, and my guess is that since those weren't brought over, that caused the item id count to be slightly smaller than the handle count. I have added an item before that without any errors, and it did fill in an unused handle in the middle without a problem. Is there anything I should look to do in the code or in the database to prevent this error from popping up again? -- Jason Sherman Systems Librarian University of Science and Arts of Oklahoma 405.574.1340 -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Item ID and handle mismatch issue
Thanks guys, sorry I apparently failed to RTFM all the way through. I should know better! I followed Helix84's advice (including shutting down tomcat), and I'm currently checking for lingering issues. On Tue, Jan 14, 2014 at 10:58 AM, helix84 heli...@centrum.sk wrote: On Jan 14, 2014 5:49 PM, Tim Donohue tdono...@duraspace.org wrote: As helix84 mentions, it won't hurt anything: Well, as long as you shut down Tomcat. https://wiki.duraspace.org/display/DSDOC4x/AIP+Backup+and+Restore#AIPBackupandRestore-RestoringEntireSite This documentation is wrong to say that you don't need to shutdown Tomcat. You do, in order to ensure that there are no new items, handles, collections, users etc. added in a transaction parallel to the one where update-sequences.sql runs. The likelihood of conflict may be small, but it easily avoidable. Updating sequences is actually a hack because sequences are meant to be transaction-safe and here we're bypassing that. -- Jason Sherman Systems Librarian University of Science and Arts of Oklahoma 405.574.1340 -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] PDF Thumbnail Resolution
Alan, You can see what I've got so far here: https://github.com/jsnshrmn/DSpace/raw/master/dspace-api/src/main/java/org/dspace/app/mediafilter/XPDF2Thumbnail.java Testing is definitely something it could use! On Tue, Nov 26, 2013 at 11:21 PM, Alan Orth alan.o...@gmail.com wrote: Jason, To clarify, you sent Scott a solution for PDF thumbnails? I'd be very interested in testing it as well. :) We're still testing XPDF and Terry's Ghostscript solution for PDF thumbnails so I'd love to compare. Cheers, Alan On 11/27/2013 02:00 AM, Jason Sherman wrote: I sent Scott a solution based of my jpeg thumbnail off-list for testing. I suppose I should have kept it on list so that everyone was on the same page. Live and learn! I can share my version if anyone is interested, but it sounds like Terry's solution does the job. On Tue, Nov 26, 2013 at 2:13 PM, Alan Orth alan.o...@gmail.com wrote: Terry, That's very interesting. I just tried your code and it works great, I compared it to the XPDF method as well as our the manual thumbnailing we've been using (embarassingly) for a few years[0]. The colors in the PDF I used as an example are a bit exaggerated, but it's not too bad and the thumbnail is much more crisp. BTW, I made a pull request[1] on your repo to add some installation / configuration instructions to your README.md. Cheers! Alan [0] http://i.imgur.com/xzH5tcL.png [1] https://github.com/Georgetown-University-Libraries/DSpaceImageMagickThumnails/pull/1 On 10/15/2013 06:34 PM, Terry Brady wrote: We had a similar need. I created a module to build thumbnails with ImageMagick and Ghost Script. If this sounds useful, you can see the code here. https://github.com/Georgetown-University-Libraries/DSpaceImageMagickThumnails Terry On Tue, Oct 15, 2013 at 11:08 AM, RENTON Scott scott.ren...@ed.ac.uk wrote: Thanks folks. I've emailed Jason as you can probably see! Scott Scott Renton Systems Developer, Library University Collections 24 Buccleuch Place, 2nd Floor Right tel: 515219 On 15/10/2013 16:02, RENTON Scott scott.ren...@ed.ac.uk wrote: Thanks Ivan I will do- in fact I will mail him now before I forget! Cheers Scott Scott Renton Systems Developer, Library University Collections 24 Buccleuch Place, 2nd Floor Right tel: 515219 On 15/10/2013 16:01, helix84 heli...@centrum.sk wrote: On Tue, Oct 15, 2013 at 4:31 PM, RENTON Scott scott.ren...@ed.ac.uk wrote: Yeah, that's the code I'm using. I'm not too surprised if the JPEG work is only for images- I would say the PDF thumbnails work is quite an obscure requirement. Anyway, if this is not changing, I would be happy to tell the users that this is the best they can get for the moment; they may decide not to implement, but that's up to them! Sorry about the inaccurate first answer. I see that both the fixed JPEGFilter/BrandedPreviewJPEGFilter and your XPDF2Thumbnail use the awt libraries to manipulate bitmaps. Therefore it might be possible to apply the same fixes to XPDF2Thumbnail as were applied to the JPEG filters. Try asking Jason Sherman jsher...@usao.edu about it, he's the author of the fixes and might be willing to do it for PDF, too. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135031iu=/4140/ostg.clktrk ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135031iu=/4140/ostg.clktrk ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https
Re: [Dspace-tech] PDF Thumbnail Resolution
Okay, I went ahead and created a pull request and a jira issue: https://jira.duraspace.org/browse/DS-1812 https://github.com/DSpace/DSpace/pull/409 On Wed, Nov 27, 2013 at 9:19 AM, Jason Sherman jsher...@usao.edu wrote: Alan, You can see what I've got so far here: https://github.com/jsnshrmn/DSpace/raw/master/dspace-api/src/main/java/org/dspace/app/mediafilter/XPDF2Thumbnail.java Testing is definitely something it could use! On Tue, Nov 26, 2013 at 11:21 PM, Alan Orth alan.o...@gmail.com wrote: Jason, To clarify, you sent Scott a solution for PDF thumbnails? I'd be very interested in testing it as well. :) We're still testing XPDF and Terry's Ghostscript solution for PDF thumbnails so I'd love to compare. Cheers, Alan On 11/27/2013 02:00 AM, Jason Sherman wrote: I sent Scott a solution based of my jpeg thumbnail off-list for testing. I suppose I should have kept it on list so that everyone was on the same page. Live and learn! I can share my version if anyone is interested, but it sounds like Terry's solution does the job. On Tue, Nov 26, 2013 at 2:13 PM, Alan Orth alan.o...@gmail.com wrote: Terry, That's very interesting. I just tried your code and it works great, I compared it to the XPDF method as well as our the manual thumbnailing we've been using (embarassingly) for a few years[0]. The colors in the PDF I used as an example are a bit exaggerated, but it's not too bad and the thumbnail is much more crisp. BTW, I made a pull request[1] on your repo to add some installation / configuration instructions to your README.md. Cheers! Alan [0] http://i.imgur.com/xzH5tcL.png [1] https://github.com/Georgetown-University-Libraries/DSpaceImageMagickThumnails/pull/1 On 10/15/2013 06:34 PM, Terry Brady wrote: We had a similar need. I created a module to build thumbnails with ImageMagick and Ghost Script. If this sounds useful, you can see the code here. https://github.com/Georgetown-University-Libraries/DSpaceImageMagickThumnails Terry On Tue, Oct 15, 2013 at 11:08 AM, RENTON Scott scott.ren...@ed.ac.uk wrote: Thanks folks. I've emailed Jason as you can probably see! Scott Scott Renton Systems Developer, Library University Collections 24 Buccleuch Place, 2nd Floor Right tel: 515219 On 15/10/2013 16:02, RENTON Scott scott.ren...@ed.ac.uk wrote: Thanks Ivan I will do- in fact I will mail him now before I forget! Cheers Scott Scott Renton Systems Developer, Library University Collections 24 Buccleuch Place, 2nd Floor Right tel: 515219 On 15/10/2013 16:01, helix84 heli...@centrum.sk wrote: On Tue, Oct 15, 2013 at 4:31 PM, RENTON Scott scott.ren...@ed.ac.uk wrote: Yeah, that's the code I'm using. I'm not too surprised if the JPEG work is only for images- I would say the PDF thumbnails work is quite an obscure requirement. Anyway, if this is not changing, I would be happy to tell the users that this is the best they can get for the moment; they may decide not to implement, but that's up to them! Sorry about the inaccurate first answer. I see that both the fixed JPEGFilter/BrandedPreviewJPEGFilter and your XPDF2Thumbnail use the awt libraries to manipulate bitmaps. Therefore it might be possible to apply the same fixes to XPDF2Thumbnail as were applied to the JPEG filters. Try asking Jason Sherman jsher...@usao.edu about it, he's the author of the fixes and might be willing to do it for PDF, too. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135031iu=/4140/ostg.clktrk ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135031iu=/4140
Re: [Dspace-tech] PDF Thumbnail Resolution
I sent Scott a solution based of my jpeg thumbnail off-list for testing. I suppose I should have kept it on list so that everyone was on the same page. Live and learn! I can share my version if anyone is interested, but it sounds like Terry's solution does the job. On Tue, Nov 26, 2013 at 2:13 PM, Alan Orth alan.o...@gmail.com wrote: Terry, That's very interesting. I just tried your code and it works great, I compared it to the XPDF method as well as our the manual thumbnailing we've been using (embarassingly) for a few years[0]. The colors in the PDF I used as an example are a bit exaggerated, but it's not too bad and the thumbnail is much more crisp. BTW, I made a pull request[1] on your repo to add some installation / configuration instructions to your README.md. Cheers! Alan [0] http://i.imgur.com/xzH5tcL.png [1] https://github.com/Georgetown-University-Libraries/DSpaceImageMagickThumnails/pull/1 On 10/15/2013 06:34 PM, Terry Brady wrote: We had a similar need. I created a module to build thumbnails with ImageMagick and Ghost Script. If this sounds useful, you can see the code here. https://github.com/Georgetown-University-Libraries/DSpaceImageMagickThumnails Terry On Tue, Oct 15, 2013 at 11:08 AM, RENTON Scott scott.ren...@ed.ac.uk wrote: Thanks folks. I've emailed Jason as you can probably see! Scott Scott Renton Systems Developer, Library University Collections 24 Buccleuch Place, 2nd Floor Right tel: 515219 On 15/10/2013 16:02, RENTON Scott scott.ren...@ed.ac.uk wrote: Thanks Ivan I will do- in fact I will mail him now before I forget! Cheers Scott Scott Renton Systems Developer, Library University Collections 24 Buccleuch Place, 2nd Floor Right tel: 515219 On 15/10/2013 16:01, helix84 heli...@centrum.sk wrote: On Tue, Oct 15, 2013 at 4:31 PM, RENTON Scott scott.ren...@ed.ac.uk wrote: Yeah, that's the code I'm using. I'm not too surprised if the JPEG work is only for images- I would say the PDF thumbnails work is quite an obscure requirement. Anyway, if this is not changing, I would be happy to tell the users that this is the best they can get for the moment; they may decide not to implement, but that's up to them! Sorry about the inaccurate first answer. I see that both the fixed JPEGFilter/BrandedPreviewJPEGFilter and your XPDF2Thumbnail use the awt libraries to manipulate bitmaps. Therefore it might be possible to apply the same fixes to XPDF2Thumbnail as were applied to the JPEG filters. Try asking Jason Sherman jsher...@usao.edu about it, he's the author of the fixes and might be willing to do it for PDF, too. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135031iu=/4140/ostg.clktrk ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135031iu=/4140/ostg.clktrk ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Alan Orth alan.o...@gmail.com http://alaninkenya.org http://mjanja.co.ke I have always wished for my computer to be as easy to use as my telephone; my wish has come true because I can no longer figure out how to use my telephone. -Bjarne Stroustrup, inventor of C++ GPG Public Key: 0xf92c4bd91084bb5de14e20be9470dd588dd1026c -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue
Re: [Dspace-tech] Inconsistent hostname in OAI interface
I got a partial fix thanks to an off-list response from Hugh Barnes
hugh.bar...@lincoln.ac.nz. Following this fixed our identifiers, though
the contents of request tag still refer to the internal IP. See below:
We have our Dspace admin outsourced, but they just fixed some “localhost”
value instances we had in our OAI-PMH output identifiers. Feel free to
share with the list if it works and no-one else has suggested this.
So our sysadmin changed a setting in config/modules/oai.cfg:
identifier.prefix = ${dspace.hostname}
(${dspace.hostname} is apparently set in dspace.cfg to the correct string
value)
The important part you may be missing is that the change doesn’t take
immediate effect. This morning (Friday here!) a restart of Dspace happened,
presumably on a cron. Don’t think that’s an everyday occurrence. Also, I
suspect a restart isn’t strictly necessary, there’s probably a script that
will do it if restarting is an issue.
Then apparently he ran “{…..}/bin/dspace oai clean-cache” after the
restart.
It was either the reboot or the clearing of the oai cache that did the
trick, because I'd tried that value for identifier.prefix at some point in
my tinkering.
I then ran
[dspace]/bin/dspace oai import -c
and the identifiers were correct.
On Thu, Oct 10, 2013 at 2:35 PM, Jason Sherman jsher...@usao.edu wrote:
I've done quite a bit of searching and haven't quite figured out a
solution to my problem. Some info:
Our dspace server uses a reserved (NATed) address.
handle.net communication and HTTP requests are run through an
internet-accessible front end server. I've really not had any issues with
it since I set it up, but now I'm interested in using the OAI interface and
I'm running into some interesting behavior.
When I do
request?verb=ListMetadataFormats
I get a response like this:
responseDate2013-10-10T10:40:03Z/responseDate
request verb=ListMetadataFormatshttps://10.x.x.x/oai/request
/request
ListMetadataFormats
metadataFormat...
Where 10.x.x.x is the internal address of the dspace server.
When I do
request?verb=ListIdentifiersmetadataPrefix=oai_dcset=com_11076_1
I get a response like this:
responseDate2013-10-10T12:11:24Z/responseDate
request verb=ListIdentifiers metadataPrefix=oai_dc
set=com_11076_1https://10.x.x.x/oai/request/request
ListIdentifiers
header
identifieroai:dspace.library.usao.edu:11076/3/identifier
Where request is still the internal address, but identifier is what I
would expect.
When I do
request?verb=ListRecordsmetadataPrefix=oai_dcset=com_11076_1
I get a response like this:
responseDate2013-10-10T12:11:30Z/responseDate
request verb=ListRecords metadataPrefix=oai_dc set=com_11076_1
https://10.0.253.47/oai/request/request
ListRecords
record
header
identifieroai:dspace.usao.edu:11076/3/identifier
Where request is still the internal address, and the identifier refers to
the hostname of the internal server on our network. This name isn't
anywhere in our dspace configs.
I've tried various combinations of the following configuration changes:
in [dspace]/config/modules/oai.cfg
ore.authoritative.source = oai
identifier.prefix = dspace.library.usao.edu
dspace.oai.url = https://dspace.library.usao.edu/xmlui
[dspace]/config/oaicat.properties
OAIHandler.baseURL = https://dspace.library.usao.edu/oai
[dspace]/bin/dspace oai import -c
Has no effect.
--
Jason Sherman
Systems Librarian
University of Science and Arts of Oklahoma
405.574.1340
--
Jason Sherman
Systems Librarian
University of Science and Arts of Oklahoma
405.574.1340
--
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60134071iu=/4140/ostg.clktrk___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
[Dspace-tech] Inconsistent hostname in OAI interface
I've done quite a bit of searching and haven't quite figured out a solution to my problem. Some info: Our dspace server uses a reserved (NATed) address. handle.net communication and HTTP requests are run through an internet-accessible front end server. I've really not had any issues with it since I set it up, but now I'm interested in using the OAI interface and I'm running into some interesting behavior. When I do request?verb=ListMetadataFormats I get a response like this: responseDate2013-10-10T10:40:03Z/responseDate request verb=ListMetadataFormatshttps://10.x.x.x/oai/request /request ListMetadataFormats metadataFormat... Where 10.x.x.x is the internal address of the dspace server. When I do request?verb=ListIdentifiersmetadataPrefix=oai_dcset=com_11076_1 I get a response like this: responseDate2013-10-10T12:11:24Z/responseDate request verb=ListIdentifiers metadataPrefix=oai_dc set=com_11076_1https://10.x.x.x/oai/request/request ListIdentifiers header identifieroai:dspace.library.usao.edu:11076/3/identifier Where request is still the internal address, but identifier is what I would expect. When I do request?verb=ListRecordsmetadataPrefix=oai_dcset=com_11076_1 I get a response like this: responseDate2013-10-10T12:11:30Z/responseDate request verb=ListRecords metadataPrefix=oai_dc set=com_11076_1 https://10.0.253.47/oai/request/request ListRecords record header identifieroai:dspace.usao.edu:11076/3/identifier Where request is still the internal address, and the identifier refers to the hostname of the internal server on our network. This name isn't anywhere in our dspace configs. I've tried various combinations of the following configuration changes: in [dspace]/config/modules/oai.cfg ore.authoritative.source = oai identifier.prefix = dspace.library.usao.edu dspace.oai.url = https://dspace.library.usao.edu/xmlui [dspace]/config/oaicat.properties OAIHandler.baseURL = https://dspace.library.usao.edu/oai [dspace]/bin/dspace oai import -c Has no effect. -- Jason Sherman Systems Librarian University of Science and Arts of Oklahoma 405.574.1340 -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60134071iu=/4140/ostg.clktrk___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] dspace 1.8.2, xmlui, and the branded JPEG media filter
helix84, Thanks for the fast and info-filled responses. You have really helped me out! On Thu, Nov 15, 2012 at 5:09 PM, helix84 heli...@centrum.sk wrote: Ouch, this hurts my eyes. What's with the variablttln names? I'd also shorten at least $context/mets:fileSec/mets:fileGrp into another variable to make it more readable. You got me there, my code was not very readable. I took your suggestions. So is there anything you'd like to improve in this part? I added in another test to ensure that lightbox code isn't produced in the absence of a branded preview bitstream. I've attached an updated template in case anyone is interested. I'm not quite following. What are you trying to achieve? 1) Didn't your numbering rules and my matching code work? 2) I thought you made it work using the title attribute instead, so what do you want to do here? 1) Yep, it's working like a champ for this case. 2) I was hoping to make a more general-purpose solution for derivative bitstreams. If I create a new media filter, I'd like to be able to mark it as a derivative and group it with the original. That way my templates could access any derived bitstream in the same way as thumbnails. It would make adding filters and adding their bitstreams to the ui much more intuitive. Maybe this is a development issue? I'd avoid modifying Java classes unless you really need to because you'll have to remember to forward-port your changes on each upgrade. The more such changes you make, the more reluctant you'll be to upgrade, trust me. Sage advice, I'll take it. I've been tracking which files I've been messing with. I think I'll revert them and move my experiments to another server. It may make more sense to try to contribute to dspace instead of managing local patches. That's a cart before the horse, though. Did you run mvn package (or mvn clean package) in [dspace-source] or in [dspace-source]/dspace? The latter would rebuild only changes in overlays and somesuch and wouldn't care about your changes in dspace-xmlui. See quick build vs. full build here: https://wiki.duraspace.org/display/DSPACE/Rebuild+DSpace Ah, maybe this was part of my issue. I made separate quick build and clean build bash scripts based off the info in the wiki. Even for the clean build, it says to start from [dspace-source]/dspace/. I updated my clean build script to run mvn clean package from [dspace-source] instead. No need to run mvn clean package, but it may save you from some nasty surprises, so I'd run it only if something doesn't work as expected after mvn package (it's just quicker). Noted. I don't think the jars you built contained your changes (see above), but there would be another gotcha - if you went around ant and did the copying manually, you would need to copy them in 2 places - both [dspace]/lib and [dspace]/webapps/xmlui/WEB-INF/lib/ Also noted. It will be easier to link to particular line numbers here if you want to point out something specific: Good point. On line 695 of the item adapter class [https://github.com/DSpace/DSpace/blob/dspace-1_8_x/dspace-xmlui/dspace-xmlui-api/src/main/java/org/dspace/app/xmlui/objectmanager/ItemAdapter.java#L695] I made the following change - if (TEXT.equals(bundle.getName()) || THUMBNAIL.equals(bundle.getName())) + if (TEXT.equals(bundle.getName()) || THUMBNAIL.equals(bundle.getName()) || BRANDED_PREVIEW.equals(bundle.getName())) I thought that would cause isDerivedBundle to be set to true for branded preview bitstreams and thus get assigned the original bitstream's group id. It doesn't seem to alter the behavior though. I thought that I had a problem with my build process, but I made your suggested directory change and I still get the same result. Branded previews are still getting unique group ids. Ideally there would be a place to register additional bundles as derived, but again, this may be creeping into a discussion more appropriate for development. Thanks again. I was fairly stuck, and your responses helped get me unstuck. -- Jason Sherman Systems Librarian University of Science and Arts of Oklahoma 405.574.1340 !-- this is from ../dri2xhtml/General-Handler.xsl -- !-- Build a single row in the bitsreams table of the item view page -- xsl:template match=mets:file xsl:param name=context select=./ tr xsl:attribute name=class xsl:textds-table-row /xsl:text xsl:if test=(position() mod 2 = 0)even /xsl:if xsl:if test=(position() mod 2 = 1)odd /xsl:if /xsl:attribute td a xsl:attribute name=href xsl:value-of select=mets:FLocat[@LOCTYPE='URL']/@xlink:href/ /xsl:attribute xsl:attribute name=title xsl:value-of select=mets:FLocat[@LOCTYPE='URL']/@xlink:title/ /xsl:attribute xsl:choose
Re: [Dspace-tech] dspace 1.8.2, xmlui, and the branded JPEG media filter
helix84,
Thanks for the information, it was very helpful. I started out
attempting a solution similar to what you suggested, but I just
couldn't easily get it to work properly. I managed to get something
working using the xlink:title attribute. I've attached my mets:file
template and a representative mets.xml file so you can see how it
works. I'm going to add another choose/when/test in there so no
lightbox code shows up unless there is definitely a preview bitstream,
but it works for my immediate needs.
As for the larger issue of branded previews not being linked to the
original bitstreams, I've had less success. This doesn't appear to be
something that handled by the media filter itself, rather it seems to
be handled by the xmlui api. For instance the ItemAdapter class
[https://svn.duraspace.org/dspace/dspace/trunk/dspace-xmlui/dspace-xmlui-api/src/main/java/org/dspace/app/xmlui/objectmanager/ItemAdapter.java]
marks THUMBNAIL and TEXT bundles as derivatives of ORIGINAL bitstreams
upon. However, this is hard coded. I tried adding in
'BRANDED_PREVIEW' to this and another class with hard-coded bundles,
and then doing a clean build and update of DSpace, but it doesn't seem
to have any effect. I also tried building new jars and dropped them
into my install, but it didn't work either. I'm not really familiar
with java development though, so I may be making a silly mistake.
On Wed, Nov 14, 2012 at 3:42 AM, helix84 heli...@centrum.sk wrote:
On Tue, Nov 13, 2012 at 9:06 PM, Jason Sherman jsher...@usao.edu wrote:
$context/mets:fileSec/mets:fileGrp[@USE='BRANDED_PREVIEW']/mets:file[@GROUPID=current()/@GROUPID]/mets:FLocat[@LOCTYPE='URL']/@xlink:href
Hi Jason,
are you using this in a template that matches match=mets:file? Send
us the whole template and the METS document you're applying it to. You
can find the METS document on your instance here (example):
/metadata/handle/123456789/100/mets.xml
CONTENT
GROUPID=group_file_12212
ID=file_12212
THUMBNAIL
GROUPID=group_file_12212
ID=file_34568
BRANDED_PREVIEW:
GROUPID=group_file_34569
ID=file_34569
This seems like it's probably a bug, but it could be somebody's
feature. What can I do to change or to work around this behavior?
OK, nice you found that out. How about doing something like this?
xsl:variable name=$thumbnail_id
select=substring($context/mets:fileSec/mets:fileGrp[@USE='THUMBNAIL']/mets:file[@GROUPID=current()/@GROUPID]/@ID,
5)/
xsl:value-of select=$context/mets:fileSec/mets:fileGrp[@USE='THUMBNAIL']/
mets:file[@GROUPID= concat('group_file_', $thumbnail_id)
]/mets:FLocat[@LOCTYPE='URL']/@xlink:href/
This is completely off the top of my head and probably contains a
million bugs, but it should give you the idea.
I found very limited documentation about creating custom media filters,
but I couldn't find how to make a copy of an existing filter and
modify it.
So you probably saw this page:
https://wiki.duraspace.org/pages/viewpage.action?pageId=27001634#TransformingDSpaceContent(MediaFilters)-CreatingCustomMediaFilters
Why not just make a copy of one of the classes here
https://github.com/DSpace/DSpace/tree/master/dspace-api/src/main/java/org/dspace/app/mediafilter
change the class name, add it to dspace.cfg and rebuild DSpace?
Shouldn't be harder than that.
Regards,
~~helix84
Compulsory reading: DSpace Mailing List Etiquette
https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
--
Jason Sherman
Systems Librarian
University of Science and Arts of Oklahoma
405.574.1340
!-- this is from ../dri2xhtml/General-Handler.xsl --
!-- Build a single row in the bitsreams table of the item view page --
xsl:template match=mets:file
xsl:param name=context select=./
tr
xsl:attribute name=class
xsl:textds-table-row /xsl:text
xsl:if test=(position() mod 2 = 0)even /xsl:if
xsl:if test=(position() mod 2 = 1)odd /xsl:if
/xsl:attribute
td
a
xsl:attribute name=href
xsl:value-of
select=mets:FLocat[@LOCTYPE='URL']/@xlink:href/
/xsl:attribute
xsl:attribute name=title
xsl:value-of
select=mets:FLocat[@LOCTYPE='URL']/@xlink:title/
/xsl:attribute
xsl:choose
xsl:when
test=string-length(mets:FLocat[@LOCTYPE='URL']/@xlink:title) 50
xsl:variable name=title_length
select=string-length(mets:FLocat[@LOCTYPE='URL']/@xlink:title)/
xsl:value-of
select=substring(mets:FLocat[@LOCTYPE='URL']/@xlink:title,1,15)/
xsl:text ... /xsl:text
xsl:value-of
select=substring(mets:FLocat[@LOCTYPE='URL']/@xlink:title,$title_length -
25,$title_length)/
/xsl:when
xsl:otherwise
[Dspace-tech] dspace 1.8.2, xmlui, and the branded JPEG media filter
Hi all, I recently got a dspace repository up and running, and I've been working on my own xmlui theme. I'm working on a lightbox-based branded preview for our images (which are mostly TIFFs). By following the online documentation, I managed to get the thumbnail and preview filters diligently creating JPEGs from our TIFF bitstreams. Heck, I've even figured out where to add in references to static files and I've done a fair amount of xsl customization. After reading in the documentation that I would need to implement xmlui previews myself, I started to do just that. However, while customizing the mets:file template I simply could not get the following xpath to match anything when it seemed like it should. I played with it for a while without success. $context/mets:fileSec/mets:fileGrp[@USE='BRANDED_PREVIEW']/mets:file[@GROUPID=current()/@GROUPID]/mets:FLocat[@LOCTYPE='URL']/@xlink:href I was able to get something if I dropped the @GROUPID limiter, however. I decided to go further up the path to see what I could find. Here's what I discovered. The original file gets a group with the same number as its file ID. The thumbnail file get its own ID number, but is put in the same group as the original file. This allows each thumbnail to be connected to the file from which it was created, not just the record. The preview filter, on the other hand, is creating a new group for each preview file. If I forcibly re-filter the record, the preview file ID and group ID increment together. After looking at a few dozen records on my system, I have concluded that each preview file belongs to a file group of exactly one member: itself. The group ID number always matches the preview file ID number. CONTENT GROUPID=group_file_12212 ID=file_12212 THUMBNAIL GROUPID=group_file_12212 ID=file_34568 BRANDED_PREVIEW: GROUPID=group_file_34569 ID=file_34569 This seems like it's probably a bug, but it could be somebody's feature. What can I do to change or to work around this behavior? I found very limited documentation about creating custom media filters, but I couldn't find how to make a copy of an existing filter and modify it. I appreciate any light you can shed on the issue, -- Jason Sherman Systems Librarian University of Science and Arts of Oklahoma 405.574.1340 -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
