Hi Jose You might want to try a command line ingest using ItemImporter or the like. I've imported a 13GB .mov file this way before with no problems.
Regards Gary Gary Browne Development Programmer Library IT Services University of Sydney ph: 9351-5946 -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Blanco, Jose Sent: Thursday, 10 January 2008 8:47 AM To: Michael Faraci; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Uploading a 10Gig File. (Blanco, Jose) The file is uncompressed. It is a mov file. The tmp dir has enough space for it. I am able to cp it there. What I have found is that if the /tmp dir does not have enough space for it, it starts to write it there and then things stop running once the dir fills up, but in this, it never tries to write it there. -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Michael Faraci Sent: Wednesday, January 09, 2008 4:36 PM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] Uploading a 10Gig File. (Blanco, Jose) Message 4. Have you checked tmp directory size. Is the file compressed (zip/tar)? I would be concerned with the uncompressed size if it is. I have never dealt with submitting a file this large, so these are just some things that come to mind. Michael Faraci Network/Systems Administrator ALC 3305 N. 3rd St. Suite 301 Abilene, Texas (325)672-7081 x4 fax (325)672-7082 -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Wednesday, January 09, 2008 11:14 AM To: dspace-tech@lists.sourceforge.net Subject: DSpace-tech Digest, Vol 21, Issue 20 Send DSpace-tech mailing list submissions to dspace-tech@lists.sourceforge.net To subscribe or unsubscribe via the World Wide Web, visit https://lists.sourceforge.net/lists/listinfo/dspace-tech or, via email, send a message with subject or body 'help' to [EMAIL PROTECTED] You can reach the person managing the list at [EMAIL PROTECTED] When replying, please edit your Subject line so it is more specific than "Re: Contents of DSpace-tech digest..." Today's Topics: 1. Re: Cannot delete collections from Dspace (Claudia J?rgen) 2. Re: Cannot delete collections from Dspace (Graham Triggs) 3. Re: Creative Commons Enigma (Richard Rodgers) 4. Uploading a 10Gig File. (Blanco, Jose) 5. Re: Getting image thumbnails (Keith Gilbertson) 6. Collection Item Mapper Search (Kyle Kaliebe) ---------------------------------------------------------------------- Message: 1 Date: Wed, 09 Jan 2008 15:02:30 +0100 From: Claudia J?rgen <[EMAIL PROTECTED]> Subject: Re: [Dspace-tech] Cannot delete collections from Dspace To: Filippos Kolovos <[EMAIL PROTECTED]> Cc: Dspace-Tech-List <dspace-tech@lists.sourceforge.net>, Dspace-general-list <[EMAIL PROTECTED]> Message-ID: <[EMAIL PROTECTED]> Content-Type: text/plain; charset=UTF-8; format=flowed Hi Filippos, are you sure you're running 1.4.2? The table "dcvalue" does not exist anymore in 1.4.2. It has been replaced in 1.4. In 1.3.2 there has been a known bug http://sourceforge.net/tracker/index.php?func=detail&aid=1333269&group_i d=19984&atid=119984 on the deletion of collections/communities containing withdrawn items. cheers Claudia Filippos Kolovos schrieb: > Dear all, > > Recently I am dealing with a problem with my Dspace installation (1.4.2). > When I try to delete a collection and/or community from Dspace the > system returns an "Internal Server Error". The explanation of this > error is shown below: > > Exception: > org.postgresql.util.PSQLException: ERROR: update or delete on "item" violates foreign key constraint "$1" on "dcvalue" > > at org.postgresql.util.PSQLException.parseServerError(PSQLException.java:13 9) > at org.postgresql.core.QueryExecutor.executeV3(QueryExecutor.java:152) > at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:100) > at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:43) > at org.postgresql.jdbc1.AbstractJdbc1Statement.execute(AbstractJdbc1Stateme nt.java:517) > at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Stateme nt.java:50) > at org.postgresql.jdbc1.AbstractJdbc1Statement.executeUpdate(AbstractJdbc1S tatement.java:273) > at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(Delega tingPreparedStatement.java:101) > at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(Delega tingPreparedStatement.java:101) > at org.dspace.storage.rdbms.DatabaseManager.updateQuery(DatabaseManager.jav a:519) > etc, etc > .......... > ......... > > The exact reason for this error seems to be that the collection id is referenced from another table, but isn't that what the > system is supposed to do when deleting a community/collection? Shouldn't all the relations be deleted and/or get handled by > the Dspace system? > > If I go manually into the Postgres and delete all the "relations" of one collection by hand then the items related to that collection remain > in the tables. I then have to do the "manual deletion" process by hand for each item, something which is not very efficient when dealing > with thousands of records (items). > > Does anyone have had this problem before? > > Kind Regards, > > -Fk > > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------ - > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for just about anything > Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketp lace > > > ------------------------------------------------------------------------ > > _______________________________________________ > DSpace-tech mailing list > DSpace-tech@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/dspace-tech ------------------------------ Message: 2 Date: Wed, 09 Jan 2008 14:03:22 +0000 From: Graham Triggs <[EMAIL PROTECTED]> Subject: Re: [Dspace-tech] Cannot delete collections from Dspace To: Filippos Kolovos <[EMAIL PROTECTED]> Cc: Dspace-Tech-List <dspace-tech@lists.sourceforge.net>, Dspace-general-list <[EMAIL PROTECTED]> Message-ID: <[EMAIL PROTECTED]> Content-Type: text/plain Filippos, When you upgraded to 1.4.x, the DCVALUE table should have been deleted after it's contents were migrated over to METADATAVALUE, and a view put in it's place. It looks like this hasn't happened, and that you still have a DCVALUE table in your database, that has it's old contents (and there is no code in 1.4.x to maintain). Check that you correctly have a METADATAVALUE table (with the correct contents), and then you should be able to safely delete the DCVALUE table (and replace it with a view - although that probably won't be necessary). Take a look at etc/database_schema_13-14.sql for more information. G On Wed, 2008-01-09 at 15:47 +0200, Filippos Kolovos wrote: > Dear all, > > Recently I am dealing with a problem with my Dspace installation > (1.4.2). > When I try to delete a collection and/or community from Dspace the > system returns an "Internal Server Error". The explanation of this > error is shown below: > > Exception: > org.postgresql.util.PSQLException: ERROR: update or delete on "item" > violates foreign key constraint "$1" on "dcvalue" > > at > org.postgresql.util.PSQLException.parseServerError(PSQLException.java:13 9) > at org.postgresql.core.QueryExecutor.executeV3(QueryExecutor.java:152) > at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:100) > at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:43) > at > org.postgresql.jdbc1.AbstractJdbc1Statement.execute(AbstractJdbc1Stateme nt.java:517) > at > org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Stateme nt.java:50) > at > org.postgresql.jdbc1.AbstractJdbc1Statement.executeUpdate(AbstractJdbc1S tatement.java:273) > at > org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(Delega tingPreparedStatement.java:101) > at > org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(Delega tingPreparedStatement.java:101) > at > org.dspace.storage.rdbms.DatabaseManager.updateQuery(DatabaseManager.jav a:519) > etc, etc > .......... > ......... > > The exact reason for this error seems to be that the collection id is > referenced from another table, but isn't that what the system is > supposed to do when deleting a community/collection? > Shouldn't all the relations be deleted and/or get handled by the > Dspace system? > > If I go manually into the Postgres and delete all the "relations" of > one collection by hand then the items related to that collection > remain in the tables. I then have to do the "manual deletion" process > by hand for each item, something which is not very efficient when > dealing with thousands of records (items). > > Does anyone have had this problem before? > > Kind Regards, > > -Fk > > -- > Filippos Kolovos > Software Systems Analyst & Engineer > M.Sc. (Eng.) in Data Communications > > Automation & Networking Department > University of Macedonia Library > Egnatia 156, P.O.Box 1591 > 540 06 Thessaloniki, Greece > > E-Mail: [EMAIL PROTECTED], > [EMAIL PROTECTED] > Profile: http://www.linkedin.com/in/filipposkolovos > ----------------------------------------------------------------------- > > > ------------------------------------------------------------------------ - > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for just about anything > Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketp lace > _______________________________________________ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech This email has been scanned by Postini. For more information please visit http://www.postini.com ------------------------------ Message: 3 Date: Wed, 09 Jan 2008 09:23:31 -0500 From: Richard Rodgers <[EMAIL PROTECTED]> Subject: Re: [Dspace-tech] Creative Commons Enigma To: Maike Dulk <[EMAIL PROTECTED]> Cc: dspace-tech <dspace-tech@lists.sourceforge.net> Message-ID: <[EMAIL PROTECTED]> Content-Type: text/plain; charset=ISO-8859-1; format="flowed" Hi Maike: A few explanations to help unravel the enigma: First, you should understand that there are two different licenses involved here, not a choice of one. The first - the deposit license - is (roughly) a licence that the depositor grants to the repository. It is not optional, and does not display to users in the item display. For many institutions, it is a legal requirement. It is a standard 'click-thru' license in the sense that the grantor can only accept or reject it (as you note). The license.default you mention contains this license text. The second - CC license - is (roughly) a license that an author grants to consumers of her work. It is entirely optional (as you note with the cc-enable property), and *does* prominently display to users in the item page. Here, the submitter *can* choose from a set of licenses that reflect the desire to control/share content. What is confusing is that they both appear in the submission work-flow (first CC, then deposit), and the the CC is a mini-workflow in itself consisting if steps in an Iframe. Many have lamented the 'brittle' Iframe, but it was the only programmatic access we had when it was first offered. Now there are 'web'service' interfaces to CC that should probably be adopted (volunteers welcome!). Hope this helps, Richard Quoting Maike Dulk <[EMAIL PROTECTED]>: > Hi > > I noticed some really strange behaviour in our DSpace installation. > > It is the Creative Commons licensing. > > When the setting in dspace.cfg is set to true > > #### Creative Commons settings ###### > # are Creative Commons licenses used in submission? > webui.submit.enable-cc = true > > the Creative Commons section of the submission process has TWO > separate steps. The first one is the one that has the iFrame that > displays the Creative Commons webpage in it, and there are two choices > that the submitter can set before accepting the license. > After that, there is another step that asks another time whether the > submitter wants to accept the Creative Commons license. This page is > much simpler and has no choices, only two buttons - either "I Grant > the License" or "I Do Not Grant the License" . > > The strange thing is that the first stage points to a version of the > CC license that is NOT the one that we want to use, and that I have > entered in the default.license config file. We want to use the > Canadian 2.5 license, but it displays - AND attaches - the generic 3.0 > CC license. > That second stage does point to the Canadian 2.5 version license - but > it is apparently discarded. > > The strangeness becomes more apparent when I do this: > > #### Creative Commons settings ###### > # are Creative Commons licenses used in submission? > webui.submit.enable-cc = false > > In that case, there is no longer the first step (the iFrame one) - but > the second step (the one with the url to the Canadian version and the > "I Grant the License" or "I Do Not Grant the License" buttons) STILL > comes up. But submissions do not have the CC license attached .. which > does make some sense, since it is disabled. > > > So I'd really like to know: > - what makes this second step appear even though the CC is disabled in > dspace.cfg? > - How can I turn it off? > - where can I make the first CC license point to the right CC license > version (Canadian 2.5) ? It is not in the default.license .. since > that is set to the Canadian one. > - if we could get the second step to work with the right CC version > that would be even better, since it is much simpler than that brittle > iFrame mechanism > > My gosh, this is *complex*. > > thanks in advance, > maike > > ---------- > Maike Dulk - Programmer / Analyst > McPherson Library, University of Victoria > (t) 250-886-5709 / (e) [EMAIL PROTECTED] > > -- Harthon gerithach aeair vilui / I hope you will have kind seas > > > ------------------------------------------------------------------------ - > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for just about anything > Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketp lace > _______________________________________________ > DSpace-tech mailing list > DSpace-tech@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/dspace-tech > ------------------------------ Message: 4 Date: Wed, 9 Jan 2008 10:30:25 -0500 From: "Blanco, Jose" <[EMAIL PROTECTED]> Subject: [Dspace-tech] Uploading a 10Gig File. To: <dspace-tech@lists.sourceforge.net> Message-ID: <[EMAIL PROTECTED] .edu> Content-Type: text/plain; charset="us-ascii" I have made the necessary changes to upload a 10 gig file. I have set: upload.max = 21001930470 I also made the code change to handle this variable being of type long. I have been successful at loading a 2.5Gig file, but when I tried to load a 10Gig file, nothing seems to happen. It doesn't hang or anything. When I hit the button "Upload" it just shows the button as being pressed, but nothing is running. I can hit other links in the page just fine to get out of the submittal page. It's as if it doesn't even try to upload the file. I don't see the file being written to the upload directory at all either. Any ideas what might be happening? Thank you! Jose ------------------------------ Message: 5 Date: Wed, 09 Jan 2008 10:34:38 -0500 From: Keith Gilbertson <[EMAIL PROTECTED]> Subject: Re: [Dspace-tech] Getting image thumbnails To: Branko Kovacevic <[EMAIL PROTECTED]> Cc: dspace-tech@lists.sourceforge.net Message-ID: <[EMAIL PROTECTED]> Content-Type: text/plain; charset=windows-1252; format=flowed Hello - The errors you posted appear to be related to filtering of PDF and Word documents. I'm not sure of the limits. For what it's worth, here are some steps you can try to build JPEG thumbnails of TIFFs in DSpace: - Enter TIFF in the DSpace bitstream format registry with mime-type image/tiff and file extensions tiff and tif - Edit the dspace.cfg file and add image/tiff and TIFF to the filter.org.dspace.app.mediafilter.JPEGFilter.inputFormats line - Download and install the Java Advanced Imaging I/O tools, currently available here: https://jai-imageio.dev.java.net/binary-builds.html . These tools contain a TIFF plugin that will allow the JPEGFilter to read the TIFF format. - Verify that the bitstreams are marked as TIFF format, and then run the filter-media script to build the JPEG thumbnails for the TIFFs. Be patient. If you see memory errors with large TIFF files, you can try increasing the "-Xmx256m" (maximum heap size) parameter in the dsrun script to resolve the problem. If you have certain types of images, you may need to write a custom filter or modify the JPEGFilter to get better results. For example, if you have large TIFF files that are primarily black and white, the JPEGFilter will favor speed over appearance when resampling the image to the thumbnail sized JPEG, and the resulting thumbnail won't look much like the original. You might need a filter that uses a different resampling method. -- Keith Systems Developer OhioLINK Branko Kovacevic wrote: > Dear All, > > So far we've been uploading jpg images into our DSpace system and had > no problems with getting thumbnails for them later. > > Unfortunately, recently after uploading a dozen of items with tiff > images (their size is between 4 and 15 Mb) couldn't get thumbnails for > them. Filter-media script returns error message. Here is the portion of > the log file, with some critical messages: > > ERROR filtering, skipping bitstream #7542 > java.io.FileNotFoundException: no such entry: "0Table" > java.io.FileNotFoundException: no such entry: "0Table" > at > org.apache.poi.poifs.filesystem.DirectoryNode.getEntry(DirectoryNode.jav a > :283) > at > org.textmining.text.extraction.WordExtractor.extractText(WordExtractor.j ava:60) > at > org.dspace.app.mediafilter.WordFilter.getDestinationStream(WordFilter.ja va:97) > at > org.dspace.app.mediafilter.MediaFilter.processBitstream > (MediaFilter.java:155) > at > org.dspace.app.mediafilter.MediaFilterManager.filterBitstream(MediaFilte rManager.java:327) > at > org.dspace.app.mediafilter.MediaFilterManager.filterItem(MediaFilterMana ger.java:296) > > at > org.dspace.app.mediafilter.MediaFilterManager.applyFiltersItem(MediaFilt erManager.java:266) > at > org.dspace.app.mediafilter.MediaFilterManager.applyFiltersAllItems(Media FilterManager.java:234) > at > org.dspace.app.mediafilter.MediaFilterManager.main(MediaFilterManager.ja va:185) > java.lang.Throwable: Warning: You did not close the PDF Document > at org.pdfbox.cos.COSDocument.finalize(COSDocument.java:384) > at gnu.gcj.runtime.FinalizerThread.run(libgcj.so.70) > java.lang.Throwable: Warning: You did not close the PDF Document > at org.pdfbox.cos.COSDocument.finalize(COSDocument.java:384) > at gnu.gcj.runtime.FinalizerThread.run (libgcj.so.70) > java.lang.Throwable: Warning: You did not close the PDF Document > at org.pdfbox.cos.COSDocument.finalize(COSDocument.java:384) > at gnu.gcj.runtime.FinalizerThread.run(libgcj.so.70) > java.lang.Throwable : Warning: You did not close the PDF Document > at org.pdfbox.cos.COSDocument.finalize(COSDocument.java:384) > at gnu.gcj.runtime.FinalizerThread.run(libgcj.so.70) > java.lang.Throwable: Warning: You did not close the PDF Document > at org.pdfbox.cos.COSDocument.finalize(COSDocument.java:384) > at gnu.gcj.runtime.FinalizerThread.run(libgcj.so.70) > FILTERED: bitstream 7682 and created > 'articles_bridging_20000615.pdf.txt' > FILTERED: bitstream 7683 and created > 'articles_sustainable_developement_20000815.pdf.txt' > GC Warning: Repeated allocation of very large block (appr. size > 20230144): > May lead to memory leak and poor performance. > FILTERED: bitstream 7684 and created > 'articles_venture_20001215.pdf.txt' > FILTERED: bitstream 7685 and created > 'articles_rethinking_20010215.pdf.txt' > FILTERED: bitstream 7686 and created > 'articles_relationship_20010515.pdf.txt' > FILTERED: bitstream 7687 and created > 'articles_org_capacity_20021115.pdf.txt' > GC Warning: Out of Memory! Returning NIL! > Exception in thread "main" java.lang.OutOfMemoryError > <<No stacktrace available>> > > Is there any limit of the file size filtering? > Any help is highly appreciated. > > Best regards, > Branko Kovacevic > > Records Coordinator > Open Society Archives > Arany Janos u. 32 > 1051 Budapest, Hungary > phone: (36-1) 327-3266 or 327-2029 > e-mail: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > website: www.osa.ceu.hu <http://www.osa.ceu.hu> > ++++++++++++++++++++++++++++ > > ------------------------------ Message: 6 Date: Wed, 9 Jan 2008 12:07:15 -0500 From: "Kyle Kaliebe" <[EMAIL PROTECTED]> Subject: [Dspace-tech] Collection Item Mapper Search To: <dspace-tech@lists.sourceforge.net> Cc: Beth Black <[EMAIL PROTECTED]> Message-ID: <[EMAIL PROTECTED]> Content-Type: text/plain; charset="us-ascii" If you go to the main page for any collection while logged in as an administrator you should see a button called "Item Mapper" under Admin Tools in the upper right hand corner. Clicking on it takes you to the Item Mapper which gives you the option of searching by author name for items to map. Our dspace installation is not carrying out this search correctly. If there is an author with the specified name, the resulting page is blank. If the author specified does not have an item in the repository, the resulting page shows an empty table. Does anyone have any ideas as to what may be causing this? Kyle Kaliebe Systems Developer/Engineer Knowledge Bank University Libraries Ohio State University [EMAIL PROTECTED] -------------- next part -------------- An HTML attachment was scrubbed... ------------------------------ ------------------------------------------------------------------------ - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketp lace ------------------------------ _______________________________________________ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech End of DSpace-tech Digest, Vol 21, Issue 20 ******************************************* ------------------------------------------------------------------------ - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketp lace _______________________________________________ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech ------------------------------------------------------------------------ - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketp lace _______________________________________________ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace _______________________________________________ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
