I think that uploading files using BASE2's http-upload can become cumbersome and prone to errors with bigger/many files. We hope to produce a lot of data in the near future and this might be a problem.
I have an idea of a "Import server side files"-plug-in. This will work in this way: 1. You upload your files to the server where BASE2 resides using your favorite (and available) file transfer (fpt, samba, etc). 2. Inside BASE2 you execute the plug-in and BASE2 moves the files so it fits with the BASE2 file system and makes necessary database records. Have any of you thought of something like it? It should be simpler to implement than a BASE2 integrated ftp server. Though it would not suit all users since it depend on having some kind of standard file transfer access to the server. Are there any big problems with this solution I have not seen? Vegard > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:basedb-devel- > [EMAIL PROTECTED] On Behalf Of Nicklas Nordborg > Sent: Friday, November 23, 2007 8:12 PM > To: BASE dev > Subject: Re: [basedb-devel] Problem uploading large zip file > > I did some test by inserting some debug statements the first thing in > the upload script. I tested this on Windows with Firefox 2.0.0.9 and > Microsoft IE 7.0 by uploading a 5.74GB (6 169 509 888 bytes) file. > > With Firefox it seems like the upload script never gets called at all. > Nothing happens and no debug output is seen. > > With IE the file starts to upload but the file is reported as 1.7 GB (1 > 874 544 014 bytes). The upload stops when that number is reached and > appears as a file in BASE (with the size 1.7 GB). In my debug output I > can see that IE is actually setting the content-length header to > 1874544014, which is approximately what you get if you just chop of the > last 4 bytes of the actual file size. > > I found a page with some information of the problem: > http://www.motobit.com/help/scptutl/pa98.htm > > So, I guess it is a browser-related problem or maybe a limitation of the > HTTP specification. You have to split the file into smaller chunks > before uploading them. > > /Nicklas > > > Nicklas Nordborg wrote: > > Adam Ameur wrote: > >> Hi, > >> > >> I encountered a problem when trying to upload a 3.6 GB zip file > >> containing raw data from about 100 high density Affymetrix arrays in > >> BASE2.4.6. In Firefox the data transfer doesn't seem to start at all. > >> Using Safari the upload got started but the file never showed up in > >> BASE. Are there any limitations in file size when uploading to BASE, or > >> is this only a browser specific problem? I know this data set is > >> extremely big but I hope that it's anyway possible to upload and store > >> it in BASE. > > > > I really don't know if it is a browser specific problem, a BASE problem > > or a limitation in the underlying operating system of the server. I > > guess most operating systems support larger files so the last not > > likely. In any case, do you see any error messages in Tomcat's log file > > that seem related to this? > > > > The only thing I can think of on the BASE side is that the content size > > is only available as an integer in the Servlet API, which can hold a max > > value around 2GB. I don't know what happens if the content size is > > larger, but I think the progress bar may get confused if it gets a > > negative value or no value at all. > > > > /Nicklas > > > > ------------------------------------------------------------------------ > - > > This SF.net email is sponsored by: Microsoft > > Defy all challenges. Microsoft(R) Visual Studio 2005. > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > > _______________________________________________ > > basedb-devel mailing list > > basedb-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/basedb-devel > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2005. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > basedb-devel mailing list > basedb-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/basedb-devel ------------------------------------------------------------------------- SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ basedb-devel mailing list basedb-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/basedb-devel
