I think FTP is the best choice for this. In regard to monitoring a directory: is there an xdmp function for getting a listing of files and directories from the file system? I didn't see that in the API documents.
Thanks, Demian On Tue, Feb 7, 2012 at 2:04 PM, Geert Josten <geert.jos...@dayon.nl> wrote: > Hi Demian, > > My guess at it is that MarkLogic would try to maintain the binary file as > request param within memory, which is limited to some size for good > reason. With 10+ concurrent requests, all uploading 2 Gb, memory would > build up very rapidly. I'm guessing the limits are related to the limits > within xdmp:document-get, which does not allow reading files larger than > 16Mb or 64Mb (depending on 32/64-bit install). > > I think I wouldn't advice to use form upload to send large files across. > Other protocols like (S)FTP are better suited for that. You could allow > such files to be uploaded to file locations through a separate (S)FTP > server on the same host. You can let MarkLogic monitor that directory > using a Scheduled Task, and perhaps leveraging Information Studio flows. > In these processes you can use different function to load such large > files, though I am not sure they are able to handle 2+Gb files.. > > Kind regards, > Geert > >> -----Oorspronkelijk bericht----- >> Van: general-boun...@developer.marklogic.com [mailto:general- >> boun...@developer.marklogic.com] Namens Demian Hess >> Verzonden: dinsdag 7 februari 2012 19:45 >> Aan: general@developer.marklogic.com >> Onderwerp: [MarkLogic Dev General] Can app server handle file uploads >>2GB? >> >> We would like to store large binary files (many EPS and high >> resolution PDFs) in MarkLogic. It would be easiest for users to create >> zips with all their files and then upload via an HTML form >> (enctype=multipart/form-data), but the size of the zip archives would >> be more than 2GB. Indeed, some individual files can get to 2GB by >> themselves. >> >> I tried POSTing a 2GB file to an app server and got a "413 Request >> Entity Too Large" error. I don't see any settings in MarkLogic to >> control max size of uploads for HTTP app servers. >> >> How efficient would ML be at handling large posts of 100MB and larger? >> Would it buffer everything in memory or would it stream to a temporary >> file on disk? Would ML 5.0 be more efficient than 4.0 at handling >> large file uploads? >> _______________________________________________ >> General mailing list >> General@developer.marklogic.com >> http://developer.marklogic.com/mailman/listinfo/general > _______________________________________________ > General mailing list > General@developer.marklogic.com > http://developer.marklogic.com/mailman/listinfo/general -- Demian Hess Avalon Consulting, LLC 527 Maple Avenue East, Suite 200, Vienna, VA 22180 Mobile: 301-943-8307 Fax: 845-367-5496 he...@avalonconsult.com _______________________________________________ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
