Re: [basex-talk] RESTXQ, multipart/form-data, out of memory saving to file

Dear Christian, With apologies for the late reply - thank you for the snapshot. I have tested today and I can now upload and save around 400MB on a 4 GB memory instance without issue. Now let me see how long before I get an even more unreasonable file size request… Thank you again, James > O

Re: [basex-talk] RESTXQ, multipart/form-data, out of memory saving to file

Hi James, Thanks for your persistence. Your observation (4 GB assigned, failing with 350 MB) made me think, and indeed the difference between handling raw post and map data was caused by an internal info output generation of map structures that does not contribute to the eventual result. With the

Re: [basex-talk] RESTXQ, multipart/form-data, out of memory saving to file

Hi Christian, Thank you for your ideas - lots for me to consider. > increase the memory that’s assigned to the JVM. I’ve currently assigned it 4GB and the file that’s failing is 350MB so I don’t think that’s going to be an easy fix. > You could try to interpret the incoming POST data with XQue

Re: [basex-talk] RESTXQ, multipart/form-data, out of memory saving to file

Hi James, There will always be an upper limit when uploading files via RESTXQ. The limit will be lower if multipart data is sent, as all single parts of your request body will additionally be wrapped into a map and (possibly) converted to another format. You could try to interpret the incoming PO

Re: [basex-talk] RESTXQ, multipart/form-data, out of memory saving to file

Hello Dirk, Thank you for such a quick response. > However, what is the actual reason to put the binary into a map? I don’t know - it’s BaseX that is doing it. For files uploaded via POST they are put into a map. I’m using the information here: http://docs.basex.org/wiki/RESTXQ#File_Uploads

Re: [basex-talk] RESTXQ, multipart/form-data, out of memory saving to file

Hello James, well, that is what I would expect BaseX to do. If you put the file in a map it needs to be in memory. For a large file your memory might run out. With your version 2) I assume you use the streaming capabilities of file:write-binary (see http://docs.basex.org/wiki/Streaming_Module fo

[basex-talk] RESTXQ, multipart/form-data, out of memory saving to file

Hello everyone, I’m trying to use BaseX RESTXQ to upload some large (around 300MB) Zip files using HTML forms (multipart/form-data), save them to disk and then process them. But I am getting server errors with large files (small files work perfectly). HTTP ERROR 500 Problem accessing /test.htm