Re: [Virtuoso-users] Parallel WebDAV uploads fail

[Rumi]

Hello Harald,

On 20-Mar-13 3:24 PM, Harald Beck wrote:

Hi,

If I upload multiple files at the same time to the WebDAV folder,
different errors occur, and I receive a 599 internal server error
response for some of the requests.

I use requests of the following form:

curl -i -T $dir/$file 
http://localhost:8890/DAV/home/username/targetfolder/$file -u username:password

I upload about 10 ttl files in parallel, ranging between 1 and 800
MB. (I also got the problem in one attempt with 4 parallel request
with files smaller than 100 MB.)

It works fine for single requests. On multple requests, I get different error 
messages (see below).

Is importing to WebDAV limited to a single request? If not, what's the problem 
and/or what can I try?

Best,
Harald

ERROR LOG 1:

12:10:30 INFO: Failed to get a buffer for a new page. Retrying.  If the failure 
repeats, an out of disk error will be signalled.  The cause of this is having 
too many buffers wired down for preread, flush or group by/hash join temp 
space.  To correct, increase the number of buffers in the configuration file.  
If this repeats in spite of having hundreds of thousands  of buffers, please 
report to support.
12:10:31 ERROR: Signaling out of disk due to failure to get a buffer.  This 
condition is not a case of running out of disk
12:10:31 ERROR: Out of disk space for database

So for ex.:

NumberOfBuffers          = 500000
MaxDirtyBuffers          = 400000



Best Regards,
Rumi

You should increase the number of buffers + MaxDirtyBuffers in your virtuoso ini file. Note that these values are approximate depending on the RAM of your machine has etc.

(Since I don't how many 'hundreds of thousands' exactly are, I omitted this 
option.)

ERROR LOG 2:

12:50:50 INFO: Checkpoint started
12:50:50 ERROR: suspect to miss a flush of L=563838 in cpt, line 905
12:50:50 ERROR: suspect to miss a flush of L=571411 in cpt, line 905
12:50:50 ERROR: Buffer 0x7fc3a1ff59b8 occupied in cpt
12:50:50 ERROR: Buffer 0x7fc3a6218f88 occupied in cpt
12:50:50 ERROR: suspect to miss a flush of L=563838 in cpt, line 905
12:50:50 ERROR: suspect to miss a flush of L=571411 in cpt, line 905
12:50:50 ERROR: suspect to miss a flush of L=563838 in cpt, line 905
12:50:50 ERROR: suspect to miss a flush of L=571411 in cpt, line 905
12:50:50 ERROR: suspect to miss a flush of L=563838 in cpt, line 905
12:50:50 ERROR: suspect to miss a flush of L=571411 in cpt, line 905
12:50:50 ERROR: suspect to miss a flush of L=563838 in cpt, line 905
12:50:50 ERROR: suspect to miss a flush of L=571411 in cpt, line 905
12:50:50 ERROR: Buffer 0x7fc3a1ff59b8 occupied in cpt
12:50:50 ERROR: Buffer 0x7fc3a1ff59b8 dirty but no remap, tree SYS_DAV_RES
12:50:50 ERROR: Buffer 0x7fc3a6218f88 occupied in cpt
12:50:50 ERROR: Buffer 0x7fc3a6218f88 dirty but no remap, tree SYS_DAV_RES
12:50:50 INFO: Checkpoint finished, log reused
12:50:50 ERROR: Buffer 0x7fc3a1ff59b8 occupied in cpt
12:50:50 ERROR: Buffer 0x7fc3a1ff59b8 dirty but no remap, tree SYS_DAV_RES
12:50:50 ERROR: Buffer 0x7fc3a6218f88 occupied in cpt
12:50:50 ERROR: Buffer 0x7fc3a6218f88 dirty but no remap, tree SYS_DAV_RES

smime.p7s
Description: S/MIME Cryptographic Signature

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users