Hi, I have some problem with the "insert" request. A large insert query uses the RAM to 100% then the SWAP and finally kill the process.
So i run a test who consist to insert 1.000, then 1.000.000, and then all the graph in a new graph. I run the test on 4 graphes. 202042 triplets, 1918433 triplets, 13608512 triplets and 38496582 triplets. The procedure: for graph in graphe1 graphe2 graphe3 graphe4 do replace <graphName> by $graph exec the request sleep 120 (to view the change in monitoring) done The 3 requests: *request1: SPARQL insert into <http://testgraph> { ?x ?p ?y } where { graph <graphName> { ?x ?p ?y } } limit 1000; *request2: SPARQL insert into <http://testgraph> { ?x ?p ?y } where { graph <graphName> { ?x ?p ?y } } limit 1000000; *request3: SPARQL insert into <http://testgraph> { ?x ?p ?y } where { graph <graphName> { ?x ?p ?y } }; The config server: AMD Opteron(tm) Processor 252 8GO RAM OpenLink Virtuoso Server Version 06.01.3127-pthreads for Linux as of Aug 3 2011 The part of virtuoso.ini conf: ; Server parameters ; [Parameters] ServerPort = 1111 LiteMode = 0 DisableUnixSocket = 1 DisableTcpSocket = 0 ServerThreads = 5 CheckpointInterval = 720 O_DIRECT = 0 NumberOfBuffers = 100000 MaxDirtyBuffers = 75000 CaseMode = 2 MaxStaticCursorRows = 5000 CheckpointAuditTrail = 1 AllowOSCalls = 0 SchedulerInterval = 10 DirsAllowed = ., /usr/share/virtuoso-opensource-6.1/vad ThreadCleanupInterval = 0 ThreadThreshold = 10 ResourcesCleanupInterval = 0 FreeTextBatchSize = 100000 SingleCPU = 0 VADInstallDir = /usr/share/virtuoso-opensource-6.1/vad/ PrefixResultNames = 0 RdfFreeTextRulesSize = 100 IndexTreeMaps = 256 TransactionAfterImageLimit = 1500000000 We have test with the configuration for our RAM system and we have modify the buffers parameters but the system SWAP and crash with a insert or delete request. I enclose the charts of the system during the test. ######################################## Fri Aug 5 10:04:57 CEST 2011 delete http://testgraph (who is empty) 1 Rows. -- 361669 msec. (SPARQL CLEAR GRAPH) Done. -- 13 msec. (delete from DB.DBA.load_list where ll_graph = ) Done. -- 0 msec. (commit WORK;) ## INSERT ## Fri Aug 5 10:10:59 CEST 2011 Count triplet tetsgraph: 0 Database Status: File size 20900216832, 2551296 pages, 2089603 free. 100000 buffers, 100000 used, 1982 dirty 0 wired down, repl age 583779 0 w. io 0 w/crsr. Disk Usage: 302560 reads avg 0 msec, 41% r 0% w last 406 s, 5718 writes, 719 read ahead, batch = 224. Autocompact 0 in 0 out, 0% saved. Fri Aug 5 10:11:43 CEST 2011 graphe1 : insert_01 : 1 Rows. -- 2136 msec. graphe2 : insert_01 : 1 Rows. -- 114 msec. graphe3 : insert_01 : 1 Rows. -- 100 msec. graphe4 : insert_01 : 1 Rows. -- 207 msec. Fri Aug 5 10:19:46 CEST 2011 Count triplet tetsgraph: 4000 (OK) Database Status: File size 20900216832, 2551296 pages, 2089415 free. 100000 buffers, 99723 used, 2169 dirty 0 wired down, repl age 578155 0 w. io 0 w/crsr. Disk Usage: 305187 reads avg 0 msec, 0% r 0% w last 485 s, 5719 writes, 743 read ahead, batch = 220. Autocompact 1556 in 837 out, 46% saved. Fri Aug 5 10:19:49 CEST 2011 graphe1 : insert_02 : 1 Rows. -- 16411 msec. graphe2 : insert_02 : 1 Rows. -- 47227 msec. graphe3 : insert_02 : 1 Rows. -- 76016 msec. graphe4 : insert_02 : 1 Rows. -- 56598 msec. Fri Aug 5 10:31:13 CEST 2011 Count triplet tetsgraph: 3202042 (OK) Database Status: File size 20900216832, 2551296 pages, 2051391 free. 100000 buffers, 100000 used, 12309 dirty 2 wired down, repl age 693723 1841 w. io 0 w/crsr. Disk Usage: 434339 reads avg 0 msec, 0% r 0% w last 688 s, 35627 writes, 1893 read ahead, batch = 143. Autocompact 20749 in 14724 out, 29% saved. Fri Aug 5 10:31:18 CEST 2011 graphe1 : insert_03 : 1 Rows. -- 7842 msec. graphe2 : insert_03 : 1 Rows. -- 96683 msec. graphe3 : insert_03 : 1 Rows. -- 1097460 msec. *** Error 08S01: [Virtuoso Driver]CL065: Lost connection to server in lines 1-2 of Top-Level: #line 1 "(console)" SPARQL insert into <tetsgraph> { ?x ?p ?y } where { graph <http://graphe4> { ?x ?p ?y } } graphe4 : insert_03 : Fri Aug 5 12:12:26 CEST 2011 ######################################## This problem is reproducible. Thanks. Maxime T.
probleme-insert.pdf
Description: Adobe PDF document
