Hi Hugh,
here you are the 'top' output:
top - 14:57:25 up 13 days, 22:52, 7 users, load average: 2.57,
1.74, 1.31
Tasks: 133 total, 1 running, 132 sleeping, 0 stopped, 0 zombie
Cpu(s): 68.0% us, 1.7% sy, 0.0% ni, 29.8% id, 0.5% wa, 0.0% hi,
0.0% si
Mem: 15901500k total, 15855120k used, 46380k free, 40364k buffers
Swap: 4096564k total, 1016240k used, 3080324k free, 2951672k cached
PID USER PR NI %CPU TIME+ %MEM VIRT RES SHR S
COMMAND 16990 root 16 0 279 378:08.04
75.7 12.0g 11g 8256 S virtuoso-t 1 root
16 0 0 0:00.66 0.0 4756 420 388 S
init 2 root RT 0 0 0:00.21
0.0 0 0 0 S migration/0 3 root 34
19 0 0:18.19 0.0 0 0 0 S
ksoftirqd/0 4 root RT 0 0 0:00.20
0.0 0 0 0 S migration/1 5 root 34
19 0 0:00.88 0.0 0 0 0 S
ksoftirqd/1 6 root RT 0 0 0:00.16
0.0 0 0 0 S migration/2 7 root 34
19 0 0:01.02 0.0 0 0 0 S
ksoftirqd/2 8 root RT 0 0 0:00.10
0.0 0 0 0 S migration/3 9 root 34
19 0 0:15.42 0.0 0 0 0 S
ksoftirqd/3 10 root 5 -10 0 0:00.00
0.0 0 0 0 S events/0 11 root 5
-10 0 0:00.01 0.0 0 0 0 S
events/1 12 root 5 -10 0 0:00.04
0.0 0 0 0 S events/2 13 root 5
-10 0 0:00.00 0.0 0 0 0 S
events/3 14 root 5 -10 0 0:00.00
0.0 0 0 0 S khelper 15 root 15
-10 0 0:00.00 0.0 0 0 0 S
kacpid 64 root 5 -10 0 0:00.00
0.0 0 0 0 S kblockd/0 65 root 5
-10 0 0:00.00 0.0 0 0 0 S
kblockd/1 66 root 5 -10 0 0:00.00
0.0 0 0 0 S kblockd/2 67 root 5
-10 0 0:00.00 0.0 0 0 0 S
kblockd/3 68 root 15 0 0 0:00.00
0.0 0 0 0 S khubd 97 root
15 0 0 1:09.28 0.0 0 0 0 S
kswapd1 98 root 15 0 0 1:27.16
0.0 0 0 0 S kswapd0 99 root 5
-10 0 0:00.00 0.0 0 0 0 S aio/0
the machine has 2 processors, more info:
$ uname -a
Linux crunch 2.6.9-67.0.15.ELsmp #1 SMP Tue Apr 22 13:58:43 EDT 2008
x86_64 x86_64 x86_64 GNU/Linux
regarding the FDsPerFile param, I deleted the entire db, so I began
from scratch setting that param to 8 (initally I tried out 32, and i
got a similar error as reported then I tried out 16, and the same
error, then 8 and it worked). Other params:
[Parameters]
ServerPort = 1112
DisableUnixSocket = 1
ServerThreads = 50
CheckpointInterval = 6000
O_DIRECT = 1
NumberOfBuffers = 1250000
MaxDirtyBuffers = 1000000
CaseMode = 2
MaxStaticCursorRows = 50000
CheckpointAuditTrail = 0
AllowOSCalls = 0
SchedulerInterval = 10
DirsAllowed = .,
/virtuoso/data/virtuoso-opensource-full/share/virtuoso/vad,
/virtuoso/data/rdf, /virtuoso/data/rdf_tc, /virtuoso/db/left,
/virtuoso/db/right
TransactionAfterImageLimit = 5000000000
ThreadCleanupInterval = 0
ThreadThreshold = 10
ResourcesCleanupInterval = 0
FreeTextBatchSize = 10000000
SingleCPU = 0
VADInstallDir =
/virtuoso/data/virtuoso-opensource-full/share/virtuoso/vad/
PrefixResultNames = 0
;TraceOn = errors, compile, exec, transact,
sql_send, user_log, failed_log
TraceOn = errors
;MaxMemPoolSize = 0
FDsPerFile = 8
[Client]
SQL_PREFETCH_ROWS = 1000
SQL_PREFETCH_BYTES = 60000
SQL_QUERY_TIMEOUT = 0
SQL_TXN_TIMEOUT = 0
[VDB]
ArrayOptimization = 0
NumArrayParameters = 10
VDBDisconnectTimeout = 1000
KeepConnectionOnFixedThread = 0
[Replication]
ServerName = crunch
ServerEnable = 1
QueueMax = 50000
..
ErrorLogLevel = 7
FileExtend = 100000
MaxCheckpointRemap = 3000000
...
thanks,
erick
Hugh Williams wrote:
Hi Erick,
Can you please provide the following:
1. The output of running the top command on the machine at the time
you are running these problem queries such that we can see the CPU,
I/O etc consumption
2. How many hard disks and CPUs are in the machine currently
3. Some of these problem queries for analysis
Your FDsPerFile has too many stripes most probably, resulting in the
process running out of descriptors. We would suggest you use only
one stripe per physical disk.
Best Regards
Hugh Williams
Professional Services
OpenLink Software
http://www.openlinksw.com
On 5 Jun 2008, at 11:33, Erick Antezana wrote:
Hello,
could anybody point out some doc/examples for doing some
performance tuning?
I have been playing (in a linux box with 16GB RAM, 1 CPU, ...) with
some
params (numberofbuffers, maxcheckpointremap, O_DIRECT, etc) as
indicated in:
http://virtuoso.openlinksw.com/wiki/main/Main/VOSScale
http://docs.openlinksw.com/virtuoso/rdfperformancetuning.html
however, our system is still pretty slow or even no responsive :-(
while
launching some SPARQL queries...
on the other hand, we're planing on buying a new server. Could anybody
recommend some specs (RAM, CPU, disks, ...) so that it could
"steadily/reasonably" cope with huge amounts of data (about 160GB
in RDF
files). Any advice for the hard disks spec (type: SCSI, ...; IO rate;
...)? config (striping, ...)?
thanks in advance!
Erick
p.s. while setting the param FDsPerFile to 16 for isntance, I've
got an
error while re-starting virtuoso:
12:15:54 Cannot open stripe on /virtuoso/db/right/db-seg32-2.db (24)
12:15:54 Server exiting
should that param be set the first time the system is started (due to
the striping?)?
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
--==================================================================
Erick Antezana http://www.cellcycleontology.org
PhD student
Tel:+32 (0)9 331 38 24 fax:+32 (0)9 3313809
VIB Department of Plant Systems Biology, Ghent University
Technologiepark 927, 9052 Gent, BELGIUM
er...@psb.ugent.be http://www.psb.ugent.be/~erant
==================================================================
