Hi Hugh,
I tried to run both queries (first surrounded by the 'explain' command)
and I've got the following error message:
The statement execution did not return a result set.
SQLState: 37000
Message: SQ074: Line 1: syntax error at 'rdf' before ':'
I also tried without the 'explain' command and I've got:
SQLState: 22023
Message: SR007: Function trim needs a string or NULL as argument 1, not an arg
of type ARRAY_OF_POINTER (193)
Moreover, I tried with the advanced tab and selecting the explain option
and I've got:
SQLState: 22023
Message: SR007: Function trim needs a string or NULL as argument 1, not an arg
of type ARRAY_OF_POINTER (193)
From the command line (isql):
SQL> explain('prefix rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#>
select distinct ?ontology
from <http://www.semantic-systems-biology.org/ontology/rdf/OBO>
where{
?term rdf:type ?ontology.
}
order by ?ontology ')Type the rest of statement, end with a semicolon
(;)> Type the rest of statement, end with a semicolon (;)> Type the rest
of statement, end with a semicolon (;)> Type the rest of statement, end
with a semicolon (;)> Type the rest of statement, end with a semicolon
(;)> Type the rest of statement, end with a semicolon (;)> ;
*** Error 37000: [Virtuoso Driver][Virtuoso Server]SQ074: Line 1: syntax
error at 'rdf' before ':'
in lines 1-7 of Top-Level:
#line 1 "(console)"
explain('prefix rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#>
select distinct ?ontology
from <http://www.semantic-systems-biology.org/ontology/rdf/OBO>
where{
?term rdf:type ?ontology.
}
order by ?ontology ');
Am I missing something?
thanks,
Erick
Hugh Williams wrote:
Hi Eric,
We have been able to see the hangs you report with the sample queries
below.
Can you please provide the output of running the Virtuoso explain
command against these 2 queries and provide this output. The explain
command provides more details on the query execution plan with
Virtuoso as detailed at:
http://docs.openlinksw.com/virtuoso/fn_explain.html#(NULL
<http://docs.openlinksw.com/virtuoso/fn_explain.html#%28NULL>)
Note you will have to use the Virtuoso isql command or
Conductor Interactive SQL UI to execute the commands which will be as
follows:
example 1:
explain('prefix rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix rdfs:<http://www.w3.org/2000/01/rdf-schema#>
prefix obo:<http://www.semantic-systems-biology.org/ontology/rdf/OBO#>
prefix goa:<http://www.semantic-systems-biology.org/ontology/rdf/GOA#>
prefix uni:<http://www.semantic-systems-biology.org/ontology/rdf/UNIPROT#>
prefix ncbi:<http://www.semantic-systems-biology.org/ontology/rdf/NCBI#>
select ?protein ?predicate ?GO_term ?GOA_association
from <http://www.semantic-systems-biology.org/ontology/rdf/OBO>
from <http://www.semantic-systems-biology.org/ontology/rdf/GOA>
where{
?prot rdf:type goa:prot. ?prot obo:has_source ncbi:NCBI_471821.
?prot rdfs:label ?protein.
?Statement rdf:subject ?prot.
?Statement rdf:predicate obo:located_in.
obo:located_in rdfs:label ?predicate.
?Statement rdf:object obo:GO_0005737.
obo:GO_0005737 rdfs:label ?GO_term.
?Statement goa:supported_by ?GOA_association.
?GOA_association obo:has_evidence_code ?eco_id.
}')
example 2:
explain('prefix rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#>
select distinct ?ontology
from <http://www.semantic-systems-biology.org/ontology/rdf/OBO>
where{
?term rdf:type ?ontology.
}
order by ?ontology ')
Best Regards
Hugh Williams
Professional Services
OpenLink Software
On 8 Jun 2008, at 09:29, Erick Antezana wrote:
Hi Hugh,
sorry, I forgot to add a couple of examples. here are some examples
that can be tested from:
http://www.cellcycleontology.org/query/38-interactive-query/82-sparql-kb
example 1:
prefix rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix rdfs:<http://www.w3.org/2000/01/rdf-schema#>
prefix obo:<http://www.semantic-systems-biology.org/ontology/rdf/OBO#>
prefix goa:<http://www.semantic-systems-biology.org/ontology/rdf/GOA#>
prefix
uni:<http://www.semantic-systems-biology.org/ontology/rdf/UNIPROT#>
prefix ncbi:<http://www.semantic-systems-biology.org/ontology/rdf/NCBI#>
select ?protein ?predicate ?GO_term ?GOA_association
from <http://www.semantic-systems-biology.org/ontology/rdf/OBO>
from <http://www.semantic-systems-biology.org/ontology/rdf/GOA>
where{
?prot rdf:type goa:prot. ?prot obo:has_source ncbi:NCBI_471821.
?prot rdfs:label ?protein.
?Statement rdf:subject ?prot.
?Statement rdf:predicate obo:located_in.
obo:located_in rdfs:label ?predicate.
?Statement rdf:object obo:GO_0005737.
obo:GO_0005737 rdfs:label ?GO_term.
?Statement goa:supported_by ?GOA_association.
?GOA_association obo:has_evidence_code ?eco_id.
}
example 2:
prefix rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#>
select distinct ?ontology
from <http://www.semantic-systems-biology.org/ontology/rdf/OBO>
where{
?term rdf:type ?ontology.
}
order by ?ontology
Since I am constantly testing it, changing the config parameters and
adding more data, the endpoint may be down during the night (CET) duo
to the entire rebuild of the system (it is build automatically from
scratch taking the latest data sources...).
Thanks,
Erick
Hugh Williams wrote:
Hi Erick,
You have not provided some sample queries for which these
performance issues are being encountered as requested in point 3 ?
Best Regards
Hugh Williams
OpenLink Software
On 6 Jun 2008, at 14:09, Erick Antezana wrote:
Hi Hugh,
here you are the 'top' output:
top - 14:57:25 up 13 days, 22:52, 7 users, load average: 2.57,
1.74, 1.31
Tasks: 133 total, 1 running, 132 sleeping, 0 stopped, 0 zombie
Cpu(s): 68.0% us, 1.7% sy, 0.0% ni, 29.8% id, 0.5% wa, 0.0%
hi, 0.0% si
Mem: 15901500k total, 15855120k used, 46380k free, 40364k
buffers
Swap: 4096564k total, 1016240k used, 3080324k free, 2951672k cached
PID USER PR NI %CPU TIME+ %MEM VIRT RES SHR S COMMAND
16990 root 16 0 279 378:08.04 75.7
12.0g 11g 8256 S virtuoso-t 1 root 16
0 0 0:00.66 0.0 4756 420 388 S init
2 root RT 0 0 0:00.21 0.0 0 0 0 S
migration/0 3 root 34 19 0 0:18.19
0.0 0 0 0 S ksoftirqd/0 4 root
RT 0 0 0:00.20 0.0 0 0 0 S migration/1
5 root 34 19 0 0:00.88 0.0 0 0 0 S
ksoftirqd/1 6 root RT 0 0 0:00.16
0.0 0 0 0 S migration/2 7 root
34 19 0 0:01.02 0.0 0 0 0 S ksoftirqd/2
8 root RT 0 0 0:00.10 0.0 0 0 0 S
migration/3 9 root 34 19 0 0:15.42
0.0 0 0 0 S ksoftirqd/3 10 root
5 -10 0 0:00.00 0.0 0 0 0 S events/0
11 root 5 -10 0 0:00.01 0.0 0 0 0 S
events/1 12 root 5 -10 0 0:00.04
0.0 0 0 0 S events/2 13 root
5 -10 0 0:00.00 0.0 0 0 0 S events/3
14 root 5 -10 0 0:00.00 0.0 0 0 0 S
khelper 15 root 15 -10 0 0:00.00
0.0 0 0 0 S kacpid 64 root
5 -10 0 0:00.00 0.0 0 0 0 S kblockd/0
65 root 5 -10 0 0:00.00 0.0 0 0 0 S
kblockd/1 66 root 5 -10 0 0:00.00
0.0 0 0 0 S kblockd/2 67 root
5 -10 0 0:00.00 0.0 0 0 0 S kblockd/3
68 root 15 0 0 0:00.00 0.0 0 0 0 S
khubd 97 root 15 0 0 1:09.28
0.0 0 0 0 S kswapd1 98 root
15 0 0 1:27.16 0.0 0 0 0 S kswapd0
99 root 5 -10 0 0:00.00 0.0 0 0 0 S aio/0
the machine has 2 processors, more info:
$ uname -a
Linux crunch 2.6.9-67.0.15.ELsmp #1 SMP Tue Apr 22 13:58:43 EDT
2008 x86_64 x86_64 x86_64 GNU/Linux
regarding the FDsPerFile param, I deleted the entire db, so I began
from scratch setting that param to 8 (initally I tried out 32, and
i got a similar error as reported then I tried out 16, and the same
error, then 8 and it worked). Other params:
[Parameters]
ServerPort = 1112
DisableUnixSocket = 1
ServerThreads = 50
CheckpointInterval = 6000
O_DIRECT = 1
NumberOfBuffers = 1250000
MaxDirtyBuffers = 1000000
CaseMode = 2
MaxStaticCursorRows = 50000
CheckpointAuditTrail = 0
AllowOSCalls = 0
SchedulerInterval = 10
DirsAllowed = .,
/virtuoso/data/virtuoso-opensource-full/share/virtuoso/vad,
/virtuoso/data/rdf, /virtuoso/data/rdf_tc, /virtuoso/db/left,
/virtuoso/db/right
TransactionAfterImageLimit = 5000000000
ThreadCleanupInterval = 0
ThreadThreshold = 10
ResourcesCleanupInterval = 0
FreeTextBatchSize = 10000000
SingleCPU = 0
VADInstallDir =
/virtuoso/data/virtuoso-opensource-full/share/virtuoso/vad/
PrefixResultNames = 0
;TraceOn = errors, compile, exec, transact,
sql_send, user_log, failed_log
TraceOn = errors
;MaxMemPoolSize = 0
FDsPerFile = 8
[Client]
SQL_PREFETCH_ROWS = 1000
SQL_PREFETCH_BYTES = 60000
SQL_QUERY_TIMEOUT = 0
SQL_TXN_TIMEOUT = 0
[VDB]
ArrayOptimization = 0
NumArrayParameters = 10
VDBDisconnectTimeout = 1000
KeepConnectionOnFixedThread = 0
[Replication]
ServerName = crunch
ServerEnable = 1
QueueMax = 50000
..
ErrorLogLevel = 7
FileExtend = 100000
MaxCheckpointRemap = 3000000
...
thanks,
erick
Hugh Williams wrote:
Hi Erick,
Can you please provide the following:
1. The output of running the top command on the machine at the
time you are running these problem queries such that we can see
the CPU, I/O etc consumption
2. How many hard disks and CPUs are in the machine currently
3. Some of these problem queries for analysis
Your FDsPerFile has too many stripes most probably, resulting in
the process running out of descriptors. We would suggest you use
only one stripe per physical disk.
Best Regards
Hugh Williams
Professional Services
OpenLink Software
http://www.openlinksw.com
On 5 Jun 2008, at 11:33, Erick Antezana wrote:
Hello,
could anybody point out some doc/examples for doing some
performance tuning?
I have been playing (in a linux box with 16GB RAM, 1 CPU, ...)
with some
params (numberofbuffers, maxcheckpointremap, O_DIRECT, etc) as
indicated in:
http://virtuoso.openlinksw.com/wiki/main/Main/VOSScale
http://docs.openlinksw.com/virtuoso/rdfperformancetuning.html
however, our system is still pretty slow or even no responsive
:-( while
launching some SPARQL queries...
on the other hand, we're planing on buying a new server. Could
anybody
recommend some specs (RAM, CPU, disks, ...) so that it could
"steadily/reasonably" cope with huge amounts of data (about 160GB
in RDF
files). Any advice for the hard disks spec (type: SCSI, ...; IO rate;
...)? config (striping, ...)?
thanks in advance!
Erick
p.s. while setting the param FDsPerFile to 16 for isntance, I've
got an
error while re-starting virtuoso:
12:15:54 Cannot open stripe on /virtuoso/db/right/db-seg32-2.db (24)
12:15:54 Server exiting
should that param be set the first time the system is started (due to
the striping?)?
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
<mailto:Virtuoso-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
--
==================================================================
Erick Antezana http://www.cellcycleontology.org
PhD student
Tel:+32 (0)9 331 38 24 fax:+32 (0)9 3313809
VIB Department of Plant Systems Biology, Ghent University
Technologiepark 927, 9052 Gent, BELGIUM
er...@psb.ugent.be <mailto:er...@psb.ugent.be>
http://www.psb.ugent.be/~erant
==================================================================
--
==================================================================
Erick Antezana http://www.cellcycleontology.org
PhD student
Tel:+32 (0)9 331 38 24 fax:+32 (0)9 3313809
VIB Department of Plant Systems Biology, Ghent University
Technologiepark 927, 9052 Gent, BELGIUM
er...@psb.ugent.be http://www.psb.ugent.be/~erant
==================================================================
