Hi Andreas, Missed that you were running 4 Virtuoso instances, how is the query workload split across the 4 instances ?
I noticed the following errors in all the logs during done attempt to restart: 09:23:02 ERROR: Unable to lock file ../var/lib/virtuoso/db/virtuoso.lck (Resource temporarily unavailable). 09:23:02 ERROR: Virtuoso is already runnning (pid 1264) 09:23:02 ERROR: This probably means you either do not have permission to start 09:23:02 ERROR: this server, or that virtuoso-t is already running. 09:23:02 ERROR: If you are absolutely sure that this is not the case, please try 09:23:02 ERROR: to remove the file ../var/lib/virtuoso/db/virtuoso.lck and start again. 09:26:41 INFO: ERRS_0 01V01 QW004 <unspec>:2891: WS.WS.SPARQL_ENDPOINT_GENERATE_FORM: Incompatible types INTEGER (189) and VARCHAR (182) in = for debug and <constant> 09:34:13 DEBUG: missed delete of name id cache benchset/d4e1a90786ac524b9f96f9d2f0506a12 0 (0x20e975b ) 09:34:16 DEBUG: missed delete of name id cache benchset/265b6be68f2030063c4c17e3778dbbe1 0 (0x2122444 ) 09:35:50 DEBUG: missed delete of name id cache benchset/8d0299f2a49cf965ac7c98101f55afaa 0 (0x2181ac6 ) But then did not occur on the following startup, do you know what happened here as it seem there was an attempt to start an already running server, but I have not seen the "DEBUG: missed delete of name id cache …” errors before … In the status_8895 output I see many occurrences of: Pending: 1126400: IER 141.87.4.9 156: IER 141.87.4.9 152: IER 141.87.4.9 148: IER 141.87.4.9 144: IER 141.87.4.9 . . . Do these remain indefinitely or do they get cleanup after a while ? Also you indicated that this errors had not been seen with the 3215 build upgrade, is this still the case ? Best Regards Hugh Williams Professional Services OpenLink Software, Inc. // http://www.openlinksw.com/ <http://www.openlinksw.com/> Weblog -- http://www.openlinksw.com/blogs/ <http://www.openlinksw.com/blogs/> LinkedIn -- http://www.linkedin.com/company/openlink-software/ <http://www.linkedin.com/company/openlink-software/> Twitter -- http://twitter.com/OpenLink <http://twitter.com/OpenLink> Google+ -- http://plus.google.com/100570109519069333827/ <http://plus.google.com/100570109519069333827/> Facebook -- http://www.facebook.com/OpenLinkSoftware <http://www.facebook.com/OpenLinkSoftware> Universal Data Access, Integration, and Management Technology Providers > On 21 Feb 2016, at 13:46, Nolle, Andreas <no...@hs-albsig.de > <mailto:no...@hs-albsig.de>> wrote: > > Hi Hugh, > > typical queries that are executed (each at any Virtuoso instance) are like > the following: > > SELECT ?x ?P0src ?P1src > WHERE { > { > SERVICE <http://141.87.4.8:8899/sparql <http://141.87.4.8:8899/sparql>> > { > ?x rdf:type <http://purl.org/ontology/bibo/Collection > <http://purl.org/ontology/bibo/Collection>> . > } . > BIND (<http://141.87.4.8:8899/sparql <http://141.87.4.8:8899/sparql>> > AS ?P0src) > } > UNION > { > ?x rdf:type <http://purl.org/ontology/bibo/Collection > <http://purl.org/ontology/bibo/Collection>> . > BIND (<http://141.87.4.8:8891/sparql <http://141.87.4.8:8891/sparql>> > AS ?P0src) > } > UNION > { > SERVICE <http://141.87.4.8:8893/sparql <http://141.87.4.8:8893/sparql>> > { > ?x rdf:type <http://purl.org/ontology/bibo/Collection > <http://purl.org/ontology/bibo/Collection>> . > } . > BIND (<http://141.87.4.8:8893/sparql <http://141.87.4.8:8893/sparql>> > AS ?P0src) > } > UNION > { > SERVICE <http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>> > { > ?x rdf:type <http://purl.org/ontology/bibo/Collection > <http://purl.org/ontology/bibo/Collection>> . > } . > BIND (<http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>> > AS ?P0src) > } . > ?x rdf:type <http://swrc.ontoware.org/ontology#Article > <http://swrc.ontoware.org/ontology#Article>> . > BIND (<http://141.87.4.8:8891/sparql <http://141.87.4.8:8891/sparql>> AS > ?P1src) > } > > Since it is not possible to use the keyword OFFSET for queries, i.e. query > parts, that would return more than 1048576 results, there are also queries > like: > > SELECT ?x ?a ?b ?P0src ?P1src > WHERE { > { > SERVICE <http://141.87.4.8:8899/sparql <http://141.87.4.8:8899/sparql>> > { > ?x <http://purl.org/dc/terms/partOf > <http://purl.org/dc/terms/partOf>> ?a . > FILTER ( ?x >= > <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp > <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) . > FILTER ( ?x < > <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp > <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) . > } . > BIND (<http://141.87.4.8:8899/sparql <http://141.87.4.8:8899/sparql>> > AS ?P0src) > } > UNION > { > SERVICE <http://141.87.4.8:8891/sparql <http://141.87.4.8:8891/sparql>> > { > ?x <http://purl.org/dc/terms/partOf > <http://purl.org/dc/terms/partOf>> ?a . > FILTER ( ?x >= > <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp > <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) . > FILTER ( ?x < > <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp > <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) . > } . > BIND (<http://141.87.4.8:8891/sparql <http://141.87.4.8:8891/sparql>> > AS ?P0src) > } > UNION > { > ?x <http://purl.org/dc/terms/partOf <http://purl.org/dc/terms/partOf>> > ?a . > FILTER ( ?x >= > <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp > <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) . > FILTER ( ?x < > <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp > <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) . > BIND (<http://141.87.4.8:8893/sparql <http://141.87.4.8:8893/sparql>> > AS ?P0src) > } > UNION > { > SERVICE <http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>> > { > ?x <http://purl.org/dc/terms/partOf > <http://purl.org/dc/terms/partOf>> ?a . > FILTER ( ?x >= > <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp > <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) . > FILTER ( ?x < > <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp > <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) . > } . > BIND (<http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>> > AS ?P0src) > BIND (<http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>> > AS ?P0src) > } . > ?b <http://www.aktors.org/ontology/portal#article-of-journal > <http://www.aktors.org/ontology/portal#article-of-journal>> ?x . > FILTER ( ?x >= > <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp > <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) . > FILTER ( ?x < <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp > <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) . > BIND (<http://141.87.4.8:8893/sparql <http://141.87.4.8:8893/sparql>> AS > ?P1src) > } > > where the filter parts are generated beforehand by querying all individuals > of a specific concept piece by piece for each 1048576 results. > > The error that occur is usually: > Virtuoso HTCLI Error HC001: Read Error in HTTP Client > > But as I remember other errors that occurred during several tests over the > last weeks are: > - No Http Response > - Transaction aborted > - Timeout > - target server failed to respond > - … > > Please find the output of status() for each Virtuoso instance (running at > port 8891, 8893, 8895, 8899) as well as the corresponding log in the zip file > at https://www.dropbox.com/s/yzmrybj7jf8m8az/files.zip?dl=0 > <https://www.dropbox.com/s/yzmrybj7jf8m8az/files.zip?dl=0>. The call of > status() was done after I received a http error from Virtuoso instance 8893. > For the same Virtuoso instance (running at port 8893) I got a timeout on > connecting by isql. Only after a restart it was possible to connect via isql. > > The reason why I assign only 16 GB per Virtuoso instance (4 x 16 GB in total) > is that each instance is take still more than 16 GB after some hours or days. > In my case I run up to 25000 queries (like the one above) at each Virtuoso > instance, but maximal 16 at one time over all instances. If you look at the > memory usage it starts normally but it increases over time as long as the > swap space is overcrowded… If this will be the case my code (running on > another server) restarts the whole server hosting the Virtuoso instances > before new SPARQL queries will be send. Please not that this behavior > occurred especially on previous versions (<7.2.2.3215) and I’m not sure if > this is still the case for version 7.2.2.3215 since I updated as recently as > two weeks ago… > > My Virtuoso instances hosting the following triple counts: > - Virtuoso instance 8891: 17765873 > - Virtuoso instance 8893: 27897291 > - Virtuoso instance 8895: 168888956 > - Virtuoso instance 8899: 72372256 > > Best > Andy > > > > Von: Hugh Williams [mailto:hwilli...@openlinksw.com > <mailto:hwilli...@openlinksw.com>] > Gesendet: Sonntag, 21. Februar 2016 02:51 > An: Nolle, Andreas <no...@hs-albsig.de <mailto:no...@hs-albsig.de>> > Cc: virtuoso-users@lists.sourceforge.net > <mailto:virtuoso-users@lists.sourceforge.net> > Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying > > Hi Andy, > > What is a typical query being executing that is result in a HTTP error and > what is the actual error ? > > When the error occurs what is the status of the server at that point , please > provide the output of running the command: > > status(); > > Are any errors reported in the “virtuoos.log” file, please provide a copy for > review ? > > You indicate having a 128GB RAM machine but memory allocated to Virtuoso is > that for a 16GB RAM machine ie NumberOfBuffer = 1360000 in the INI file, was > there a specific reason for this ? > > How many triples are hosted in the Virtuoso instance , please provide the > output of the count query: > > select count(*) where {?s ?p ?o} > > Best Regards > Hugh Williams > Professional Services > OpenLink Software, Inc. // http://www.openlinksw.com/ > <http://www.openlinksw.com/> > Weblog -- http://www.openlinksw.com/blogs/ > <http://www.openlinksw.com/blogs/> > LinkedIn -- http://www.linkedin.com/company/openlink-software/ > <http://www.linkedin.com/company/openlink-software/> > Twitter -- http://twitter.com/OpenLink <http://twitter.com/OpenLink> > Google+ -- http://plus.google.com/100570109519069333827/ > <http://plus.google.com/100570109519069333827/> > Facebook -- http://www.facebook.com/OpenLinkSoftware > <http://www.facebook.com/OpenLinkSoftware> > Universal Data Access, Integration, and Management Technology Providers > > > > On 20 Feb 2016, at 15:47, Nolle, Andreas <no...@hs-albsig.de > <mailto:no...@hs-albsig.de>> wrote: > > Hi folks, > > I’ve set up four Virtuoso instances (Version 7.2.2.3215) on an Ubuntu 14.04.4 > LTS server with 12 cores and 128 GB memory. > If I run several (federated) queries in parallel (maximal 16 at a time), > where some of the queries may take several minutes, it may happened that a > SPARQL endpoint return some HTTP error. If I rerun the same query some > seconds later, it will ordinarily be evaluated correct and the requested > results are returned. > I’ve already tried to solve this infrequent issue but without any success. > Can you please give me some recommendations? > Maybe my configuration may comprise some not well-chosen parameter? > > > Please find below my virtuoso.ini: > > [Database] > DatabaseFile = ../var/lib/virtuoso/db/virtuoso.db > ErrorLogFile = ../var/lib/virtuoso/db/virtuoso.log > LockFile = ../var/lib/virtuoso/db/virtuoso.lck > TransactionFile = ../var/lib/virtuoso/db/virtuoso.trx > xa_persistent_file = ../var/lib/virtuoso/db/virtuoso.pxa > ErrorLogLevel = 7 > FileExtend = 200 > MaxCheckpointRemap = 150000 > Striping = 0 > TempStorage = TempDatabase > > [TempDatabase] > DatabaseFile = ../var/lib/virtuoso/db/virtuoso-temp.db > TransactionFile = ../var/lib/virtuoso/db/virtuoso-temp.trx > MaxCheckpointRemap = 9000 > Striping = 0 > > [Parameters] > ServerPort = 1112 > LiteMode = 0 > DisableUnixSocket = 1 > DisableTcpSocket = 0 > MaxClientConnections = 10000 > ServerThreads = 1000 > CheckpointInterval = 60 > O_DIRECT = 0 > CaseMode = 2 > MaxStaticCursorRows = 5000 > CheckpointAuditTrail = 0 > AllowOSCalls = 0 > SchedulerInterval = 10 > DirsAllowed = ., ../share/virtuoso/vad,../saleem/dumps, > /opt/benchset/bibsonomy > ThreadCleanupInterval = 1 > ThreadThreshold = 1000 > ResourcesCleanupInterval = 1 > FreeTextBatchSize = 100000 > SingleCPU = 0 > VADInstallDir = ../share/virtuoso/vad/ > PrefixResultNames = 0 > RdfFreeTextRulesSize = 100 > IndexTreeMaps = 512 > MaxMemPoolSize = 200000000 > PrefixResultNames = 0 > MacSpotlight = 0 > MaxQueryMem = 8G ; memory allocated to query processor > VectorSize = 1000 ; initial parallel query vector > (array of query operations) size > MaxVectorSize = 2000000 ; query vector size threshold. > AdjustVectorSize = 0 > ThreadsPerQuery = 24 > AsyncQueueMaxThreads = 36 > NumberOfBuffers = 1360000 > MaxDirtyBuffers = 1000000 > TraceOn = errors > CallstackOnException = 1 > MaxSortedTopRows = 10000000 > DefaultIsolation = 2 > > [HTTPServer] > ServerPort = 8891 > ServerRoot = ../var/lib/virtuoso/vsp > MaxClientConnections = 10000 > DavRoot = DAV > EnabledDavVSP = 0 > HTTPProxyEnabled = 0 > TempASPXDir = 0 > DefaultMailServer = localhost:25 > ServerThreads = 1000 > MaxKeepAlives = 100 > KeepAliveTimeout = 10 > MaxCachedProxyConnections = 10 > ProxyConnectionCacheTimeout = 15 > HTTPThreadSize = 280000 > HttpPrintWarningsInOutput = 0 > Charset = UTF-8 > MaintenancePage = atomic.html > EnabledGzipContent = 1 > > [AutoRepair] > BadParentLinks = 0 > > [Client] > SQL_PREFETCH_ROWS = 100 > SQL_PREFETCH_BYTES = 16000 > SQL_QUERY_TIMEOUT = 0 > SQL_TXN_TIMEOUT = 0 > > [VDB] > ArrayOptimization = 0 > NumArrayParameters = 10 > VDBDisconnectTimeout = 1000 > KeepConnectionOnFixedThread = 0 > > [Replication] > ServerName = db-AKSWDESKTOP > ServerEnable = 1 > QueueMax = 50000 > > [Striping] > Segment1 = 8G, db-seg1-1.db, db-seg1-2.db > Segment2 = 8G, db-seg2-1.db, db-seg2-2.db > > [Zero Config] > ServerName = virtuoso (AKSWDESKTOP) > > [Mono] > > [URIQA] > DynamicLocal = 0 > DefaultHost = localhost:8890 > > [SPARQL] > DefaultGraph = http://test.hs-albsig.de/benchset > <http://test.hs-albsig.de/benchset> > ResultSetMaxRows = 10000000 > DefaultQuery = select distinct ?Concept where {[] a ?Concept} > LIMIT 100 > DeferInferenceRulesInit = 0 ; controls inference rules loading > > [Plugins] > LoadPath = ../lib/virtuoso/hosting > > > Best regards > Andy Nolle > > ------------------------------------------------------------------------------ > Site24x7 APM Insight: Get Deep Visibility into Application Performance > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > Monitor end-to-end web transactions and take corrective actions now > Troubleshoot faster and improve end-user experience. Signup Now! > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________ > > <http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________> > Virtuoso-users mailing list > Virtuoso-users@lists.sourceforge.net > <mailto:Virtuoso-users@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/virtuoso-users > <https://lists.sourceforge.net/lists/listinfo/virtuoso-users>
smime.p7s
Description: S/MIME cryptographic signature
------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________ Virtuoso-users mailing list Virtuoso-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/virtuoso-users