Hi Andreas, Missed that you were running 4 Virtuoso instances, how is the query workload split across the 4 instances ?
I noticed the following errors in all the logs during done attempt to restart:
09:23:02 ERROR: Unable to lock file ../var/lib/virtuoso/db/virtuoso.lck
(Resource temporarily unavailable).
09:23:02 ERROR: Virtuoso is already runnning (pid 1264)
09:23:02 ERROR: This probably means you either do not have permission to start
09:23:02 ERROR: this server, or that virtuoso-t is already running.
09:23:02 ERROR: If you are absolutely sure that this is not the case, please try
09:23:02 ERROR: to remove the file ../var/lib/virtuoso/db/virtuoso.lck and
start again.
09:26:41 INFO: ERRS_0 01V01 QW004 <unspec>:2891:
WS.WS.SPARQL_ENDPOINT_GENERATE_FORM: Incompatible types INTEGER (189) and
VARCHAR (182) in = for debug and <constant>
09:34:13 DEBUG: missed delete of name id cache
benchset/d4e1a90786ac524b9f96f9d2f0506a12 0 (0x20e975b )
09:34:16 DEBUG: missed delete of name id cache
benchset/265b6be68f2030063c4c17e3778dbbe1 0 (0x2122444 )
09:35:50 DEBUG: missed delete of name id cache
benchset/8d0299f2a49cf965ac7c98101f55afaa 0 (0x2181ac6 )
But then did not occur on the following startup, do you know what happened here
as it seem there was an attempt to start an already running server, but I have
not seen the "DEBUG: missed delete of name id cache …” errors before …
In the status_8895 output I see many occurrences of:
Pending:
1126400: IER 141.87.4.9
156: IER 141.87.4.9
152: IER 141.87.4.9
148: IER 141.87.4.9
144: IER 141.87.4.9
.
.
.
Do these remain indefinitely or do they get cleanup after a while ?
Also you indicated that this errors had not been seen with the 3215 build
upgrade, is this still the case ?
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
<http://www.openlinksw.com/>
Weblog -- http://www.openlinksw.com/blogs/ <http://www.openlinksw.com/blogs/>
LinkedIn -- http://www.linkedin.com/company/openlink-software/
<http://www.linkedin.com/company/openlink-software/>
Twitter -- http://twitter.com/OpenLink <http://twitter.com/OpenLink>
Google+ -- http://plus.google.com/100570109519069333827/
<http://plus.google.com/100570109519069333827/>
Facebook -- http://www.facebook.com/OpenLinkSoftware
<http://www.facebook.com/OpenLinkSoftware>
Universal Data Access, Integration, and Management Technology Providers
> On 21 Feb 2016, at 13:46, Nolle, Andreas <no...@hs-albsig.de
> <mailto:no...@hs-albsig.de>> wrote:
>
> Hi Hugh,
>
> typical queries that are executed (each at any Virtuoso instance) are like
> the following:
>
> SELECT ?x ?P0src ?P1src
> WHERE {
> {
> SERVICE <http://141.87.4.8:8899/sparql <http://141.87.4.8:8899/sparql>>
> {
> ?x rdf:type <http://purl.org/ontology/bibo/Collection
> <http://purl.org/ontology/bibo/Collection>> .
> } .
> BIND (<http://141.87.4.8:8899/sparql <http://141.87.4.8:8899/sparql>>
> AS ?P0src)
> }
> UNION
> {
> ?x rdf:type <http://purl.org/ontology/bibo/Collection
> <http://purl.org/ontology/bibo/Collection>> .
> BIND (<http://141.87.4.8:8891/sparql <http://141.87.4.8:8891/sparql>>
> AS ?P0src)
> }
> UNION
> {
> SERVICE <http://141.87.4.8:8893/sparql <http://141.87.4.8:8893/sparql>>
> {
> ?x rdf:type <http://purl.org/ontology/bibo/Collection
> <http://purl.org/ontology/bibo/Collection>> .
> } .
> BIND (<http://141.87.4.8:8893/sparql <http://141.87.4.8:8893/sparql>>
> AS ?P0src)
> }
> UNION
> {
> SERVICE <http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>>
> {
> ?x rdf:type <http://purl.org/ontology/bibo/Collection
> <http://purl.org/ontology/bibo/Collection>> .
> } .
> BIND (<http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>>
> AS ?P0src)
> } .
> ?x rdf:type <http://swrc.ontoware.org/ontology#Article
> <http://swrc.ontoware.org/ontology#Article>> .
> BIND (<http://141.87.4.8:8891/sparql <http://141.87.4.8:8891/sparql>> AS
> ?P1src)
> }
>
> Since it is not possible to use the keyword OFFSET for queries, i.e. query
> parts, that would return more than 1048576 results, there are also queries
> like:
>
> SELECT ?x ?a ?b ?P0src ?P1src
> WHERE {
> {
> SERVICE <http://141.87.4.8:8899/sparql <http://141.87.4.8:8899/sparql>>
> {
> ?x <http://purl.org/dc/terms/partOf
> <http://purl.org/dc/terms/partOf>> ?a .
> FILTER ( ?x >=
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) .
> FILTER ( ?x <
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) .
> } .
> BIND (<http://141.87.4.8:8899/sparql <http://141.87.4.8:8899/sparql>>
> AS ?P0src)
> }
> UNION
> {
> SERVICE <http://141.87.4.8:8891/sparql <http://141.87.4.8:8891/sparql>>
> {
> ?x <http://purl.org/dc/terms/partOf
> <http://purl.org/dc/terms/partOf>> ?a .
> FILTER ( ?x >=
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) .
> FILTER ( ?x <
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) .
> } .
> BIND (<http://141.87.4.8:8891/sparql <http://141.87.4.8:8891/sparql>>
> AS ?P0src)
> }
> UNION
> {
> ?x <http://purl.org/dc/terms/partOf <http://purl.org/dc/terms/partOf>>
> ?a .
> FILTER ( ?x >=
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) .
> FILTER ( ?x <
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) .
> BIND (<http://141.87.4.8:8893/sparql <http://141.87.4.8:8893/sparql>>
> AS ?P0src)
> }
> UNION
> {
> SERVICE <http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>>
> {
> ?x <http://purl.org/dc/terms/partOf
> <http://purl.org/dc/terms/partOf>> ?a .
> FILTER ( ?x >=
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) .
> FILTER ( ?x <
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) .
> } .
> BIND (<http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>>
> AS ?P0src)
> BIND (<http://141.87.4.8:8895/sparql <http://141.87.4.8:8895/sparql>>
> AS ?P0src)
> } .
> ?b <http://www.aktors.org/ontology/portal#article-of-journal
> <http://www.aktors.org/ontology/portal#article-of-journal>> ?x .
> FILTER ( ?x >=
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp
> <http://bibsonomy.org/uri/bibtexkey/conf/icmcs/NagatomoYOIT02/dblp>> ) .
> FILTER ( ?x < <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp
> <http://bibsonomy.org/uri/bibtexkey/conf/wpmc/EndoMA14/dblp>> ) .
> BIND (<http://141.87.4.8:8893/sparql <http://141.87.4.8:8893/sparql>> AS
> ?P1src)
> }
>
> where the filter parts are generated beforehand by querying all individuals
> of a specific concept piece by piece for each 1048576 results.
>
> The error that occur is usually:
> Virtuoso HTCLI Error HC001: Read Error in HTTP Client
>
> But as I remember other errors that occurred during several tests over the
> last weeks are:
> - No Http Response
> - Transaction aborted
> - Timeout
> - target server failed to respond
> - …
>
> Please find the output of status() for each Virtuoso instance (running at
> port 8891, 8893, 8895, 8899) as well as the corresponding log in the zip file
> at https://www.dropbox.com/s/yzmrybj7jf8m8az/files.zip?dl=0
> <https://www.dropbox.com/s/yzmrybj7jf8m8az/files.zip?dl=0>. The call of
> status() was done after I received a http error from Virtuoso instance 8893.
> For the same Virtuoso instance (running at port 8893) I got a timeout on
> connecting by isql. Only after a restart it was possible to connect via isql.
>
> The reason why I assign only 16 GB per Virtuoso instance (4 x 16 GB in total)
> is that each instance is take still more than 16 GB after some hours or days.
> In my case I run up to 25000 queries (like the one above) at each Virtuoso
> instance, but maximal 16 at one time over all instances. If you look at the
> memory usage it starts normally but it increases over time as long as the
> swap space is overcrowded… If this will be the case my code (running on
> another server) restarts the whole server hosting the Virtuoso instances
> before new SPARQL queries will be send. Please not that this behavior
> occurred especially on previous versions (<7.2.2.3215) and I’m not sure if
> this is still the case for version 7.2.2.3215 since I updated as recently as
> two weeks ago…
>
> My Virtuoso instances hosting the following triple counts:
> - Virtuoso instance 8891: 17765873
> - Virtuoso instance 8893: 27897291
> - Virtuoso instance 8895: 168888956
> - Virtuoso instance 8899: 72372256
>
> Best
> Andy
>
>
>
> Von: Hugh Williams [mailto:hwilli...@openlinksw.com
> <mailto:hwilli...@openlinksw.com>]
> Gesendet: Sonntag, 21. Februar 2016 02:51
> An: Nolle, Andreas <no...@hs-albsig.de <mailto:no...@hs-albsig.de>>
> Cc: virtuoso-users@lists.sourceforge.net
> <mailto:virtuoso-users@lists.sourceforge.net>
> Betreff: Re: [Virtuoso-users] infrequent errors on parallel querying
>
> Hi Andy,
>
> What is a typical query being executing that is result in a HTTP error and
> what is the actual error ?
>
> When the error occurs what is the status of the server at that point , please
> provide the output of running the command:
>
> status();
>
> Are any errors reported in the “virtuoos.log” file, please provide a copy for
> review ?
>
> You indicate having a 128GB RAM machine but memory allocated to Virtuoso is
> that for a 16GB RAM machine ie NumberOfBuffer = 1360000 in the INI file, was
> there a specific reason for this ?
>
> How many triples are hosted in the Virtuoso instance , please provide the
> output of the count query:
>
> select count(*) where {?s ?p ?o}
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc. // http://www.openlinksw.com/
> <http://www.openlinksw.com/>
> Weblog -- http://www.openlinksw.com/blogs/
> <http://www.openlinksw.com/blogs/>
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> <http://www.linkedin.com/company/openlink-software/>
> Twitter -- http://twitter.com/OpenLink <http://twitter.com/OpenLink>
> Google+ -- http://plus.google.com/100570109519069333827/
> <http://plus.google.com/100570109519069333827/>
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> <http://www.facebook.com/OpenLinkSoftware>
> Universal Data Access, Integration, and Management Technology Providers
>
>
>
> On 20 Feb 2016, at 15:47, Nolle, Andreas <no...@hs-albsig.de
> <mailto:no...@hs-albsig.de>> wrote:
>
> Hi folks,
>
> I’ve set up four Virtuoso instances (Version 7.2.2.3215) on an Ubuntu 14.04.4
> LTS server with 12 cores and 128 GB memory.
> If I run several (federated) queries in parallel (maximal 16 at a time),
> where some of the queries may take several minutes, it may happened that a
> SPARQL endpoint return some HTTP error. If I rerun the same query some
> seconds later, it will ordinarily be evaluated correct and the requested
> results are returned.
> I’ve already tried to solve this infrequent issue but without any success.
> Can you please give me some recommendations?
> Maybe my configuration may comprise some not well-chosen parameter?
>
>
> Please find below my virtuoso.ini:
>
> [Database]
> DatabaseFile = ../var/lib/virtuoso/db/virtuoso.db
> ErrorLogFile = ../var/lib/virtuoso/db/virtuoso.log
> LockFile = ../var/lib/virtuoso/db/virtuoso.lck
> TransactionFile = ../var/lib/virtuoso/db/virtuoso.trx
> xa_persistent_file = ../var/lib/virtuoso/db/virtuoso.pxa
> ErrorLogLevel = 7
> FileExtend = 200
> MaxCheckpointRemap = 150000
> Striping = 0
> TempStorage = TempDatabase
>
> [TempDatabase]
> DatabaseFile = ../var/lib/virtuoso/db/virtuoso-temp.db
> TransactionFile = ../var/lib/virtuoso/db/virtuoso-temp.trx
> MaxCheckpointRemap = 9000
> Striping = 0
>
> [Parameters]
> ServerPort = 1112
> LiteMode = 0
> DisableUnixSocket = 1
> DisableTcpSocket = 0
> MaxClientConnections = 10000
> ServerThreads = 1000
> CheckpointInterval = 60
> O_DIRECT = 0
> CaseMode = 2
> MaxStaticCursorRows = 5000
> CheckpointAuditTrail = 0
> AllowOSCalls = 0
> SchedulerInterval = 10
> DirsAllowed = ., ../share/virtuoso/vad,../saleem/dumps,
> /opt/benchset/bibsonomy
> ThreadCleanupInterval = 1
> ThreadThreshold = 1000
> ResourcesCleanupInterval = 1
> FreeTextBatchSize = 100000
> SingleCPU = 0
> VADInstallDir = ../share/virtuoso/vad/
> PrefixResultNames = 0
> RdfFreeTextRulesSize = 100
> IndexTreeMaps = 512
> MaxMemPoolSize = 200000000
> PrefixResultNames = 0
> MacSpotlight = 0
> MaxQueryMem = 8G ; memory allocated to query processor
> VectorSize = 1000 ; initial parallel query vector
> (array of query operations) size
> MaxVectorSize = 2000000 ; query vector size threshold.
> AdjustVectorSize = 0
> ThreadsPerQuery = 24
> AsyncQueueMaxThreads = 36
> NumberOfBuffers = 1360000
> MaxDirtyBuffers = 1000000
> TraceOn = errors
> CallstackOnException = 1
> MaxSortedTopRows = 10000000
> DefaultIsolation = 2
>
> [HTTPServer]
> ServerPort = 8891
> ServerRoot = ../var/lib/virtuoso/vsp
> MaxClientConnections = 10000
> DavRoot = DAV
> EnabledDavVSP = 0
> HTTPProxyEnabled = 0
> TempASPXDir = 0
> DefaultMailServer = localhost:25
> ServerThreads = 1000
> MaxKeepAlives = 100
> KeepAliveTimeout = 10
> MaxCachedProxyConnections = 10
> ProxyConnectionCacheTimeout = 15
> HTTPThreadSize = 280000
> HttpPrintWarningsInOutput = 0
> Charset = UTF-8
> MaintenancePage = atomic.html
> EnabledGzipContent = 1
>
> [AutoRepair]
> BadParentLinks = 0
>
> [Client]
> SQL_PREFETCH_ROWS = 100
> SQL_PREFETCH_BYTES = 16000
> SQL_QUERY_TIMEOUT = 0
> SQL_TXN_TIMEOUT = 0
>
> [VDB]
> ArrayOptimization = 0
> NumArrayParameters = 10
> VDBDisconnectTimeout = 1000
> KeepConnectionOnFixedThread = 0
>
> [Replication]
> ServerName = db-AKSWDESKTOP
> ServerEnable = 1
> QueueMax = 50000
>
> [Striping]
> Segment1 = 8G, db-seg1-1.db, db-seg1-2.db
> Segment2 = 8G, db-seg2-1.db, db-seg2-2.db
>
> [Zero Config]
> ServerName = virtuoso (AKSWDESKTOP)
>
> [Mono]
>
> [URIQA]
> DynamicLocal = 0
> DefaultHost = localhost:8890
>
> [SPARQL]
> DefaultGraph = http://test.hs-albsig.de/benchset
> <http://test.hs-albsig.de/benchset>
> ResultSetMaxRows = 10000000
> DefaultQuery = select distinct ?Concept where {[] a ?Concept}
> LIMIT 100
> DeferInferenceRulesInit = 0 ; controls inference rules loading
>
> [Plugins]
> LoadPath = ../lib/virtuoso/hosting
>
>
> Best regards
> Andy Nolle
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________
>
> <http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________>
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
> <mailto:Virtuoso-users@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
> <https://lists.sourceforge.net/lists/listinfo/virtuoso-users>
smime.p7s
Description: S/MIME cryptographic signature
------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________ Virtuoso-users mailing list Virtuoso-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/virtuoso-users
