Re: [Virtuoso-users] Debugging JDBC Virtuoso connection
Hugh,
I figured it out. I used interactive Python to prove the port was open, and
then discovered my own web app was overriding the serverName I set in the
Tomcat resource.
From: Hugh Williams [hwilli...@openlinksw.com]
Sent: Thursday, August 11, 2016 9:47 AM
To: Davis, Daniel (NIH/NLM) [C]
Cc: virtuoso-users
Subject: Re: [Virtuoso-users] Debugging JDBC Virtuoso connection
Hi Daniel,
So the same application using the same JDBC connect string work on localhost
where Virtuoso runs, but not from remote client ?
I once had same problem when java runtime was trying to make ipv6 connection
from remote host and was getting “network is unreachable” type error and had to
disable ivp6 on the localhost machine Virtuoso was running on. So wondering if
you might be having similar problem ? I trace the cause with strace using
command like:
strace -f -v -o -s 1500 java -cp "virtjdbc4.jar:.”
sample.java
which showed something like:
10738 connect(8, {sa_family=AF_INET6, sin6_port=htons(1168), inet_pton(AF_INET6,
":::127.0.0.1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EN
ETUNREACH (Network is unreachable)
BTW, I assume you JDBC connect string is using the actual machine hostname or
IP address and not “localhost” from when is was just being run locally ?
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
On 10 Aug 2016, at 16:48, Davis, Daniel (NIH/NLM) [C]
mailto:daniel.da...@nih.gov>> wrote:
I’ve done my UI using Java Spring, but so far, I’ve been doing it as an old
dinosaur, running my build and my changes on the Linux host where Virtuoso is
running.
Now, I’m trying to use Virtuoso remotely from another host.
>From Python, I can connect to port , but when I try to connect to Virtuoso
>from Java, it does not seem to be working.
I would prefer to enable some logging from Virtuoso JDBC to see what the
problem is – it is throwing connection refused, but the connection should be
open.
What is it actually trying to connect with?
Thanks,
Dan Davis, Systems/Applications Architect (Contractor),
Office of Computer and Communications Systems,
National Library of Medicine, NIH
--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports.
http://sdm.link/zohodev2dev___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
Re: [Virtuoso-users] Debugging JDBC Virtuoso connection
Hi Daniel,
So the same application using the same JDBC connect string work on localhost
where Virtuoso runs, but not from remote client ?
I once had same problem when java runtime was trying to make ipv6 connection
from remote host and was getting “network is unreachable” type error and had to
disable ivp6 on the localhost machine Virtuoso was running on. So wondering if
you might be having similar problem ? I trace the cause with strace using
command like:
strace -f -v -o -s 1500 java -cp "virtjdbc4.jar:.”
sample.java
which showed something like:
10738 connect(8, {sa_family=AF_INET6, sin6_port=htons(1168), inet_pton(AF_INET6,
":::127.0.0.1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EN
ETUNREACH (Network is unreachable)
BTW, I assume you JDBC connect string is using the actual machine hostname or
IP address and not “localhost” from when is was just being run locally ?
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
> On 10 Aug 2016, at 16:48, Davis, Daniel (NIH/NLM) [C]
> wrote:
>
> I’ve done my UI using Java Spring, but so far, I’ve been doing it as an old
> dinosaur, running my build and my changes on the Linux host where Virtuoso is
> running.
> Now, I’m trying to use Virtuoso remotely from another host.
>
> From Python, I can connect to port , but when I try to connect to
> Virtuoso from Java, it does not seem to be working.
>
> I would prefer to enable some logging from Virtuoso JDBC to see what the
> problem is – it is throwing connection refused, but the connection should be
> open.
> What is it actually trying to connect with?
>
> Thanks,
>
> Dan Davis, Systems/Applications Architect (Contractor),
> Office of Computer and Communications Systems,
> National Library of Medicine, NIH
>
> --
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports.
> http://sdm.link/zohodev2dev___
>
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
>
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
>
smime.p7s
Description: S/MIME cryptographic signature
--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
Re: [Virtuoso-users] Long query - "Lost connection to server"
Hi Pantelis,
152M triple graphs is quite large to be performing such an operation on which
will require a significant amount of memory to complete. What is the total
number of triples in Virtuoso and how many of the buffers are in use when the
database is in use, which can be seen by running the status(); command from
isql. As you may be able to reduce the “NumberOfBuffers” param for database
workig set to make more memory available to the system for performing such
large insert queries.
You should also ensure the following params are set in the INI file to ensure
unused resource/threads are cleaned up immediately to maximise available memory:
[Parameters]
ThreadCleanupInterval= 1
ResourcesCleanupInterval = 1
...
See, http://docs.openlinksw.com/virtuoso/dbadm/
The server log does not show anything useful …
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
> On 10 Aug 2016, at 09:44, Pantelis Natsiavas wrote:
>
> Thank you Hugh for your answer.
>
> I have actually already used the respective pragma before the SPARUL query.
>
> SPARQL DEFINE sql:log-enable 3
> INSERT
>
> The documentation of the virtuoso functions is not clear for me. However, I
> think I have already done what you suggest. Right?
>
> The graph the SPARUL is run against is rather big. It is the
> "ClinicalTrials.gov" graph of the Bio2RDF, namely 152.725.529 triplets. My VM
> has 32GBs RAM and the memory consumption of the virtuoso process starts from
> 40% and constantly increases until the break down (I have seen 70% memory
> consumption but I cannot be certain for the peak). When the operation breaks
> down, the memory is not fully released. It drops down to around 40%. Even
> though in the log file it is shown that virtuoso restarts, I have to restart
> the VM in order to get my full memory back.
>
> I have repeated the process today and I am attaching the respective log file
> part (I though it might be useful). Perhaps you can understand more from the
> log, or you could give me instructions for more detailed logging.
>
> Please note that I have already followed in the instructions regarding the
> buffering (General Memory Usage Settings) and the swapping (Linux-only --
> "swappiness") on the
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning.
>
> I really appreciate your help.
>
> Kind regards,
> Pantelis Natsiavas
>
> 2016-08-10 4:09 GMT+03:00 Hugh Williams :
> Hi Pantelis,
>
> What is the memory consumption of Virtuoso whilst running the insert query
> and what is the size of the graph the insert is being performed against ? As
> note for SPARUL operation against graph with large amounts of data it is
> recommended queries are performed in row-wise auto commit mode to reduce
> memory consumption which case otherwise be depleted causing hangs or the
> server crash if memory cannot be allocated, with the Virtuoso log_enable
> function as detailed at:
>
> http://docs.openlinksw.com/virtuoso/fn_log_enable.html
>
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksGuideDeleteLargeGraphs
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc. // http://www.openlinksw.com/
> Weblog -- http://www.openlinksw.com/blogs/
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> Twitter -- http://twitter.com/OpenLink
> Google+ -- http://plus.google.com/100570109519069333827/
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> Universal Data Access, Integration, and Management Technology Providers
>
>> On 9 Aug 2016, at 12:11, Pantelis Natsiavas wrote:
>>
>> I have just retried the above long-time query and checked the virtuoso.log.
>> I see that after 45 minutes I get the following log entries (I executed the
>> query around 10.00):
>>
>> 10:24:23 * Monitor: High disk read (2)
>> 10:41:23 * Monitor: High disk read (2)
>> 10:43:24 * Monitor: High disk read (2)
>> 10:43:41 mmap failed with 12
>> 10:43:45 mmap failed with 12
>> 10:43:45 mmap failed with 12
>> 10:43:45 mmap failed with 12
>> 10:43:46 mmap failed with 12
>> 10:43:46 mmap failed with 12
>> 10:43:46 GPF: Dkpool.c:1634 could not allocate memory with mmap
>>
>> The same messages (more or less) had in the previous "lost connection"
>> incident.
>>
>> Could somebody provide some hints?
>>
>> Kind regards,
>> Pantelis Natsiavas
>>
>>
>> 2016-08-09 9:58 GMT+03:00 Pantelis Natsiavas :
>> Hi.
>>
>> I am trying to execute a SPARQL INSERT-WHERE taking a lot time to complete.
>>
>> INSERT {
