Re: [Virtuoso-users] System setup guidelines

[Hugh Williams]

Hi Dan,

The main places Virtuoso setup and tuning information can be found are:

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#ptune
http://docs.openlinksw.com/virtuoso/rdfperformancetuning.html

I saw the git issue and commit you made regarding the NFS issue you encountered 
which we shall be reviewing.

The only issue I have come across with NFS was with regards to file locking 
required by the virtuoso.lck, thus an INI file param is provided to specify an 
alternative location for it as detailed at:

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#ini_Database

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 6 Apr 2016, at 19:57, Davis, Daniel (NIH/NLM) [C]  
> wrote:
> 
> Maybe I’m missing something, but is there any place in the documentation I 
> simple list of the things you should do to setup Virtuoso:
> ·Don’t use NFS
> ·If you do use NFS, don’t use soft mounts
> ·The change to add “vm.swappiness = 10” to /etc/sysctl.conf that is 
> mentioned in the RDF performance tuning chapter.
> ·Any changes to limits, e.g. setrlimit/ulimit for the virtuoso user?
>  
> Thanks,
>  
> Dan Davis, Systems/Applications Architect (Contractor),
> Office of Computer and Communications Systems,
> National Library of Medicine, NIH
>  
> --
> ___
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users



smime.p7s
Description: S/MIME cryptographic signature
--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] OAuth request_token endpoint returns OAuth Verification Failed: Bad Signature

[Hugh Williams]

Hi Sunday,

What you really need is fine-grained ACLs using the Virtuoso Authentication 
Layer (VAL) for controlling user access to data/graphs, which is a commercial 
only feature ie not part of the open source product. See the following 
presentation on this feature:


http://www.slideshare.net/kidehen/how-virtuoso-enables-attributed-based-access-controls

We are in the process of updating the online documentation on OAuth integration 
with VAL, but there are some details on how this can be used with the Virtuoso 
sponger middleware at:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/SpongerOAuthSupport

There is a programmers guide for OAuth documented at:


http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtuosoOAuthProgrammersGuide

and is applicable to open source.

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 6 Apr 2016, at 18:08, Sunday Ayandokun  wrote:
> 
> Hi Hughs,
> 
> Once again, thanks. I checked out the updated docs and I was able to use the 
> /oauth endpoint.
> 
> I have a scenario I will like to get more insight on it.
> 
> This is my scenario, 
> I have an application that connects to users personal graph on virtuosos, 
> which allows individual user manages their data.
> 
> My implementation should allow other external applications to access user 
> data when the user delegate that application to do so via oauth access. 
> 
> My initial thought is to make use of the oauth/ access mechanism for each 
> user to grant access to their data. 
> 
> But I see that from here http://localhost:8890/oauth/oauth_apps.vspx I can 
> only generate token for SPARQL and Discussion applications.
> 
> I want to be able to generate consumer_key and consumer_secret for each 
> application that wants to fetch users data.
> 
> Questions
> 1. Is there any other way to achieve this with oauth in virtuosos.
> 2. Is it possible to make a request to /oauth to programmatically generate 
> the consumer_key and consumer_secret for different applications(developers). 
> 
> 
> Kind Regards.
> Sunday Ayandokun
> University of Tartu
> Tel: +37256724770
> Skype: sunday.ayandokun
> 
>> On Mar 31, 2016, at 3:07 PM, Hugh Williams  wrote:
>> 
>> Hi Sunday,
>> 
>> The 
>> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtOAuthSPARQL  
>> was pointing to and old ods framework vad and has been updated to point to 
>> the latest one for the 7.2.2 release. Note also that a different ods 
>> framework vad is require for open source and commercial as indicated on the 
>> page also. 
>> 
>> You can also build the required ods framework vad if compiling from open 
>> source archive by passing the --enable-ods-vad configure option …
>> 
>> Best Regards
>> Hugh Williams
>> Professional Services
>> OpenLink Software, Inc.  //  http://www.openlinksw.com/
>> Weblog   -- http://www.openlinksw.com/blogs/
>> LinkedIn -- http://www.linkedin.com/company/openlink-software/
>> Twitter  -- http://twitter.com/OpenLink
>> Google+  -- http://plus.google.com/100570109519069333827/
>> Facebook -- http://www.facebook.com/OpenLinkSoftware
>> Universal Data Access, Integration, and Management Technology Providers
>> 
>> 
>> 
>>> On 30 Mar 2016, at 21:29, Sunday Ayandokun  
>>> wrote:
>>> 
>>> Hello,
>>> 
>>> I am having issue connecting to the OAuth to get token.
>>> 
>>> I installed the ods_framework_dav.vad package and generated SPARQL 
>>> application token and secret following the example here 
>>> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtOAuthSPARQL
>>> 
>>> 
>>> I read this http://docs.openlinksw.com/virtuoso/voauth.html to understand 
>>> how Oauth request should be made to Virtuoso Oauth endpoint.
>>> 
>>> 
>>> http://localhost:8890/OAuth/request_token?oauth_version=1.0_nonce=72180780940317254641459358880_timestamp=1459299214_consumer_key=client_key_signature_method=HMAC-SHA1_signature=KmF8ngGHSBQJIDHTDw%2FWP%2Fdqgmc%3D
>>> 
>>> I am using a python library to sign the request.
>>> 
>>> client_key= token
>>> client_secret = secret
>>> 
>>> client = oauth1.Client(client_key, client_secret=client_secret)
>>> uri, headers, body = client.sign(request_token_url)
>>> 
>>> I am getting this response 'OAuth Verification Failed: Bad Signature'
>>> 
>>> 
>>> Kindly point me to the best approach.
>>> 
>>> 
>>> Kind Regards.
>>> Sunday Ayandokun
>>> University of Tartu
>>> Tel: +37256724770
>>> Skype: sunday.ayandokun
>>> 
>>> 

[Virtuoso-users] System setup guidelines

[Davis, Daniel (NIH/NLM) [C]]

Maybe I'm missing something, but is there any place in the documentation I 
simple list of the things you should do to setup Virtuoso:

*Don't use NFS

*If you do use NFS, don't use soft mounts

*The change to add "vm.swappiness = 10" to /etc/sysctl.conf that is 
mentioned in the RDF performance tuning chapter.

*Any changes to limits, e.g. setrlimit/ulimit for the virtuoso user?

Thanks,

Dan Davis, Systems/Applications Architect (Contractor),
Office of Computer and Communications Systems,
National Library of Medicine, NIH

--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

Re: [Virtuoso-users] OAuth request_token endpoint returns OAuth Verification Failed: Bad Signature

[Sunday Ayandokun]

Hi Hughs,

Once again, thanks. I checked out the updated docs and I was able to use the 
/oauth endpoint.

I have a scenario I will like to get more insight on it.

This is my scenario, 
I have an application that connects to users personal graph on virtuosos, which 
allows individual user manages their data.

My implementation should allow other external applications to access user data 
when the user delegate that application to do so via oauth access. 

My initial thought is to make use of the oauth/ access mechanism for each user 
to grant access to their data. 

But I see that from here http://localhost:8890/oauth/oauth_apps.vspx 
 I can only generate token for 
SPARQL and Discussion applications.

I want to be able to generate consumer_key and consumer_secret for each 
application that wants to fetch users data.

Questions
1. Is there any other way to achieve this with oauth in virtuosos.
2. Is it possible to make a request to /oauth to programmatically generate the 
consumer_key and consumer_secret for different applications(developers). 


Kind Regards.
Sunday Ayandokun
University of Tartu
Tel: +37256724770
Skype: sunday.ayandokun

> On Mar 31, 2016, at 3:07 PM, Hugh Williams  wrote:
> 
> Hi Sunday,
> 
> The 
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtOAuthSPARQL 
>   
> was pointing to and old ods framework vad and has been updated to point to 
> the latest one for the 7.2.2 release. Note also that a different ods 
> framework vad is require for open source and commercial as indicated on the 
> page also. 
> 
> You can also build the required ods framework vad if compiling from open 
> source archive by passing the --enable-ods-vad configure option …
> 
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc.  //  http://www.openlinksw.com/ 
> 
> Weblog   -- http://www.openlinksw.com/blogs/ 
> 
> LinkedIn -- http://www.linkedin.com/company/openlink-software/ 
> 
> Twitter  -- http://twitter.com/OpenLink 
> Google+  -- http://plus.google.com/100570109519069333827/ 
> 
> Facebook -- http://www.facebook.com/OpenLinkSoftware 
> 
> Universal Data Access, Integration, and Management Technology Providers
> 
> 
> 
>> On 30 Mar 2016, at 21:29, Sunday Ayandokun > > wrote:
>> 
>> Hello,
>> 
>> I am having issue connecting to the OAuth to get token.
>> 
>> I installed the ods_framework_dav.vad package and generated SPARQL 
>> application token and secret following the example here 
>> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtOAuthSPARQL 
>> 
>> 
>> 
>> I read this http://docs.openlinksw.com/virtuoso/voauth.html 
>>  to understand how Oauth 
>> request should be made to Virtuoso Oauth endpoint.
>> 
>> 
>> http://localhost:8890/OAuth/request_token?oauth_version=1.0_nonce=72180780940317254641459358880_timestamp=1459299214_consumer_key=client_key_signature_method=HMAC-SHA1_signature=KmF8ngGHSBQJIDHTDw%2FWP%2Fdqgmc%3D
>>  
>> 
>> 
>> I am using a python library to sign the request.
>> 
>> client_key= token
>> client_secret = secret
>> 
>> client = oauth1.Client(client_key, client_secret=client_secret)
>> uri, headers, body = client.sign(request_token_url)
>> 
>> I am getting this response 'OAuth Verification Failed: Bad Signature'
>> 
>> 
>> Kindly point me to the best approach.
>> 
>> 
>> Kind Regards.
>> Sunday Ayandokun
>> University of Tartu
>> Tel: +37256724770
>> Skype: sunday.ayandokun
>> 
>> --
>> Transform Data into Opportunity.
>> Accelerate data analysis in your applications with
>> Intel Data Analytics Acceleration Library.
>> Click to learn more.
>> http://pubads.g.doubleclick.net/gampad/clk?id=278785471=/4140___
>>  
>> 
>> Virtuoso-users mailing list
>> Virtuoso-users@lists.sourceforge.net 
>> 
>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
> 

--

Re: [Virtuoso-users] list graph query performance in V7.2.2 ?

[Hugh Williams]

Hi Thomas,

You can use the Virtuoso PL Debugger to see what procedures/statements are 
running on the server continuously as follows:

$ isql  dba  -D
OpenLink Interactive PL Debugger (Virtuoso).
Type EXIT to exit, HELP for help
Debug session estabilished to 
DEBUG> info threads
@:4 in DB.DBA.test_proc1 () at 11
DEBUG>

See the following documentation of additional details on usage:

http://docs.openlinksw.com/virtuoso/pldebugger.html#pldebugger

With regards to the "SR175: Uniqueness violation : Violating unique index 
VSPX_SESSION on table DB.DBA.VSPX_SESSION. Transaction killed.” messages in 
your log, the DB.DBA.VSPX_SESSION table does not hold for a long period of 
time, thus you can try dropping the table and restarting Virtuoso which would 
automatically recreate it. As ever please ensure you have a Full backup in 
place first …

Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc.  //  http://www.openlinksw.com/
Weblog   -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter  -- http://twitter.com/OpenLink
Google+  -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers



> On 5 Apr 2016, at 15:24, Thomas Michaux  wrote:
> 
> ok, glad to know you're aware of these kind of issues...
> 
> don't know if it can help you but before your suggestion (when i had 
> AdjustVectorSize = 1) i was experimenting "waits" issues (message " * 
> Monitor: Locks are held for a long time" in the logs), i tried to 
> investigate with stats query (select top 10 * from sys_l_stat order by 
> waits desc;) and was only able to notice a lot of wait time was on 
> VSPX_SESSION index at the top of stats locks results.
> 
> ...and after your suggestion (AdjustVectorSize = 0), i dont' know if 
> it's linked, but i noticed these messages in the logs :
> 
> 12:56:52 OpenLink Virtuoso Universal Server
> 12:56:52 Version 07.20.3215-pthreads for Linux as of Mar 29 2016
> 12:56:52 uses parts of OpenSSL, PCRE, Html Tidy
> 12:57:02 Database version 3126
> 12:57:02 SQL Optimizer enabled (max 1000 layouts)
> 12:57:04 Compiler unit is timed at 0.000205 msec
> 12:57:21 Roll forward started
> 12:57:21 SQL Error: 23000 : SR175: Uniqueness violation : Violating 
> unique index VSPX_SESSION on table DB.DBA.VSPX_SESSION. Transaction killed.
> 12:57:21 In roll forward batch commit failed code 5
> 12:57:21 Rfwd error: 23000: SR175: Uniqueness violation : Violating 
> unique index VSPX_SESSION on table DB.DBA.VSPX_SESSION. Transaction killed.
> 12:57:21 SQL Error: 23000 : SR175: Uniqueness violation : Violating 
> unique index VSPX_SESSION on table DB.DBA.VSPX_SESSION. Transaction killed.
> 12:57:21 101 transactions, 10072 bytes replayed (100 %)
> 12:57:21 Roll forward complete
> 12:57:22 Checkpoint started
> 12:57:36 Checkpoint finished, new log is 
> /usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso20160405125721.trx
> 12:57:38 HTTP/WebDAV server online at 8890
> 12:57:38 Server online at  (pid 20868)
> 13:57:40 Checkpoint started
> 13:57:40 Checkpoint finished, new log is 
> /usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso20160405135739.trx
> 14:57:42 Checkpoint started
> 14:57:42 Checkpoint finished, new log is 
> /usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso20160405145741.trx
> 15:57:44 Checkpoint started
> 15:57:45 Checkpoint finished, new log is 
> /usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso20160405155743.trx
> 
> Thomas
> 
> 
> Le 05/04/2016 13:20, Hugh Williams a écrit :
>> Hi Thomas,
>> 
>> Yes, "AdjustVectorSize = 0” as it is the new default in 7.2.2 both in the 
>> INI files and if not set,  following similar problems display/counting of 
>> graphs and queries with long execution times …
>> 
>> Best Regards
>> Hugh Williams
>> Professional Services
>> OpenLink Software, Inc.  //  http://www.openlinksw.com/
>> Weblog   -- http://www.openlinksw.com/blogs/
>> LinkedIn -- http://www.linkedin.com/company/openlink-software/
>> Twitter  -- http://twitter.com/OpenLink
>> Google+  -- http://plus.google.com/100570109519069333827/
>> Facebook -- http://www.facebook.com/OpenLinkSoftware
>> Universal Data Access, Integration, and Management Technology Providers
>> 
>> 
>> 
>>> On 5 Apr 2016, at 12:03, Thomas Michaux  wrote:
>>> 
>>> yes it's improving (i added some grpah meanwhile so 46 rows is the correct 
>>> number) :
>>> 
>>> SPARQL SELECT DISTINCT ?g WHERE { GRAPH ?g { ?s ?p ?o } };
>>> 46 Rows. -- 1337 msec.
>>> 
>>> DB.DBA.SPARQL_SELECT_KNOWN_GRAPHS();
>>> 46 Rows. -- 434 msec.
>>> 
>>> I had already VectorSize   = 1000 in virtuoso.ini but I had 
>>> AdjustVectorSize = 1 so changing to 0 has an effect
>>> 
>>> We have 232 973 825, read only use case mainly, is it ok to keep 
>>> 

Re: [Virtuoso-users] obtaining a copy/dump of an online SPARQL endpoint in a nice manner

[Jörn Hees]

On 31 Mar 2016, at 22:10, Kingsley Idehen  wrote:
> 
> How are you arriving at data devoid or metadata about its origins?

Link rot... Endpoint is still working, dumps aren't...


> You would be better served, ultimately, instantiating a dedicated
> Virtuoso instance in the cloud for your specific needs. This instance
> could load datasets from wherever, using some of the existing endpoints
> (DBpedia and others) as a mechanism for exposing provenance data etc..

How would i do that? I'm not sure i fully understand... We have a Virtuoso 
instance in our local network for our working group already.
I usually manually load it with the dumps of datasets i want to run my 
algorithms against so i don't impede the public endpoints operations / get 
blocked for doing too many requests.


> There is no nice way of trying to dump all the data from an existing
> SPARQL endpoint.

If there is no nice way, is there any way at all?

I mean I can't be the first one who is interested in all triples a graph on a 
remote endpoint contains (even if that graph is big).


Best,
Jörn


--
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users