Re: [Virtuoso-users] SPARQL Linked Data: howto configure sponger behind IRI deref for variables

2010-08-03 Thread Kingsley Idehen 

Jörn Hees wrote:

On Tuesday 03 August 2010, Kingsley Idehen wrote:
  

[...]
Jörn Hees wrote:


If I use:
  DEFINE get:proxy "localhost:8890"

with this query:
  DEFINE input:grab-all "yes"
  DEFINE input:grab-depth 1
  SELECT DISTINCT ?p ?lp ?o ?lo
  WHERE {
  
 ?p ?o.

OPTIONAL { ?p rdfs:label ?lp. }
OPTIONAL { ?o rdfs:label ?lo. }
   
   }


[...]

Cheers,
Jörn
  

All,

Have you digested this tutorial re. use of Sponger for de-referencing IRIs?

See:
http://virtuoso.openlinksw.com/presentations/SPARQL_Tutorials/SPARQL_Tutori
als_Part_7/SPARQL_Tutorials_Part_7.html



Yes, actually that's what lead me to the query above.
I looked all of the pragmas up in 
http://docs.openlinksw.com/virtuoso/rdfiridereferencing.html and now went 
through it again, but can't find how it helps with my problem.


What am I missing out?
  


I have the suspicion that cache handling is actually disabled via your 
INI due to default settings.


Kingsley

Cheers,
Jörn

--
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
  



--

Regards,

Kingsley Idehen	  
President & CEO 
OpenLink Software 
Web: http://www.openlinksw.com

Weblog: http://www.openlinksw.com/blog/~kidehen
Twitter/Identi.ca: kidehen

Re: [Virtuoso-users] SPARQL Linked Data: howto configure sponger behind IRI deref for variables

2010-08-03 Thread Kingsley Idehen 

Jörn Hees wrote:

On Tuesday 03 August 2010, Ivan Mikhailov wrote:
  

What should be added to existing extension API configurable via
"define input:grab-resolver ..."
and "define input:grab-loader ..."
as they're described in
http://docs.openlinksw.com/virtuoso/rdfiridereferencing.html#rdfinputgrab



Sorry, saw your email too late.

Thanks for pointing met o input:grab-resolver, I think that's what I want...
Is there a HowTo for writing such an own grab-resolver method if you would 
want all IRIs to be resolved to your local store if they exist, use them; if 
not, fetch them and store them locally; not refetching anything more often 
than weekly? (searched your docs, but nothing found when I enter 
"RDF_GRAB_RESOLVER_DEFAULT"?!?)


Currently when calling the Proxy like this: 
http://localhost:8890/about/rdf/http/dbpedia.org/resource/Barack_Obama

it won't use the local DBpedia dump, as the graphs aren't "materialized":
I only have one graph from the DBpedia dump in the local store: 
"http://dbpedia.org";, all proxy-calls seem to introduce new graphs such as: 
"http://dbpedia.org/resource/Barack_Obama"; (and judging from the time they 
take it seems as if they don't use the local store) .

The same counts for all SPARQL-Queries doing a IRI-deref.

So would it be a good idea to substitute http://dbpedia.org with 
http://localhost:8890/ in the grab resolver?
Can I perhaps also do this for the proxy in total? (i.e., rewriting 
http://localhost:8890/about/rdf/http://dbpedia.org/resource/Barack_Obama to 
http://localhost:8890/resource/Barack_Obama )
  


What are your Virtuoso INI settings? The behavior you describe basically 
implies that cache handling is disabled.


The [SPARQL] section of the INI is what's relevant.

Kingsley

Cheers,
Jörn

--
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
  



--

Regards,

Kingsley Idehen	  
President & CEO 
OpenLink Software 
Web: http://www.openlinksw.com

Weblog: http://www.openlinksw.com/blog/~kidehen
Twitter/Identi.ca: kidehen

Re: [Virtuoso-users] SPARQL Linked Data: howto configure sponger behind IRI deref for variables

2010-08-03 Thread Jörn Hees 
On Tuesday 03 August 2010, Kingsley Idehen wrote:
> [...]
> Jörn Hees wrote:
> > If I use:
> >   DEFINE get:proxy "localhost:8890"
> > 
> > with this query:
> >   DEFINE input:grab-all "yes"
> >   DEFINE input:grab-depth 1
> >   SELECT DISTINCT ?p ?lp ?o ?lo
> >   WHERE {
> >   
> >  ?p ?o.
> > OPTIONAL { ?p rdfs:label ?lp. }
> > OPTIONAL { ?o rdfs:label ?lo. }
> >
> >}
> > 
> > [...]
> > 
> > Cheers,
> > Jörn
> 
> All,
> 
> Have you digested this tutorial re. use of Sponger for de-referencing IRIs?
> 
> See:
> http://virtuoso.openlinksw.com/presentations/SPARQL_Tutorials/SPARQL_Tutori
> als_Part_7/SPARQL_Tutorials_Part_7.html

Yes, actually that's what lead me to the query above.
I looked all of the pragmas up in 
http://docs.openlinksw.com/virtuoso/rdfiridereferencing.html and now went 
through it again, but can't find how it helps with my problem.

What am I missing out?

Cheers,
Jörn

Re: [Virtuoso-users] SPARQL Linked Data: howto configure sponger behind IRI deref for variables

2010-08-03 Thread Jörn Hees 
On Tuesday 03 August 2010, Ivan Mikhailov wrote:
> What should be added to existing extension API configurable via
> "define input:grab-resolver ..."
> and "define input:grab-loader ..."
> as they're described in
> http://docs.openlinksw.com/virtuoso/rdfiridereferencing.html#rdfinputgrab

Sorry, saw your email too late.

Thanks for pointing met o input:grab-resolver, I think that's what I want...
Is there a HowTo for writing such an own grab-resolver method if you would 
want all IRIs to be resolved to your local store if they exist, use them; if 
not, fetch them and store them locally; not refetching anything more often 
than weekly? (searched your docs, but nothing found when I enter 
"RDF_GRAB_RESOLVER_DEFAULT"?!?)

Currently when calling the Proxy like this: 
http://localhost:8890/about/rdf/http/dbpedia.org/resource/Barack_Obama
it won't use the local DBpedia dump, as the graphs aren't "materialized":
I only have one graph from the DBpedia dump in the local store: 
"http://dbpedia.org";, all proxy-calls seem to introduce new graphs such as: 
"http://dbpedia.org/resource/Barack_Obama"; (and judging from the time they 
take it seems as if they don't use the local store) .
The same counts for all SPARQL-Queries doing a IRI-deref.

So would it be a good idea to substitute http://dbpedia.org with 
http://localhost:8890/ in the grab resolver?
Can I perhaps also do this for the proxy in total? (i.e., rewriting 
http://localhost:8890/about/rdf/http://dbpedia.org/resource/Barack_Obama to 
http://localhost:8890/resource/Barack_Obama )

Cheers,
Jörn

Re: [Virtuoso-users] SPARQL Linked Data: howto configure sponger behind IRI deref for variables

2010-08-03 Thread Kingsley Idehen 

Jörn Hees wrote:

On Tuesday 03 August 2010, Nathan wrote:
  

Aldo Bucchi wrote:


Hi Ivan, Jorn,

I've run into similar requirements. A catch-all solution to this could
be to surface the ability to specify a custom deref procedure

define input:iri_deref "DB.DBA.MY_CUSTOM_IRI_DEREF"

This should get all the context to decide whether to perform a fetch
and also have access to the sponger infrastructure.

This is a broad and raw idea, I know. The point is that the current
visibility into what happens under the covers is too low. Of course,
in an ideal world, sources should play along with the Linked Data
rules ( including specifying caching TTL ) but in reality you might
  

and ETag's!



want to override some sources, implement specific caching strategies,
etc.

Spongers provide some flexibility in the "how do I get stuff from a
IRI" part. But perhaps the controller part could also be "extensible".
  

FWIW, +1 unless you can simply designate sponger to use a proxy, which
would solve all use cases from what I can tell.



Yupp, I definitely agree. AFAIK there currently is no possibility to make use 
of let's say a local DBpedia mirror in case you're using the IRI-dereferencing 
options:


If I use:
  DEFINE get:proxy "localhost:8890"
with this query:
  DEFINE input:grab-all "yes"
  DEFINE input:grab-depth 1
  SELECT DISTINCT ?p ?lp ?o ?lo
  WHERE {
 ?p ?o.
OPTIONAL { ?p rdfs:label ?lp. }
OPTIONAL { ?o rdfs:label ?lo. }
   }
I'm told that get:proxy only makes sense with a FROM clause. So is there any 
possibility to use a proxy (e.g., the current datastore's URL rewriting) for 
IRI dereferencing, or perhaps even Aldo's suggestion of a special 
DB.DBA.MY_CUSTOM_IRI_DEREF ?


Cheers,
Jörn
  


All,

Have you digested this tutorial re. use of Sponger for de-referencing IRIs?


See: 
http://virtuoso.openlinksw.com/presentations/SPARQL_Tutorials/SPARQL_Tutorials_Part_7/SPARQL_Tutorials_Part_7.html



Kingsley


--
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
  



--

Regards,

Kingsley Idehen	  
President & CEO 
OpenLink Software 
Web: http://www.openlinksw.com

Weblog: http://www.openlinksw.com/blog/~kidehen
Twitter/Identi.ca: kidehen

Re: [Virtuoso-users] SPARQL Linked Data: howto configure sponger behind IRI deref for variables

2010-08-03 Thread Jörn Hees 
On Tuesday 03 August 2010, Nathan wrote:
> Aldo Bucchi wrote:
> > Hi Ivan, Jorn,
> > 
> > I've run into similar requirements. A catch-all solution to this could
> > be to surface the ability to specify a custom deref procedure
> > 
> > define input:iri_deref "DB.DBA.MY_CUSTOM_IRI_DEREF"
> > 
> > This should get all the context to decide whether to perform a fetch
> > and also have access to the sponger infrastructure.
> > 
> > This is a broad and raw idea, I know. The point is that the current
> > visibility into what happens under the covers is too low. Of course,
> > in an ideal world, sources should play along with the Linked Data
> > rules ( including specifying caching TTL ) but in reality you might
> 
> and ETag's!
> 
> > want to override some sources, implement specific caching strategies,
> > etc.
> > 
> > Spongers provide some flexibility in the "how do I get stuff from a
> > IRI" part. But perhaps the controller part could also be "extensible".
> 
> FWIW, +1 unless you can simply designate sponger to use a proxy, which
> would solve all use cases from what I can tell.

Yupp, I definitely agree. AFAIK there currently is no possibility to make use 
of let's say a local DBpedia mirror in case you're using the IRI-dereferencing 
options:

If I use:
  DEFINE get:proxy "localhost:8890"
with this query:
  DEFINE input:grab-all "yes"
  DEFINE input:grab-depth 1
  SELECT DISTINCT ?p ?lp ?o ?lo
  WHERE {
 ?p ?o.
OPTIONAL { ?p rdfs:label ?lp. }
OPTIONAL { ?o rdfs:label ?lo. }
   }
I'm told that get:proxy only makes sense with a FROM clause. So is there any 
possibility to use a proxy (e.g., the current datastore's URL rewriting) for 
IRI dereferencing, or perhaps even Aldo's suggestion of a special 
DB.DBA.MY_CUSTOM_IRI_DEREF ?

Cheers,
Jörn

Re: [Virtuoso-users] SPARQL Linked Data: howto configure sponger behind IRI deref for variables

2010-08-03 Thread Ivan Mikhailov 
Hello Aldo,

On Mon, 2010-08-02 at 20:36 -0400, Aldo Bucchi wrote:
> Hi Ivan, Jorn,
> 
> A catch-all solution to this could
> be to surface the ability to specify a custom deref procedure
> 
> define input:iri_deref "DB.DBA.MY_CUSTOM_IRI_DEREF"
> 
> This should get all the context to decide whether to perform a fetch
> and also have access to the sponger infrastructure.

What should be added to existing extension API configurable via
"define input:grab-resolver ..."
and "define input:grab-loader ..."
as they're described in
http://docs.openlinksw.com/virtuoso/rdfiridereferencing.html#rdfinputgrab
?

Best Regards,

Ivan Mikhailov
OpenLink Software
http://virtuoso.openlinksw.com

Re: [Virtuoso-users] Examples of how to connect SQL from PHP (under apache)?

2010-08-03 Thread Kingsley Idehen 

Nathan wrote:

Matthew Tedder wrote:
  

Ok..

Maybe I am seeing how it works now.  I am not used to ODBC (usually, mysql,
postgresql, or mssql) and saw various examples under google searches
explicitly stating the database driver with such as "driver=..." in the odbc
connect statements.



Hi Matthew,

One PHP Dev to another, you can use the odbc functions:
   http://php.net/manual/en/book.uodbc.php

or you can use PDO with the ODBC drivers
   http://php.net/manual/en/ref.pdo-odbc.php

ODBC (Open Database Connectivity) can be used to connect to any 
database, not just Virtuoso - you can connect to MySQL through ODBC, 
Postgres through ODBC, and afaik virtually every other database out there.


Noted you've got a nice reply from Kingsley who's nailed all the facts, 
and honestly he's right - ODBC is a standardized API for DBMS solutions, 
if we all used ODBC rather than client specific connectors we'd have 
vendor portability and wouldn't need those 'clever' abstraction layers 
like PDO.
  


One additional comment. Nothing stops PDO sitting atop ODBC.

This page shows how pathetic things have become re. Open Data Access 
comprehension: http://www.php.net/manual/en/pdo.drivers.php :-(



Kingsley

Best & Good luck,

Nathan

--
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

  



--

Regards,

Kingsley Idehen	  
President & CEO 
OpenLink Software 
Web: http://www.openlinksw.com

Weblog: http://www.openlinksw.com/blog/~kidehen
Twitter/Identi.ca: kidehen

Re: [Virtuoso-users] SPARQL Linked Data: howto configure sponger behind IRI deref for variables

2010-08-03 Thread Nathan 

Aldo Bucchi wrote:

Hi Ivan, Jorn,

I've run into similar requirements. A catch-all solution to this could
be to surface the ability to specify a custom deref procedure

define input:iri_deref "DB.DBA.MY_CUSTOM_IRI_DEREF"

This should get all the context to decide whether to perform a fetch
and also have access to the sponger infrastructure.

This is a broad and raw idea, I know. The point is that the current
visibility into what happens under the covers is too low. Of course,
in an ideal world, sources should play along with the Linked Data
rules ( including specifying caching TTL ) but in reality you might


and ETag's!


want to override some sources, implement specific caching strategies,
etc.

Spongers provide some flexibility in the "how do I get stuff from a
IRI" part. But perhaps the controller part could also be "extensible".


FWIW, +1 unless you can simply designate sponger to use a proxy, which 
would solve all use cases from what I can tell.


Best,

Nathan

Re: [Virtuoso-users] Examples of how to connect SQL from PHP (under apache)?

2010-08-03 Thread Nathan 

Matthew Tedder wrote:

Ok..

Maybe I am seeing how it works now.  I am not used to ODBC (usually, mysql,
postgresql, or mssql) and saw various examples under google searches
explicitly stating the database driver with such as "driver=..." in the odbc
connect statements.


Hi Matthew,

One PHP Dev to another, you can use the odbc functions:
  http://php.net/manual/en/book.uodbc.php

or you can use PDO with the ODBC drivers
  http://php.net/manual/en/ref.pdo-odbc.php

ODBC (Open Database Connectivity) can be used to connect to any 
database, not just Virtuoso - you can connect to MySQL through ODBC, 
Postgres through ODBC, and afaik virtually every other database out there.


Noted you've got a nice reply from Kingsley who's nailed all the facts, 
and honestly he's right - ODBC is a standardized API for DBMS solutions, 
if we all used ODBC rather than client specific connectors we'd have 
vendor portability and wouldn't need those 'clever' abstraction layers 
like PDO.


Best & Good luck,

Nathan

Re: [Virtuoso-users] SPARQL Linked Data: howto configure sponger behind IRI deref for variables

2010-08-03 Thread Aldo Bucchi 
Hi Ivan, Jorn,

I've run into similar requirements. A catch-all solution to this could
be to surface the ability to specify a custom deref procedure

define input:iri_deref "DB.DBA.MY_CUSTOM_IRI_DEREF"

This should get all the context to decide whether to perform a fetch
and also have access to the sponger infrastructure.

This is a broad and raw idea, I know. The point is that the current
visibility into what happens under the covers is too low. Of course,
in an ideal world, sources should play along with the Linked Data
rules ( including specifying caching TTL ) but in reality you might
want to override some sources, implement specific caching strategies,
etc.

Spongers provide some flexibility in the "how do I get stuff from a
IRI" part. But perhaps the controller part could also be "extensible".

As usual, the smartest way to go about this is wait until there are
enough use cases. I see virtuoso-users is picking up some speed ;)

Regards,
A





On Sat, Jul 31, 2010 at 12:18 AM, Ivan Mikhailov
 wrote:
> Hello Jörn,
>
>> is there any way to configure the sponger behind this to
>> 1. cache its lookups, so don't retrieve graphs already present in the local
>> quad store (the DEFINE get:soft "soft" seems to be what I want, but that only
>> works for graphs mentioned in a FROM or FROM NAMED clause),
>
> It caches things by default and control the expiration of cached copies
> by returned HTTP header lines. If a remote tells that the resource
> should not be cached or sets very small expiration time then the cache
> lifetime can not be lengthen. define get:refresh can force earlier
> reloading but can not delay it: it sets maximum allowed refresh time,
> not a strict value.
>
>> 2. do n-fetches in parallel and/or,
>
> It tries to fetch in parallel, via async_queue-s of "width" 8.
> Unfortunately that happens not in all cases.
>
>> 3. tell me if some object turned out not to be a linked data resource, but
>> just an URL (e.g., in the triple: (dbpedia:Barack_Obama dbpprop:reference
>> ) )
>
> I'll think what can be done. Right now there's nothing ready-to-use,
> because we follow Squids strategy and cache only good results, not
> failures. However it's possible to make sample RDF view to render data
> from table DB.DBA.SYS_HTTP_SPONGE and get access to status of currently
> cached copies.
>
> Best Regards,
>
> Ivan Mikhailov
> OpenLink Software
> http://virtuoso.openlinksw.com
>
>
>
> --
> The Palm PDK Hot Apps Program offers developers who use the
> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> of $1 Million in cash or HP Products. Visit us here for more details:
> http://p.sf.net/sfu/dev2dev-palm
> ___
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
>



-- 
Aldo Bucchi
@aldonline
skype:aldo.bucchi
http://aldobucchi.com/

PRIVILEGED AND CONFIDENTIAL INFORMATION
This message is only for the use of the individual or entity to which it is
addressed and may contain information that is privileged and confidential. If
you are not the intended recipient, please do not distribute or copy this
communication, by e-mail or otherwise. Instead, please notify us immediately by
return e-mail.