hi there again.. i've pushed our changes to our company fork on github..
my commit is this: https://github.com/semantic-web-company/incubator-rya/commit/2b788ea84848251ea06951ea3371010e319abfb6 any feedback would be highly welcome (an integration even more :-) ) krj *Jürgen Jakobitsch* Senior Technical Consultant Semantic Web Company GmbH EU: +43-14021235 <+43%201%204021235> US: (415) 800-3776 Mobile: +43-676-6212710 <+43%20676%206212710> https://www.poolparty.biz https://www.semantic-web.com *Download E-Book*: Introducing Semantic AI <https://www.poolparty.biz/machine-learning-meets-semantics/> Am Mi., 5. Dez. 2018 um 21:00 Uhr schrieb Jürgen Jakobitsch < [email protected]>: > hi, yo it's an older version.. > > we've already upgraded a couple of versions as well and are integrating > the sparql endpoint in our own rya fork.. > > there was hardly any reaction to the fist version, so i'm happy that the > discussion is starting again. > i'll make sure to commit our current version during the next days and you > can have a look at it. > > krj > > *Jürgen Jakobitsch* > Senior Technical Consultant > Semantic Web Company GmbH > EU: +43-14021235 <+43%201%204021235> > US: (415) 800-3776 > Mobile: +43-676-6212710 <+43%20676%206212710> > https://www.poolparty.biz > https://www.semantic-web.com > > *Download E-Book*: Introducing Semantic AI > <https://www.poolparty.biz/machine-learning-meets-semantics/> > > > Am Mi., 5. Dez. 2018 um 20:12 Uhr schrieb Maxim Kolchin < > [email protected]>: > >> Hi Jurgen, >> >> > hi there, we've implemented the whole sparql endpoint anew. >> >> Thank you for the work! >> >> But unfortunately, I wasn't able to use the code you published, because >> it's based on libraries which aren't used in Rya from the master branch, >> like OpenRDF Sesame which was replaced by RDF4J. Is it because you use an >> older code base? >> >> So I'd spent some time to integrate Rya with RDF4J Workbench and it >> worked. >> You can find the code in >> https://github.com/DataFabricRus/incubator-rya/tree/datafabric-patches. >> The >> branch contains a lot of unrelated changes, so it may be hard to find >> exact >> changes we've made to integrate RDF4J Workbench, but the starting point is >> the web module. Maybe it'll help someone. >> >> > However, that information is not used when building the Accumulo queries >> in ParallelEvaluationStrategyImpl.java, which is something I will still >> need to address. >> >> I didn't test that with RDF4J Workbench, so my mail probably unhelpful for >> you, sorry. >> >> Maxim >> >> On Wed, Dec 5, 2018 at 8:53 PM Boris Pelakh < >> [email protected]> >> wrote: >> >> > Yes, the deficiency in the evaluation engine is the problem I am trying >> to >> > address. As per the SPARQL 1.1. protocol, the default-graph-uri and >> > named-graph-uri query parameters are supposed to override any graph >> names >> > provided in the query itself. The Rdf4j workbench, which provides a very >> > nice web UI, uses the options to implement some of its navigation >> > functionality. >> > >> > The RDF4J SAIL interface implemented by Rya provides for this exact >> > functionality by allowing the specification of a Dataset on any query >> > object. However, that information is not used when building the Accumulo >> > queries in ParallelEvaluationStrategyImpl.java, which is something I >> will >> > still need to address. >> > >> > Boris Pelakh >> > Ontologist, Developer, Software Architect >> > [email protected] >> > +1-321-243-3804 >> > >> > >> > -----Original Message----- >> > From: Jürgen Jakobitsch <[email protected]> >> > Sent: Wednesday, December 5, 2018 12:19 PM >> > To: [email protected] >> > Subject: Re: SPARQL endpoint [Was: Dataset not applied to queries] >> > >> > no, i didn't include that parameter.. >> > >> > usually this parameter is anyway included in the SPARQL query itself as >> > FROM or FROM NAMED. >> > >> > mind that depending on the situation you might have to rewrite the >> query.. >> > >> > also mind that there are some application specifics involved, since we >> > also have DEFAULT keyword in SPARQL and finally also mind that the >> > evaluation strategy (the actual SPARQL engine here) must make use of the >> > graph constraint, which i think isn't done at the moment >> > >> > krj >> > >> > >> > *Jürgen Jakobitsch* >> > Senior Technical Consultant >> > Semantic Web Company GmbH >> > EU: +43-14021235 <+43%201%204021235> >> > US: (415) 800-3776 >> > Mobile: +43-676-6212710 <+43%20676%206212710> https://www.poolparty.biz >> > https://www.semantic-web.com >> > >> > *Download E-Book*: Introducing Semantic AI < >> > https://www.poolparty.biz/machine-learning-meets-semantics/> >> > >> > >> > Am Mi., 5. Dez. 2018 um 17:56 Uhr schrieb Boris Pelakh < >> > [email protected]>: >> > >> > > Hi, Jurgen >> > > >> > > I looked at your source, and you do not appear to have addressed the >> > > default-graph-uri and named-graph-uri constraints defined in the >> > > SPARQL 1.1 protocol >> > > (https://www.w3.org/TR/sparql11-protocol/#protocol). Is this >> addressed >> > somewhere other than SPARQLController.java? >> > > >> > > Boris Pelakh >> > > Ontologist, Developer, Software Architect >> > > [email protected] >> > > +1-321-243-3804 >> > > >> > > >> > > -----Original Message----- >> > > From: Jürgen Jakobitsch <[email protected]> >> > > Sent: Wednesday, December 5, 2018 10:45 AM >> > > To: [email protected] >> > > Subject: Re: SPARQL endpoint [Was: Dataset not applied to queries] >> > > >> > > hi there, we've implemented the whole sparql endpoint anew. >> > > >> > > some of the features from the old endpoint are still missing, but i >> > > think they need some re-engineering anyway, like send a user as a url >> > > paramter is not exactly a good idea.. >> > > >> > > i'm including a mail from i wrote from earlier this year (around >> > > april/may) >> > > >> > > hi, >> > > >> > > as mentioned in previous mails, we've setup a first draft of the >> > > proposed upgrade of rya's sparql endpoint.. again note, that we've >> > > setup a small debug/demo maven project using a sesame memory store to >> > > demonstrate the proposed changes. if the proposed upgrade of the >> > > sparql endpoint is welcome we would incorporate this with existing rya >> > > code to include all required rya features. >> > > features: >> > > >> > > 1. full support for content-negotiation (essential within the >> > > linkeddata/semantic web realm) 2. SPARQL Service Description 3. more >> > > HTTP Status Codes (202 accepted, 400 bad request, 200 ok) 4. all RDF >> > > serialization formats available natively from sesame 5. customizable >> > > web endpoint (via a sesame MAVWriter (ModelAndView) + JSPs >> > > + Styles) >> > > 6. valid XHTML for the endpoint >> > > 7. valid XHTML+RDFa in a browser for construct queries 8. a future >> > > rdf4j upgrade will not seriously affect the whole endpoint >> > > >> > > the project is very easy to build and you can checkout all features on >> > > your local machine easily (examples are provided). the maven build >> > > will include a tomcat hosting this sample endpoint, descriptions and >> > > code can be found >> > > here: >> > > >> > > https://github.com/jj-semantic-web-com/rya-sparql-endpoint >> > > >> > > any feedback, pointers, comments and wishes are highly welcome! >> > > >> > > related JIRA task: https://issues.apache.org/jira/browse/RYA-477 >> > > >> > > kr jürgen >> > > >> > > >> > > >> > > >> > > >> > > *Jürgen Jakobitsch* >> > > Senior Technical Consultant >> > > Semantic Web Company GmbH >> > > EU: +43-14021235 <+43%201%204021235> >> > > US: (415) 800-3776 >> > > Mobile: +43-676-6212710 <+43%20676%206212710> >> > > https://www.poolparty.biz https://www.semantic-web.com >> > > >> > > *Download E-Book*: Introducing Semantic AI < >> > > https://www.poolparty.biz/machine-learning-meets-semantics/> >> > > >> > > >> > > Am Mi., 5. Dez. 2018 um 16:28 Uhr schrieb Adina Crainiceanu < >> > > [email protected] >> > > >: >> > > >> > > > Hi Jurgen, >> > > > >> > > > If you see this message, can you please reply? >> > > > >> > > > Boris, if you create a JIRA account you should be able to add >> > > > comments and "watch" the issue to be notified of changes. >> > > > >> > > > Adina >> > > > On Wed, Dec 5, 2018 at 9:41 AM Boris Pelakh >> > > > <[email protected] >> > > > > >> > > > wrote: >> > > > >> > > > > Hi, Adina, >> > > > > >> > > > > >> > > > > Thanks for the pointer, and I wish I had seen this prior to >> > > > > starting my effort. My work is essentially complete with the >> > > > > exception of the aforementioned defect in applying default and >> > > > > named graph filters, and it covers pretty much that entire Jira >> > > > > issue. I have no way of contacting Jurgen, as I only have read >> > > > > access to the JIRA instance you linked, but >> > > > it >> > > > > seems that no work has been done on that 'In Progress' issue for >> > > > > more >> > > > than >> > > > > 6 months. >> > > > > >> > > > > >> > > > > Boris >> > > > > >> > > > > ________________________________ >> > > > > From: Adina Crainiceanu <[email protected]> >> > > > > Sent: Wednesday, December 5, 2018 9:32:17 AM >> > > > > To: [email protected] >> > > > > Subject: Re: Dataset not applied to queries >> > > > > >> > > > > Hi Boris, >> > > > > >> > > > > There is some work currently being done on the SPARQL endpoint by >> > > > > Jurgen >> > > > in >> > > > > RYA-477 https://issues.apache.org/jira/browse/RYA-477 >> > > > > Let's make sure we discuss there ideas and design decisions on the >> > > > > dev list. I would hate if someone's work gets duplicated. In >> > > > > general, please check whether a JIRA issue for the bug/desired >> > > > > feature does not already exist >> > > > > https://issues.apache.org/jira/browse/RYA. If it does, >> collaborate >> > > > > with the person assigned to that issue (if any) or assign the >> > > > > issue to yourself (if none assigned). If not, create the issue and >> > > > > assign it to yourself. If you cannot assign an issue to yourself, >> > > > > send a message to >> > > > the >> > > > > dev list with the Jira username, so you can be added to the list >> > > > > of contributors in JIRA. >> > > > > >> > > > > Looking forward to your contributions! >> > > > > Adina >> > > > > >> > > > > >> > > > > >> > > > > On Tue, Dec 4, 2018 at 11:16 AM Puja Valiyil <[email protected]> >> > > wrote: >> > > > > >> > > > > > I can definitely take a look, but Aaron would also be a good >> > > > > > person if >> > > > > its >> > > > > > mostly using Accumulo backed Rya. You're touching core Rya >> > > > > > classes so >> > > > > the >> > > > > > more eyes the better! >> > > > > > >> > > > > > On Tue, Dec 4, 2018 at 10:14 AM Boris Pelakh < >> > > > > > [email protected]> >> > > > > > wrote: >> > > > > > >> > > > > > > Thanks, Puja, that's very helpful. I am making the changes in >> > > > > > > a >> > > > forked >> > > > > > > repo, would I assign pull requests to you once I am finished? >> > > > > > > >> > > > > > > Boris Pelakh >> > > > > > > Ontologist, Developer, Software Architect >> > > > > > > [email protected] >> > > > > > > +1-321-243-3804 >> > > > > > > >> > > > > > > >> > > > > > > -----Original Message----- >> > > > > > > From: Puja Valiyil <[email protected]> >> > > > > > > Sent: Tuesday, December 4, 2018 10:12 AM >> > > > > > > To: [email protected] >> > > > > > > Subject: Re: Dataset not applied to queries >> > > > > > > >> > > > > > > Hi Boris, >> > > > > > > The execution of the query plan is done by the >> > > > > > ParallelEvaluationStrategy: >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> https://github.com/apache/incubator-rya/blob/master/sail/src/main/ja >> > > > va >> > > > /org/apache/rya/rdftriplestore/evaluation/ParallelEvaluationStrategy >> > > > Im >> > > > pl.java >> > > > > > > >> > > > > > > It looks like whoever did the rdf4j upgrade updated that class >> > > > > > > to >> > > > take >> > > > > > the >> > > > > > > dataset object in its constructor, but it doesn't seem to be >> > > > > > > used >> > > > > > anywhere >> > > > > > > in query evaluation. That would be the class to modify, but >> > > > > > > I'm >> > > > still >> > > > > > not >> > > > > > > clear on what the changes would look like since I'm not >> > > > > > > familiar with >> > > > > > what >> > > > > > > the contract is supposed to be. >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > On Tue, Dec 4, 2018 at 9:45 AM Boris Pelakh < >> > > > > > [email protected] >> > > > > > > > >> > > > > > > wrote: >> > > > > > > >> > > > > > > > So, a >> > > > > /sparql?query=SELECT...WHERE...&default-graph-uri=my-named-graph >> > > > > > > > is logically equivalent to >> > > > > > > > >> > > > > > > > SELECT ... >> > > > > > > > FROM <my-named-graph> >> > > > > > > > WHERE ... >> > > > > > > > >> > > > > > > > (overriding whatever FROM/FROM NAMED clauses actually >> > > > > > > > specified in >> > > > > the >> > > > > > > > query) I looked at >> > > > > > > > >> > > > > org.apache.rya.rdftriplestore.RdfCloudTripleStoreConnection.evalua >> > > > > te >> > > > > In >> > > > > > > > ternal(), and the dataset containing the overriding graph >> > > > > > > > names is passed in, but does not seem to be honored, i.e if >> > > > > > > > I have the following data: >> > > > > > > > >> > > > > > > > ns:g1 { ns:p1 ns:pred ns:o1 } >> > > > > > > > ns:g2 { ns:p2 ns:pred ns:o2 } >> > > > > > > > >> > > > > > > > and I execute 'SELECT COUNT(?s) where { ?s ns:pred ?o }' >> > > > > > > > with default-graph-uri set to ns:g1, I expect to get 1, but >> I >> > get 2. >> > > > > > > > >> > > > > > > > I am assuming that there is someplace where the TupleExpr >> > > > > > > > passed to evaluateInternal gets transformed into a scan >> > > > > > > > pattern for Accumulo, after the right index (spo/ops/pso) is >> > > > > > > > identified. If I remember correctly, S/P/O is used the the >> > > > > > > > row ID, and the graph is the >> > > > column >> > > > > > > > name (or family?). So the dataset should modify the scan >> > > > > > > > filter if >> > > > > > > applied correctly. >> > > > > > > > >> > > > > > > > Boris Pelakh >> > > > > > > > Ontologist, Developer, Software Architect >> > > > > > > > [email protected] >> > > > > > > > +1-321-243-3804 >> > > > > > > > >> > > > > > > > >> > > > > > > > -----Original Message----- >> > > > > > > > From: Puja Valiyil <[email protected]> >> > > > > > > > Sent: Tuesday, December 4, 2018 9:31 AM >> > > > > > > > To: [email protected] >> > > > > > > > Subject: Re: Dataset not applied to queries >> > > > > > > > >> > > > > > > > Hi Boris, >> > > > > > > > Unfortunately I'm not very familiar with that method -- can >> > > > > > > > you provide some more details into exactly what its supposed >> > > > > > > > to >> > > do? >> > > > > > > > All of the connectors for implementing the sail api are in >> > > > > > > > the rya.sail, and the main classes to look at are >> > > > > > > > RdfCloudTripleStore, RyaSailRepository, and >> > > > > > > > RdfCloudTripleStoreConnection. All of these classes all >> > > > > > > > extend abstract implementations SAIL provides out of >> > > > the >> > > > > > > > box, so typically the way you extend them to do more stuff >> > > > > > > > is to >> > > > look >> > > > > > > > at the Sail implementation and customize it for Rya's data >> > > > > structures. >> > > > > > > > If you need any help or want to bounce any implementation >> > > > > > > > ideas off >> > > > > of >> > > > > > > > anyone, please send out an email over the dev list. Just let >> > > > > > > > us >> > > > know >> > > > > > > > how we can help! >> > > > > > > > Thanks, >> > > > > > > > Puja >> > > > > > > > >> > > > > > > > >> > > > > > > > On Tue, Dec 4, 2018 at 9:19 AM Boris Pelakh >> > > > > > > > <[email protected] >> > > > > > > > > >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > > > I am in the process of implementing a SPARQL 1.1-compliant >> > > > endpoint >> > > > > > > > > for web.rya so that Rya can be used as a federated >> > > > > > > > > repository >> > > > with >> > > > > > > > > RDF4J workbench. >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > One of requirements is to be able to pass >> > > > > > > > > default-graph-uri and named-graph-uri filters external to >> > > > > > > > > the query to override the >> > > > graph >> > > > > > > > > filter in the query itself (see >> > > > > > > > > https://www.w3.org/TR/sparql11-protocol/#query-operation >> ). >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > In the sail API this is accomplished by setting the >> > > > > > > > > dataset on >> > > > the >> > > > > > > > > query object (see >> > > > > > > > > >> > > > > >> http://docs.rdf4j.org/javadoc/2.2/org/eclipse/rdf4j/query/Operation. >> > > > > > > > > ht ml#setDataset-org.eclipse.rdf4j.query.Dataset-), >> > > > > > > > > but that setting does not seem to be honored by the Rya >> > > > > > > > > SAIL implementation. Any ideas on where I can start >> > > > > > > > > digging to correct >> > > > > > this? >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > (I have also tried wrapping the Rya repository connection >> > > > > > > > > in a ContextAwareConnection and setting readContexts, but >> > > > > > > > > the effect >> > > > is >> > > > > > > > > the same). >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > *Boris Pelakh* >> > > > > > > > > >> > > > > > > > > *Ontologist, Developer, Software Architect* >> > > > > > > > > >> > > > > > > > > *[email protected] >> > > > > > > > > <[email protected]>* >> > > > > > > > > >> > > > > > > > > *+1-321-243-3804* >> > > > > > > > > >> > > > > > > > > [image: SemanticArtsLogo] >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > -- >> > > > > Dr. Adina Crainiceanu >> > > > > Associate Professor >> > > > > Computer Science Department >> > > > > United States Naval Academy >> > > > > 410-293-6822 >> > > > > [email protected] >> > > > > http://www.usna.edu/Users/cs/adina/ >> > > > > >> > > > >> > > > >> > > > -- >> > > > Dr. Adina Crainiceanu >> > > > Associate Professor >> > > > Computer Science Department >> > > > United States Naval Academy >> > > > 410-293-6822 >> > > > [email protected] >> > > > http://www.usna.edu/Users/cs/adina/ >> > > > >> > > >> > >> >
