20 years of Jena @ ApacheCon

[Claude Warren]

Greetings,

It is 20 years since Jena was introduced to the world and we are trying to
put together a program track at ApacheCon to celebrate!

If you have a story to tell, a module to describe, or a project to crow
about, please submit a proposal [1] and be sure to use the Jena name in the
title or description.

There are only a few days left, so please hurry.  If you have any questions
please reach out to me.

Thank you,
Claude

[1] https://acna2020.jamhosted.net/cfp-submit.html

Re: 20 years of Jena @ ApacheCon

[Claude Warren]

I forgot to state that proposals should be submitted in the
"General/Other"  category

On Thu, Jul 9, 2020 at 5:10 PM Claude Warren  wrote:

> Greetings,
>
> It is 20 years since Jena was introduced to the world and we are trying to
> put together a program track at ApacheCon to celebrate!
>
> If you have a story to tell, a module to describe, or a project to crow
> about, please submit a proposal [1] and be sure to use the Jena name in the
> title or description.
>
> There are only a few days left, so please hurry.  If you have any
> questions please reach out to me.
>
> Thank you,
> Claude
>
> [1] https://acna2020.jamhosted.net/cfp-submit.html
>

ApacheCon@Home Jena Track

[Claude Warren]

We have a track at ApacheCon in which we can celebrate 20 years of Jena.

I would like someone who has been on the project to give a  Janus View of
Jena.  Janus being the Roman god that looks both to the past and the future.

If anyone wants to give a talk about:
1) Your favorite component/module/extension (e.g. GeoSPARQL -- thx Marco,
Permissions, building reasoners, RDF diff, QueryBuilder, etc.)
2) How you are using/have used Jena in your project.
3) How you use/have used RDF in your projects.

Please submit a proposal at [1] or drop me a line to discuss off-list.

Claude

ApacheCon@home 2020 - Semantic Graph BoF

[Claude Warren]

Greetings,

ApacheCon is almost upon us.  This year it is online and free.  So please
make plans to attend.

This year Apache Jena is hosting a Semantic Graph "Birds of a Feather"
session[1] as part of the Jena track.  Please come join us and discuss all
things Semantic Graph.

[1] https://www.apachecon.com/acah2020/tracks/jena.html

ApacheCon 2021

[Claude Warren]

There is a call for tracks for ApacheCon 2021.  Do we want to propose a
track again this year?  Perhaps we could get some presentations from users.

FYI:  I gave an impromptu talk about Jena at the Apache "table" during
FOSDEM at the beginning of this month.

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

use of * in import

[Claude Warren]

I noticed that the changes to querybuilder converted a bunch of java.util.x
import to java.util.*

I have always heard that the asterisk should not be used in the import
statement as it can cause conflicts that are very hard to debug when new
versions of the specific library are updated and conflicting classes added.

Is there a reason to use the '*' import?

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: use of * in import

[Claude Warren]

Actually, I was asking if we has a standard or recommendation one way or
the other.I'll take a look again tomorrow, I didn't have much time to
review.  If you need to proceed without my approval, feel free.

Claude

On Wed, Feb 17, 2021 at 10:52 PM Andy Seaborne  wrote:

>
>
> On 17/02/2021 17:37, Claude Warren wrote:
> > I noticed that the changes to querybuilder converted a bunch of
> java.util.x
> > import to java.util.*
>
> This is what PR review is for. It does help to be in-context.
>
> I have updated the PR - can you approve it now please?
>
> I was in the code because I had to make lot of changes to redo the
> direct calls to Binding implementation classes.
>
>  Andy
>
> > I have always heard that the asterisk should not be used in the import
> > statement as it can cause conflicts that are very hard to debug when new
> > versions of the specific library are updated and conflicting classes
> added.
> >
> > Is there a reason to use the '*' import?
> >
> > Claude
> >
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: ApacheCon 2021

[Claude Warren]

not hearing any suggestions on this topic. I will drop it.

Thx

On Wed, Feb 17, 2021 at 5:41 PM Marco Neumann 
wrote:

> I am in touch with the Apache spatial group about the 2021 slots, and it
> looks like I will dedicate a strand to linked data and geosparql with jena
> here in spatial track.
>
> Marco
>
>
>
> On Wed, Feb 17, 2021 at 11:43 AM Claude Warren  wrote:
>
> > There is a call for tracks for ApacheCon 2021.  Do we want to propose a
> > track again this year?  Perhaps we could get some presentations from
> users.
> >
> > FYI:  I gave an impromptu talk about Jena at the Apache "table" during
> > FOSDEM at the beginning of this month.
> >
> > Claude
> >
> > --
> > I like: Like Like - The likeliest place on the web
> > <http://like-like.xenei.com>
> > LinkedIn: http://www.linkedin.com/in/claudewarren
> >
>
>
> --
>
>
> ---
> Marco Neumann
> KONA
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: ApacheCon 2021

[Claude Warren]

arrrgg.  What was I thinking??  I don't know either.  I meant to ping this
topic to see if there was any interest before dropping it.  So if you are
interested in a Jena topic at ApacheCon this year let me know and I will
propose it and drive it forward as best I can.

Claude

On Thu, Feb 18, 2021 at 6:50 PM Claude Warren  wrote:

> not hearing any suggestions on this topic. I will drop it.
>
> Thx
>
> On Wed, Feb 17, 2021 at 5:41 PM Marco Neumann 
> wrote:
>
>> I am in touch with the Apache spatial group about the 2021 slots, and it
>> looks like I will dedicate a strand to linked data and geosparql with jena
>> here in spatial track.
>>
>> Marco
>>
>>
>>
>> On Wed, Feb 17, 2021 at 11:43 AM Claude Warren  wrote:
>>
>> > There is a call for tracks for ApacheCon 2021.  Do we want to propose a
>> > track again this year?  Perhaps we could get some presentations from
>> users.
>> >
>> > FYI:  I gave an impromptu talk about Jena at the Apache "table" during
>> > FOSDEM at the beginning of this month.
>> >
>> > Claude
>> >
>> > --
>> > I like: Like Like - The likeliest place on the web
>> > <http://like-like.xenei.com>
>> > LinkedIn: http://www.linkedin.com/in/claudewarren
>> >
>>
>>
>> --
>>
>>
>> ---
>> Marco Neumann
>> KONA
>>
>
>
> --
> I like: Like Like - The likeliest place on the web
> <http://like-like.xenei.com>
> LinkedIn: http://www.linkedin.com/in/claudewarren
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: ApacheCon 2021

[Claude Warren]

My talk at FOSDEM was ok.  Basically just talked about what Jena is and
does.  Turns out that is a lot to say.   It was all off the cuff and more
like a radio show than anything else.


On Thu, Feb 18, 2021 at 7:37 PM Andy Seaborne  wrote:

>
>
> On 17/02/2021 11:43, Claude Warren wrote:
> > FYI:  I gave an impromptu talk about Jena at the Apache "table" during
> > FOSDEM at the beginning of this month.
>
> How did that go?
>
>  Andy
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: ApacheCon 2021

[Claude Warren]

ApacheCon date is not set but it normally is at the end of Sept.

Nothing is set yet as tracks are just being established.

Last year we did a track that comprised 4 talks and a chat session (I
forget what that was called).  we got some folks from Rya attending as
well.  We might consider asking Rya if they would like to join us in an
RDF/Linked data track.  However, what I would need is commitment from the
team for 4 talks, or 2 talks and an agreement to ask Rya to join us in an
RDF/Linked data track.

Thoughts?
Claude

On Thu, Feb 18, 2021 at 7:00 PM Martynas Jusevičius 
wrote:

> We might be interested :) When is it and what does it take?
>
> On Thu, Feb 18, 2021 at 7:52 PM Claude Warren  wrote:
> >
> > arrrgg.  What was I thinking??  I don't know either.  I meant to ping
> this
> > topic to see if there was any interest before dropping it.  So if you are
> > interested in a Jena topic at ApacheCon this year let me know and I will
> > propose it and drive it forward as best I can.
> >
> > Claude
> >
> > On Thu, Feb 18, 2021 at 6:50 PM Claude Warren  wrote:
> >
> > > not hearing any suggestions on this topic. I will drop it.
> > >
> > > Thx
> > >
> > > On Wed, Feb 17, 2021 at 5:41 PM Marco Neumann  >
> > > wrote:
> > >
> > >> I am in touch with the Apache spatial group about the 2021 slots, and
> it
> > >> looks like I will dedicate a strand to linked data and geosparql with
> jena
> > >> here in spatial track.
> > >>
> > >> Marco
> > >>
> > >>
> > >>
> > >> On Wed, Feb 17, 2021 at 11:43 AM Claude Warren 
> wrote:
> > >>
> > >> > There is a call for tracks for ApacheCon 2021.  Do we want to
> propose a
> > >> > track again this year?  Perhaps we could get some presentations from
> > >> users.
> > >> >
> > >> > FYI:  I gave an impromptu talk about Jena at the Apache "table"
> during
> > >> > FOSDEM at the beginning of this month.
> > >> >
> > >> > Claude
> > >> >
> > >> > --
> > >> > I like: Like Like - The likeliest place on the web
> > >> > <http://like-like.xenei.com>
> > >> > LinkedIn: http://www.linkedin.com/in/claudewarren
> > >> >
> > >>
> > >>
> > >> --
> > >>
> > >>
> > >> ---
> > >> Marco Neumann
> > >> KONA
> > >>
> > >
> > >
> > > --
> > > I like: Like Like - The likeliest place on the web
> > > <http://like-like.xenei.com>
> > > LinkedIn: http://www.linkedin.com/in/claudewarren
> > >
> >
> >
> > --
> > I like: Like Like - The likeliest place on the web
> > <http://like-like.xenei.com>
> > LinkedIn: http://www.linkedin.com/in/claudewarren
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: ApacheCon 2021

[Claude Warren]

Can you run a demo?

A demo and a look ahead to V4 would be nice.

I can talk about the permissions layer or the query builder, but it would
be nice to have a presentation showing the Jena in a real world application.

@Andy Seaborne  would you be willing to talk about V4 and
the changes therein?

Perhaps we should take this discussion to the users list and see if there
is interest over there.

Thoughts?
Claude

On Fri, Feb 19, 2021 at 8:54 PM Marco Neumann 
wrote:

> I'd like to see a demo session.
>
> On Thu, Feb 18, 2021 at 6:52 PM Claude Warren  wrote:
>
> > arrrgg.  What was I thinking??  I don't know either.  I meant to ping
> this
> > topic to see if there was any interest before dropping it.  So if you are
> > interested in a Jena topic at ApacheCon this year let me know and I will
> > propose it and drive it forward as best I can.
> >
> > Claude
> >
> > On Thu, Feb 18, 2021 at 6:50 PM Claude Warren  wrote:
> >
> > > not hearing any suggestions on this topic. I will drop it.
> > >
> > > Thx
> > >
> > > On Wed, Feb 17, 2021 at 5:41 PM Marco Neumann  >
> > > wrote:
> > >
> > >> I am in touch with the Apache spatial group about the 2021 slots, and
> it
> > >> looks like I will dedicate a strand to linked data and geosparql with
> > jena
> > >> here in spatial track.
> > >>
> > >> Marco
> > >>
> > >>
> > >>
> > >> On Wed, Feb 17, 2021 at 11:43 AM Claude Warren 
> > wrote:
> > >>
> > >> > There is a call for tracks for ApacheCon 2021.  Do we want to
> propose
> > a
> > >> > track again this year?  Perhaps we could get some presentations from
> > >> users.
> > >> >
> > >> > FYI:  I gave an impromptu talk about Jena at the Apache "table"
> during
> > >> > FOSDEM at the beginning of this month.
> > >> >
> > >> > Claude
> > >> >
> > >> > --
> > >> > I like: Like Like - The likeliest place on the web
> > >> > <http://like-like.xenei.com>
> > >> > LinkedIn: http://www.linkedin.com/in/claudewarren
> > >> >
> > >>
> > >>
> > >> --
> > >>
> > >>
> > >> ---
> > >> Marco Neumann
> > >> KONA
> > >>
> > >
> > >
> > > --
> > > I like: Like Like - The likeliest place on the web
> > > <http://like-like.xenei.com>
> > > LinkedIn: http://www.linkedin.com/in/claudewarren
> > >
> >
> >
> > --
> > I like: Like Like - The likeliest place on the web
> > <http://like-like.xenei.com>
> > LinkedIn: http://www.linkedin.com/in/claudewarren
> >
>
>
> --
>
>
> ---
> Marco Neumann
> KONA
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

read permissions on data.

[Claude Warren]

I started looking at the read permissions on graphs issue that was raised
today.

It seems to me that if we change the functioning of graph.find() then we
need to change graph.contains() and graph.size() accordingly.

This led me to look at the Model based classes, and there we find a number
of iterators, lists of properties, hasX() methods, etc.  All of which
currently throw the ReadDeniedException.

Changing these methods will change the default operation in the wild.
Something that is OK for v4 but in v3 I think it should stay the same.

So I am thinking that it might make sense to specify HardReadErrors (the
current throw the exception we have now) or SoftReadErrors (return empty
iterators, false for hasX() and so on).  At first I thought of putting this
in a context, but it could be added to the SecurityEvaluator.  Since the
SecurityEvaluator is an interface I would add it with a default of
HardReadErrors and allow implementations to override that.

I think this might be the best way forward, though there will be a lot of
change in the permissions code base.  Does anyone see an issue with this
approach or a better approach?

Claude



-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: read permissions on data.

[Claude Warren]

Since the flag would be in the SecurityEvaluator you could conceivably have
soft in one place and hard in another.

Does the above conversation mean that we should look at soft errors from
Writes?  (e.g. not report an error if the user is not permitted)
And do we then consider soft errors for Updates separately?  The only place
an update occurs is in the Model code, this is explained in the notes about
the difference between SecuredModel and Model on a SecuredGraph.

For now I think we should do the soft read error flag and leave the rest
for when it is requested.

Claude

On Tue, Mar 2, 2021 at 3:11 PM Rob Vesse  wrote:

> Updates and soft mode is an interesting conundrum
>
> In my $dayjob system update operations are all resource centric i.e. you
> are either creating a new resource or overwriting an existing one.  So for
> update we do use hard errors for non-permissible updates because it still
> doesn't let you infer any data.  There's no update operations in our system
> that are conditional/calculated as in the SPARQL Update case.  Aside, most
> of the resources in this system are owned by a specific user and by default
> you can only write to your own "space" within the system.  If you try to
> write to another users "space" the error is same whether that other user
> exists or not because your default permissions are minimal.  About the only
> obvious side channel I can think of in our system is that you can infer the
> existence of another user if the other user has modified their permissions
> from the system defaults.
>
> Whereas here we're talking about much more fine grained permissions.  And
> I think what you are alluding to here is that SPARQL is trickier because
> you can do INSERT {} WHERE {} or DELETE {} WHERE {} (and of course the
> combined form).  A clever user could perhaps craft an update that takes
> advantage of soft errors in the WHERE {} clause.
>
> There's definitely a contention here between where the authentication
> (AuthC) and authorization (AuthZ) pieces happen complicated by the fact you
> need to solve both the in-memory case and the HTTP case.  It may be that
> different machinery is needed in different places.
>
> Rob
>
> On 02/03/2021, 11:53, "Andy Seaborne"  wrote:
>
>
> On 02/03/2021 10:04, Rob Vesse wrote:
> > No I like that approach
> >
> > In a system we're building currently in $dayjob we do much the same
> thing.  If a user doesn't have permission to see the data it just looks to
> that user like that data does not exist so they get null, empty/truncated
> lists, false etc.  From an information security perspective soft errors are
> actually more secure because they don't provide side channels.  With a hard
> error users can explicitly distinguish between data that isn't there and
> data that is there but to which they don’t have access i.e. a clever user
> can infer the existence of data under a hard error setup.  With soft errors
> the two cases are indistinguishable.
> >
> > Jena 4 is an opportunity to make breaking changes so even if you
> make the soft vs hard error behaviour configurable I would suggest making
> the default be soft error and document the change as part of whatever Jena
> 3 -> 4 migration guide we end up writing.  Then people who are using the
> permissions modules and upgrading would be forced to understand the change,
> the reasoning behind it and make a decision about how to migrate based on
> their own use cases.  If you leave the default as-is those users could
> carry on none the wiser without realising they are missing out on potential
> security benefits.
> >
> > Rob
>
> jena-fuseki-access might be interesting for that.
>
> For TDB, it puts in a filter right in at the bottom as the quads come
> off the database itself, filtering by NodeId of the graph. Works for
> defaultUnionGraph as well.
>
> TIM could do the same but doesn't.
>
> It could filter on anything in the quad but per graph was the
> requirement.
>
> Authentication is part of the HTTP request cycle.
> It's for SPARQL operations and hooks into the operation processors.
>
> The core dataset and QueryExecution work is not Fuseki dependent.
>
> For other datasets, it builds a view datasets that only have the
> graphs
> the request has access to. That makes it work with defaultUnionGraph
> across a mix of graphs.  Filtering quads is harder for a mixes dataset
> -
> have to put back a quads view, filter and undo that work which at the
> low level looked expensive and it wasn't the primary requirement for
> the
> work.
>
> Ha

autoclosable iterator

[Claude Warren]

I know that we had discussed adding autoclosable to the iterators but
decided not to.  I find that I am doing a lot of try/finally-close
manipulation with extended iterators.  Rather than rehashing the previous
discussion (I'll assume we made the correct decision before), I would like
to suggest an AutoClosableExtendedIterator that extends both
ExtendedIterator and AutoClosable.

Is there a reason not to do this?

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

RDF / Linked data track @ ApacheCon

[Claude Warren]

Sorry for the cross posting.

I am the chair for the RDF and Linked data track at ApacheCon this year.  I
am looking for a few speakers to talk about RDF and or Linked data in
projects, particularly with respect to the use of Apache libraries, tool
kits, etc.  If you are interested please drop me a line and I will make
sure you are notified directly when the call for papers opens.

Thank you,
Claude

RDF-* and permissions

[Claude Warren]

Greetings,

RDF-* seems like it may cause problems for permissions.

>From what I have seen we take a statement and convert that to a node where
the label is the statement (or the triple?).  But there does not seem to be
a way to differentiate the RDF-* edge nodes from other Nodes.

My question arises around the following:

Let's say there are 2 statements in the model
(s,p,o) and (x,y, (s,p,o)) where the (x,y,(s,p,o)) is an RDF-* statement
about (s,p,o).

If a user does not have access to see (s,p,o) they probably should not be
able to see (x,y,(s,p,o)) either.  Without a way to distinguish the
RDF-*Node from a regular resource I can't do the filtering.  The best I can
hope for is that the SecuritEngine implementation can, but I expect that
will have problems too.

Does anyone with RDF-* background see a way around this?

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Towards Jena4

[Claude Warren]

I am adding the softReadRestriction processing to the 3.x version
currently.  I think it will directly translate to 4.x, and it gives us the
ability to make softReadRestriction the default setting rather than the way
it is set in V3.

I am only about 1/2 way through the code -- there are some bits that were
thrown in that I think opened up some security holes.  Since it is an
"Extra" we should just not release Permissions until it is ready and not
have it delay the rest of 4.x

Claude

On Wed, Mar 10, 2021 at 2:45 PM Andy Seaborne  wrote:

>
>
> On 05/02/2021 11:11, Andy Seaborne wrote:
> > 3.17.0 was 2020-12-01
> > so the usual timescale is end Feb - end March.
>
> I just checked the release and apart from two javadoc warnings, it is
> looking good.
>
> All-
> Now is a good time to try out 4.0.0-SNAPSHOT in your downstream builds.
>
> Claude - jena-permissions - A I understand it, there isn't anything
> outside of that module needed. It is up-to-date with the interface
> changes elsewhere in Jena.
>
>
> Implementation of RDF-star (tracking the current community work).
> That isn't to say RDF-star is fixed and stable but the code should
> follow the direction that has emerged.
>
> 
> Tickets resolved for Jena 4.0.0:
> https://s.apache.org/jena-4.0.0-jira
>
>
> * Require Java11
>
> * Retire SDB
> * Retire jena-text-es
> * Retire jena-elephas
>
> * IRIx abstraction
>This may tighten up some illegal use of URIs
>  While accepted currently,  %XX - where XX is not hex - is illegal.
>URIs must have 2+ character NID and non-empty NSS
>  "urn:x:" is illegal twice.
>This may tighten up in the future.
>Details of IRIs: https://afs.github.io/rdf-iri-syntax.html
>
> * RDF-star implementation
>
> * jena-core: rename RDFReader as RDFReaderI
> and RDFWriter a RDFWriterI to avoid name clashes
>
> * Deprecation cleanup [ongoing...]
>
> * Remove property functions:
>  list:listMember list:listIndex, list:listLength
> Use list:member, list:index, list:length
>
> * Single command line tool package:
> jena.cmd.* -> org.apache.jena.cmd.*
>
> * Lucene upgrade to 8.8.0.
>
> * Other dependency upgrades (except jetty 10/11 - wait until our Java11
> move has gone OK)
>
> * The default git branch is 'main'
>


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Towards Jena4

[Claude Warren]

For now, I think releasing without Permissions unless Permissions get
completed before the V4 release.  I don't want to put out the old version
and then reverse the basic functioning of the default hard/soft read.  That
change just feels like it should be at a version boundary.

On Wed, Mar 10, 2021 at 6:47 PM Andy Seaborne  wrote:

>
>
> On 10/03/2021 18:18, Claude Warren wrote:
> > I am adding the softReadRestriction processing to the 3.x version
> > currently.  I think it will directly translate to 4.x, and it gives us
> the
> > ability to make softReadRestriction the default setting rather than the
> way
> > it is set in V3.
>
> Hope it does.
>
> To ease the merge: jena-permission has changes in 4.0.0 for interface
> changes elsewhere. It is up-to-date on main.
>
> > I am only about 1/2 way through the code -- there are some bits that were
> > thrown in that I think opened up some security holes.  Since it is an
> > "Extra" we should just not release Permissions until it is ready and not
> > have it delay the rest of 4.x
>
> Agreed. Modules have to evolve at their natural pace.
>
> We can skip it from 4.0.0 or release in no-softReadRestriction form. I
> don't mind - your choice.
>
>  Andy
>
> >
> > Claude
> >
> > On Wed, Mar 10, 2021 at 2:45 PM Andy Seaborne  wrote:
> >
> >>
> >>
> >> On 05/02/2021 11:11, Andy Seaborne wrote:
> >>> 3.17.0 was 2020-12-01
> >>> so the usual timescale is end Feb - end March.
> >>
> >> I just checked the release and apart from two javadoc warnings, it is
> >> looking good.
> >>
> >> All-
> >> Now is a good time to try out 4.0.0-SNAPSHOT in your downstream builds.
> >>
> >> Claude - jena-permissions - A I understand it, there isn't anything
> >> outside of that module needed. It is up-to-date with the interface
> >> changes elsewhere in Jena.
> >>
> >>
> >> Implementation of RDF-star (tracking the current community work).
> >> That isn't to say RDF-star is fixed and stable but the code should
> >> follow the direction that has emerged.
> >>
> >> 
> >> Tickets resolved for Jena 4.0.0:
> >> https://s.apache.org/jena-4.0.0-jira
> >>
> >>
> >> * Require Java11
> >>
> >> * Retire SDB
> >> * Retire jena-text-es
> >> * Retire jena-elephas
> >>
> >> * IRIx abstraction
> >> This may tighten up some illegal use of URIs
> >>   While accepted currently,  %XX - where XX is not hex - is
> illegal.
> >> URIs must have 2+ character NID and non-empty NSS
> >>   "urn:x:" is illegal twice.
> >> This may tighten up in the future.
> >> Details of IRIs: https://afs.github.io/rdf-iri-syntax.html
> >>
> >> * RDF-star implementation
> >>
> >> * jena-core: rename RDFReader as RDFReaderI
> >>  and RDFWriter a RDFWriterI to avoid name clashes
> >>
> >> * Deprecation cleanup [ongoing...]
> >>
> >> * Remove property functions:
> >>   list:listMember list:listIndex, list:listLength
> >>  Use list:member, list:index, list:length
> >>
> >> * Single command line tool package:
> >>  jena.cmd.* -> org.apache.jena.cmd.*
> >>
> >> * Lucene upgrade to 8.8.0.
> >>
> >> * Other dependency upgrades (except jetty 10/11 - wait until our Java11
> >> move has gone OK)
> >>
> >> * The default git branch is 'main'
> >>
> >
> >
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: autoclosable iterator

[Claude Warren]

In the permissions code I have to call close on all the ExtendedIterators
because I have no idea what is behind them.

I tried to add Autoclosable as an interface using a dynamic proxy in the
hopes that all those iterators that extend ExtendedIterator would still
function properly.  But there was a conflict between the close methods as
you noted.

I was unable to come up with a solution.

Claude


On Sat, Mar 6, 2021 at 8:33 PM Andy Seaborne  wrote:

> Could give some details please?
>
> I have tried some things out and they don't give a warning unless close
> is explicitly present, unlike InputStream.
>
> Current guess:  because ExtendedIterator.close is not the same as
> AutoCloseable.close due to throws.
>
>
> It's a multi-facet area. Is this primarily for exceptions? Early exit of
> a loop? Within jena-permissions - obviously you can do what you think is
> the right thing.
>
>
> What about ExtendedIterator.forEach? That closes on exception.
>
> Have you seen the "org.apache.jena.riot.other.G" library which is
> functions to work with graphs that do the close thing.  That was my
> solution to making sure lcose was called. We could move (some of) that
> to jena-core.
>
> Is there an interaction with "var"?
>
>  Andy
>
> On 06/03/2021 17:56, Claude Warren wrote:
> > I know that we had discussed adding autoclosable to the iterators but
> > decided not to.  I find that I am doing a lot of try/finally-close
> > manipulation with extended iterators.  Rather than rehashing the previous
> > discussion (I'll assume we made the correct decision before), I would
> like
> > to suggest an AutoClosableExtendedIterator that extends both
> > ExtendedIterator and AutoClosable.
> >
> > Is there a reason not to do this?
> >
> > Claude
> >
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: autoclosable iterator

[Claude Warren]

I think that if the solution was any simpler it would have bit me.

Change ClosableIterator to extend AutoCloseable as well as Iterator.
mark ClosableIterator.close() with an @Override annotation and done.

While Autoclosable throws an exception the ClosableIterator does not so the
try with resources works.

I tried this and then changed
FinderUtils.contains() to be coded as

@Override
public boolean contains(TriplePattern pattern) {
try (ClosableIterator it = find(pattern))
{
return it.hasNext();
}
}

and the IDE (Eclipse) does not complain.  So I think it will just work.

Re: autoclosable iterator

[Claude Warren]

My thought in all this was that the try with resources is now standard in
Java so we should try to support it.

Perhaps it makes sense to do the change in V4.
We can clean up our code so that there are no warnings.
Since the issues are warnings only, it should be manageable for our users.

Claude

On Thu, Mar 11, 2021 at 10:48 AM Andy Seaborne  wrote:

> I'm not clear which problem we are discussing:
>
> 1/ Ensuring inside jena-permissions that iterators are properly closed.
>
> 2/ Ensuring use code using jena-permissions (i.e
> SecuredGraph/SecuredModel by name) closes iterators properly.
>
> 3/ Ensuring app code that uses the Graph or Model interface (and so may
> be Secured* but it does not know) closes iterators properly.
>
> If (3) isn't any change going to impact existing use code?
>
> What's more, it does not seem to work
>
> On 10/03/2021 21:56, Claude Warren wrote:
> > I think that if the solution was any simpler it would have bit me.
> >
> > Change ClosableIterator to extend AutoCloseable as well as Iterator.
> > mark ClosableIterator.close() with an @Override annotation and done.
> >
> > While Autoclosable throws an exception the ClosableIterator does not so
> the
> > try with resources works.
> >
> > I tried this and then changed
> > FinderUtils.contains() to be coded as
> >
> > @Override
> >  public boolean contains(TriplePattern pattern) {
> >  try (ClosableIterator it = find(pattern))
> >  {
> >  return it.hasNext();
> >  }
> >  }
> >
> > and the IDE (Eclipse) does not complain.  So I think it will just work.
>
> because writing
>
> public boolean contains(TriplePattern pattern) {
>ExtendedIterator it = find(pattern);
>try {
>   return it.hasNext();
>} finally { it.close(); }
> }
>
> is accident-prone?
>
>
> I changed ClosableIterator as you describe and got hundreds of warnings
> from the codebase, and it will happen to user code.
>
> 1/
> A warning happens on code that is currently correctly written old style as
>
>  ExtendedIterator<> iter =
>  try {
>...
>  } finally { iter.close(); }
>
> Eclipse "quick-fix" does not do a good job on this.
>
> The other "old style" code is where the iterator runs the the end
> (exceptions aside).
>
> 2/
> Sometimes there isn't a warning!
>
> Warning: as expected:
>  public static void main(String...a) throws FileNotFoundException {
>  InputStream in = new FileInputStream("");
>  }
>
> No warning:  
>  static ExtendedIterator foo() { return null; }
>  public static void main(String...a) {
>  ExtendedIterator iter = foo();
>  }
>
> Warning:
>  static ExtendedIterator foo() { return null; }
>  public static void main(String...a) {
>  ExtendedIterator iter = foo();
>  iter.close();
>  }
>
> Weird.
> That somewhat negates the utility.
>
>  Andy
>
>

-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Towards Jena4

[Claude Warren]

 https://issues.apache.org/jira/browse/JENA-2075

Might need a resolution before 4.0 (or perhaps it is already resolved
there?)

On Mon, Mar 22, 2021 at 2:46 PM Andy Seaborne  wrote:

> I was just about ready to build 4.0.0.
>
> Then JENA-2074 came in (c.f. JENA-2061).
>
> I'm looking at fixing that by buffering the output for RDF/XML only
> before sending an HTTP response. It's not new but now the root problem
> has arisen twice recently.
>
> RDF/XML is more prone to problems because:
>
> (1) XML 1.0 has unwritable characters
> (2) RDF/XML has unwritable RDF
>
> After JENA-2074 - and excluding Tomcat-embedded getting included
> (THRIFT-5375) which i have just noticed by chance - 4.0.0 should be
> ready from my point of view.
>
> Anything else?
>
>  Andy
>
> On 05/02/2021 11:11, Andy Seaborne wrote:
> > 3.17.0 was 2020-12-01
> > so the usual timescale is end Feb - end March.
> >
> > Personally, I'd rather let it slip this time if there are any stability
> > issues. That said, we've had a ticket reporting one bug due to clearup
> > so some downstream is at least running with the latest code. (The
> > version number in GIT and in JIRA isn't updated yet.)
> >
> > Jena4:
> >
> > * Java11 [done]
> >   the users@ feedback was this is enough to merit "jena4".
> >
> > * Rename default branch as "main"
> >
> > * Retire SDB
> >
> > * Retire jena-text-es
> >
> > * Deprecation cleanup [ongoing...]
> >
> > * IRIx abstraction
> >(should this become package: org.apache.jena.iri?
> > and rename the old one?)
> >This may tighten up some illegal use of URIs (%XX -
> >  where XX is not hex).
> >
> > * jena-core: rename RDFReader as RDFReaderI
> >and RDFWriter a RDFWriterI to avoid name clashes
> >
> > * Upgrades (except jetty 10/11 - wait until our Java11 move has gone OK)
> >
> > jena-elephas does not build on Java11.
> >
> > What else, especially things that are changes?
> >
> > As ever - contributions welcome!
> >
> >  Andy
>


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: RDF / Linked data track @ ApacheCon

[Claude Warren]

The call for presentations for ApacheCod closes  *Monday, May 3rd, 2021
8:00 AM* (UTC). *Please do not wait until the last minute*.

If we want to have an RDF / Linked data track we need talks.

Please submit your proposal at https://acah2021.jamhosted.net/

Thank you,
Claude


On Sat, Mar 6, 2021 at 9:42 PM Claude Warren  wrote:

> Sorry for the cross posting.
>
> I am the chair for the RDF and Linked data track at ApacheCon this year.
> I am looking for a few speakers to talk about RDF and or Linked data in
> projects, particularly with respect to the use of Apache libraries, tool
> kits, etc.  If you are interested please drop me a line and I will make
> sure you are notified directly when the call for papers opens.
>
> Thank you,
> Claude
>

Jena Permissions 4.0 / changelog?

[Claude Warren]

I have completed the initial coding of the Permissions to add the HardError
flag (so that the union code will work without a strange
SecurityEvaluator.  But I want to note the addition of the flag as it
changes the old default operation.I don't find a changelog in our
codebase.  Are there any recommendation for how to note this?  Also, any
instructions for releasing a component after the main code?

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

SPARQL question

[Claude Warren]

Is there a difference between

CONSTRUCT
  {
?s ?p ?o .
  }
WHERE
  { { WHERE
{ ?s  ?p  ?o}
}
?s2  ?p2  ?o2
  }

and
CONSTRUCT
  {
?s ?p ?o .
  }
WHERE
  { { SELECT  *
  WHERE
{ ?s  ?p  ?o}
}
?s2  ?p2  ?o2
  }

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Pull merge process?

[Claude Warren]

I created a pull request.  Andy reviewed and approved it.  Am i responsible
for performing the merge or is that required to be done by a 3rd party?

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Query Pattern question

[Claude Warren]

Is there any effective difference (execution difference) between

ElementGroup [ ElementPathBlock[  ], ElementPathBlock[  ] ]

and

ElementPathBlock[ ,  ]


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Weird parsing difference

[Claude Warren]

If I use an ARQParser to parse the string "?x < 10"
I get an E_LessThan object with the expected var and and XMLSchema#integer

If I use NodeFactory.createLiteral(LiteralLabelFactory.createTypedLiteral(
10 )) I get the XMLSchema#int.

This means that the following fails
{noformat}
E_LessThan lt1 = new E_LessThan( new ExprVar(Var.alloc( "x" )),
NodeFactory.createLiteral(LiteralLabelFactory.createTypedLiteral( 10 ));
E_LessThan lt2 = ARQParser.parse( "?x < 10" )

ElementFilter ef1 = new ElementFilter( lt1 );
ElementFilter ef2 = new ElementFilter( lt2 );

assertTrue( ef1.equalTo( ef2 ) );
{noformat}

It fails because ElementFilter.equalTo executes lt1.equalsBySyntax( lt2 )
and by syntax they are different.  Even though they will return the same
results for any set of data.

Should E_LessThan and other similar functions do something more akin to the
NodeValue.compare() to determine if the functions are equal?

I'm not sure what a good solution would be here but I think this is a
problem.

comments?

Claude






Now this doesn't make a difference during evaluation as they both evaluate
as BitInteger.  However, when comparing the


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: About JENA-2339 - security related

[Claude Warren]

I am sorry that I am coming to this _VERY_ late.

I don't understand why Permissions can not be used.  It does address
individual graphs.  It does handle union graphs properly.  It does handle
the difference between graph update and model update.  It calls back to the
Security engine to effectively say "Can this user perform this operation on
this IRI"  It also asks   "Can this user perform this operation on
this/these triples in this IRI".

I don't understand what queries you are trying to execute that can not be
answered.

Claude



On Mon, Aug 15, 2022 at 2:27 PM Andy Seaborne  wrote:

> There is one Jena user - it's Vilnis (for Iotics).
>
> Your use cases - whatever they are - are for the current product and
> will evolve. Whether the way you propose will support the evolution of
> the use cases in the future, say the next 5 years, is unclear (and I
> think quite unlikely both on security features because product feature
> evolve, and on wanting to working with spatial or text datasets).  Jena
> tries to give stability.
>
> The essence of the PR is ~30 lines in SecurityContextDynamic.
> The rest is rearranging the plumbing to have a magic user.
> This does not need DatasetGraphAccessControl.
>
> This could be in a custom query processor extending SPARQL_QueryDataset
>   overriding decideDataset delivered as Fuseki Module. (You can override
> the standard query processor (1 line of code) if you want all query
> services to have this, or all for a particular service (2 lines of
> code), or be a new endpoint that offers only SPARQL query over a view of
> the dataset. The latter is better for you because you can put API
> security on the endpoint. It's a opt-in, drop-in extension, to a
> standard distribution Fuseki/Main.
>
> The amount of code reuse from SecurityContextView is 20 lines maybe via
> SecurityContextView.filterTDB and the functionality could made into a
> function.
>
> Now your usage is not a security issue for the Fuseki server as the HTTP
> request interface is not changed. No interaction with GSP.
>
> So Iotics add their own query processor to a standard Fuseki server and
> can evolve the extension. Configuring the network for the extension is
> the responsibility of the Iotics deployment.
>
> The extension might even be interesting to other Jena users not as
> security feature but as for the dynamic view capability.
>
>  >>> using a "SELECT {} 1" query, and
>  >>> adding a certain set of graphs makes the queries on my laptop take:
>  >>> ~600 graphs ~115ms
>  >>> ~1500 graphs ~162ms
>  >>> ~3k graphs ~240ms
>  >>> ~6k graphs ~400ms
>  >>
>  >> That's an illustration of the current system but we don't know what
>  >> is the cause of the cost.
>  >>
>  >> What piece of the code is taking the time?
>  >> Maybe the right thing to do is make it faster.
>  >
>  > I haven't looked into this in great detail, but from my understanding
>  > the time taken is a combination of a) parsing the input of allowed
>  > graphs and b) generating a new SecurityContext (holding a hashmap of
>  > said graphs). If providing a set of allowed graphs in the proposed way
>  > is not a no-go, I'm happy to dig into where the cost is exactly.
>
> We haven't seen the queries you were making. It is difficult to believe
> that Java takes >100ms to build a 6K entry hash map.
>
> You mentioned that request line gets too long. True for GET but a SPARQL
> query request could be sent as a HTML form
> (application/x-www-form-urlencoded) so listing the graph using
> ?default-graph-uri=/?named-graph-uri= can be much larger than the
> practical GET limits.
>
>  Andy
>
>
>

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Updates for sparql.org

[Claude Warren]

Greetings,

After several attempts the sparql.org site is automated.  There was a
recent announcement from infrastructure indicating that they would be
automating reboots to handle system updates.  The changes listed below were
undertaken to ensure that the sparql.org site would restart properly.

I modified
https://github.com/apache/infrastructure-p6/blob/production/data/nodes/jena-vm.apache.org.yaml
to add the uu_asf entry to send reboot notifications to dev@jena.a.o and
added proxy pass info to the sparql-ssl section.

On the jena-vm.apache.org server (where sparql.org is alised) I created an
/etc/jena directory and
copied files from /home/andy/sparqler to /etc/jena/sparqler.

Updates for sparqler should now be performed in the /etc/jena/sparqler
directory.

I created sparqler.service starts after apache2.service and uses
"ExecStart=/etc/jena/sparqler/run-sparqler" and "ExecStop=/usr/bin/pkill -f
fuseki" to start and stop the Fuseki engine after setting
"Environment=BACKGROUND=0" to force operation in the foreground.

I created a symbolic link from  /lib/systemd/system/sparqler.service to
/etc/jena/sparqler.service and executed `systemctl enable sparqler.service`
to ensure the system starts and stops as required during rebooting.

We might want to consider adding sparqler.service as a fuseki.service into
one of the subprojects as an example of how to start/stop using systemd.

This should resolve any issues with rebooting the sparql.org system.

Claude

Re: Updates for sparql.org

[Claude Warren]

that can be done by editing
https://github.com/apache/infrastructure-p6/blob/production/data/nodes/jena-vm.apache.org.yaml
to do the redirect if someone wants to take it on.

On Thu, Mar 30, 2023 at 2:18 PM Martynas Jusevičius 
wrote:

> Maybe http://sparql.org should redirect to https://sparql.org?
>
> On Thu, Mar 30, 2023 at 3:03 PM Andy Seaborne  wrote:
> >
> > Claude,
> >
> > Thank you for doing this. https works now as well which is great.
> >
> > One question: the ASF Infra email mentioned
> >
> > On 29/03/2023 12:43, Claude Warren wrote:
> > > Greetings,
> > >
> > > After several attempts the sparql.org site is automated.  There was a
> > > recent announcement from infrastructure indicating that they would be
> > > automating reboots to handle system updates.  The changes listed below
> were
> > > undertaken to ensure that the sparql.org site would restart properly.
> > >
> > > I modified
> > >
> https://github.com/apache/infrastructure-p6/blob/production/data/nodes/jena-vm.apache.org.yaml
> > > to add the uu_asf entry to send reboot notifications to dev@jena.a.o
> and
> > > added proxy pass info to the sparql-ssl section.
> >
> > If the reboots become too frequent (I hope not) emal can be redirected
> > but i think for now, dev@ is fine.
> >
> > FYI all: the server runs Ubuntu 20.04.6 LTS
> >
> > > On the jena-vm.apache.org server (where sparql.org is alised) I
> created an
> > > /etc/jena directory and
> > > copied files from /home/andy/sparqler to /etc/jena/sparqler.
> > >
> > > Updates for sparqler should now be performed in the /etc/jena/sparqler
> > > directory.
> >
> > And I've moved the old stuff into a dumping area to make sure I see that
> > the upgrade should be done elsewhere.
> >
> > >
> > > I created sparqler.service starts after apache2.service and uses
> > > "ExecStart=/etc/jena/sparqler/run-sparqler" and
> "ExecStop=/usr/bin/pkill -f
> > > fuseki" to start and stop the Fuseki engine after setting
> > > "Environment=BACKGROUND=0" to force operation in the foreground.
> > >
> > > I created a symbolic link from  /lib/systemd/system/sparqler.service to
> > > /etc/jena/sparqler.service and executed `systemctl enable
> sparqler.service`
> > > to ensure the system starts and stops as required during rebooting.
> > >
> > > We might want to consider adding sparqler.service as a fuseki.service
> into
> > > one of the subprojects as an example of how to start/stop using
> systemd.
> >
> > Yes - where do you suggest?
> >
> > Either jena-fuseki2/examples or jena-fuseki2/jena-fuseki-main/sparqler
> > (and the latter could usefully be moved out .. but separate matter).
> >
> > > This should resolve any issues with rebooting the sparql.org system.
> > >
> > > Claude
> > >
> >
> > Works great!
> >
> >  Andy
>


-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Updates for sparql.org

[Claude Warren]

Does anyone know how to ensure that the text "## Licensed under the terms
of http://www.apache.org/licenses/LICENSE-2.0"; in the sparqler.service file
will pass the RAT test.

On Thu, Mar 30, 2023 at 5:09 PM Claude Warren  wrote:

> that can be done by editing
> https://github.com/apache/infrastructure-p6/blob/production/data/nodes/jena-vm.apache.org.yaml
> to do the redirect if someone wants to take it on.
>
> On Thu, Mar 30, 2023 at 2:18 PM Martynas Jusevičius <
> marty...@atomgraph.com> wrote:
>
>> Maybe http://sparql.org should redirect to https://sparql.org?
>>
>> On Thu, Mar 30, 2023 at 3:03 PM Andy Seaborne  wrote:
>> >
>> > Claude,
>> >
>> > Thank you for doing this. https works now as well which is great.
>> >
>> > One question: the ASF Infra email mentioned
>> >
>> > On 29/03/2023 12:43, Claude Warren wrote:
>> > > Greetings,
>> > >
>> > > After several attempts the sparql.org site is automated.  There was a
>> > > recent announcement from infrastructure indicating that they would be
>> > > automating reboots to handle system updates.  The changes listed
>> below were
>> > > undertaken to ensure that the sparql.org site would restart properly.
>> > >
>> > > I modified
>> > >
>> https://github.com/apache/infrastructure-p6/blob/production/data/nodes/jena-vm.apache.org.yaml
>> > > to add the uu_asf entry to send reboot notifications to dev@jena.a.o
>> and
>> > > added proxy pass info to the sparql-ssl section.
>> >
>> > If the reboots become too frequent (I hope not) emal can be redirected
>> > but i think for now, dev@ is fine.
>> >
>> > FYI all: the server runs Ubuntu 20.04.6 LTS
>> >
>> > > On the jena-vm.apache.org server (where sparql.org is alised) I
>> created an
>> > > /etc/jena directory and
>> > > copied files from /home/andy/sparqler to /etc/jena/sparqler.
>> > >
>> > > Updates for sparqler should now be performed in the /etc/jena/sparqler
>> > > directory.
>> >
>> > And I've moved the old stuff into a dumping area to make sure I see that
>> > the upgrade should be done elsewhere.
>> >
>> > >
>> > > I created sparqler.service starts after apache2.service and uses
>> > > "ExecStart=/etc/jena/sparqler/run-sparqler" and
>> "ExecStop=/usr/bin/pkill -f
>> > > fuseki" to start and stop the Fuseki engine after setting
>> > > "Environment=BACKGROUND=0" to force operation in the foreground.
>> > >
>> > > I created a symbolic link from  /lib/systemd/system/sparqler.service
>> to
>> > > /etc/jena/sparqler.service and executed `systemctl enable
>> sparqler.service`
>> > > to ensure the system starts and stops as required during rebooting.
>> > >
>> > > We might want to consider adding sparqler.service as a fuseki.service
>> into
>> > > one of the subprojects as an example of how to start/stop using
>> systemd.
>> >
>> > Yes - where do you suggest?
>> >
>> > Either jena-fuseki2/examples or jena-fuseki2/jena-fuseki-main/sparqler
>> > (and the latter could usefully be moved out .. but separate matter).
>> >
>> > > This should resolve any issues with rebooting the sparql.org system.
>> > >
>> > > Claude
>> > >
>> >
>> > Works great!
>> >
>> >  Andy
>>
>
>
> --
> LinkedIn: http://www.linkedin.com/in/claudewarren
>


-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

Mea Culpa

[Claude Warren]

I was not following the dev list and made a change to main while a release
vote was underway.
I opened JENA-2352 [1] and closed it with a pull request [2] which was
reviewed and I merged.

The defect is major and the change is minor, but I will leave it to Andy to
decide if I should back it out.

Claude

[1] https://issues.apache.org/jira/browse/JENA-2352
[2] https://github.com/apache/jena/pull/1848
-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

[Update] Reboot noise

[Claude Warren]

At this time there is no way to reduce the noise from the reboot messages
short of creating a new mailing list.  If there is a strong desire to do
that then I will start the process. However, while the noise is a bit
higher I think that it is handy to know when the system has been updated.

Claude

dataset union query.

[Claude Warren]

is there a GRAPH name for the union of the models in a dataset?

I have tried: ASK FROM  { { 
(){+}  }}

now assuming that there is a  in one of the models of the dataset
it should return "true"

Am I missing something? If not, I think I have found a bug.

-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: dataset union query.

[Claude Warren]

OK.  PBKAC.  But I would like to know if there is a standard name for the
Union of the graphs in the dataset rather than the arq specific one.

On Fri, Nov 17, 2023 at 6:29 PM Claude Warren  wrote:

> is there a GRAPH name for the union of the models in a dataset?
>
> I have tried: ASK FROM  { { 
> (){+}  }}
>
> now assuming that there is a  in one of the models of the dataset
> it should return "true"
>
> Am I missing something? If not, I think I have found a bug.
>
> --
> LinkedIn: http://www.linkedin.com/in/claudewarren
>


-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

Collection of paths?

[Claude Warren]

RDF Collection provides a mechanism to create a list of Nodes.
Is there a similar construct to create a list of Paths?
I don't see one.

Claude

-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

process question.

[Claude Warren]

I haven't been around for awhile so I have a process question.
How many reviews are required before code can be merged?

Claude

-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

[geosparql] How can I specify a planar geometry?

[Claude Warren]

I want to use the geospatial capabilities of Jena but without the global
shape.  I am looking for a planar geometry (flat, like a small scale map).
Is there a way to specify this?

Claude

-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: [geosparql] How can I specify a planar geometry?

[Claude Warren]

Yes just x and y on a flat surface.

On Sat 6 Jan 2024, 00:45 Marco Neumann,  wrote:

> Can you give an example? you just want to use x and y instead of WGS84
> coordinates?
>
> On Fri, Jan 5, 2024 at 11:28 PM Claude Warren  wrote:
>
> > I want to use the geospatial capabilities of Jena but without the global
> > shape.  I am looking for a planar geometry (flat, like a small scale
> map).
> > Is there a way to specify this?
> >
> > Claude
> >
> > --
> > LinkedIn: http://www.linkedin.com/in/claudewarren
> >
>
>
> --
>
>
> ---
> Marco Neumann
>

Re: [geosparql] How can I specify a planar geometry?

[Claude Warren]

Background:  I am building an "autonomous chair side table" for my wife.
The goal is to have the table carry her tea from the kitchen to whatever
room she is in.  As part of this project I am building a reasoning system
that builds a map based on various inputs (that I call scanners).  I store
all the data in an RDF graph, because all data is a graph and I need to be
able to annotate it in ways I have not discovered yet.

So basically I get sensor data that tells me an obstacle appears at a
location.  That goes into the system initially as a point, however
processing may group it with other points using a convex hull calculation
to convert a point cloud to an obstacle.

I need to be able to determine if a path of a specific with (modeled as a
line with a rounded-end buffer) can traverse the space between 2 points
without touching the obstacles.

So the functions I have been playing with are:

   - Geof.SF_OVERLAPS
   - Geof.SF_INTERSECTS
   - Geof.SF_TOUCHES
   - Geof.DISTANCE_NAME
   - SpatialExtension.NEARBY

I may just revert to writing a couple of filter functions to do what I
want, but I was trying to learn the existing library.

Claude

On Sat, Jan 6, 2024 at 10:47 AM Marco Neumann 
wrote:

> In terms of geo you would talk about some sort of equidistant projection
> that is available via the CRS/SRS.This is currently implemented with Apache
> SIS in Jena.
>
> But it sounds like geo isn't really your thing here, so in conclusion
> geosparql isn't either as it's all about geo data.
>
> What type of functions (access methods) would you like to perform on the
> data?
>
>
>
> On Sat, Jan 6, 2024 at 8:24 AM Claude Warren  wrote:
>
> > Yes just x and y on a flat surface.
> >
> > On Sat 6 Jan 2024, 00:45 Marco Neumann,  wrote:
> >
> > > Can you give an example? you just want to use x and y instead of WGS84
> > > coordinates?
> > >
> > > On Fri, Jan 5, 2024 at 11:28 PM Claude Warren 
> wrote:
> > >
> > > > I want to use the geospatial capabilities of Jena but without the
> > global
> > > > shape.  I am looking for a planar geometry (flat, like a small scale
> > > map).
> > > > Is there a way to specify this?
> > > >
> > > > Claude
> > > >
> > > > --
> > > > LinkedIn: http://www.linkedin.com/in/claudewarren
> > > >
> > >
> > >
> > > --
> > >
> > >
> > > ---
> > > Marco Neumann
> > >
> >
>
>
> --
>
>
> ---
> Marco Neumann
>


-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: [geosparql] How can I specify a planar geometry?

[Claude Warren]

Marco,

Is your earlier code available in a repository somewhere?  I'd love to get
a head start on the functions.

Claude

On Sat, Jan 6, 2024 at 12:43 PM Marco Neumann 
wrote:

> yes, a lot of the geosparql functions will push you towards geo.
>
> It might be better to just reuse the existing low level functions. My first
> spatial implementation for Jena in 2006 made use of the JTS Topology Suite.
>
> JTS is still used in Jena today to run some of the low level evaluation
> mentioned above but can also conveniently be used for X,Y data. I believe
> this might just do the job for your use case.
>
> Marco
>
>
>
>
>
>
>
>  .
>
> On Sat, Jan 6, 2024 at 11:15 AM Claude Warren  wrote:
>
> > Background:  I am building an "autonomous chair side table" for my wife.
> > The goal is to have the table carry her tea from the kitchen to whatever
> > room she is in.  As part of this project I am building a reasoning system
> > that builds a map based on various inputs (that I call scanners).  I
> store
> > all the data in an RDF graph, because all data is a graph and I need to
> be
> > able to annotate it in ways I have not discovered yet.
> >
> > So basically I get sensor data that tells me an obstacle appears at a
> > location.  That goes into the system initially as a point, however
> > processing may group it with other points using a convex hull calculation
> > to convert a point cloud to an obstacle.
> >
> > I need to be able to determine if a path of a specific with (modeled as a
> > line with a rounded-end buffer) can traverse the space between 2 points
> > without touching the obstacles.
> >
> > So the functions I have been playing with are:
> >
> >- Geof.SF_OVERLAPS
> >- Geof.SF_INTERSECTS
> >- Geof.SF_TOUCHES
> >- Geof.DISTANCE_NAME
> >- SpatialExtension.NEARBY
> >
> > I may just revert to writing a couple of filter functions to do what I
> > want, but I was trying to learn the existing library.
> >
> > Claude
> >
> > On Sat, Jan 6, 2024 at 10:47 AM Marco Neumann 
> > wrote:
> >
> > > In terms of geo you would talk about some sort of equidistant
> projection
> > > that is available via the CRS/SRS.This is currently implemented with
> > Apache
> > > SIS in Jena.
> > >
> > > But it sounds like geo isn't really your thing here, so in conclusion
> > > geosparql isn't either as it's all about geo data.
> > >
> > > What type of functions (access methods) would you like to perform on
> the
> > > data?
> > >
> > >
> > >
> > > On Sat, Jan 6, 2024 at 8:24 AM Claude Warren  wrote:
> > >
> > > > Yes just x and y on a flat surface.
> > > >
> > > > On Sat 6 Jan 2024, 00:45 Marco Neumann, 
> > wrote:
> > > >
> > > > > Can you give an example? you just want to use x and y instead of
> > WGS84
> > > > > coordinates?
> > > > >
> > > > > On Fri, Jan 5, 2024 at 11:28 PM Claude Warren 
> > > wrote:
> > > > >
> > > > > > I want to use the geospatial capabilities of Jena but without the
> > > > global
> > > > > > shape.  I am looking for a planar geometry (flat, like a small
> > scale
> > > > > map).
> > > > > > Is there a way to specify this?
> > > > > >
> > > > > > Claude
> > > > > >
> > > > > > --
> > > > > > LinkedIn: http://www.linkedin.com/in/claudewarren
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > >
> > > > > ---
> > > > > Marco Neumann
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > >
> > > ---
> > > Marco Neumann
> > >
> >
> >
> > --
> > LinkedIn: http://www.linkedin.com/in/claudewarren
> >
>
>
> --
>
>
> ---
> Marco Neumann
>


-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: [geosparql] How can I specify a planar geometry?

[Claude Warren]

Perhaps we should think about a Geometry SPARQL extension.

On Sat, Jan 6, 2024 at 1:51 PM Marco Neumann 
wrote:

> It used to be all on Google Code. I believe Google folded that site
> a few years ago.  I will have a look through my backups. But to be frank it
> would be straightforward to implement nowadays and the old code would be
> pre Jena 3.
>
> You might be able to get at some of the pre 2007 stuff by way of
> archive.org
> https://web.archive.org/web/20110710100953/http://geosparql.org/
>
> Keep in mind I came with a background in 3D queries back in 1996. So the
> Jena 2D spatial stuff was quick for me to implement. The most challenging
> part was to hookup the functions to the RDF query engine back in 2002 with
> what was available in RDQL (a precursor to SPARQL) in Jena. This part is
> well documented nowadays with what is sometimes dubbed as Jena property
> functions or magic properties in SPARQL.
>
> I am currently working again with a 3D extension to SPARQL but this is
> limited to a custom implementation (and not in line with OGC GeoSPARQL) in
> a client project.
>
> Let's discuss in more detail, you might be flying in no time.
>
> Marco
>
>
>
> On Sat, Jan 6, 2024 at 12:00 PM Claude Warren  wrote:
>
> > Marco,
> >
> > Is your earlier code available in a repository somewhere?  I'd love to
> get
> > a head start on the functions.
> >
> > Claude
> >
> > On Sat, Jan 6, 2024 at 12:43 PM Marco Neumann 
> > wrote:
> >
> > > yes, a lot of the geosparql functions will push you towards geo.
> > >
> > > It might be better to just reuse the existing low level functions. My
> > first
> > > spatial implementation for Jena in 2006 made use of the JTS Topology
> > Suite.
> > >
> > > JTS is still used in Jena today to run some of the low level evaluation
> > > mentioned above but can also conveniently be used for X,Y data. I
> believe
> > > this might just do the job for your use case.
> > >
> > > Marco
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >  .
> > >
> > > On Sat, Jan 6, 2024 at 11:15 AM Claude Warren 
> wrote:
> > >
> > > > Background:  I am building an "autonomous chair side table" for my
> > wife.
> > > > The goal is to have the table carry her tea from the kitchen to
> > whatever
> > > > room she is in.  As part of this project I am building a reasoning
> > system
> > > > that builds a map based on various inputs (that I call scanners).  I
> > > store
> > > > all the data in an RDF graph, because all data is a graph and I need
> to
> > > be
> > > > able to annotate it in ways I have not discovered yet.
> > > >
> > > > So basically I get sensor data that tells me an obstacle appears at a
> > > > location.  That goes into the system initially as a point, however
> > > > processing may group it with other points using a convex hull
> > calculation
> > > > to convert a point cloud to an obstacle.
> > > >
> > > > I need to be able to determine if a path of a specific with (modeled
> > as a
> > > > line with a rounded-end buffer) can traverse the space between 2
> points
> > > > without touching the obstacles.
> > > >
> > > > So the functions I have been playing with are:
> > > >
> > > >- Geof.SF_OVERLAPS
> > > >- Geof.SF_INTERSECTS
> > > >- Geof.SF_TOUCHES
> > > >- Geof.DISTANCE_NAME
> > > >- SpatialExtension.NEARBY
> > > >
> > > > I may just revert to writing a couple of filter functions to do what
> I
> > > > want, but I was trying to learn the existing library.
> > > >
> > > > Claude
> > > >
> > > > On Sat, Jan 6, 2024 at 10:47 AM Marco Neumann <
> marco.neum...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > In terms of geo you would talk about some sort of equidistant
> > > projection
> > > > > that is available via the CRS/SRS.This is currently implemented
> with
> > > > Apache
> > > > > SIS in Jena.
> > > > >
> > > > > But it sounds like geo isn't really your thing here, so in
> conclusion
> > > > > geosparql isn't either as it's all about geo data.
> > > > >
> > > > > What type 

Re: [VOTE] Apache Jena 5.0.0

[Claude Warren]

+1 checked as per checklist.  I also run the 5.0.0 in a Raspberry Pi
project and have had no problems there.

On Sat, Mar 16, 2024 at 7:34 PM Andy Seaborne  wrote:

> Hi,
>
> Here is a vote on the release of Apache Jena version 5.0.0.
>
>  Release Vote
>
> This vote will be open until at least
>
>  Wednesday 20th March 2024 at 08:00 UTC
>
> Please vote to approve this release:
>
>  [ ] +1 Approve the release
>  [ ]  0 Don't care
>  [ ] -1 Don't release, because ...
>
> Everyone, not just committers, is invited to test and vote.
> Please download and test the proposed release. See the checklist below.
>
> Staging repository:
>https://repository.apache.org/content/repositories/orgapachejena-1063
>
> Proposed dist/ area:
>https://dist.apache.org/repos/dist/dev/jena/
>
> Keys:
>https://svn.apache.org/repos/asf/jena/dist/KEYS
>
> Git commit (browser URL):
>https://github.com/apache/jena/commit/f475cdc84a
>
> Git Commit Hash:
>f475cdc84a85e48c22a2c6487141e2d782c10517
>
> Git Commit Tag:
>jena-5.0.0
>
> If you expect to check the release but the time limit does not work
> for you, please email within the schedule above.
>
>  Andy
>
>
>  About Jena5 
>
> == General
>
> Issues since Jena 4.10.0:
>
>https://s.apache.org/jena-5.0.0-issues
>
> which includes the ones specifically related to Jena5:
>
>https://github.com/apache/jena/issues?q=label%3Ajena5
>
>
> ** Java Requirement
>
> Java 17 or later is required.
> Java 17 language constructs now are used in the codebase.
>
> ** Language tags
>
> Language tags become are case-insensitive unique.
>
> "abc"@EN and "abc"@en are the same RDF term.
>
> Internally, language tags are formatted using the algorithm of RFC 5646.
>
> Examples "@en", "@en-GB", "@en-Latn-GB".
>
> SPARQL LANG(?literal) will return a formatted language tag.
>
> Data stored in TDB using language tags must be reloaded.
>
> ** Term graphs
>
> Graphs are now term graphs in the API or SPARQL. That is, they do not
> match "same value" for some of the java mapped datatypes. The model API
> already normalizes values written.
>
> TDB1, TDB2 keep their value canonicalization during data loading.
>
> A legacy value-graph implementation can be obtained from GraphMemFactory.
>
> ** RRX - New RDF/XML parser
>
> RRX is the default RDF/XML parser. It is a replacement for ARP.
> RIOT uses RRX.
>
> The ARP parser is still temporarily available for transition assistance.
>
> ** Remove support for JSON-LD 1.0
>
> JSON-LD 1.1, using Titanium-JSON-LD, is the supported version of JSON-LD.
>
> https://github.com/filip26/titanium-json-ld
>
> ** Turtle/Trig Output
>
> "PREFIX" and "BASE" are output by default for Turtle and TriG output.
>
> ** Misc
>
> There is now a release BOM for Jena artifacts - artifact
> org.apache.jena:jena-bom
>
> There are now OWASP CycloneDX SBOM for Jena artifacts.
> https://github.com/CycloneDX
>
>
>  API Users
>
> ** Deprecation removal
>
> There has been a clearing out of deprecated functions, methods and
> classes. This includes the deprecations in Jena 4.10.0 added to show
> code that is being removed in Jena5.
>
> ** QueryExecutionFactory
>
> QueryExecutionFactory is simplified to cover commons cases only; it
> becomes a way to call the general QueryExecution builders which are
> preferred and provide all full query execution setup controls.
>
> Local execution builder:
> QueryExecution.create()...
>
> Remote execution builder:
> QueryExecution.service(URL)...
>
> ** QueryExecution variable substitution
>
> Using "substitution", where the query is modified by replacing one or
> more variables by RDF terms, is now preferred to using "initial
> bindings", where query solutions include (var,value) pairs.
>
> "substitution" is available for all queries, local and remote, not just
> local executions.
>
> Rename TDB1 packages org.apache.jena.tdb -> org.apache.jena.tdb1
>
> The update to slf4j 2.x means any use of log4j should use artifact
> "log4j-slf4j2-impl" (was "log4j-slf4j-impl").
>
>
>  Fuseki Users
>
> Fuseki: Uses the jakarta namespace for servlets and Fuseki has been
> upgraded to use Eclipse Jetty12.
>
> Apache Tomcat10 or later, is required for running the WAR file.
> Tomcat 9 or earlier will not work.
>
>
> ---
>
>
> Checking:
>
> + are the GPG signatures fine?
> + are the checksums correct?
> + is there a source archive?
> + can the source archive be built?
>(NB This requires a "mvn install" first time)
> + is there a correct LICENSE and NOTICE file in each artifact
>(both source and binary artifacts)?
> + does the NOTICE file contain all necessary attributions?
> + have any licenses of dependencies changed due to upgrades?
> if so have LICENSE and NOTICE been upgraded appropriately?
> + does the tag/commit in the SCM contain reproducible sources?
>


-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: [ANN] New PMC member : Arne Bernhardt

[Claude Warren]

Arne,

Welcome to the fold.

Claude

On Fri, Apr 5, 2024 at 6:09 PM Andy Seaborne  wrote:

> We are pleased to announce that Arne has accepted an invitation to join
> the Apache Jena PMC.
>
> Please welcome Arne to this role!
>
>  Andy
>


-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

Anyone going to Community over Code EU?

[Claude Warren]

I will be attending Community over Code EU (June 3-5) in Bratislava.  Is
anyone else here planning on going?  If so do we have enough people to do a
birds of a feather session and perhaps find some more interested parties?

Claude

Re: [VOTE][LAZY] Accept jena-ontapi into the Apache Jena codebase

[Claude Warren]

+1

On Fri, May 3, 2024 at 9:56 PM Arne Bernhardt 
wrote:

> +1
>
> Andy Seaborne  schrieb am Fr., 3. Mai 2024, 19:17:
>
> > This is a lazy consensus VOTE to accept a PR for a new module
> > jena-ontapi that enhances Apache Jena with OWL2 support.
> >
> > https://github.com/apache/jena/pull/2420
> > from @sszuev
> >
> > This vote is open until
> >
> > 05:00am UTC Wednesday 8th May 2024
> >
> > The code in this PR is within a new jena-ontapi module except for two
> > changes in the top Jena POM to add the module into the build.
> >
> > There is no change to org.apache.jena.ontology code in jena-core.
> >
> > There are no new dependencies for downstream user/application code.
> >
> > PR README.md (temporary link)
> >
> >
> >
> https://github.com/apache/jena/blob/f1584c53c9834a38248dbfca1121214c8cdff4d8/jena-ontapi/README.md
> >
> > Previous discussion:
> > https://lists.apache.org/thread/yr9q394fssr0mvgxvrskynmhjlz0g33x
> >
> >  Andy
> >
>


-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

SPDX, Rats and Configs --- oh my

[Claude Warren]

Greetings,

I saw a note from Andy awhile back about exploring SPDX tag usage in Rat.
I am currently working on Rat to make it much more configurable.  Recent
changes include the ability to detect SPDX license statements and an
upcoming change that will check licenses found in archives (e.g. jars in a
lib dir).

My question is, Is there something, some knob or lever or action, that
could be added to Rat, that would help process the Jena releases?

Is there any such change for any other project you are working on?

Note: there have been lots of changes.  Defining licenses is now simply
including a configuration file, licenses can be excluded, Copyright and
SPDX specific tests can be added to license checks.  Checks can be either
required or prohibited.  Checks can be grouped with "all" or "any".

Any input wolud be appreciated.
Claude

-- 
LinkedIn: http://www.linkedin.com/in/claudewarren

QueryBuilder

[Claude Warren]

I have code for a query builder that I would like to contribute... but I
don't see a good place to put the code.

It is not Jena specific.

it has the following methods

addFilter()
addGraph()
addOptional()
addOrderBy()
addPrefix()
addSubQuery()
addUnion()
addVar()
addWhere()
build()
clone()

the build() method creates a SELECT statement string.

the clone() makes a deep copy so it is easy to create a base query in code
and then extend it.

Anybody have any idea if and where it might go in the Jena source? Perhaps
the jena-commons experimental code?

If we decide that it doesn't belong, I'll just create a git hub project for
it.

Claude
-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: QueryBuilder

[Claude Warren]

Looking back at the code again.. it does use the Jena Node and Resource
clases so it is Jena specific.

It could build an ARQ syntax tree -- I just didn't go that far with it.

I think there is a need for a Commons and have created jena-commons in the
Experimental branch.  Currently that only contains the Apache commons
Configuration implementation that uses Models to store the configuration
options.

I could add this there.

I don't have a handle on git structure so I won't comment on that, but will
read and learn.

Claude


On Tue, Jul 22, 2014 at 1:33 PM, Andy Seaborne  wrote:

>
>
>
>
> On 21/07/14 20:34, Claude Warren wrote:
>
>> I have code for a query builder that I would like to contribute... but I
>> don't see a good place to put the code.
>>
>> It is not Jena specific.
>>
>
> So it's string based?
>
> Can it (could it) build an ARQ syntax tree?  That avoid the need to parse,
> and makes it harder to have injection issues (accidental or not so
> accidental).
>
>
>
>> it has the following methods
>>
>> addFilter()
>> addGraph()
>> addOptional()
>> addOrderBy()
>> addPrefix()
>> addSubQuery()
>> addUnion()
>> addVar()
>> addWhere()
>> build()
>> clone()
>>
>> the build() method creates a SELECT statement string.
>>
>> the clone() makes a deep copy so it is easy to create a base query in code
>> and then extend it.
>>
>> Anybody have any idea if and where it might go in the Jena source? Perhaps
>> the jena-commons experimental code?
>>
>
> Let's think this through for the general case.  The question must be how
> this fits into releases.  Separate release?  In conjunction with the main
> release?  If it's just source code hosting (no releases), github may be a
> better choice.  Releases have a special status.
>
> It's also most as if "we" (the wider community inc users) would benefit
> from something like Apache Commons.  What else might be a candidate for
> such a thing?
>
> How might we structure the things so that the design works for git as well?
>
> One possibility is that there is a number of different areas "trunk",
> "commons"(or some such area like "modules", "main", ...)
>
> But git will clone everything unlike svn that can pick and choose (isthat
> correct for git these days?)
>
> What I think we need to have a clear setup as to what's in the main
> releases and what's an additional area so that people don't get the
> expectation that an additional module is going to get released every main
> release.
>
>
>
>> If we decide that it doesn't belong, I'll just create a git hub project
>> for
>> it.
>>
>> Claude
>>
>>
> WDYT?
>
> Andy
>
>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Other build jobs

[Claude Warren]

Jenna Commons was supposed to build the jena-commons experimental code.

Jena New Tests were supposed to build the experimental new tests.




On Tue, Jul 22, 2014 at 12:49 PM, Andy Seaborne  wrote:

> I have found two triggered builds - "Jena New Tests" and "Jena Commons" -
> both of which don't build and haven't for several days.
>
> But they build the same codebase? What's the difference? (Claude?)
>
> Andy
>



-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

TDB transactions

[Claude Warren]

Are there any tools to help with debugging multi-threaded TDB Transactions?

I know that I must have some sort of weird error as I create a transaction
but when I try to commit I get the "Not active" errors. :(

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

TDB inconsistency warning.

[Claude Warren]

What should I be looking for if I find this error?
WARN org.slf4j.impl.Log4jLoggerAdapter  - Inconsistency: base.allocOffset()
= 4121905 : allocOffset = 4121855

I suspect that it is some sort of write without transaction or something
like that.

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Other build jobs

[Claude Warren]

I suppose I set them up incorrectly.  Any idea how to make it do that?


On Tue, Jul 22, 2014 at 7:48 PM, Andy Seaborne  wrote:

> On 22/07/14 18:42, Claude Warren wrote:
>
>> Jenna Commons was supposed to build the jena-commons experimental code.
>>
>> Jena New Tests were supposed to build the experimental new tests.
>>
>>
> Looking at Jena Commons =>
>
> Repository URL
> http://svn.apache.org/repos/asf/jena/Experimental/new-test
>
> which looks like the wrong URL.
>
> And the build itself:
> [ERROR] java.lang.UnsupportedClassVersionError
>
>
> Why don't these jobs email commits@ when they break?  It would help me,
> at least, to know if some downstream has failed due to some trunk change.
>
> Andy
>
>
>
>>
>>
>> On Tue, Jul 22, 2014 at 12:49 PM, Andy Seaborne  wrote:
>>
>>  I have found two triggered builds - "Jena New Tests" and "Jena Commons" -
>>> both of which don't build and haven't for several days.
>>>
>>> But they build the same codebase? What's the difference? (Claude?)
>>>
>>>  Andy
>>>
>>>
>>
>>
>>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Strange TDB transaction error

[Claude Warren]

I have seen this error 2x now (or at least I think I have).  I am able to
duplicate it but I don't have a small self contained example (yet)

In the current code I have a couple of threads working on a single Dataset.
 I am using transactions and as far as I can tell I am in a transaction
when the error occurs.

I begin the transaction, add a triple to a named model which eventually
causes a BPTreeNode.split() and a BPTreeNode.promote().  However, none of
the transactions are in ACTIVE state.  They are all COMMITTED.

in NodeTupleTableConcrete.addRow() the dsPolicy.writeCounter = 1
but the tupleTable.indexes[0].index.recordsMgr.blockMgr.transaction is in
the COMMITTED state.

Does anyone have any hints for how to debug this problem?

Claude
-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

transactions and model locks.

[Claude Warren]

Are there any known issues mixing transactions on datasources and model
locks (e.g. model.enterCriticalSection(Lock.WRITE)) ?


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: transactions and model locks.

[Claude Warren]

I have lots of moving parts but here goes an attempt at a description.

I have a dataset with multiple named graphs.

I am running a modified (reworked?) version of Fuseki -- Old version but
modified to use JAXB and provide some extra restful functionality to
support queries and jquery clients.

I have 2 threads running in a ScheduledExecutorService.

The Runnables those threads execute do some work (external rest calls) and
then update a named graph with the results.  The Runnables have the dataset
and a resource.   I start a transaction on the dataset and update the
resource. then commit.

To make things more difficult the resource is wrapped with an PA4RDF
annotated object.

Sounds like I need to change the process to send in the URI of the resource
and the model name so that I can get the object within the transaction.

Thx,
Claude





On Wed, Jul 23, 2014 at 1:51 PM, Andy Seaborne  wrote:

> On 23/07/14 11:37, Claude Warren wrote:
>
>> Are there any known issues mixing transactions on datasources and model
>> locks (e.g. model.enterCriticalSection(Lock.WRITE)) ?
>>
>>
>>
> No. Well, not known to me :-)
>
> But I would hazard a guess you have encountered something.
>
> Could you describe your setup?  I get snippets from your emails but not an
> overall picture.
>
> Are you really trying to do multithreads inside one transaction?
>
> Note - it is necessary to get the model inside a transaction and not use
> it across transaction boundaries.
>
> (It's very visible API change to fix this :-()
>
> Andy
>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Joinable Transactions?

[Claude Warren]

As you can probably guess from my recent posts here, I am working with
transactions, datasets, and web applications.

In Spring there is this concept of "joining a transaction".  Basically it
works like this:

When calling begin()

If a transaction is in progress and is the proper type (Write request must
be write, Read request may be Read or Write) use it.

if a transaction is in progress and is not the proper type (write request
with read transaction in progress) throw an exception

if no transaction is in progress start one of the proper type.

when calling commit()

if we joined the transaction do nothing.

if we started the transaction commit it.

when calling abort()/rollback()
if we joined the transaction throw an exception
if we created the  transaction abort rollback the transaction.

In our special case of end() I think we would do something like commit().

I have a class that handles this by wrapping the real dataset and providing
this functionality via thread local variables.

usage case is something like:

In a web service we have a query and a dataset to query against.  We want
to stream the results to the client without creating the potentially large
file locally.

Assume a QueryResult object that comprises the result set from a query and
the dataset it was run against.


   1. service call comes in to a JAX-RS annotated service
   2. service starts a READ transaction.
   3. execute the query
   4. returns a QueryResult (see above)
   5. system uses a @Provider annotated class to format the result as per
   the request.
   6. the transaction.end() is called.


As you can see the transaction is started in one object and is logically
not complete until the end of an second object.

If anyone is interested I'll put the class in the jena-commons.



-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

detecting changes in a dataset.

[Claude Warren]

Is there a listener like interface for datasets? I can't find it.  I need
to detect when any of the models in the dataset change, or when models are
added or removed.

Is there anything like that?

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: detecting changes in a dataset.

[Claude Warren]

I think the Eclipse functionality is "Generate Delegate Methods"

create a class that has an instance variable of some type.

select the instance variable.
right click,
source -> "Generate Delegate Methods"

Is that what you were thinking of?




On Thu, Jul 24, 2014 at 12:25 PM, Andy Seaborne  wrote:

> On 24/07/14 12:03, Claude Warren wrote:
>
>> Is there a listener like interface for datasets? I can't find it.  I need
>> to detect when any of the models in the dataset change, or when models are
>> added or removed.
>>
>> Is there anything like that?
>>
>> Claude
>>
>>
> DatasetGraphMonitor for DatasetGraph for montoring add/remove quads.
>
> There is nothing for Dataset to specifically handle whole model operations
> but the same style with a intercepting wrapper class is easy to do (I wish
> Eclipse had "generate wrapper" functionality)
>
> Andy
>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Joinable Transactions?

[Claude Warren]

The detached result set takes more memory, no?  So for large results on a
heavily loaded system this is an issue.  In addition creating a detached
set takes time, so in systems where response times are critical it may
makes sense to return the streaming result set.




On Thu, Jul 24, 2014 at 12:40 PM, Andy Seaborne  wrote:

> I won't override begin/commit/end to do this.
>
> FWIW I think the idea that "commit()" does not actually cause a persistent
> change on disk is bad.   The contract on commit() is ACID, specifically D
> so ".commit() ; System.exit(0) - restart" smeans teh app is guarantteed to
> see the change.
>
> This is just reference counting?
> What happens if the nesting never un-nests?
>
> Better is to separate concepts and have a counter - that is a samll
> mamangement library for trasnactions:
>
> TransactionLevelMgt mgr = new TransactionLevelMgt(dataset) ;
>
> mgr.beginLevel(dataset, WRITE) ;
> mgr.endLevel() ;
> mgr.abort() ;
>
> and endLevel does a "commit()" when the level count returns to zero.
>
>
> > Assume a QueryResult object that comprises the result set from
> > a query and the dataset it was run against.
>
> Surely the right way is to create a detached result set? (no tied to the
> transaction or the query execution)
>
> Andy
>
>
> On 24/07/14 10:32, Claude Warren wrote:
>
>> As you can probably guess from my recent posts here, I am working with
>> transactions, datasets, and web applications.
>>
>> In Spring there is this concept of "joining a transaction".  Basically it
>> works like this:
>>
>> When calling begin()
>>
>> If a transaction is in progress and is the proper type (Write request must
>> be write, Read request may be Read or Write) use it.
>>
>> if a transaction is in progress and is not the proper type (write request
>> with read transaction in progress) throw an exception
>>
>> if no transaction is in progress start one of the proper type.
>>
>> when calling commit()
>>
>> if we joined the transaction do nothing.
>>
>> if we started the transaction commit it.
>>
>> when calling abort()/rollback()
>> if we joined the transaction throw an exception
>> if we created the  transaction abort rollback the transaction.
>>
>> In our special case of end() I think we would do something like commit().
>>
>> I have a class that handles this by wrapping the real dataset and
>> providing
>> this functionality via thread local variables.
>>
>> usage case is something like:
>>
>> In a web service we have a query and a dataset to query against.  We want
>> to stream the results to the client without creating the potentially large
>> file locally.
>>
>> Assume a QueryResult object that comprises the result set from a query and
>> the dataset it was run against.
>>
>>
>> 1. service call comes in to a JAX-RS annotated service
>> 2. service starts a READ transaction.
>> 3. execute the query
>> 4. returns a QueryResult (see above)
>> 5. system uses a @Provider annotated class to format the result as per
>> the request.
>> 6. the transaction.end() is called.
>>
>>
>>
>> As you can see the transaction is started in one object and is logically
>> not complete until the end of an second object.
>>
>> If anyone is interested I'll put the class in the jena-commons.
>>
>>
>>
>>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Joinable Transactions?

[Claude Warren]

For systems that do not have nested commits the joined commit means that
the outer most commit does the commit.

That is the contract.




On Thu, Jul 24, 2014 at 12:40 PM, Andy Seaborne  wrote:

> I won't override begin/commit/end to do this.
>
> FWIW I think the idea that "commit()" does not actually cause a persistent
> change on disk is bad.   The contract on commit() is ACID, specifically D
> so ".commit() ; System.exit(0) - restart" smeans teh app is guarantteed to
> see the change.
>
> This is just reference counting?
> What happens if the nesting never un-nests?
>
> Better is to separate concepts and have a counter - that is a samll
> mamangement library for trasnactions:
>
> TransactionLevelMgt mgr = new TransactionLevelMgt(dataset) ;
>
> mgr.beginLevel(dataset, WRITE) ;
> mgr.endLevel() ;
> mgr.abort() ;
>
> and endLevel does a "commit()" when the level count returns to zero.
>
>
> > Assume a QueryResult object that comprises the result set from
> > a query and the dataset it was run against.
>
> Surely the right way is to create a detached result set? (no tied to the
> transaction or the query execution)
>
> Andy
>
>
> On 24/07/14 10:32, Claude Warren wrote:
>
>> As you can probably guess from my recent posts here, I am working with
>> transactions, datasets, and web applications.
>>
>> In Spring there is this concept of "joining a transaction".  Basically it
>> works like this:
>>
>> When calling begin()
>>
>> If a transaction is in progress and is the proper type (Write request must
>> be write, Read request may be Read or Write) use it.
>>
>> if a transaction is in progress and is not the proper type (write request
>> with read transaction in progress) throw an exception
>>
>> if no transaction is in progress start one of the proper type.
>>
>> when calling commit()
>>
>> if we joined the transaction do nothing.
>>
>> if we started the transaction commit it.
>>
>> when calling abort()/rollback()
>> if we joined the transaction throw an exception
>> if we created the  transaction abort rollback the transaction.
>>
>> In our special case of end() I think we would do something like commit().
>>
>> I have a class that handles this by wrapping the real dataset and
>> providing
>> this functionality via thread local variables.
>>
>> usage case is something like:
>>
>> In a web service we have a query and a dataset to query against.  We want
>> to stream the results to the client without creating the potentially large
>> file locally.
>>
>> Assume a QueryResult object that comprises the result set from a query and
>> the dataset it was run against.
>>
>>
>> 1. service call comes in to a JAX-RS annotated service
>> 2. service starts a READ transaction.
>> 3. execute the query
>> 4. returns a QueryResult (see above)
>> 5. system uses a @Provider annotated class to format the result as per
>> the request.
>> 6. the transaction.end() is called.
>>
>>
>>
>> As you can see the transaction is started in one object and is logically
>> not complete until the end of an second object.
>>
>> If anyone is interested I'll put the class in the jena-commons.
>>
>>
>>
>>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Module Structuring and Release Cycles

[Claude Warren]

I am a fan of JRE as it ensure that everything works together at that point
in time.

That being said, perhaps we should define what is "Jena Core".  As opposed
to jena-core, "Jena Core" are the components that provide the basic
functionality that we want to promote for Jena.  At this time it is
probably everthing.  However, if we proceed with something like
jena-commons  as a collection of code that provides functionality that
might be useful for developers using the Jena Core but not required -- then
it could be outside the JRE envelope.

If we provide different configurations for fuseki (just thinking outside
the box here) then those might be outside the JRE envelope.

If we end up with a bunch of graph or model implementations (e.g. JDBC
based, Hadoop based, etc) then they might be outside of the JRE envelope.

I think we determine what goes inside the envelope by the following
checklist:

1) does it implement a W3 RDF based recommendation (e.g. SPARQL)
2) is it required by any component that provides #1
3) is it required to provide a reasonable out of the box working
implementation (e.g. TDB)
4) (optional) is it directly tied to and significantly affected by versions
of the above (e.g. jena-security)

If it ticks a box in the above then it is in.  If not then it is out.  If
security end up outside the box, I would like a way to match security
version with release versions and get the download pages updated, etc.

Just my thoughts,
Claude


If we provide


On Wed, Jul 30, 2014 at 10:46 AM, Andy Seaborne  wrote:

> In trying to reply to this, I keep coming back to teh release structure
> and whether we want to JRE. (JRE = Just Release Everything)
>
> This affects the repo structure (not so much the modules).  It would be
> nice for people to check out just a part of the project.  It's possible in
> git but it's convoluted [1][2].
>
> If there are multiple git repos, I don't see how JRE would work except as
> doing each repo in turn producing multiple source artifacts.  There isn't a
> single source artifact produced this way. (We could write our build process
> that glues things togther).
>
> I think we want to have JRE without excess plumbing and maintenance of a
> build process.  The partial fetch of Jena is an nice-to-have feature but
> not at the cost of making other things harder.
>
> So it seems to be that we have one (git) repo.
>
> Within the one repo, we split up the modules into sub-projects.  We
> document/link to ways to pull part of the repo.
>
> The only module that might benefit from being separate is jena-iri which
> is very stable.
>
> Does that make sense?
>
> Andy
>
> [1] http://jasonkarns.com/blog/subdirectory-checkouts-with-
> git-sparse-checkout/
> [2] http://briancoyner.github.io/blog/2013/06/05/git-sparse-checkout/
>
>
>
> On 23/07/14 13:48, Andy Seaborne wrote:
>
>> Rob,
>>
>> Excellent
>>
>> I hadn't realised that TLP could have several git repos and for some
>> reason I was thinking it discouraged.  (No idea where that belief came
>> from as it's clearly wrong.)
>>
>> It solves the big problem I was noodling on - how to be able to work on
>> some of Jena without having to have everything cloned.  It would (might)
>> be tolerable for us committers but opaque for people coming to jena
>> initially.
>>
>>  Andy
>>
>> On 23/07/14 11:02, Rob Vesse wrote:
>>
>>> Hey All
>>>
>>> Following up on the recent discussion points Andy raised around module
>>> structuring as the project expands its scope and modules do we want to
>>> think
>>> about reorganising our repository and how we do releases?
>>>
>>> For discussion lets consider splitting the current modules up into
>>> several
>>> sub-projects like so:
>>>
>>> * commons - jena-iri and Claude's proposed jena-commons module
>>> * core - jena-core, jena-arq, jena-tdb, jena-security
>>> * search - jena-text, jena-spatial
>>> * fuseki - jena-fuseki
>>> * jdbc - All JDBC modules
>>> * sdb - jena-sdb
>>> * client - Stephen's experimental jena-client module
>>> * hadoop - The jena-hadoop-rdf modules
>>> (This is just an illustration, what goes in which sub-project can be
>>> decided
>>> later, this is intended to stimulate discussion rather than to be a
>>> concrete
>>> proposal).
>>>
>>> So rather than having a flat trunk structure with each module in the
>>> root we
>>> would have a structure where each sub-project has a directory in trunk
>>> with
>>> its modules located under it.  In this structure jena-parent then
>>> lives in
>>> the top level directory.
>>>
>>> With such a structure we could then consider having different release
>>> cadences for different sub-projects.  So when we do a new core release we
>>> would not necessarily have to release everything else at the same time
>>> (though in most cases I suspect we would want to), however if a
>>> sub-project
>>> wants to make an interim release in the meantime there would be
>>> nothing to
>>> stop this happening e.g. making a critical bug fix, tracking a
>>> sub-proj

packaging

[Claude Warren]

I just realized that the jena-security jar is in the apache-jena-libs
package but not the apache-jena-.zip|tgz

I must have misconfigured something way back when.  If someone can point me
to the proper options I'll make the fix.

thx,
Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Jena on Nosql

[Claude Warren]

Andrea,

I am not certain that Cassandra will work for a Jena graph store because of
the requirement for the primary key column.  But...

The easiest mechanism for extension is to extend GraphBase.  As the
documentation states:

GraphBase is an implementation of Graph that provides some convenient base
functionality for Graph implementations.

Subtypes of GraphBase must provide performAdd(Triple),
performDelete(Triple), graphBaseFind(TripleMatch,TripleAction), and
graphBaseSize(). GraphBase provides default implementations of the other
methods, including the other finds (on top of that one), a simple-minded
prepare, and contains. GraphBase also handles the event-listening and
registration interfaces.
When a GraphBase is closed, future operations on it may throw an exception.

Once that is working you can override other methods to provide better
performance.

Claude


On Fri, Aug 1, 2014 at 6:57 PM, Andrea Gazzarini 
wrote:

> Hi devs,
> for a project where I'm working on, I'd need to use Jena over a nosql
> storage for storing and queryjng rdf, specifically using Cassandra. So
> after googling a bit I wasn't able to find something that fits my needs.
>
> On top of that, i'd have the following questions:
>
> - do you know if there's something ready to use?
>
> - in case the answer to the previous question is negative, I'm actively
> participating in the development of CumulusRDF [1], an RDF store that uses
> Cassandra...but unfortunately is not using Jena :( so, here's the
> question...do you think a jena-cassandra could be a useful contribution? I
> could reuse something did for that project  (appropriately translated into
> jena domain). In case, could you please give me some advice (e.g. extension
> points on the jena api) to start?
>
> Best,
> Andrea
>
> [1] https://code.google.com/p/cumulusrdf
>



-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

SPIs and Testing

[Claude Warren]

I think I have some time again to look at testing some more.  The last time
I dove into it I learned quite a bit, but I don't think I improved on the
existing testing at all.

However, with time to reflect and movement forward of the system I think
that perhaps we should think about defining the SPI and what the contracts
are for the SPI.

I would like to take a crack at the following:

1) getting consensus on what classes/interfaces are part of the basic SPI
(e.g. Graph interface / GraphBase class)
2) document the contracts for those classes. (e.g. adding a triple to graph
triggers notification of listeners)
3) write contract tests for those classes/interfaces. (
https://github.com/Claudenw/junit-contracts)  This should result in a set
of tests that are simple to extend/implement and will ensure that
implementations implement the entire contract correctly.

With this in mind, what is the best forum for this discussion?  Who is
interested in participating in the discussions?  Who is interested in
helping to build the contract tests?

I was thinking about the wiki as a place to hold the discussions.

Thoughts?

Claude


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: SPIs and Testing

[Claude Warren]

Perhaps we can use the wiki to record consensus as they are reached.

We might consider moving from traditional wrappers to dynamic proxies to do
add in functionality as this allows the lower level functionality to stil
lbe visible through the wrapper.

Claude


On Thu, Aug 7, 2014 at 3:45 PM, Andy Seaborne  wrote:

> May be discussion is better on dev@ while the wiki is good for creating
> and collecting long term material but poor for discussion.
>
> I'm interested as time permits.
>
> It would be good to have these tests - example: I'd like to consider a
> change (jena3) to the implementation of graph events to have the basic
> graph not event-aware and add a wrapper to do the event triggers. It would
> make extensions cleaner. The most basic graph interface is little more than
> add/delete/find.  Contract tests would pin down external functionality.
>
>     Andy
>
>
> On 06/08/14 18:46, Claude Warren wrote:
>
>> I think I have some time again to look at testing some more.  The last
>> time
>> I dove into it I learned quite a bit, but I don't think I improved on the
>> existing testing at all.
>>
>> However, with time to reflect and movement forward of the system I think
>> that perhaps we should think about defining the SPI and what the contracts
>> are for the SPI.
>>
>> I would like to take a crack at the following:
>>
>> 1) getting consensus on what classes/interfaces are part of the basic SPI
>> (e.g. Graph interface / GraphBase class)
>> 2) document the contracts for those classes. (e.g. adding a triple to
>> graph
>> triggers notification of listeners)
>> 3) write contract tests for those classes/interfaces. (
>> https://github.com/Claudenw/junit-contracts)  This should result in a set
>> of tests that are simple to extend/implement and will ensure that
>> implementations implement the entire contract correctly.
>>
>> With this in mind, what is the best forum for this discussion?  Who is
>> interested in participating in the discussions?  Who is interested in
>> helping to build the contract tests?
>>
>> I was thinking about the wiki as a place to hold the discussions.
>>
>> Thoughts?
>>
>> Claude
>>
>>
>>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Graph SPI Contract

[Claude Warren]

This is a message stack for Graph SPI Contract testing.  It covers only the
Jena 2 Graph Contract.  This an attempt to document the current Graph
contract.  Any correction should specify the bullet point number.

   1. add() -- technically from GraphAdd
  1. when a triple is added to a graph all registered listeners must
  receive an (add graph triple) message
  2. subsequent graph.contains( triple ) must return true.
  3. If add is performed within a transaction the listeners are not
  notified until after the commit.
  4. If graph is read only (Capabilities.addAllowed() returns false)
  must throw AddDeniedException
   2. clear()
  1. If the graph can be empty (Capabilities.canBeEmpty()) there should
  be no triples returned from find( Triple.ANY )
  2. If the graph can not be empty there should only be the elements
  that were present when the graph was created.
  3. if delete is not allowed (Capabilities.canDelete() is
  false) clear() must throw DeleteDeniedException
  3. close()
  1. after close isClosed() should return true
  2. calling close on closed graph should not throw an exception.
  3. calling any Graph method other than close() on a closed graph
  should throw a ClosedException
   4. contains()
  1. returns true if the graph contains the specified triple.
 1. Node.ANY will match any node in the position.
  2. if the graph supports transactions and a transaction is in
  progress the graph will only not show any triples that only exist within
  the transaction.
   5. delete()
  1. if delete is not allowed (Capabilities.canDelete() is false)
  delete() must throw DeleteDeniedException
  2. when a triple is deleted from  a graph all registered listeners
  must receive an (delete graph triple) message
  3. subsequent graph.contains( triple ) must return false.
  4. If add is performed within a transaction the listeners are not
  notified until after the commit.
   6. dependsOn()
  1. true if this graph's content depends on the other graph. May be
  pessimistic (ie return true if it's not sure). Typically true
when a  graph
  is a composition of other graphs, eg union.
   7. find()
  1. returns an iterator of triples that match the specified triple.
   8. getBulkUpdateHandler() -- deprecated / removed -- no tests
   9. getCapabilities()
  1. must not return null.
  2. capabilities must match other results.
 1. if not addAllowed() , add must throw exception
 2. if not deleteAllowed(),
1. delete must throw exception
2. clear must throw exception
 3. if iteratorRemoveAllowed(), iterator from find must allow
 remove()
 4. if canBeEmpty()
1. initial construction must be empty()
2. clear() must be empty.
 3. must pass Capabilities contract tests.
   10. getEventManager()
  1. May not return null
  2. Listeners registered with event manager must be notified of
  changes.
  3. EventManager must pass GraphEventManager contract test.
   11. getPrefixMapping()
  1. May not be null
  2. changes to the prefixes managed by the PrefixMapping returned
   getPrefixMapping() must be reflected in all other PrefixMapping classes
  from the same graph.
  3. Changes made to a prefix mapping within a transaction are visible
  outside of the transaction and are not rolled back by the transaction.
  4. PrefixMapping  must pass the PrefixMapping contract test
   12. getStatisticsHandler()
  1. may be null
  2. if not null must pass the GraphStatisticsHandler contract test.
  3. all GraphStatisticsHandlers returned must pass handler.equals(
  handler2 )
   13. getTransactionHandler()
  1. may not be null
  2. must pass the TransactionHandler contract test.
   14. isClosed()
  1. must return false when the graph is created.
  2. must return true after the close() has been called.
   15. isEmpty()
  1. must return true when graph is created if
  Capabilities.canBeEmpty() is true
  2. must not return true after triples are added
  3. must return true after all triples are deleted if
  Capabilities.canBeEmpty() is true.
  4. must return true after clear() if Capabilities.canBeEmpty() is
  true.
   16. isIsomorphicWith() -- from (
   
http://www.w3.org/TR/2014/REC-rdf11-concepts-20140225/#section-graph-equality):
Two RDF graphs G and G' are isomorphic (that is, they have an identical
   form) if there is a bijection M between the sets of nodes of the two
   graphs, such that:
  1. M maps blank nodes to blank nodes.
  2. M(lit)=lit for all RDF literals lit which are nodes of G.
  3. M(iri)=iri for all IRIs iri which are nodes of G.
  4. The triple ( s, p, o ) is in G if and only if the triple ( M(s),
  p, M(o) ) is in G'
   17. remove()
  1. when a triple is r

Re: Graph SPI Contract (Transactions)

[Claude Warren]

On Mon, Aug 11, 2014 at 6:19 PM, Andy Seaborne  wrote:

>
>
> Transactions:
>
> The text around transactions does not distinguish being inside or outside
> a transaction.
>
> There are 2 base kinds of graphs - ones in datasets (views) and standalone
> ones, then things like InfGraph and other added functionality. Transactions
> on view graphs need to be defined in the context of the dataset because
> transactions are connected.
>
>
Using databases as a transaction example.  There are multiple types of
transactions -- I don't know if we want to get into supporting or
identifying the type of transaction supported by a graph, but...
In the current case (Jena 2.12.x) whith 2 threads T1 and T2.

T1 begins a write transaction
T1 add to graph

T2 begins a read transaction (is this possible -- I think so)
Can T2 find the triples written by T1 (note that the transaction is not yet
committed)?

T1 commits the transaction
Can T2 now find the triples written by T1?

T2 ends transaction (just for completeness)


Claude
-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Graph SPI Contract (Transactions)

[Claude Warren]

I just realized that we don't have any distinction between read and write
transactions at the graph level.  Perhaps we should add that in V3.

For V2 the transaction must imply a write lock and my example above can not
be done at the graph level.

It can be executed at a level that provides a view type graph.

So do view level graphs support transactions?

if so does the TransactionHandler.begin() start a write transaction?







On Tue, Aug 12, 2014 at 9:18 AM, Andy Seaborne  wrote:

> On 12/08/14 08:58, Claude Warren wrote:
>
>> On Mon, Aug 11, 2014 at 6:19 PM, Andy Seaborne  wrote:
>>
>>
>>>
>>> Transactions:
>>>
>>> The text around transactions does not distinguish being inside or outside
>>> a transaction.
>>>
>>> There are 2 base kinds of graphs - ones in datasets (views) and
>>> standalone
>>> ones, then things like InfGraph and other added functionality.
>>> Transactions
>>> on view graphs need to be defined in the context of the dataset because
>>> transactions are connected.
>>>
>>>
>>>
> The point here is that several graphs may be in one transaction but other
> graphs (other datasets) may not.
>
>
>  Using databases as a transaction example.  There are multiple types of
>> transactions -- I don't know if we want to get into supporting or
>> identifying the type of transaction supported by a graph, but...
>> In the current case (Jena 2.12.x) whith 2 threads T1 and T2.
>>
>> T1 begins a write transaction
>> T1 add to graph
>>
>> T2 begins a read transaction (is this possible -- I think so)
>> Can T2 find the triples written by T1 (note that the transaction is not
>> yet
>> committed)?
>>
>
> No
>
>
>  T1 commits the transaction
>> Can T2 now find the triples written by T1?
>>
>
> No
>
> The isolation level in TDB transactions is serializable, not read
> committed.  TDB transaction are not lock based nor do they ever cause
> system aborts due to unresolvable contention [*].
>
> T2's view of the database is all commits up until then (so not T1) and
> does not change.  That would be read committed.
>
>
>
>> T2 ends transaction (just for completeness)
>>
>
> And if T3 starts, it sees T1 updates and T2 still does not. see them.
>
> Given the nature of storing triples/quads and not entities (meaningful
> rows in the data abstraction which is what SQL tends to do), the lower
> isolation levels can have rather strange effects.  And phantom reads are
> horrendous.
>
> Andy
>
> PS 4.2 does not differentiate between inside and outside the transaction
> so the isolation level is not relevant.
>
> [*] That would change if we have begin() with no read/write indicator at
> the point of transaction promotion from read to write, an abort would be
> possible if the DB is inconsistent with that happening.
>
>
>>
>> Claude
>>
>>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Graph SPI Contract

[Claude Warren]

I think the contract has to cover multi-threaded possibilities.  However,
for the most part the document I originally proposed is the view from
within a single thread.

I agree that graphAdd serves no purpose and go as far as saying it should
be removed in Jena 3.

Think that defining the add with the listener will clarify the contract,
but we need clarification of the Listener contract later.

I think that the current process is:

   1. triple added to or deleted from graph
   2. listeners notified

I think that this is correct but that we need to add that exceptions in the
listeners may not raise and add denied exception.  I believe that the
contract with listeners is:

   1. they are notified after the event they are listening for has been
   completed.  That they are not notified if an Exception is thrown in the add.
   2. if a listener throws an exception it will not undo the add or delete.
   3. I believe that: #1 means that the listeners would be notified at the
   commit of a transaction, so listeners are guaranteed to have messages
   queued by the end of the commit (if present) or at the end of add (if no
   transaction is present).

This does lead to the possibility that a graph implementation may need to
notify other components within the transaction that the add or delete was
completed -- I am not certain that this is needed but raise the point here
for further discussion if necessary.

So the full process for an add is

   1. begin add( triple )
   2. if adding is not allowed (Capabilities.addAllowed() returns false)
   throw AddDeniedException.
   3. add to the underlying storage system, may throw an exception.
  1. If a checked exception is thrown wrap it in an AddDeniedException.
  4. if not in a transaction notify listeners of add
   5. end add(triple)
   6. begin commit if in transaction
   7. commit the change so that it is visible to outside of the transaction.
   8. notify listeners of add.
   9. end commit.

If that it the case then the full process for a delete is

   1. begin delete( triple )
   2. if deleting is not allowed (Capabilities.deleteAllowed() returns
   false) throw DeleteDeniedException.
   3. delete from the underlying storage system, may throw an exception.
  1. If a checked exception is thrown wrap it in a
  DeleteDeniedException.
  4. if not in a transaction notify listeners of delete
   5. end delete(triple)
   6. begin commit if in transaction
   7. commit the change so that it is visible to outside of the transaction.
   8. notify listeners of delete.
   9. end commit.

As for the find process

   1. returns an ExtendedIterator of triples that match the specified
   triple.
   2. If inside a transaction all uncommited triples are candidates for
   matching.

The iterator may throw a ConcurrentModificationException in conditions
outlined by
http://docs.oracle.com/javase/7/docs/api/java/util/ConcurrentModificationException.html
with the following caveat:

   - If the find is taking place within a transaction and the current
   thread has not modified the underlying data the
   ConcurrentModificationException may not be thrown.


Thoughs?
Claude





On Mon, Aug 11, 2014 at 6:19 PM, Andy Seaborne  wrote:

> On 08/08/14 22:13, Claude Warren wrote:
>
>> This is a message stack for Graph SPI Contract testing.  It covers only
>> the
>> Jena 2 Graph Contract.  This an attempt to document the current Graph
>> contract.  Any correction should specify the bullet point number.
>>
>
> Overall:
>
> Getting the exact contract is hard and I'm assuming this is only for
> single-threaded code.
>
> Maybe start with a subset of Graph
>
> .add
> .delete
> .find
>
> then add listeners into the picture
> then define other operations in terms of the primitives:
>
> .contains
> .remove
> .clear
>
> Transactions:
>
> The text around transactions does not distinguish being inside or outside
> a transaction.
>
> There are 2 base kinds of graphs - ones in datasets (views) and standalone
> ones, then things like InfGraph and other added functionality. Transactions
> on view graphs need to be defined in the context of the dataset because
> transactions are connected.
>
>
>  1. add() -- technically from GraphAdd
>>
>
> IMO The "GraphAdd" interface serves no purpose.
>
> 1. when a triple is added to a graph all registered listeners must
>>
>>receive an (add graph triple) message
>>
>
> It's hard to define listeners:
>
>   Does a listener see the graph before or after the triple is added?
>   Is a listener called if AddDeniedException is raised?
>   Can a listener cause AddDeniedException to be raised?
>   Is the listener guaranted to have been called by the
> time add() returns?
>
> hence the suggestion of starting with just the basic ope

Re: Graph SPI Contract

[Claude Warren]

Are there any more thoughts on this discussion?  If not then I am going to
assume this is basically correct and document it on the wiki, build a
contract test for this bit, and start on the bits of the Graph interface we
have not agreed on yet.

Claude



On Sun, Aug 17, 2014 at 11:43 AM, Claude Warren  wrote:

> I think the contract has to cover multi-threaded possibilities.  However,
> for the most part the document I originally proposed is the view from
> within a single thread.
>
> I agree that graphAdd serves no purpose and go as far as saying it should
> be removed in Jena 3.
>
> Think that defining the add with the listener will clarify the contract,
> but we need clarification of the Listener contract later.
>
> I think that the current process is:
>
>1. triple added to or deleted from graph
>2. listeners notified
>
> I think that this is correct but that we need to add that exceptions in
> the listeners may not raise and add denied exception.  I believe that the
> contract with listeners is:
>
>1. they are notified after the event they are listening for has been
>completed.  That they are not notified if an Exception is thrown in the 
> add.
>2. if a listener throws an exception it will not undo the add or
>delete.
>3. I believe that: #1 means that the listeners would be notified at
>the commit of a transaction, so listeners are guaranteed to have messages
>queued by the end of the commit (if present) or at the end of add (if no
>transaction is present).
>
> This does lead to the possibility that a graph implementation may need to
> notify other components within the transaction that the add or delete was
> completed -- I am not certain that this is needed but raise the point here
> for further discussion if necessary.
>
> So the full process for an add is
>
>1. begin add( triple )
>2. if adding is not allowed (Capabilities.addAllowed() returns false)
>throw AddDeniedException.
>3. add to the underlying storage system, may throw an exception.
>   1. If a checked exception is thrown wrap it in an
>   AddDeniedException.
>   4. if not in a transaction notify listeners of add
>5. end add(triple)
>6. begin commit if in transaction
>7. commit the change so that it is visible to outside of the
>transaction.
>8. notify listeners of add.
>9. end commit.
>
> If that it the case then the full process for a delete is
>
>1. begin delete( triple )
>2. if deleting is not allowed (Capabilities.deleteAllowed() returns
>false) throw DeleteDeniedException.
>3. delete from the underlying storage system, may throw an exception.
>   1. If a checked exception is thrown wrap it in a
>   DeleteDeniedException.
>   4. if not in a transaction notify listeners of delete
>5. end delete(triple)
>6. begin commit if in transaction
>7. commit the change so that it is visible to outside of the
>transaction.
>8. notify listeners of delete.
>9. end commit.
>
> As for the find process
>
>1. returns an ExtendedIterator of triples that match the specified
>triple.
>2. If inside a transaction all uncommited triples are candidates for
>matching.
>
> The iterator may throw a ConcurrentModificationException in conditions
> outlined by
> http://docs.oracle.com/javase/7/docs/api/java/util/ConcurrentModificationException.html
> with the following caveat:
>
>- If the find is taking place within a transaction and the current
>thread has not modified the underlying data the
>ConcurrentModificationException may not be thrown.
>
>
> Thoughs?
> Claude
>
>
>
>
>
> On Mon, Aug 11, 2014 at 6:19 PM, Andy Seaborne  wrote:
>
>> On 08/08/14 22:13, Claude Warren wrote:
>>
>>> This is a message stack for Graph SPI Contract testing.  It covers only
>>> the
>>> Jena 2 Graph Contract.  This an attempt to document the current Graph
>>> contract.  Any correction should specify the bullet point number.
>>>
>>
>> Overall:
>>
>> Getting the exact contract is hard and I'm assuming this is only for
>> single-threaded code.
>>
>> Maybe start with a subset of Graph
>>
>> .add
>> .delete
>> .find
>>
>> then add listeners into the picture
>> then define other operations in terms of the primitives:
>>
>> .contains
>> .remove
>> .clear
>>
>> Transactions:
>>
>> The text around transactions does not distinguish being inside or outside
>> a transaction.
>>
>> There are 2 base kinds of graphs - ones in datasets (vi

Re: Graph SPI Contract

[Claude Warren]

Andy,

 I think we agree on transactions.

I think the difference is in the understanding of when listeners are
triggered.

I realize that all current implementations of listeners appear to be on a
single thread.  But I did not realize that was a requirement of the
listener interface.  (yes that would be part of the listener contract
test).  I would think that a listener could place messages on a queue and
that would be sufficient to meet the listener interface -- but that would
mean the state within a transaction would be visible outside of the
transaction.

I have wanted the ability for one thread to be notified when another thread
completed a transaction -- basically when the changes became visible.  But
for now that appears to be outside the scope of a listener.

As listeners are same thread callbacks, does this mean that when a
transaction is rolled back the listeners must be notified to undo the
previous notifications -- for example

begin Tx
add T1
listener notified of add T1
rollback Tx
listener notified of delete T1

If not then we need to document this case.

To be honest it is this issue that made me think that listeners should be
notified after the transaction committed.  If listeners are notified after
commit then they can also be on different threads.

Claude





On Sat, Aug 23, 2014 at 4:22 PM, Andy Seaborne  wrote:

> Claude,
>
> We seem to have different understandings about transactions.
>
> I see a transaction (as in ACID) as defining a scope or view of the system
> (or valid state of the system). Within a transaction changes happen only
> from actions of the transaction, not outside.
>
> A transaction sees a consistent state of the world - transactions are
> serialized onto the time line and appear to happen instantaneously as a
> single unit.  From outside, nothing changes until all of a sudden all
> changes are made at once.  These are "serializable" isolation which is the
> ideal.
>
> Weaker forms of isolation exist but they have unpredictable effects. For
> us, a find() is a range query so even isolation level "repeatable reads"
> can cause a find to see a state of the storage that never existed in any
> application view point.
>
> http://en.wikipedia.org/wiki/Isolation_%28database_systems%29
>
> We aren't considering nested transactions.
>
> add() just needs to define what add() does inside a transaction.  At some
> later time, all the change become visible.
>
> == add()
>
> Pre condition:
>   graph exists
>
> action:
>   add(t)
>
> Post condition:
> in the view of the transaction for the current thread:
>   if no exception
> graph contains t
>   else if AddDeniedException
> graph contains t if and only if the graph contained t before.
>
> Listeners are same-thread callbacks so they are in the same transaction as
> the update.  Complex systems on top of this are out of scope. Jena provides
> the building block.
>
>
> On 17/08/14 11:43, Claude Warren wrote:
>
>> I think the contract has to cover multi-threaded possibilities.  However,
>> for the most part the document I originally proposed is the view from
>> within a single thread.
>>
>
>
> For non-transactional, multi-threaded systems, I don't think anything
> needs to said except "don't!!" - or rather "single view" or else all bets
> are off.  Failure modes are way too implementation specific - even across
> JVMs (see IBM vs oracle JVMs for HashMap as we have know here).
>
> Jena in-memory is read-concurrent safe.
> http://jena.apache.org/documentation/notes/concurrency-howto.html
>
> Even that is non-trivial to provide in the inference engine.
>
>
>  I agree that graphAdd serves no purpose and go as far as saying it should
>> be removed in Jena 3.
>>
>
> Yes.
>
>  Think that defining the add with the listener will clarify the contract,
>> but we need clarification of the Listener contract later.
>>
>> I think that the current process is:
>>
>> 1. triple added to or deleted from graph
>> 2. listeners notified
>>
>>
>> I think that this is correct but that we need to add that exceptions in
>> the
>> listeners may not raise and add denied exception.
>>
>
> s/and/an/ ?  If so - yes.
>
> How about:
>
> 1. listeners should not raise exceptions.
> 2. If they do (outside the contract), the exception should be (logged and)
> dropped.
>
> It seems odd to me to have an exception and the triple be added.
>
>   I believe that the
>> contract with listeners is:
>>
>> 1. they are notified after the event they are listening for has been
>>
>> completed.  That they are not notified if an Exceptio

Re: Graph SPI Contract

[Claude Warren]

I added information on
https://cwiki.apache.org/confluence/display/JENA/SPI+Contract+Details to
cover this discussion.  Please review and comment.



On Sun, Aug 24, 2014 at 2:19 PM, Andy Seaborne  wrote:

> On 23/08/14 18:07, Claude Warren wrote:
>
>> Andy,
>>
>>   I think we agree on transactions.
>>
>> I think the difference is in the understanding of when listeners are
>> triggered.
>>
>> I realize that all current implementations of listeners appear to be on a
>> single thread.  But I did not realize that was a requirement of the
>> listener interface.  (yes that would be part of the listener contract
>> test).  I would think that a listener could place messages on a queue and
>> that would be sufficient to meet the listener interface -- but that would
>> mean the state within a transaction would be visible outside of the
>> transaction.
>>
>
> Fine - that is building a higher-level mechanism on top of the basic
> callback system.
>
> The fact you can leak information from inside a transaction is nothing
> special here.  You can do it normally just by missing bytes across
> threads.  Don't!  or at least, do with care.
>
> You can in JDBC as well.  Use multiple connections. Or store in a
> variable, abort the transaction and start again.
>
>
>  I have wanted the ability for one thread to be notified when another
>> thread
>> completed a transaction -- basically when the changes became visible.  But
>> for now that appears to be outside the scope of a listener.
>>
>
> It's certainly outside the scope of an add() - transactions are groups of
> add/delete.
>
> You want transaction monitoring - callbacks on transaction operations.
>
>
>  As listeners are same thread callbacks, does this mean that when a
>> transaction is rolled back the listeners must be notified to undo the
>> previous notifications -- for example
>>
>> begin Tx
>> add T1
>> listener notified of add T1
>> rollback Tx
>> listener notified of delete T1
>>
>
> I'd say "no".  T1 is not deleted (there is no delete() call).
>
> You have passed information out of a transaction.
>
>
>  If not then we need to document this case.
>>
>> To be honest it is this issue that made me think that listeners should be
>> notified after the transaction committed.  If listeners are notified after
>> commit then they can also be on different threads.
>>
>
> That is a higher level abstraction - we should be able to support writing
> that but not provide it as the one design. There are many different designs
> which is why listeners were made the basic building block on which
> applications can do what they want.
>
> Andy
>
>
>> Claude
>>
>>
>>
>>
>>
>> On Sat, Aug 23, 2014 at 4:22 PM, Andy Seaborne  wrote:
>>
>>  Claude,
>>>
>>> We seem to have different understandings about transactions.
>>>
>>> I see a transaction (as in ACID) as defining a scope or view of the
>>> system
>>> (or valid state of the system). Within a transaction changes happen only
>>> from actions of the transaction, not outside.
>>>
>>> A transaction sees a consistent state of the world - transactions are
>>> serialized onto the time line and appear to happen instantaneously as a
>>> single unit.  From outside, nothing changes until all of a sudden all
>>> changes are made at once.  These are "serializable" isolation which is
>>> the
>>> ideal.
>>>
>>> Weaker forms of isolation exist but they have unpredictable effects. For
>>> us, a find() is a range query so even isolation level "repeatable reads"
>>> can cause a find to see a state of the storage that never existed in any
>>> application view point.
>>>
>>> http://en.wikipedia.org/wiki/Isolation_%28database_systems%29
>>>
>>> We aren't considering nested transactions.
>>>
>>> add() just needs to define what add() does inside a transaction.  At some
>>> later time, all the change become visible.
>>>
>>> == add()
>>>
>>> Pre condition:
>>>graph exists
>>>
>>> action:
>>>add(t)
>>>
>>> Post condition:
>>> in the view of the transaction for the current thread:
>>>if no exception
>>>  graph contains t
>>>else if AddDeniedException
>>>  graph contains t if and only if the graph contained t before.
>>>
>>> Listeners 

prepared statements in Jena

[Claude Warren]

I recall a discussion of prepared statements in Jena awhile back.  Now my
google search has turned up using bind to do something like a prepared
statement, but for some reason I thought there was a proposal to do real
prepared statement with replacable parameters.  Somthing that would work
with service calls as the parameters would be replaced before the statement
executed.

Does anyone else remember such a conversation?  Does anyone know of such a
solution?

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

idea: streaming union

[Claude Warren]

I have had several cases where I have unions of service calls.  since the
service calls take time and since the operation is not sensitive to the
order of the items being unioned I was thinking of implementing a union
operation for service endpoints such that both endpoints would be opened
and the results fed into the result stream as they are returned.  This
wiould involve a polling mechanism on the connections to see if there was a
result available.  First available data element is added to the result and
the polling continues.

pseudo code something like

create a list of connections.
loop:
 poll each connection
if connection finished close
if connection has data add to result
 if there are connections still open go to loop;

I think that this would improve performance when used with multiple service
calls.  It may also improve performance across slower storeage devices.

Thoughts?

Claude


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: prepared statements in Jena

[Claude Warren]

I have a QueryBuilder that currently builds a query string.  Andy had asked
if I could make it build the query object so that we didn't have to parse
the string.  One of the features of the current method is that it allows
you to use the String.format() parameters and replace them before calling
the parser.  I am looking for a way to do that while building the query
objects.  I had thought about allowing replacement of the variables with
concrete values similar to what the parameterized-sparql-strings does.




On Tue, Sep 2, 2014 at 1:27 PM, Rob Vesse  wrote:

> Claude
>
> There is the ParameterizedSparqlString functionality which is essentially
> just textual parameterization -
> https://jena.apache.org/documentation/query/parameterized-sparql-strings.ht
> ml - so it is somewhat limited but that has existed in ARQ for a year or
> two now.
>
> There was some talk about doing the parameterisation at the query/algebra
> level.  Certainly some of the plumbing that supports parameterisation at
> the algebra level already exists because it is used by optimisations
>
> Maybe you could elaborate more on what you intend?
>
> Rob
>
> On 02/09/2014 11:10, "Claude Warren"  wrote:
>
> >I recall a discussion of prepared statements in Jena awhile back.  Now my
> >google search has turned up using bind to do something like a prepared
> >statement, but for some reason I thought there was a proposal to do real
> >prepared statement with replacable parameters.  Somthing that would work
> >with service calls as the parameters would be replaced before the
> >statement
> >executed.
> >
> >Does anyone else remember such a conversation?  Does anyone know of such a
> >solution?
> >
> >Claude
> >
> >--
> >I like: Like Like - The likeliest place on the web
> ><http://like-like.xenei.com>
> >LinkedIn: http://www.linkedin.com/in/claudewarren
>
>
>
>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: prepared statements in Jena

[Claude Warren]

You misunderstood me.  I was saying the current implementation supports
String.format() type replacable parameters.  I was just looking for a way
to do replacable parameters inside a querybuilder.


On Tue, Sep 2, 2014 at 2:58 PM, Rob Vesse  wrote:

> I'm not sure that using String.format() style parameters makes sense
> because SPARQL expects values to be in certain restricted formats and
> values not matching those formats would be invalid
>
> For example consider the format specifier %,d I.e integer with thousands
> separator.  Values printed with that format into a SPARQL string would not
> always be valid integers according to SPARQL and depending on where they
> were inserted would not necessarily be immediate syntax errors either but
> might only manifest as unexpected runtime behaviour
>
> Rob
>
> On 02/09/2014 13:51, "Claude Warren"  wrote:
>
> >I have a QueryBuilder that currently builds a query string.  Andy had
> >asked
> >if I could make it build the query object so that we didn't have to parse
> >the string.  One of the features of the current method is that it allows
> >you to use the String.format() parameters and replace them before calling
> >the parser.  I am looking for a way to do that while building the query
> >objects.  I had thought about allowing replacement of the variables with
> >concrete values similar to what the parameterized-sparql-strings does.
> >
> >
> >
> >
> >On Tue, Sep 2, 2014 at 1:27 PM, Rob Vesse  wrote:
> >
> >> Claude
> >>
> >> There is the ParameterizedSparqlString functionality which is
> >>essentially
> >> just textual parameterization -
> >>
> >>https://jena.apache.org/documentation/query/parameterized-sparql-strings
> .
> >>ht
> >> ml - so it is somewhat limited but that has existed in ARQ for a year or
> >> two now.
> >>
> >> There was some talk about doing the parameterisation at the
> >>query/algebra
> >> level.  Certainly some of the plumbing that supports parameterisation at
> >> the algebra level already exists because it is used by optimisations
> >>
> >> Maybe you could elaborate more on what you intend?
> >>
> >> Rob
> >>
> >> On 02/09/2014 11:10, "Claude Warren"  wrote:
> >>
> >> >I recall a discussion of prepared statements in Jena awhile back.  Now
> >>my
> >> >google search has turned up using bind to do something like a prepared
> >> >statement, but for some reason I thought there was a proposal to do
> >>real
> >> >prepared statement with replacable parameters.  Somthing that would
> >>work
> >> >with service calls as the parameters would be replaced before the
> >> >statement
> >> >executed.
> >> >
> >> >Does anyone else remember such a conversation?  Does anyone know of
> >>such a
> >> >solution?
> >> >
> >> >Claude
> >> >
> >> >--
> >> >I like: Like Like - The likeliest place on the web
> >> ><http://like-like.xenei.com>
> >> >LinkedIn: http://www.linkedin.com/in/claudewarren
> >>
> >>
> >>
> >>
> >>
> >
> >
> >--
> >I like: Like Like - The likeliest place on the web
> ><http://like-like.xenei.com>
> >LinkedIn: http://www.linkedin.com/in/claudewarren
>
>
>
>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Jena3 - proposal

[Claude Warren]

Sounds good to me.

I would like to see Jena 3 added to Jira so we can begin to log requested
features against it (e.g. remove the GraphAdd interface)




On Wed, Sep 3, 2014 at 11:47 AM, Andy Seaborne  wrote:

> I had a go at refactoring to have both org.apache.jena and com.hp.hpl.jena
> APIs (my target was the RDF and ontology APIs).  It looks possible but
> isn't trivial.  Some classes link the API to the SPI and that then gets a
> bit messy, which equates to it's not a perfect drop in replacement.
>
> But I also got to ask myself about whether this really is an advantage to
> users and the project.
>
> A/ If there is a legacy, when will it go away?
>
> B/ If there are multiple classes called "Model", it can be quite confusing
> for people.  Autocomplete is already polluted enough.
>
> C/ There isn't one system to use because using Jena involves choosing
> different setups via jars included, which in itself may cause a steady
> stream of questions.
>
> And with changes like RDF 1.1 persistent data, there are changes anyway
> that a major version change reflects.
>
> So just making the package rename once and for all seems to me to be just
> as helpful.  It's a bump at the point of jena2 -> jena3 but the bump is
> going to happen if the legacy adapter ever goes away.
>
> A quick migration route for user code is to convert all the imports:
>
> s/import \s+com.hp.hpl.jena/import org.apache.jena/g
>
> in your favorite editor (including Eclipse).
>
> Obviously, some corner cases, like full names for classes, do not work but
> I think they will be few and far between.
>
> Whatever we do, we end up with parallel codebases to apply bug fixes to.
> The only practical way round that that I can see is to delay making the
> package rename as late as possible.  That does not seem very attractive.
>
> Data namespaces are untouched.  We can consider them separately, on a
> case-by-case basis.
>
> So ...
>
> 1/ Move the code to git
> 2/ Repackage com.hp.hpl.jena -> org.apache.jena
> 3/ Declare jena-3.0-alpha-1 (not a release - just POM version set.).
> 4/ Starting making changes
>(like RDF 1.1 details - invalidates persistent data).
> ...
> X/ Release jena-3.0.0 (when we feel it is ready).
>
> Andy
>



-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

TripleMatch: jena3 removal candidate?

[Claude Warren]

It seems to me that TripleMatch falls in the same category as GraphAdd.  An
interface for no defined purpose.  I propose it as a candidate for removal
in Jena3.  Does anyone have an objection?  If not I'll open the Jira ticket
for it.

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

SDB viability

[Claude Warren]

Greetings,

I am designing an application and I want to use some semantic technologies
to solve some tricky queries.  However, the environment I am in does not
support storing directly to disk (you have to use one of their services) so
TDB can not be used.  I know that we no longer do any work on SDB but does
anyone know if it still functions with the current code 2.x codebase?

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Query Builder

[Claude Warren]

Greetings,

I have created a QueryBuilder that builds a query without constructing the
text version first.  The classes (AskBuilder, ConstructBuilder, and
SelectBuilder) allows you to construct the query by adding Construct,
Dataset, Prolog Select SolutionModifer and Where clauses.

In addition there is some templating in that variables anywhere in the
query structure can be replaced by calling setVar( var, node ) before
calling the build() method.

The code is currently hosted at https://github.com/Claudenw/querybuilder

I have not completed the setVar() tests yet, but I think that the code is
stable enough for people to play with.

Claude

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Query Builder

[Claude Warren]

I started looking at it this AM.  I think the proper solution is to accept
"*" as a var in the selecthandler.  I will look at it over the weekend and
will add documentation (javadoc at least)



On Fri, Sep 19, 2014 at 10:14 AM, Andy Seaborne  wrote:

> On 19/09/14 05:18, Bruno P. Kinoshita wrote:
>
>> Hello Claude,
>>
>> I didn't understand what QueryBuilder was supposed to do at first, or how
>> to use it. Luckily there are tests in the project, kudos for writing those,
>> very helpful. I liked the idea, and for users familiar with Java Jooq, PHP
>> and Ruby ActiveRecord that's definitely an intuitive API.
>>
>> In order to test it, I first created some dummy data.
>>
>> PREFIX foaf:  
>> PREFIX :  
>>
>> INSERT DATA
>> {
>> :bruno foaf:name "Bruno" .
>> :jorge foaf:name "Jorge" .
>> :bruno :brotherOf :jorge
>> }
>>
>> Then I retrieved the data with a simple SELECT.
>>
>> PREFIX foaf:  
>> PREFIX :  
>>
>> SELECT *
>> {
>> ?a ?b ?c
>> }
>>
>> -
>> | a  | b  | c   |
>> =
>> | :bruno | foaf:name  | "Bruno" |
>> | :bruno | :brotherOf | :jorge  |
>> | :jorge | foaf:name  | "Jorge" |
>> -
>>
>> I tried to recreate the same query with QueryBuilder, but alas it didn't
>> work. I have probably made something really stupid [1]. My output is always:
>>
>> PREFIX  : 
>> PREFIX  foaf: 
>>
>> WHERE
>>{ ?a  ?b  ?c . }
>>
>> I just wanted to validate that I could recreate the same query with
>> QueryBuilder. But I couldn't figure how that works. Looking at
>> WhereClauseTest, I thought that addVar("*") would result in "SELECT *". The
>> prefixes is not keeping the order it was added, thus that's probably not an
>> issue.
>>
>> The SelectBuilder throws ParseException, which extends Exception. I think
>> **if** the Query Builder is supposed to be used by programmers for writing
>> their Web applications or similar, then perhaps it could extend a
>> RuntimeException?
>>
>> Just my 0.002 cents
>>
>> Thanks
>> Bruno
>>
>> [1] https://gist.github.com/kinow/b50a5d3b875f2155b7bb
>>
>
> Ditto.
>
> (I have just pushed a fix to Wherehandler.testTriple.)
>
> Andy
>
>
>


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Query Builder

[Claude Warren]

I have added license and javadoc comments as well as a readme.md  In
addition the above mentioned  bug is now fixed and addVar( "*" ) works as
expected.

On Fri, Sep 19, 2014 at 1:24 PM, Claude Warren  wrote:

> I started looking at it this AM.  I think the proper solution is to accept
> "*" as a var in the selecthandler.  I will look at it over the weekend and
> will add documentation (javadoc at least)
>
>
>
> On Fri, Sep 19, 2014 at 10:14 AM, Andy Seaborne  wrote:
>
>> On 19/09/14 05:18, Bruno P. Kinoshita wrote:
>>
>>> Hello Claude,
>>>
>>> I didn't understand what QueryBuilder was supposed to do at first, or
>>> how to use it. Luckily there are tests in the project, kudos for writing
>>> those, very helpful. I liked the idea, and for users familiar with Java
>>> Jooq, PHP and Ruby ActiveRecord that's definitely an intuitive API.
>>>
>>> In order to test it, I first created some dummy data.
>>>
>>> PREFIX foaf:  <http://xmlns.com/foaf/0.1/>
>>> PREFIX :  <http://example.org/>
>>>
>>> INSERT DATA
>>> {
>>> :bruno foaf:name "Bruno" .
>>> :jorge foaf:name "Jorge" .
>>> :bruno :brotherOf :jorge
>>> }
>>>
>>> Then I retrieved the data with a simple SELECT.
>>>
>>> PREFIX foaf:  <http://xmlns.com/foaf/0.1/>
>>> PREFIX :  <http://example.org/>
>>>
>>> SELECT *
>>> {
>>> ?a ?b ?c
>>> }
>>>
>>> -
>>> | a  | b  | c   |
>>> =
>>> | :bruno | foaf:name  | "Bruno" |
>>> | :bruno | :brotherOf | :jorge  |
>>> | :jorge | foaf:name  | "Jorge" |
>>> -
>>>
>>> I tried to recreate the same query with QueryBuilder, but alas it didn't
>>> work. I have probably made something really stupid [1]. My output is always:
>>>
>>> PREFIX  : <http://example.org/>
>>> PREFIX  foaf: <http://xmlns.com/foaf/0.1/>
>>>
>>> WHERE
>>>{ ?a  ?b  ?c . }
>>>
>>> I just wanted to validate that I could recreate the same query with
>>> QueryBuilder. But I couldn't figure how that works. Looking at
>>> WhereClauseTest, I thought that addVar("*") would result in "SELECT *". The
>>> prefixes is not keeping the order it was added, thus that's probably not an
>>> issue.
>>>
>>> The SelectBuilder throws ParseException, which extends Exception. I
>>> think **if** the Query Builder is supposed to be used by programmers for
>>> writing their Web applications or similar, then perhaps it could extend a
>>> RuntimeException?
>>>
>>> Just my 0.002 cents
>>>
>>> Thanks
>>> Bruno
>>>
>>> [1] https://gist.github.com/kinow/b50a5d3b875f2155b7bb
>>>
>>
>> Ditto.
>>
>> (I have just pushed a fix to Wherehandler.testTriple.)
>>
>> Andy
>>
>>
>>
>
>
> --
> I like: Like Like - The likeliest place on the web
> <http://like-like.xenei.com>
> LinkedIn: http://www.linkedin.com/in/claudewarren
>



-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Query Builder

[Claude Warren]

Andy,

1 and 2 implemented.
3 was another issue with the testTriple() and has been fixed.

Tests were added.

On Sat, Sep 20, 2014 at 6:50 PM, Andy Seaborne  wrote:

> Hi Claude,
>
> 1/ Suggestion
>if there is no addVar then the query defaults to SELECT *.
>
> 2/ Suggestion
>Strip any trailing ":" from the prefix name so
>   .addPrefix(":", "http://example/";)
>   works.
>
> 3/  I tried:
>
> SelectBuilder sb = new SelectBuilder() ;
> sb.addVar("*")
> .addPrefix("", "http://example/";)
> .addWhere(":S", "?p", "?o") ;
> Query query = sb.build() ;
> System.out.println(query) ;
>
> and got:
> -
> Exception in thread "main" java.lang.IllegalArgumentException: Predicate
> (?p) must be a URI , variable, or a wildcard.
> Is a prefix missing?  Prefix must be defined before use.
>
> org.apache.jena.arq.querybuilder.handlers.WhereHandler.testTriple(
> WhereHandler.java:124)
> org.apache.jena.arq.querybuilder.handlers.WhereHandler.addWhere(
> WhereHandler.java:129)
> org.apache.jena.arq.querybuilder.SelectBuilder.
> addWhere(SelectBuilder.java:194)
> org.apache.jena.arq.querybuilder.SelectBuilder.
> addWhere(SelectBuilder.java:206)
> dev.QBuild.main(QBuild.java:34)
> -
> while
> ("?s", "?p", "?o")  works.
> Andy
>
>
> On 19/09/14 22:51, Claude Warren wrote:
>
>> I have added license and javadoc comments as well as a readme.md  In
>> addition the above mentioned  bug is now fixed and addVar( "*" ) works as
>> expected.
>>
>> On Fri, Sep 19, 2014 at 1:24 PM, Claude Warren  wrote:
>>
>>  I started looking at it this AM.  I think the proper solution is to
>>> accept
>>> "*" as a var in the selecthandler.  I will look at it over the weekend
>>> and
>>> will add documentation (javadoc at least)
>>>
>>>
>>>
>>> On Fri, Sep 19, 2014 at 10:14 AM, Andy Seaborne  wrote:
>>>
>>>  On 19/09/14 05:18, Bruno P. Kinoshita wrote:
>>>>
>>>>  Hello Claude,
>>>>>
>>>>> I didn't understand what QueryBuilder was supposed to do at first, or
>>>>> how to use it. Luckily there are tests in the project, kudos for
>>>>> writing
>>>>> those, very helpful. I liked the idea, and for users familiar with Java
>>>>> Jooq, PHP and Ruby ActiveRecord that's definitely an intuitive API.
>>>>>
>>>>> In order to test it, I first created some dummy data.
>>>>>
>>>>> PREFIX foaf:  <http://xmlns.com/foaf/0.1/>
>>>>> PREFIX :  <http://example.org/>
>>>>>
>>>>> INSERT DATA
>>>>> {
>>>>> :bruno foaf:name "Bruno" .
>>>>> :jorge foaf:name "Jorge" .
>>>>> :bruno :brotherOf :jorge
>>>>> }
>>>>>
>>>>> Then I retrieved the data with a simple SELECT.
>>>>>
>>>>> PREFIX foaf:  <http://xmlns.com/foaf/0.1/>
>>>>> PREFIX :  <http://example.org/>
>>>>>
>>>>> SELECT *
>>>>> {
>>>>> ?a ?b ?c
>>>>> }
>>>>>
>>>>> -
>>>>> | a  | b  | c   |
>>>>> =
>>>>> | :bruno | foaf:name  | "Bruno" |
>>>>> | :bruno | :brotherOf | :jorge  |
>>>>> | :jorge | foaf:name  | "Jorge" |
>>>>> -
>>>>>
>>>>> I tried to recreate the same query with QueryBuilder, but alas it
>>>>> didn't
>>>>> work. I have probably made something really stupid [1]. My output is
>>>>> always:
>>>>>
>>>>> PREFIX  : <http://example.org/>
>>>>> PREFIX  foaf: <http://xmlns.com/foaf/0.1/>
>>>>>
>>>>> WHERE
>>>>> { ?a  ?b  ?c . }
>>>>>
>>>>> I just wanted to validate that I could recreate the same query with
>>>>> QueryBuilder. But I couldn't figure how that works. Looking at
>>>>> WhereClauseTest, I thought that addVar("*") would result in "SELECT
>>>>> *". The
>>>>> prefixes is not keeping the order it was added, thus that's probably
>>>>> not an
>>>>> issue.
>>>>>
>>>>> The SelectBuilder throws ParseException, which extends Exception. I
>>>>> think **if** the Query Builder is supposed to be used by programmers
>>>>> for
>>>>> writing their Web applications or similar, then perhaps it could
>>>>> extend a
>>>>> RuntimeException?
>>>>>
>>>>> Just my 0.002 cents
>>>>>
>>>>> Thanks
>>>>> Bruno
>>>>>
>>>>> [1] https://gist.github.com/kinow/b50a5d3b875f2155b7bb
>>>>>
>>>>>
>>>> Ditto.
>>>>
>>>> (I have just pushed a fix to Wherehandler.testTriple.)
>>>>
>>>>  Andy
>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>> I like: Like Like - The likeliest place on the web
>>> <http://like-like.xenei.com>
>>> LinkedIn: http://www.linkedin.com/in/claudewarren
>>>
>>>
>>
>>
>>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Query Builder

[Claude Warren]

fixed.

On Sat, Sep 20, 2014 at 10:57 PM, Andy Seaborne  wrote:

> On 20/09/14 21:27, Claude Warren wrote:
>
>> Andy,
>>
>> 1 and 2 implemented.
>>
>
> Off by one error :-)
>
> SelectBuilder sb = new SelectBuilder() ;
> sb.addPrefix("abcd:", "http://example/";) ;
> Query query = sb.build() ;
> System.out.println(query) ;
>
> ==> PREFIX  abc:  <http://example/>
>
> and
>
> .addPrefix(":", "http://example/";) ;
> ==>
> "String index out of range"
>
>
> In
> private String canonicalPfx(String x) {
> if (x.endsWith(":"))
> return x.substring(0, x.length() - 1);
> return x;
> }
>
> -1 (it was -2)
>
> Andy
>
>
>  3 was another issue with the testTriple() and has been fixed.
>>
>> Tests were added.
>>
>> On Sat, Sep 20, 2014 at 6:50 PM, Andy Seaborne  wrote:
>>
>>  Hi Claude,
>>>
>>> 1/ Suggestion
>>> if there is no addVar then the query defaults to SELECT *.
>>>
>>> 2/ Suggestion
>>> Strip any trailing ":" from the prefix name so
>>>.addPrefix(":", "http://example/";)
>>>works.
>>>
>>> 3/  I tried:
>>>
>>>  SelectBuilder sb = new SelectBuilder() ;
>>>  sb.addVar("*")
>>>  .addPrefix("", "http://example/";)
>>>  .addWhere(":S", "?p", "?o") ;
>>>  Query query = sb.build() ;
>>>  System.out.println(query) ;
>>>
>>> and got:
>>> -
>>> Exception in thread "main" java.lang.IllegalArgumentException: Predicate
>>> (?p) must be a URI , variable, or a wildcard.
>>> Is a prefix missing?  Prefix must be defined before use.
>>>
>>> org.apache.jena.arq.querybuilder.handlers.WhereHandler.testTriple(
>>> WhereHandler.java:124)
>>> org.apache.jena.arq.querybuilder.handlers.WhereHandler.addWhere(
>>> WhereHandler.java:129)
>>> org.apache.jena.arq.querybuilder.SelectBuilder.
>>> addWhere(SelectBuilder.java:194)
>>> org.apache.jena.arq.querybuilder.SelectBuilder.
>>> addWhere(SelectBuilder.java:206)
>>> dev.QBuild.main(QBuild.java:34)
>>> -
>>> while
>>> ("?s", "?p", "?o")  works.
>>>  Andy
>>>
>>>
>>> On 19/09/14 22:51, Claude Warren wrote:
>>>
>>>  I have added license and javadoc comments as well as a readme.md  In
>>>> addition the above mentioned  bug is now fixed and addVar( "*" ) works
>>>> as
>>>> expected.
>>>>
>>>> On Fri, Sep 19, 2014 at 1:24 PM, Claude Warren 
>>>> wrote:
>>>>
>>>>   I started looking at it this AM.  I think the proper solution is to
>>>>
>>>>> accept
>>>>> "*" as a var in the selecthandler.  I will look at it over the weekend
>>>>> and
>>>>> will add documentation (javadoc at least)
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Sep 19, 2014 at 10:14 AM, Andy Seaborne 
>>>>> wrote:
>>>>>
>>>>>   On 19/09/14 05:18, Bruno P. Kinoshita wrote:
>>>>>
>>>>>>
>>>>>>   Hello Claude,
>>>>>>
>>>>>>>
>>>>>>> I didn't understand what QueryBuilder was supposed to do at first, or
>>>>>>> how to use it. Luckily there are tests in the project, kudos for
>>>>>>> writing
>>>>>>> those, very helpful. I liked the idea, and for users familiar with
>>>>>>> Java
>>>>>>> Jooq, PHP and Ruby ActiveRecord that's definitely an intuitive API.
>>>>>>>
>>>>>>> In order to test it, I first created some dummy data.
>>>>>>>
>>>>>>> PREFIX foaf:  <http://xmlns.com/foaf/0.1/>
>>>>>>> PREFIX :  <http://example.org/>
>>>>>>>
>>>>>>> INSERT DATA
>>>>>>> {
>>>>>>> :bruno foaf:name "Bruno" .
>>>>>>> :jorge foaf:name "Jorge" .
>>>>>>> :bruno :brotherOf :jorge
>>>>>>> }
>>>

Re: Migration from svn to git

[Claude Warren]

I think I have anything to do as the only downstream job I think I have is
the new tests, they are in experimental and that is still on SVN right?

Claude


On Mon, Sep 22, 2014 at 6:29 PM, Andy Seaborne  wrote:

> On 21/09/14 19:17, Andy Seaborne wrote:
>
>> On 19/09/14 10:52, Andy Seaborne wrote:
>>
>>> The request is:
>>>
>>> https://issues.apache.org/jira/browse/INFRA-8367
>>>
>>
> Things should now work.
>
> Both svn and git are now read-write.
>
> Old svn trunk is still there - just don't use it! - as soon as we're
> confident everything works, we can "svn rm" it to hide it.
>
> I have some changes for the website to refer to git, not svn, and I'm in
> the process of getting the Jenkins jobs to run again (line endings :-( for
> Windows) so there may be some more noise from those.  I have patchy amounts
> of time this week.
>
> If you find anything wrong, drop a note to INFRA-8367.  Geoff has been
> very helpful and responsive ... and this was his first svn to git migration.
>
> (Claude - can I assume you'll take care of the downstream Jenkins jobs you
> have?)
>
> Andy
>
>
>


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: 2.12.1 Release?

[Claude Warren]

I would like to add the querybuilder code.  However that is not critical
and could be released in 2.12.2.  The code tree is all org.apache.jena so
it should merge quickly.  test cases are included.  but there has been
minimal testing.

On Mon, Sep 29, 2014 at 11:56 AM, Rob Vesse  wrote:

> Now we are past the Git migration do people think we are in a position to
> put out a 2.12.1 release?
>
> I'd like to cut a release candidate tomorrow if nobody objects?
>
> Cheers,
>
> Rob
>
> On 19/09/2014 10:20, "Andy Seaborne"  wrote:
>
> >On 18/09/14 15:51, Rob Vesse wrote:
> >> All
> >>
> >> Would people be interested in us putting out a 2.12.1 release in the
> >>next
> >> couple of weeks?
> >>
> >> There are various bug fixes and API changes we would like to have
> >>available
> >> in a stable release and start exercising in our stack ASAP.  What work
> >>is
> >> ongoing that people would like to get done prior to a new release?
> >>
> >> It may be worth waiting to do the release after conducting the move to
> >>git
> >> (since it looks like that vote will pass) so we can iron out any kinks
> >>in
> >> the release process that it might introduce.
> >>
> >> I am happy to volunteer as RM for this release
> >>
> >> Rob
> >
> >Works for me.
> >
> >There will be Fuseki2 in trunk but not in the build.  It just means
> >source-release gets extra stuff in it.
> >
> >   Andy
> >
>
>
>
>
>


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: [] Release Jena 2.12.1 and Fuseki 1.1.1

[Claude Warren]

Looking at the code I can't figure out how any of them pass.  I'll spend
more time on it this evening.

On Wed, Oct 1, 2014 at 11:14 AM, Andy Seaborne  wrote:

> I'm having problems building the source-release with java8.
>
> * jena-security consistently fails with java8 (1.8.0_20)
> * It does build with java7
> * The git tag source builds with java8.
>
> Claude - what causes this test failure?
>
> And is there a reason why the tests can't use an in-memory TDB dataset? It
> will speed things up (make the tests not crash into each other if run in
> parallel).
>
> Andy
>
> (I'll continue with other checking)
>
> java version "1.8.0_20"
> Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
> Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
>
>
> The first failure is:
>
> Running org.apache.jena.security.graph.TDBGraphTest
> Tests run: 384, Failures: 0, Errors: 16, Skipped: 0,
> Time elapsed: 24.311 sec <<< FAILURE!
> - in org.apache.jena.security.graph.TDBGraphTest
> testGetPrefixMapping[C:true R:true U:false D:true
> force:true](org.apache.jena.security.graph.TDBGraphTest)  Time elapsed:
> 0.067 sec  <<< ERROR!
> java.lang.reflect.InvocationTargetException: null
>   at sun.reflect.GeneratedMethodAccessor502.invoke(Unknown Source)
>   at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:483)
>   at org.apache.jena.security.graph.SecuredPrefixMappingTest.runTests(
> SecuredPrefixMappingTest.java:78)
>   at org.apache.jena.security.graph.MemGraphTest.testGetPrefixMapping(
> MemGraphTest.java:268)
>   at sun.reflect.GeneratedMethodAccessor570.invoke(Unknown Source)
>
> ...junit on the stack .
>
> Caused by: java.lang.AssertionError:
> Should have thrown AccessDenied Exception
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.apache.jena.security.graph.SecuredPrefixMappingTest.
> testWithDefaultMappings(SecuredPrefixMappingTest.java:401)
>at sun.reflect.GeneratedMethodAccessor502.invoke(Unknown Source)
>at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>at java.lang.reflect.Method.invoke(Method.java:483)
>at org.apache.jena.security.graph.SecuredPrefixMappingTest.runTests(
> SecuredPrefixMappingTest.java:78)
>at org.apache.jena.security.graph.MemGraphTest.testGetPrefixMapping(
> MemGraphTest.java:268)
>at sun.reflect.GeneratedMethodAccessor570.invoke(Unknown Source)
>
> There are 16 of these errors:
>
> Edited...
>
> testGetPrefixMapping[C:true R:true U:false D:true force:true]
> testGetPrefixMapping[C:true R:true U:false D:true force:false]
> testGetPrefixMapping[C:true R:true U:false D:false force:true]
> testGetPrefixMapping[C:true R:true U:false D:false force:false]
> testGetPrefixMapping[C:true R:false U:false D:true force:true]
> testGetPrefixMapping[C:true R:false U:false D:true force:false]
> testGetPrefixMapping[C:true R:false U:false D:false force:true]
> testGetPrefixMapping[C:true R:false U:false D:false force:false]
> testGetPrefixMapping[C:false R:true U:false D:true force:true]
> testGetPrefixMapping[C:false R:true U:false D:true force:false]
> testGetPrefixMapping[C:false R:true U:false D:false force:true]
> testGetPrefixMapping[C:false R:true U:false D:false force:false]
> testGetPrefixMapping[C:false R:false U:false D:true force:true]
> testGetPrefixMapping[C:false R:false U:false D:true force:false]
> testGetPrefixMapping[C:false R:false U:false D:false force:true]
> testGetPrefixMapping[C:false R:false U:false D:false force:false]
>



-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: [] Release Jena 2.12.1 and Fuseki 1.1.1

[Claude Warren]

Seems like that change only applies to hashSet and it made no contract
concerning the order of retrieval.  Thx for the heads up though.

On Wed, Oct 1, 2014 at 7:19 PM, Kristian Rosenvold 
wrote:

> I believe Set iteration order has changed for java 8, at least for some
> sets. This can break brittle tests...
>
> Kristian
> 1. Okt. 2014 12:14 skrev "Andy Seaborne"  følgende:
>
> > I'm having problems building the source-release with java8.
> >
> > * jena-security consistently fails with java8 (1.8.0_20)
> > * It does build with java7
> > * The git tag source builds with java8.
> >
> > Claude - what causes this test failure?
> >
> > And is there a reason why the tests can't use an in-memory TDB dataset?
> It
> > will speed things up (make the tests not crash into each other if run in
> > parallel).
> >
> > Andy
> >
> > (I'll continue with other checking)
> >
> > java version "1.8.0_20"
> > Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
> > Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
> >
> >
> > The first failure is:
> >
> > Running org.apache.jena.security.graph.TDBGraphTest
> > Tests run: 384, Failures: 0, Errors: 16, Skipped: 0,
> > Time elapsed: 24.311 sec <<< FAILURE!
> > - in org.apache.jena.security.graph.TDBGraphTest
> > testGetPrefixMapping[C:true R:true U:false D:true
> > force:true](org.apache.jena.security.graph.TDBGraphTest)  Time elapsed:
> > 0.067 sec  <<< ERROR!
> > java.lang.reflect.InvocationTargetException: null
> >   at sun.reflect.GeneratedMethodAccessor502.invoke(Unknown Source)
> >   at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:43)
> >   at java.lang.reflect.Method.invoke(Method.java:483)
> >   at org.apache.jena.security.graph.SecuredPrefixMappingTest.runTests(
> > SecuredPrefixMappingTest.java:78)
> >   at org.apache.jena.security.graph.MemGraphTest.testGetPrefixMapping(
> > MemGraphTest.java:268)
> >   at sun.reflect.GeneratedMethodAccessor570.invoke(Unknown Source)
> >
> > ...junit on the stack .
> >
> > Caused by: java.lang.AssertionError:
> > Should have thrown AccessDenied Exception
> >   at org.junit.Assert.fail(Assert.java:88)
> >   at org.apache.jena.security.graph.SecuredPrefixMappingTest.
> > testWithDefaultMappings(SecuredPrefixMappingTest.java:401)
> >at sun.reflect.GeneratedMethodAccessor502.invoke(Unknown Source)
> >at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:43)
> >at java.lang.reflect.Method.invoke(Method.java:483)
> >at org.apache.jena.security.graph.SecuredPrefixMappingTest.runTests(
> > SecuredPrefixMappingTest.java:78)
> >at org.apache.jena.security.graph.MemGraphTest.testGetPrefixMapping(
> > MemGraphTest.java:268)
> >at sun.reflect.GeneratedMethodAccessor570.invoke(Unknown Source)
> >
> > There are 16 of these errors:
> >
> > Edited...
> >
> > testGetPrefixMapping[C:true R:true U:false D:true force:true]
> > testGetPrefixMapping[C:true R:true U:false D:true force:false]
> > testGetPrefixMapping[C:true R:true U:false D:false force:true]
> > testGetPrefixMapping[C:true R:true U:false D:false force:false]
> > testGetPrefixMapping[C:true R:false U:false D:true force:true]
> > testGetPrefixMapping[C:true R:false U:false D:true force:false]
> > testGetPrefixMapping[C:true R:false U:false D:false force:true]
> > testGetPrefixMapping[C:true R:false U:false D:false force:false]
> > testGetPrefixMapping[C:false R:true U:false D:true force:true]
> > testGetPrefixMapping[C:false R:true U:false D:true force:false]
> > testGetPrefixMapping[C:false R:true U:false D:false force:true]
> > testGetPrefixMapping[C:false R:true U:false D:false force:false]
> > testGetPrefixMapping[C:false R:false U:false D:true force:true]
> > testGetPrefixMapping[C:false R:false U:false D:true force:false]
> > testGetPrefixMapping[C:false R:false U:false D:false force:true]
> > testGetPrefixMapping[C:false R:false U:false D:false force:false]
> >
>



-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Problems building.

[Claude Warren]

I found the bug that Andy reported with the TDB Security tests and made
some changes.  However, when I attempted to compile and test the entire
build on my rather underpowered machine I came across this error:

Running com.hp.hpl.jena.test.TestPackage
 WARN [main] (MoreTests.java:169) - Encoding mismatch tests not executed on
platform with default UTF encoding.
Tests run: 8667, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
1,006.349 sec <<< FAILURE! - in com.hp.hpl.jena.test.TestPackage
testWithOWLMemMicroRuleInfModel(com.hp.hpl.jena.reasoner.rulesys.test.ConcurrencyTest)
Time elapsed: 3.16 sec  <<< FAILURE!
java.lang.AssertionError: Deadlock detected!
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
com.hp.hpl.jena.reasoner.rulesys.test.ConcurrencyTest.doTestConcurrency(ConcurrencyTest.java:183)
at
com.hp.hpl.jena.reasoner.rulesys.test.ConcurrencyTest.runConcurrencyTest(ConcurrencyTest.java:87)
at
com.hp.hpl.jena.reasoner.rulesys.test.ConcurrencyTest.testWithOWLMemMicroRuleInfModel(ConcurrencyTest.java:190)


I am retesting  now before I head to work and so should be able to report
if this is a one off (as I suspect) or not this evening.

The changes I made were in the security test classes so I would not expect
any interaction with the core tests.

Claude


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

unlocking a locked prefixmapping

[Claude Warren]

Currently there is no way to unlock a locked prefix mapping I assume this
is by design. Does anyone remember why?

-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: [] Release Jena 2.12.1 and Fuseki 1.1.1 (RC 2)

[Claude Warren]

I found an issue in the source.  Though I don't know how it ever worked.  I
have built from scratch.

On Fri, Oct 3, 2014 at 4:17 PM, Andy Seaborne  wrote:

> I'm still having problems with jena-security from the source-release.
>
> From the git tag works and the file trees for jena-security check as
> identical (diff -ur).
>
> The difference from rc1 is that now java-7-openjdk fails (more often than
> not) and java-8-oracle fails every time.  I would like to believe it is a
> problem at my end but I am failing to come up with any theories as to what
> it might be.
>
> Has anyone else succeeded in building from the source-release?  On ubuntu?
>
> All the other checks are fine.
>
> Andy
>
> java version "1.8.0_20"
> Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
> Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
>
> java version "1.7.0_65"
> OpenJDK Runtime Environment (IcedTea 2.5.2) (7u65-2.5.2-3~14.04)
> OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)
>
>


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: [] Release Jena 2.12.1 and Fuseki 1.1.1 (RC 2)

[Claude Warren]

I think the changes I just checked in will fix the problem.  I was able to
build and test on Ubuntu using Java 8.

On Fri, Oct 3, 2014 at 7:22 PM, Claude Warren  wrote:

> I found an issue in the source.  Though I don't know how it ever worked.
> I have built from scratch.
>
> On Fri, Oct 3, 2014 at 4:17 PM, Andy Seaborne  wrote:
>
>> I'm still having problems with jena-security from the source-release.
>>
>> From the git tag works and the file trees for jena-security check as
>> identical (diff -ur).
>>
>> The difference from rc1 is that now java-7-openjdk fails (more often than
>> not) and java-8-oracle fails every time.  I would like to believe it is a
>> problem at my end but I am failing to come up with any theories as to what
>> it might be.
>>
>> Has anyone else succeeded in building from the source-release?  On ubuntu?
>>
>> All the other checks are fine.
>>
>> Andy
>>
>> java version "1.8.0_20"
>> Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
>> Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
>>
>> java version "1.7.0_65"
>> OpenJDK Runtime Environment (IcedTea 2.5.2) (7u65-2.5.2-3~14.04)
>> OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)
>>
>>
>
>
> --
> I like: Like Like - The likeliest place on the web
> <http://like-like.xenei.com>
> LinkedIn: http://www.linkedin.com/in/claudewarren
>



-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Security Tests

[Claude Warren]

Andy,

Have you attempted to build with the new tests I checked in?  If so did it
work?  If it did I will close the open issue.

Thanks,
Claude


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Security Tests

[Claude Warren]

The problem was with test code and was a misplaced not in an if statement
(as I said, I don't know how it would have worked in the first place).  I
also don't know how it would work for the in-memory tests but not the TDB
tests.  To quote Alice: "Curiouser and Curiouser"

Claude

On Sat, Oct 4, 2014 at 11:39 PM, Andy Seaborne  wrote:

> On 04/10/14 19:29, Claude Warren wrote:
>
>> Andy,
>>
>> Have you attempted to build with the new tests I checked in?  If so did
>> it work?  If it did I will close the open issue.
>>
>
> Claude,
>
> Haven't tried (just kicking off a build now .. master builds for me with
> Java8) but I'm not clear from the changes how it relates to my experience.
>
> But ... it was working for me from the git tag and master, just not
> working from the source-release despite the files being identical (of
> course, some things still differ like the file order in directories is
> different leading to differences with looking at files).
>
> We're also not seeing problems on Jenkins.  The spite of recent Jenkins
> failures were (1) migrating to git then (2) the build servers are under a
> lot of stress and they have just been rebuilt (ubuntu*) and new servers
> made available to the general pool (H*) all of which caused some
> configuration problems.
>
> What was the problem with jena-security?  And would it induce
> non-deterministic behaviour?
>
> Andy
>
>
>> Thanks,
>> Claude
>>
>>
>> --
>> I like: Like Like - The likeliest place on the web
>> <http://like-like.xenei.com>
>> LinkedIn: http://www.linkedin.com/in/claudewarren
>>
>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Security Tests

[Claude Warren]

The issue was in SecuredPrefixMappingTest.java  around line 62 is should
read

if (m.isAnnotationPresent(Test.class))

but it did read

if ( ! m.isAnnotationPresent(Test.class))



On Sat, Oct 4, 2014 at 11:39 PM, Andy Seaborne  wrote:

> On 04/10/14 19:29, Claude Warren wrote:
>
>> Andy,
>>
>> Have you attempted to build with the new tests I checked in?  If so did
>> it work?  If it did I will close the open issue.
>>
>
> Claude,
>
> Haven't tried (just kicking off a build now .. master builds for me with
> Java8) but I'm not clear from the changes how it relates to my experience.
>
> But ... it was working for me from the git tag and master, just not
> working from the source-release despite the files being identical (of
> course, some things still differ like the file order in directories is
> different leading to differences with looking at files).
>
> We're also not seeing problems on Jenkins.  The spite of recent Jenkins
> failures were (1) migrating to git then (2) the build servers are under a
> lot of stress and they have just been rebuilt (ubuntu*) and new servers
> made available to the general pool (H*) all of which caused some
> configuration problems.
>
> What was the problem with jena-security?  And would it induce
> non-deterministic behaviour?
>
> Andy
>
>
>> Thanks,
>> Claude
>>
>>
>> --
>> I like: Like Like - The likeliest place on the web
>> <http://like-like.xenei.com>
>> LinkedIn: http://www.linkedin.com/in/claudewarren
>>
>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

TDB, inferencing and OOM

[Claude Warren]

I have the following code


data = TDBFactory.createDataset( dir.getAbsolutePath() );
data.begin( ReadWrite.WRITE);
// read 2 files into the model (approx 3.5K triples)
Model dataM = loadDir( ctxt, "/WEB-INF/resources/rdf/documents");
// read http://www.w3.org/2009/08/skos-reference/skos.rdf
Model schemaM = loadDir( ctxt, "/WEB-INF/resources/rdf/schemas" );
Reasoner reasoner = ReasonerRegistry.getOWLMiniReasoner();
reasoner = reasoner.bindSchema(schemaM);

infModel = ModelFactory.createInfModel(reasoner, dataM);
data.commit();

// DEBUGGING CODE
String realPath = ctxt.getRealPath("/WEB-INF/resources/");
File f2 = new File(realPath, "data.rdf");

// OOM occures here
infModel.write(new FileOutputStream(f2));


So my questions are:


1) Should in inferencing be done within a write transaction and if so how
does one assure that all inferencing will be done within a write
transaction?  In this case I would expect that all inferencing would be
done in the infModel.write() call but in the general case I may not be
making that call.

2) Shoudn't the inferencing be writing to the TDB datastore?  If so why is
there an OOM error?

Many thanks,
Claude


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

Re: [VOTE] Release Jena 2.12.1 and Fuseki 1.1.1 (RC 2)

[Claude Warren]

+1
Checked on Ubuntu

On Mon, Oct 6, 2014 at 11:23 AM, Rob Vesse  wrote:

> +1
>
> Rob
>
> ---
>
> Checking needed:
>
> + does everything work on Linux?
>
> Not tested
>
> + does everything work on MS Windows?
>
> Not tested
>
> + does everything work on OS X?
>
> Yes
>
> + is the GPG signature fine?
>
> Yes
>
> + are the checksums correct?
>
> Yes
>
> + is there a source archive?
>
> Yes
>
> + can the source archive really be built?
>
> Yes
>
> + is there a correct LICENSE and NOTICE file in each artifact
> (both source and binary artifacts)?
>
> Yes
>
> + does the NOTICE file contain all necessary attributions?
>
> Yes
>
> + have any licenses of dependencies changed due to upgrades?
>  if so have LICENSE and NOTICE been upgraded appropriately?
>
> Yes
>
> + does the tag in the SCM contain reproducible sources?
>
>
> Yes
>
> On 05/10/2014 12:09, "Andy Seaborne"  wrote:
>
> >
> >
> >
> >> Please vote to approve this release:
> >>
> >>[ ] +1 Approve the release
> >
> >+1
> >
> >>[ ]  0 Don't care
> >>[ ] -1 Don't release, because ...
> >
> >
> >Give the problems with building are identified as an issue with the
> >testing, not the main code, I am happy to release this build.
> >
> >>
> >> Checking needed:
> >>
> >> + does everything work on Linux?
> >
> >Only problems were the security tests.
> >
> >> + does everything work on MS Windows?
> >> + does everything work on OS X?
> >
> >> + is the GPG signature fine?
> >Yes
> >> + are the checksums correct?
> >Yes
> >> + is there a source archive?
> >Yes
> >> + can the source archive really be built?
> >Yes, it can be built.
> >
> >> + is there a correct LICENSE and NOTICE file in each artifact
> >>  (both source and binary artifacts)?
> >Yes
> >> + does the NOTICE file contain all necessary attributions?
> >Yes
> >> + have any licenses of dependencies changed due to upgrades?
> >>   if so have LICENSE and NOTICE been upgraded appropriately?
> >Yes
> >> + does the tag in the SCM contain reproducible sources?
> >Yes
> >>
> >>
> >>
> >>
> >>
> >
>
>
>
>
>


-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren