[GitHub] jena pull request #458: JENA-1481, JENA-1586: Delete databases and expel fro...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/458#discussion_r209309616
  
--- Diff: 
jena-fuseki2/jena-fuseki-core/src/main/java/org/apache/jena/fuseki/mgt/ActionDatasets.java
 ---
@@ -288,28 +294,89 @@ protected void execDeleteItem(HttpAction action) {
 if ( ! action.getDataAccessPointRegistry().isRegistered(name) )
 ServletOps.errorNotFound("No such dataset registered: "+name);
 
+// This acts as a lock. 
 systemDSG.begin(ReadWrite.WRITE) ;
 boolean committed = false ;
+
 try {
 // Here, go offline.
 // Need to reference count operations when they drop to zero
 // or a timer goes off, we delete the dataset.
 
 DataAccessPoint ref = 
action.getDataAccessPointRegistry().get(name) ;
+
 // Redo check inside transaction.
 if ( ref == null )
 ServletOps.errorNotFound("No such dataset registered: 
"+name);
+
+String filename = name.startsWith("/") ? name.substring(1) : 
name;
+List configurationFiles = 
FusekiSystem.existingConfigurationFile(filename);
+if  ( configurationFiles.size() != 1 ) {
+// This should not happen.
+action.log.warn(format("[%d] There are %d configuration 
files, not one.", action.id, configurationFiles.size()));
+ServletOps.errorOccurred(
+format(
+"There are %d configuration files, not one. Delete 
not performed; clearup of the filesystem needed.",
+action.id, configurationFiles.size()));
+}
+
+String cfgPathname = configurationFiles.get(0);
+
+// Delete configuration file.
+// Once deleted, server restart will not have the database. 
+FileOps.deleteSilent(cfgPathname);
 
-// Make it invisible to the outside.
+// Get before removing.
+DatasetGraph database = ref.getDataService().getDataset();
+// Make it invisible in this running server.
 action.getDataAccessPointRegistry().remove(name);
-// Delete configuration file.
-// Should be only one, undo damage if multiple.
-String filename = name.startsWith("/") ? name.substring(1) : 
name;
-
FusekiSystem.existingConfigurationFile(filename).stream().forEach(FileOps::deleteSilent);
-// Leave the database in place. if it is in /databases/, 
recreating the
-// configuration will make the database reappear. This is 
intentional.
-// Deleting a large database by accident is a major mistake.
 
+// JENA-1481 & JENA-1586 : Delete the database.
+// Delete the database for real only when it is in the server 
"run/databases"
+// area. Don't delete databases that reside elsewhere. We do 
delete the
+// configuration file, so the databases will not be associated 
with the server
+// anymore.
+
+boolean tryToRemoveFiles = true;
+
+// Text databases.
+// Close the in-JVM objects for Lucene index and databases. 
+// Do not delete files; at least for the lucene index, they 
are likely outside the run/databases. 
+if ( database instanceof DatasetGraphText ) {
+DatasetGraphText dbtext = (DatasetGraphText)database;
+database = dbtext.getBase();
+dbtext.getTextIndex().close();
+tryToRemoveFiles = false ;
+}
+
+boolean isTDB1 = 
org.apache.jena.tdb.sys.TDBInternal.isTDB1(database);
+boolean isTDB2 = 
org.apache.jena.tdb2.sys.TDBInternal.isTDB2(database);
+
+if ( ( isTDB1 || isTDB2 ) ) {
+// JENA-1586: Remove database from the process.
+if ( isTDB1 )
+org.apache.jena.tdb.sys.TDBInternal.expel(database);
+if ( isTDB2 )
+org.apache.jena.tdb2.sys.TDBInternal.expel(database);
+
+// JENA-1481: Really delete files.
+// Find the database files (may not be any - e.g. 
in-memory).
+Path pDatabase = 
FusekiSystem.dirDatabases.resolve(filename);
+if ( tryToRemoveFiles && Files.exists(pDatabase)) {
+try { 
+if ( Files.isSymbo

[GitHub] jena pull request #458: JENA-1481, JENA-1586: Delete databases and expel fro...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/458#discussion_r209307965
  
--- Diff: jena-base/src/main/java/org/apache/jena/atlas/io/IO.java ---
@@ -364,4 +369,31 @@ public static String uniqueFilename(String directory, 
String base, String ext) {
 return null ;
 }
 }
+
+/** Delete everything from a {@code Path} start point, including the 
path itself.
+ * This function works on files or directories.
+ * This function does not follow symbolic links.
+ */  
+public static void deleteAll(Path start) {
+// Walks down the tree and deletes directories on the way backup.
+try { 
+Files.walkFileTree(start, new SimpleFileVisitor() {
--- End diff --

Could this `new SimpleFileVisitor` be broken out as a `static` inner class? 


---

[GitHub] jena pull request #449: JENA-1578

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/449#discussion_r205813086
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/query/ParameterizedSparqlString.java ---
@@ -1734,4 +1740,243 @@ public String toString() {
 }
 
 }
+
+/**
+ * Assign a varName with a multiple items and whether to include
+ * parenthesis.
+ *
+ * @param varName
+ * @param items
+ * @param isParenthesisNeeded
+ */
+public void setValues(String varName, Collection 
items, boolean isParenthesisNeeded) {
+this.valuesReplacements.put(varName, new ValueReplacement(varName, 
items, isParenthesisNeeded));
+}
+
+/**
+ * Assign a varName with a multiple items.
+ * Can be used to assign multiple values to a single variable or single
+ * value to multiple variables (if using a List) in the SPARQL 
query.
+ * See setGroupedValues to assign multiple values to multiple 
variables.
+ *
+ * @param varName
+ * @param items
+ */
+public void setValues(String varName, Collection 
items) {
+setValues(varName, items, false);
+}
+
+/**
+ * Assign a varName with a single item and whether to include 
parenthesis.
+ *
+ * @param varName
+ * @param item
+ * @param isParenthesisNeeded
+ */
+public void setValues(String varName, RDFNode item, boolean 
isParenthesisNeeded) {
+setValues(varName, Arrays.asList(item), isParenthesisNeeded);
+}
+
+/**
+ * Assign a varName with a single item.
+ *
+ * @param varName
+ * @param item
+ */
+public void setValues(String varName, RDFNode item) {
+setValues(varName, Arrays.asList(item), false);
+}
+
+/**
+ * Sets a map of varNames and their items.
+ *
+ * @param valuesItems
+ */
+public void setValues(Map> 
valuesItems) {
+for (String varName : valuesItems.keySet()) {
--- End diff --

Just a style thing, really, but I believe this method could be just 
`valuesItems.forEach(this::setValues)`.


---

[GitHub] jena pull request #449: JENA-1578

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/449#discussion_r205813451
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/query/ParameterizedSparqlString.java ---
@@ -1734,4 +1740,243 @@ public String toString() {
 }
 
 }
+
+/**
+ * Assign a varName with a multiple items and whether to include
+ * parenthesis.
+ *
+ * @param varName
+ * @param items
+ * @param isParenthesisNeeded
+ */
+public void setValues(String varName, Collection 
items, boolean isParenthesisNeeded) {
+this.valuesReplacements.put(varName, new ValueReplacement(varName, 
items, isParenthesisNeeded));
+}
+
+/**
+ * Assign a varName with a multiple items.
+ * Can be used to assign multiple values to a single variable or single
+ * value to multiple variables (if using a List) in the SPARQL 
query.
+ * See setGroupedValues to assign multiple values to multiple 
variables.
+ *
+ * @param varName
+ * @param items
+ */
+public void setValues(String varName, Collection 
items) {
+setValues(varName, items, false);
+}
+
+/**
+ * Assign a varName with a single item and whether to include 
parenthesis.
+ *
+ * @param varName
+ * @param item
+ * @param isParenthesisNeeded
+ */
+public void setValues(String varName, RDFNode item, boolean 
isParenthesisNeeded) {
+setValues(varName, Arrays.asList(item), isParenthesisNeeded);
+}
+
+/**
+ * Assign a varName with a single item.
+ *
+ * @param varName
+ * @param item
+ */
+public void setValues(String varName, RDFNode item) {
+setValues(varName, Arrays.asList(item), false);
+}
+
+/**
+ * Sets a map of varNames and their items.
+ *
+ * @param valuesItems
+ */
+public void setValues(Map> 
valuesItems) {
+for (String varName : valuesItems.keySet()) {
+Collection items = valuesItems.get(varName);
+setValues(varName, items);
+}
+}
+
+/**
+ * All varNames in the map will use the same approach to parenthesis.
+ *
+ * @param valuesItems
+ * @param isParenthesisNeeded
+ */
+public void setValues(Map> 
valuesItems, Boolean isParenthesisNeeded) {
+for (String varName : valuesItems.keySet()) {
+Collection items = valuesItems.get(varName);
+setValues(varName, items, isParenthesisNeeded);
--- End diff --

Similarly, `valuesItems.forEach((varName, item)->setValues(varName, items, 
isParenthesisNeeded))`, and the same below.


---

[GitHub] jena pull request #448: JENA-1575: tdbloader - unnecessarily specific class ...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/448#discussion_r202350272
  
--- Diff: jena-tdb/src/main/java/org/apache/jena/tdb/TDBLoader.java ---
@@ -219,11 +211,14 @@ public final void setGenerateStats(boolean 
generateStats)
 
 // These are the basic operations for TDBLoader.
 
-private static void loadGraph$(GraphNonTxnTDB graph, InputStream 
input, boolean showProgress, boolean collectStats) {
+private static void loadGraph$(GraphTDB graph, InputStream input, 
boolean showProgress, boolean collectStats) {
+
+DatasetGraphTDB dsgtdb = 
TDBInternal.getBaseDatasetGraphTDB(graph.getDatasetGraphTDB());
--- End diff --

Just for my education, why is `getBaseDatasetGraphTDB` coming in here? What 
case would it meet that `graph.getDatasetGraphTDB()` by itself wouldn't?


---

Old internal notes?

[ajs6f]

Hey, Jena-folks,

I found this page: 
https://jena.apache.org/documentation/notes/jena-internals.html

while looking for something else, and after reading it, I wonder if it's a good 
thing to keep in the public documentation. It seems more than a bit 
out-of-date, since it refers to types that don't exist and plans that don't 
seemed to have been carried out.

Am I missing something, or shall I move that page to the Cwiki instance as 
historical-only?

ajs6f

Re: [jira] [Commented] (JENA-1568) Don't use JUnit3 TestCase for in SSE tests

[ajs6f]

I'm not surprised. We do some clever things in tests! My suggestion is really 
just to add a ticket (an "epic" or whatever) to track overall migration status, 
so that we do at least have it on the table as a conscious long-term goal.

ajs6f

> On Jun 28, 2018, at 11:08 AM, Claude Warren  wrote:
> 
> I remember munging around in Junit3 -> Junit4 conversion.  It was not at
> all pretty.  There are some constructs we use that do not map easily
> (dynamicly generated test suites comes to mind).  But I agree we should
> migrate.
> 
> On Thu, Jun 28, 2018 at 3:11 PM, A. Soroka (JIRA)  wrote:
> 
>> 
>>[ https://issues.apache.org/jira/browse/JENA-1568?page=
>> com.atlassian.jira.plugin.system.issuetabpanels:comment-
>> tabpanel=16526333#comment-16526333 ]
>> 
>> A. Soroka commented on JENA-1568:
>> -
>> 
>> [~andy.seaborne], is this an example of a larger program of migration we'd
>> like to make? JUnit5 is out, and while I'm _not_ suggesting we tangle with
>> that now, it might be nice to uniformly use 4.
>> 
>>> Don't use JUnit3 TestCase for in SSE tests
>>> ---
>>> 
>>>Key: JENA-1568
>>>URL: https://issues.apache.org/jira/browse/JENA-1568
>>>Project: Apache Jena
>>> Issue Type: Improvement
>>>   Affects Versions: Jena 3.8.0
>>>   Reporter: Andy Seaborne
>>>   Assignee: Andy Seaborne
>>>   Priority: Minor
>>>Fix For: Jena 3.9.0
>>> 
>>> 
>>> Follows on from JENA-1566.
>> 
>> 
>> 
>> --
>> This message was sent by Atlassian JIRA
>> (v7.6.3#76005)
>> 
> 
> 
> 
> -- 
> I like: Like Like - The likeliest place on the web
> <http://like-like.xenei.com>
> LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Towards Jena 3.8.0

[ajs6f]

Andy, I can do it as well, but what would be really great would be for someone 
who hasn't done it to enjoy the opportunity!

It's very straightforward (we have extensive notes [1]) and a great way to 
learn about some of Jena's machinery beyond the codebase itself.

ajs6f

[1] https://cwiki.apache.org/confluence/display/JENA/Release+Process

> On Jun 22, 2018, at 11:29 AM, Andy Seaborne  wrote:
> 
> Looks like we're ready.
> 
> I'll do the release hopefully early next week unless anyone else volunteers.
> 
>Andy
> 
> On 10/06/18 19:32, Andy Seaborne wrote:
>> Status:
>> Looks like these PRs are complete and waiting to go in,pending reviews.
>> PR#430: JENA-1556 text:query multilingual enhancements
>> PR#431: JENA-1559: Alt PrefixMapping implementation TDB1 and TDB2.
>> PR#432: JENA-1560: PrefixMappingUtils
>> PR#433: JENA-1561: Enable TDB2 in Fuseki UI and on the command line.
>> "It would be nice":
>> JENA-1562 (TDB2 Graph.size slowness) -- a silly mistake more than anything 
>> else.  WIP. I should be able to do that one soon.
>> JENA-1563 (Neptune/JSON results)
>> Looks to be a localized change.
>> Anything else?
>> Andy
>> On 06/06/18 12:37, Bruno P. Kinoshita wrote:
>>> +1
>>> 
>>> I'm adding a post-it to make sure I will help reviewing the release, and 
>>> also going through the documentation to see if there's anything that needs 
>>> updating (I remember at least one page I think I used 3.7.1 instead of 3.8 
>>> as the @since for a feature).
>> Thanks for getting that done.
>>> 
>>> 
>>> Cheers
>>> Bruno
>>> 
>>> 
>>> From: Andy Seaborne 
>>> To: "dev@jena.apache.org" 
>>> Sent: Wednesday, 6 June 2018 11:32 PM
>>> Subject: Towards Jena 3.8.0
>>> 
>>> 
>>> 
>>> Let's look at a Jena 3.8.0 release - there are some significant new
>>> 
>>> items in this release.
>>> 
>>> 
>>> JIRA report:
>>> 
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311220=12343042
>>>  
>>> 
>>> 
>>> --- Headlines:
>>> 
>>> 
>>> ** JENA-632:  JSON templated SPARQL queries.
>>> 
>>> 
>>> http://jena.staging.apache.org/documentation/query/generate-json-from-sparql.html
>>>  
>>> 
>>> 
>>> 
>>> JENA-1542: Integrate Lucene index in transaction lifecycle (TDB1, TDB2).
>>> 
>>> 
>>> JENA-1550: Parallel bulk loader for TDB2
>>> 
>>> http://jena.staging.apache.org/documentation/tdb2/tdb2_cmds.html
>>> 
>>> 
>>> --- Dependency changes
>>> 
>>> 
>>> Removed:
>>> 
>>> 
>>> org.apache.xerces is no longer a dependency.
>>> 
>>> Remove xercesImpl-2.11.0.jar
>>> 
>>> Remove xml-apis-1.4.01.jar
>>> 
>>> 
>>> 
>>> Added:
>>> 
>>> 
>>> Add Apache Commons Compress : commons-compress 1.17
>>> 
>>> 
>>> https://lists.apache.org/thread.html/40ebcb548cd2cb6d404d150cc1c919605689cf242ae17fe9e47191b1@%3Cdev.jena.apache.org%3E
>>>  
>>> 
>>> 
>>> Updated:
>>> 
>>> 
>>>jsonldjava 0.11.1 ==> 0.12.0
>>> 
>>>jackson 2.9.0 ==> 2.9.5 (addresses CVE-2018-5968)
>>> 
>>>httpclient 4.5.3 ==> 4.5.5
>>> 
>>>httpcore  4.4.6 ==> 4.4.9
>>> 
>>>Shared guava update 21.0 ==> 21.1-jre
>>> 
>>> 
>>> Tests:
>>> 
>>>com.jayway.awaitility::1.7.0 ==> org.awaitility.awaitility::3.1.0
>>> 
>>>org.objenesis:objenesis:jar: 2.1 ==> 2.6
>>> 
>>> Build:
>>> 
>>>maven-surefire-plugin: 2.20.1 ==> 2.21.0
>>> 
>>> 
>>> 
>>>  System changes:
>>> 
>>> 
>>> JENA-1537: Remove xerces
>>> 
>>> 
>>> JENA-1525 / Christopher Johnson
>>> 
>>> Java Automatic Module Names
>>> 
>>> 
>>> JENA-1524 / Christopher Johnson
>>> 
>>> Split package
>>> 
>>> 
>>> Note to repacking and deep system integrations:
>>> 
>>> 
>>> Package "org.apache.jena.system" was split across jars.
>>> 
>>> There are now two packages:
>>> 
>>> 
>>> "org.apache.jena.sys"
>>> 
>>> "org.apache.jena.system"
>>> 
>>> 
>>> and "sys" contains the system service loader code.
>>> 
>>> 
>>> JenaSystem.init() has migrated, with deprecated proxy, from
>>> 
>>> "org.apache.jena.system" to "org.apache.jena.sys"
>>> 
>>> 
>>> ** NB ServiceLoader file change **
>>> 
>>> 
>>> The ServiceLoader interface for system initialization is now:
>>> 
>>> 
>>> org.apache.jena.sys.JenaSubsystemLifecycle
>>> 
>>> 
>>> --- Other Changes
>>> 
>>> 
>>> JENA-1544: Consistent FROM/FROM NAMED naming handling
>>> 
>>> 
>>> JENA-1519: OpExt / Jeremy Coulon
>>> 
>>> 
>>> JENA-1488: SelectiveFoldingFilter for jena-text
>>> 

Re: Commit workflow

[ajs6f]

As for gitbox, the last result of my investigation was that we won't 
_currently_ have the same JIRA integrations that we now have, because INFRA 
hasn't done that yet. (And they would love to have some help doing that!)

I left the matter there. For my money, we might do well to revisit the JIRA 
integrations and make the switch...

ajs6f

> On Jun 20, 2018, at 9:52 AM, Aaron Coburn  wrote:
> 
> Thanks Andy, this is very helpful.
> 
> Best,
> Aaron
> 
>> On Jun 20, 2018, at 6:02 AM, Andy Seaborne  wrote:
>> 
>> Aaron, Chris, all,
>> 
>> One thing to mention - with the master repo on Apache hardware and PRs on 
>> the mirror (which we don't have write access to) there is a workflow for 
>> merging into the Apache github repo:
>> 
>> https://cwiki.apache.org/confluence/display/JENA/Commit+Workflow+for+Github-ASF
>> 
>>   Andy
>> 
>> (gitbox?)
> 

Re: Contribution proposal for Jena: support of a datatype for quantity values

[ajs6f]

See comments in-line...

ajs6f

> On Jun 15, 2018, at 9:49 AM, Maxime Lefrançois  
> wrote:
> 
> Dear all,
> 
> Regarding our contribution proposal to enable extensions to override SPARQL 
> operators in Jena
> 
> We finally got the agreement from our institution to contribute to the Apache 
> foundation.
> Question 1: what is the procedure to upload the form?

If we're talking about:

https://www.apache.org/licenses/cla-corporate.txt

then you can just scan and email a PDF to secret...@apache.org. There are other 
means of submission at that URL.

> About the how, I would like to discuss first with you
> 
> In a nutshell this is what I was thinking about:
> 
> Add use of the standard Java Service Provider API to load things 
> automatically found in the classpath:
> 
> - In TypeMapper --> a method that uses the Service Provider API to find more 
> Datatypes

Should this be a method, or rather additional behavior for getTypeByName, etc.? 
Are you thinking of something like "void getMoreMappings()" which would check 
for more available datatypes?

> - Datatype subclasses are not for just one URI, but could be for a set of URIs

Would that be true of Java types, as well?

> - ValueSpaceClassification should not be an enum any more --> maybe use a 
> class ValueSpace ...
> - should add some interface like NodeValueComparator, with some methods like:
>  - canCompare(ValueSpace vs, ValueSpace vs)
>  - sameAs(NodeValue nv, NodeValue nv)
>  - compare(NodeValue nv, NodeValue nv)

Should this return a Comparator instead? (Thinking of sorting.)

>  - add(NodeValue nv, NodeValue nv)
>  - substract(NodeValue nv, NodeValue nv)
> - in NodeValue class, method sameAs(NodeValue nv1, NodeValue nv2) and 
> compare(...) should  uses the Service Provider API to find 
> NodeValueComparators in the classpath
> - in class NodeValueOps, method divisionNV(NodeValue nv1, NodeValue nv2), 
> multiplicationNV(...) additionNV(...)  , subtractionNV(...)   should  uses 
> the Service Provider API to find more NodeValueComparators in the classpath

Hm. Is there some way this could happen via a lookup in TypeMapper? I'd rather 
not see too many paths to the same service impls...

> Any thoughts about this?

Yes: thank you so much for doing this excellent work!

> Best regards,
> Maxime Lefrançois
> 
> 
> 
> Le sam. 7 avr. 2018 à 15:13, ajs6f  a écrit :
> 
>> We're (well, Andy is) working on 3.7.0 now. We've been trying to maintain
>> a 6-month or so release cadence, so you've hit a really good time to begin
>> this work. That having been said, I don't think anyone would say that we
>> are especially stringent about it, so I wouldn't worry too much about the
>> timing myself.
>> 
>> ajs6f
>> 
>>> On Apr 6, 2018, at 9:36 AM, Maxime Lefrançois 
>> wrote:
>>> 
>>> Well,
>>> 
>>> I think I have a pretty clear idea how I would do this. We would end up
>>> using a registery like for custom functions or datatypes.
>>> That registry would contain an ordered list of SPARQL operator handlers,
>>> pre-filled by one for handling XSD datatypes.
>>> 
>>> I am currently requesting the right to fill the Apache individual
>>> contributor license agreement.
>>> 
>>> What would be the timeline if we wanted this shipped in the next release?
>>> 
>>> Best,
>>> Maxime
>>> 
>>> Le mar. 3 avr. 2018 à 15:30, ajs6f  a écrit :
>>> 
>>>> I agree. I can imagine plenty of use cases for such a powerful pair of
>>>> extension points.
>>>> 
>>>> Maxime, how can we help you attack that work? Is there a design that is
>>>> already clear to you? Are there any blockers we can help remove?
>>>> 
>>>> ajs6f
>>>> 
>>>>> On Mar 28, 2018, at 5:08 AM, Rob Vesse  wrote:
>>>>> 
>>>>> I think work towards Option 2 would be the most valuable to the
>> community
>>>>> 
>>>>> 
>>>>> 
>>>>> The SPARQL specification allows for the overloading of any
>>>> operator/expression where the spec currently defines the evaluation to
>> be
>>>> an error so extending operators is a natural and valid extension point
>> to
>>>> provide
>>>>> 
>>>>> 
>>>>> 
>>>>> The Terms of Use for UCUM would probably need us to obtain a licensing
>>>> assessment from Apache Legal as it is a non-standard OSS license even if
>>>> the code that implements it is under BSD (which is fine from an Apache
>>>> perspe

[GitHub] jena pull request #432: JENA-1560: PrefixMappingUtils

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/432#discussion_r194564667
  
--- Diff: 
jena-core/src/main/java/org/apache/jena/util/PrefixMappingUtils.java ---
@@ -0,0 +1,362 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.util;
+
+import java.util.* ;
+import java.util.function.Consumer ;
+import java.util.stream.Collectors ;
+
+import org.apache.jena.atlas.lib.SetUtils ;
+import org.apache.jena.atlas.lib.Trie ;
+import org.apache.jena.graph.Graph ;
+import org.apache.jena.graph.Node ;
+import org.apache.jena.graph.Triple ;
+import org.apache.jena.graph.impl.WrappedGraph;
+import org.apache.jena.rdf.model.Model ;
+import org.apache.jena.shared.PrefixMapping ;
+import org.apache.jena.shared.impl.PrefixMappingImpl ;
+
+public class PrefixMappingUtils {
+/**
+ * Return a read-only graph that has the same data (RDF triples) as 
the one given, but has a
+ * prefix mapping that only includes "in use " prefixes.
+ * 
+ * The prefix mappings of the two graphs are not connected. 
+ * Later changes to the prefix mapping of the original graph are not 
reflected in the returned graph.
+ * Modifications to the triples conatained in the underlying graph are 
reflected.   
--- End diff --

Typo "conatained" => "contained"


---

[GitHub] jena pull request #432: JENA-1560: PrefixMappingUtils

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/432#discussion_r194566823
  
--- Diff: 
jena-core/src/main/java/org/apache/jena/util/PrefixMappingUtils.java ---
@@ -0,0 +1,362 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.util;
+
+import java.util.* ;
+import java.util.function.Consumer ;
+import java.util.stream.Collectors ;
+
+import org.apache.jena.atlas.lib.SetUtils ;
+import org.apache.jena.atlas.lib.Trie ;
+import org.apache.jena.graph.Graph ;
+import org.apache.jena.graph.Node ;
+import org.apache.jena.graph.Triple ;
+import org.apache.jena.graph.impl.WrappedGraph;
+import org.apache.jena.rdf.model.Model ;
+import org.apache.jena.shared.PrefixMapping ;
+import org.apache.jena.shared.impl.PrefixMappingImpl ;
+
+public class PrefixMappingUtils {
+/**
+ * Return a read-only graph that has the same data (RDF triples) as 
the one given, but has a
+ * prefix mapping that only includes "in use " prefixes.
+ * 
+ * The prefix mappings of the two graphs are not connected. 
+ * Later changes to the prefix mapping of the original graph are not 
reflected in the returned graph.
+ * Modifications to the triples conatained in the underlying graph are 
reflected.   
+ */
+public static Graph graphInUsePrefixMapping(Graph graph) {
+final PrefixMapping prefixMapping = calcInUsePrefixMapping(graph) ;
+prefixMapping.lock() ;
+Graph graph2 = new WrappedGraph(graph) {
+@Override
+public void performAdd(Triple triple)
+{ throw new UnsupportedOperationException() ; }
+
+@Override
+public void performDelete(Triple triple)
+{ throw new UnsupportedOperationException() ; }
+
+@Override
+public PrefixMapping getPrefixMapping() {
+return prefixMapping ;
+}
+} ;
+return graph2 ;
+}
+
+/**
+ * Analyse the graph to see which prefixes of the graph are in use.
+ * 
+ * In the case of overlapping prefixes (where one prefix declaration 
is has an initial
+ * URI string which matches another prefix declaration), all are 
included, though
+ * they may not be used when printing (that depends on the output 
process). In effect,
+ * this process has "false positives".
+ * 
+ * This function does not calculate new prefixes.
+ * 
+ * @see #calcInUsePrefixMappingTTL(Graph)
+ */
+public static PrefixMapping calcInUsePrefixMapping(Graph graph) {
+PrefixMapping prefixMapping = graph.getPrefixMapping() ;
+if ( prefixMapping == null )
+return null ;
+return calcInUsePrefixMapping(graph, prefixMapping) ;
+}
+
+/**
+ * Analyse the graph to see which prefixes of the given {@link 
PrefixMapping} are in
+ * use.
+ * 
+ * In the case of overlapping prefixes (where one prefix declaration 
is has an initial
+ * URI string which matches another prefix declaration), all are 
included, though
+ * they may not be used when printing (that depends on the output 
process). In effect,
+ * this process has "false positives".
+ * 
+ * This function does not calculate new prefixes.
+ * 
+ * @see #calcInUsePrefixMappingTTL(Graph, PrefixMapping)
+ */
+public static PrefixMapping calcInUsePrefixMapping(Graph graph, 
PrefixMapping prefixMapping) {
+
+/* Method:
+ * 
+ * For each URI in the data, look it up in the trie.
+ * to see if has a declared prefix.
+ * 
+ * Exit early if every prefix is accounted for. 
+

[GitHub] jena pull request #432: JENA-1560: PrefixMappingUtils

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/432#discussion_r194567840
  
--- Diff: 
jena-core/src/main/java/org/apache/jena/util/PrefixMappingUtils.java ---
@@ -0,0 +1,362 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.util;
+
+import java.util.* ;
+import java.util.function.Consumer ;
+import java.util.stream.Collectors ;
+
+import org.apache.jena.atlas.lib.SetUtils ;
+import org.apache.jena.atlas.lib.Trie ;
+import org.apache.jena.graph.Graph ;
+import org.apache.jena.graph.Node ;
+import org.apache.jena.graph.Triple ;
+import org.apache.jena.graph.impl.WrappedGraph;
+import org.apache.jena.rdf.model.Model ;
+import org.apache.jena.shared.PrefixMapping ;
+import org.apache.jena.shared.impl.PrefixMappingImpl ;
+
+public class PrefixMappingUtils {
+/**
+ * Return a read-only graph that has the same data (RDF triples) as 
the one given, but has a
+ * prefix mapping that only includes "in use " prefixes.
+ * 
+ * The prefix mappings of the two graphs are not connected. 
+ * Later changes to the prefix mapping of the original graph are not 
reflected in the returned graph.
+ * Modifications to the triples conatained in the underlying graph are 
reflected.   
+ */
+public static Graph graphInUsePrefixMapping(Graph graph) {
+final PrefixMapping prefixMapping = calcInUsePrefixMapping(graph) ;
+prefixMapping.lock() ;
+Graph graph2 = new WrappedGraph(graph) {
+@Override
+public void performAdd(Triple triple)
+{ throw new UnsupportedOperationException() ; }
+
+@Override
+public void performDelete(Triple triple)
+{ throw new UnsupportedOperationException() ; }
+
+@Override
+public PrefixMapping getPrefixMapping() {
+return prefixMapping ;
+}
+} ;
+return graph2 ;
+}
+
+/**
+ * Analyse the graph to see which prefixes of the graph are in use.
+ * 
+ * In the case of overlapping prefixes (where one prefix declaration 
is has an initial
+ * URI string which matches another prefix declaration), all are 
included, though
+ * they may not be used when printing (that depends on the output 
process). In effect,
+ * this process has "false positives".
+ * 
+ * This function does not calculate new prefixes.
+ * 
+ * @see #calcInUsePrefixMappingTTL(Graph)
+ */
+public static PrefixMapping calcInUsePrefixMapping(Graph graph) {
+PrefixMapping prefixMapping = graph.getPrefixMapping() ;
+if ( prefixMapping == null )
+return null ;
+return calcInUsePrefixMapping(graph, prefixMapping) ;
+}
+
+/**
+ * Analyse the graph to see which prefixes of the given {@link 
PrefixMapping} are in
+ * use.
+ * 
+ * In the case of overlapping prefixes (where one prefix declaration 
is has an initial
+ * URI string which matches another prefix declaration), all are 
included, though
+ * they may not be used when printing (that depends on the output 
process). In effect,
+ * this process has "false positives".
+ * 
+ * This function does not calculate new prefixes.
+ * 
+ * @see #calcInUsePrefixMappingTTL(Graph, PrefixMapping)
+ */
+public static PrefixMapping calcInUsePrefixMapping(Graph graph, 
PrefixMapping prefixMapping) {
+
+/* Method:
+ * 
+ * For each URI in the data, look it up in the trie.
+ * to see if has a declared prefix.
+ * 
+ * Exit early if every prefix is accounted for. 
+

[GitHub] jena pull request #432: JENA-1560: PrefixMappingUtils

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/432#discussion_r194564937
  
--- Diff: 
jena-core/src/main/java/org/apache/jena/util/PrefixMappingUtils.java ---
@@ -0,0 +1,362 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.util;
+
+import java.util.* ;
+import java.util.function.Consumer ;
+import java.util.stream.Collectors ;
+
+import org.apache.jena.atlas.lib.SetUtils ;
+import org.apache.jena.atlas.lib.Trie ;
+import org.apache.jena.graph.Graph ;
+import org.apache.jena.graph.Node ;
+import org.apache.jena.graph.Triple ;
+import org.apache.jena.graph.impl.WrappedGraph;
+import org.apache.jena.rdf.model.Model ;
+import org.apache.jena.shared.PrefixMapping ;
+import org.apache.jena.shared.impl.PrefixMappingImpl ;
+
+public class PrefixMappingUtils {
+/**
+ * Return a read-only graph that has the same data (RDF triples) as 
the one given, but has a
+ * prefix mapping that only includes "in use " prefixes.
--- End diff --

Might want to put a link here to `calcInUsePrefixMapping` below just in 
case someone wants to the precise meaning of "in use". 


---

[GitHub] jena pull request #432: JENA-1560: PrefixMappingUtils

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/432#discussion_r194568224
  
--- Diff: 
jena-core/src/main/java/org/apache/jena/util/PrefixMappingUtils.java ---
@@ -0,0 +1,362 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.util;
+
+import java.util.* ;
+import java.util.function.Consumer ;
+import java.util.stream.Collectors ;
+
+import org.apache.jena.atlas.lib.SetUtils ;
+import org.apache.jena.atlas.lib.Trie ;
+import org.apache.jena.graph.Graph ;
+import org.apache.jena.graph.Node ;
+import org.apache.jena.graph.Triple ;
+import org.apache.jena.graph.impl.WrappedGraph;
+import org.apache.jena.rdf.model.Model ;
+import org.apache.jena.shared.PrefixMapping ;
+import org.apache.jena.shared.impl.PrefixMappingImpl ;
+
+public class PrefixMappingUtils {
+/**
+ * Return a read-only graph that has the same data (RDF triples) as 
the one given, but has a
+ * prefix mapping that only includes "in use " prefixes.
+ * 
+ * The prefix mappings of the two graphs are not connected. 
+ * Later changes to the prefix mapping of the original graph are not 
reflected in the returned graph.
+ * Modifications to the triples conatained in the underlying graph are 
reflected.   
+ */
+public static Graph graphInUsePrefixMapping(Graph graph) {
+final PrefixMapping prefixMapping = calcInUsePrefixMapping(graph) ;
+prefixMapping.lock() ;
+Graph graph2 = new WrappedGraph(graph) {
+@Override
+public void performAdd(Triple triple)
+{ throw new UnsupportedOperationException() ; }
+
+@Override
+public void performDelete(Triple triple)
+{ throw new UnsupportedOperationException() ; }
+
+@Override
+public PrefixMapping getPrefixMapping() {
+return prefixMapping ;
+}
+} ;
+return graph2 ;
+}
+
+/**
+ * Analyse the graph to see which prefixes of the graph are in use.
+ * 
+ * In the case of overlapping prefixes (where one prefix declaration 
is has an initial
+ * URI string which matches another prefix declaration), all are 
included, though
+ * they may not be used when printing (that depends on the output 
process). In effect,
+ * this process has "false positives".
+ * 
+ * This function does not calculate new prefixes.
+ * 
+ * @see #calcInUsePrefixMappingTTL(Graph)
+ */
+public static PrefixMapping calcInUsePrefixMapping(Graph graph) {
+PrefixMapping prefixMapping = graph.getPrefixMapping() ;
+if ( prefixMapping == null )
+return null ;
+return calcInUsePrefixMapping(graph, prefixMapping) ;
+}
+
+/**
+ * Analyse the graph to see which prefixes of the given {@link 
PrefixMapping} are in
+ * use.
+ * 
+ * In the case of overlapping prefixes (where one prefix declaration 
is has an initial
+ * URI string which matches another prefix declaration), all are 
included, though
+ * they may not be used when printing (that depends on the output 
process). In effect,
+ * this process has "false positives".
+ * 
+ * This function does not calculate new prefixes.
+ * 
+ * @see #calcInUsePrefixMappingTTL(Graph, PrefixMapping)
+ */
+public static PrefixMapping calcInUsePrefixMapping(Graph graph, 
PrefixMapping prefixMapping) {
+
+/* Method:
+ * 
+ * For each URI in the data, look it up in the trie.
+ * to see if has a declared prefix.
+ * 
+ * Exit early if every prefix is accounted for. 
+

[GitHub] jena pull request #431: JENA-1559: Alternative PrefixMapping implementation;...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/431#discussion_r194399107
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/graph/PrefixMappingOver.java ---
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.graph;
+
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.function.BiConsumer;
+
+import org.apache.jena.iri.IRI;
+import org.apache.jena.riot.system.PrefixMap;
+
+public class PrefixMappingOver extends PrefixMappingBase {
--- End diff --

Can't speak for @rvesse, but "Adaptor" is even better to my eye.


---

[GitHub] jena pull request #434: JENA-1562: Fix for Graph.size() for TDB2 graphs

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/434#discussion_r194398578
  
--- Diff: 
jena-db/jena-tdb2/src/main/java/org/apache/jena/tdb2/store/GraphViewSwitchable.java
 ---
@@ -79,16 +83,34 @@ public TransactionHandler getTransactionHandler() {
 /** Return the {@code DatasetGraphSwitchable} we are viewing. */
 @Override
 public DatasetGraphSwitchable getDataset() {
-return getx() ;
+return getx();
 }
 
 /** Return the {@code Graph} from the underlying switchable.
  *  Do not hold onto this reference across switches. 
  */
 public Graph getGraph() {
-return getx().getGraph(getGraphName()) ;
+return getx().getGraph(getGraphName());
 }
 
+// Super uses find. Override to call GraphTDB.size()
+@Override
+protected int graphBaseSize() {
+if ( isDefaultGraph() )
+return getDSG().getDefaultGraphTDB().size();
+return getDSG().getGraphTDB(getGraphName()).size();
+}
+
+private DatasetGraphTDB getDSG() {
+return ((DatasetGraphTDB)(getx().get()));
+}
+
+private static Function, Tuple> 
project4TupleTo3Tuple = item -> {
--- End diff --

I'm totally confused. The method above reads:
```
private DatasetGraphTDB getDSG() {
return ((DatasetGraphTDB)(getx().get()));
}
```
Where is `project4TupleTo3Tuple` used there?


---

[GitHub] jena pull request #431: JENA-1559: Alternative PrefixMapping implementation;...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/431#discussion_r194364163
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/graph/PrefixMappingOver.java ---
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.graph;
+
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.function.BiConsumer;
+
+import org.apache.jena.iri.IRI;
+import org.apache.jena.riot.system.PrefixMap;
+
+public class PrefixMappingOver extends PrefixMappingBase {
--- End diff --

+1


---

[GitHub] jena pull request #434: JENA-1562: Fix for Graph.size() for TDB2 graphs

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/434#discussion_r194269455
  
--- Diff: jena-arq/src/main/java/org/apache/jena/sparql/core/GraphView.java 
---
@@ -165,6 +165,9 @@ public void performDelete( Triple t ) {
 Node o = t.getObject() ;
 dsg.delete(g, s, p, o) ;
 }
+
+// Subclasses may wish to provide graphBaseSize otherwise GraphBase 
uses find()  
--- End diff --

Maybe this is just where I like to find things, but I'd rather `@Override 
protected int graphBaseSize() { return super.graphBaseSize(); }` with this as a 
Javadoc comment. 


---

Re: New dependency : Apache Commons Compress

[ajs6f]

This all sounds quite reasonable, but I have one question: why would we include 
awaitility in the downloads? Isn't it only used for test code?

ajs6f

> On Jun 3, 2018, at 1:53 PM, Andy Seaborne  wrote:
> 
> JENA-1554 (support bz2 compression support) / PR#427 includes adding a new 
> dependency org.apache.commons:commons-compress.
> 
> There is a knock-on effect: commons-compress depends on 
> org.objenesis:objenesis and a version that the enforcer considers 
> incompatible with that coming via com.jayway.awaitility:awaitility 
> (scope=test).
> 
> com.jayway.awaitility:awaitility is quite out of date - it is now 
> org.awaitility:awaitility - so that is upgraded.
> 
> All ALv2 license.
> 
> The binary downloads will include commons-compress, awaitility and objenesis 
> (all quite small).
> 
>Andy
> 
> https://github.com/apache/jena/pull/426
> https://issues.apache.org/jira/browse/JENA-1554

Re: Jena Eyeball [was: JENA-1541]

[ajs6f]

+1.

ajs6f

> On May 21, 2018, at 1:44 PM, Bruno P. Kinoshita <ki...@apache.org> wrote:
> 
> Sounds good to me
> +1
> 
> Sent from Yahoo Mail on Android 
> 
>  On Tue, 22 May 2018 at 5:08, Andy Seaborne<a...@apache.org> wrote:   To pull 
> this into a plan ...
> 
> The Eyeball code is already elsewhere (in Apache SVN - keep that link).
> 
> Proposal:
> 
> * Unlink from the tools page.
> * Upgarde the notice on all Eyeball pages to "This page is historical 
> for information only - there is no Apache release of Eyeball".
> * Remove links to JIRA, SF distribution (broken anyway) and mention of 
> help/support.
> 
> PMC - OK?
> 
> Andy
> 
> On 11/05/18 14:10, Chris Dollin wrote:
>> (Sorry for previous empty reply, hit wrong button)
>> 
>> I'm the original developer of Eyeball. It is several years since any work
>> has been done on it, and I suspect that it is now obsolete, that is,
>> that there other tools with equivalent functionality (though I don't
>> know what they are).
>> 
>> Putting it somewhere designated "Attic" with a red label saying
>> "use at your own risk" seems reasonable.
>> 
>> Chris
>> 
>> On 11 May 2018 at 13:57, Chris Dollin <chris.dol...@epimorphics.com> wrote:
>> 
>>> 
>>> 
>>> On 11 May 2018 at 13:11, Andy Seaborne <a...@apache.org> wrote:
>>> 
>>>> Edward found Eyeball via:
>>>> 
>>>> https://jena.apache.org/documentation/tools/eyeball-getting-started.html
>>>> 
>>>> Eyeball is mentioned:
>>>> 
>>>> documentation/tools/index.mdtext
>>>> documentation/tools/eyeball-guide.mdtext
>>>> documentation/tools/eyeball-getting-started.mdtext
>>>> documentation/tools/eyeball-manual.mdtext
>>>> 
>>>> The tools page has schemagen and eyeball on it.
>>>> 
>>>> eyeball-getting-started says "file a JIRA".
>>>> 
>>>> What should we do?
>>>> If it is not released, do we retire it from the documentation?
>>>> 
>>>>   Andy
>>>> 
>>>> 
>>>> On 04/05/18 11:39, Edward (JIRA) wrote:
>>>> 
>>>>> Edward created JENA-1541:
>>>>> 
>>>>> 
>>>>> Summary: Jena Eyeball - ant test fails with TestCase Error
>>>>> Key: JENA-1541
>>>>> URL: https://issues.apache.org/jira/browse/JENA-1541
>>>>> Project: Apache Jena
>>>>> Issue Type: Question
>>>>> Components: Eyeball, Jena
>>>>>   Reporter: Edward
>>>>> 
>>>>> 
>>>>> Hello,
>>>>> 
>>>>> I'm trying to get Jena Eyeball running. I wanted to do the Tutorial on
>>>>> [here|[https://jena.apache.org/documentation/tools/eyeball-g
>>>>> etting-started.html],] but it failed with following error:
>>>>> {code:java}
>>>>>   [junit] Testcase: warning(junit.framework.TestSuite$1):FAILED
>>>>>   [junit] Class com.hp.hpl.jena.eyeball.inspec
>>>>> tors.test.TestMoreOwlSyntaxInspector has no public constructor
>>>>> TestCase(String name) or TestCase()
>>>>>   [junit] junit.framework.AssertionFailedError: Class
>>>>> com.hp.hpl.jena.eyeball.inspectors.test.TestMoreOwlSyntaxInspector has
>>>>> no public constructor TestCase(String name) or TestCase()
>>>>>   [junit]
>>>>>   [junit]
>>>>> 
>>>>> BUILD FAILED
>>>>> /**/Downloads/eyeball-2.3/build.xml:146: Test
>>>>> com.hp.hpl.jena.eyeball.inspectors.test.TestMoreOwlSyntaxInspector
>>>>> failed
>>>>> {code}
>>>>> The tutorial says if the ant test doesn't pass, I should file a Jira
>>>>> Issue. So that's what I am doing.
>>>>> Help would be much appreciated!
>>>>> 
>>>>> 
>>>>> Greetings,
>>>>> 
>>>>> Edward
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> This message was sent by Atlassian JIRA
>>>>> (v7.6.3#76005)
>>>>> 
>>>>> 
>>> 
>>> 
>>> --
>>> "What I don't understand is this ..."  Trevor Chaplin, /The Beiderbeck
>>> Affair/
>>> 
>>> Epimorphics Ltd, http://www.epimorphics.com
>>> Registered address: Court Lodge, 105 High Street, Portishead, Bristol BS20
>>> 6PT
>>> Epimorphics Ltd. is a limited company registered in England (number
>>> 7016688)
>>> 
>> 
>> 
>> 
> 

[GitHub] jena pull request #423: Tweaks

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/423#discussion_r189391950
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/expr/nodevalue/NodeFunctions.java 
---
@@ -254,28 +254,33 @@ public static String lang(Node node) {
 }
 
 //  langMatches
+/** LANGMATCHES - first argument is a language string */ 
--- End diff --

Ah, I didn't know that, but at least I just learnt about `LANGMATCHES`!


---

[GitHub] jena pull request #423: Tweaks

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/423#discussion_r189388128
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/expr/nodevalue/NodeFunctions.java 
---
@@ -254,28 +254,33 @@ public static String lang(Node node) {
 }
 
 //  langMatches
+/** LANGMATCHES - first argument is a language string */ 
 public static NodeValue langMatches(NodeValue nv, NodeValue nvPattern) 
{
 return langMatches(nv, nvPattern.getString()) ;
 }
 
+/** LANGMATCHES - first argument is a language string */ 
--- End diff --

Ditto.


---

[GitHub] jena pull request #423: Tweaks

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/423#discussion_r189388102
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/expr/nodevalue/NodeFunctions.java 
---
@@ -254,28 +254,33 @@ public static String lang(Node node) {
 }
 
 //  langMatches
+/** LANGMATCHES - first argument is a language string */ 
--- End diff --

This seems like a `@param`?
```
/** LANGMATCHES 
  @param nv a language string
*/ 
```


---

[GitHub] jena pull request #423: Tweaks

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/423#discussion_r189388383
  
--- Diff: 
jena-db/jena-dboe-transaction/src/main/java/org/apache/jena/dboe/transaction/txn/TransactionCoordinator.java
 ---
@@ -88,6 +89,22 @@
 // See also {@code lock} which protects the datastructures during 
transaction management.  
 private ReadWriteLock exclusivitylock = new ReentrantReadWriteLock() ;
 
+// The version is the serialization point for a transaction.
+// All transactions on the same view of the data get the same 
serialization point.
+
+// A read transaction can be promoted if writer does not start
+// This TransactionCoordinator provides Serializable, Read-lock-free
+// execution. With no item locking, a read can only be promoted
+// if no writer started since the reader started or if it is "read 
committed",
+// seeing changes made since it started and comitted at the poiont of 
promotion.
+
+/* The version of the data - incremented when transaction commits.
+ * This is the version with repest to the last commited transaction.
+ * Aborts do not cause the data version to advance. 
+ * This counterr never goes backwards.
--- End diff --

typo `counterr` => `counter`


---

Re: parsing and matching triples

[ajs6f]

> On May 11, 2018, at 6:48 PM, Andy Seaborne <a...@apache.org> wrote:
> On 11/05/18 20:35, ajs6f wrote:
>>> On May 9, 2018, at 3:31 AM, Dave Reynolds <dave.e.reyno...@gmail.com> wrote:
>>> On 08/05/18 16:55, ajs6f wrote:
>>>> Forking a thread off to dev@:
>>>> Do we have a global policy about where null is accepted as a wildcard? I 
>>>> know it works in at least some places...
> 
> You have mentioned one place - any others?

I don't understand-- it was you who mentioned the places it is accepted, not 
me. I just pointed out that for at least some of those places, it is not 
documented. Some places are worse, e.g. DatasetGraph:

/** Find matching quads in the dataset - may include wildcards, Node.ANY or null
* @see Graph#find(Triple)
*/
public Iterator find(Quad quad) ;

"wildcards, Node.ANY or null"? But Node.ANY and null _are_ the wildcards and 
they all behave the same way...?! The Javadocs for Graph just don't mention 
null as parameter at all.

I'm happy to make a PR for this-- I just want to make sure I have it straight 
myself. 

> The big pain with nulls is returned values leading to NPEs.
> That's not the case here, which is usage for arguments to functions/methods.

null _is_ a pain here. Accidentally passing a null reference into a method 
where it will be interpreted as wildcard can (e.g.) delete unexpected things or 
return a much larger query result than expected, _without any error_. That 
would be prevented by a null-object pattern. I understand that you may not have 
seen these problems, but I have. I'm not trying to argue that they deserve a 
solution that disrupts deployments (as Dave described) but that doesn't mean 
these kinds of problems don't exist.

>  This overlaps with the debate around Optional as argument.

No one is suggesting introducing Optional at all, or changing any return types, 
or removing null from the whole API or from Java (!). My question was pretty 
clear: are we keeping null-as-wildcard for the extant API, or requiring it for 
all new API, and Dave just answered it as I would: no, new API doesn't have to 
continue the pattern. I'll try to get in a PR for Graph/DatasetGraph Javadocs 
shortly.

ajs6f

> 
>   Andy
> 
> 
> 
>> ajs6f
>>> Dave
>>> 
>>>>> On May 8, 2018, at 11:51 AM, Andy Seaborne <a...@apache.org> wrote:
>>>>> 
>>>>> Barry,
>>>>> 
>>>>> As a general concept "matching" happens at different levels.
>>>>> 
>>>>> Triple.match corresponds to the matching done by Graph.find - RDF terms 
>>>>> (URI, bnode, literal) match exactly, and Node.ANY is a wildcard.
>>>>> 
>>>>> Triple t1 = Triple.ANY;
>>>>> Triple t2 = SSE.parseTriple("(:s :p :o)");
>>>>> t1.matches(t2) -> true
>>>>> t2.matches(t1) -> false
>>>>> 
>>>>> Variables are a concept for SPARQL - and matches usefully need to return 
>>>>> which variable matched which RDF Term.
>>>>> 
>>>>> Triple patterns match against graphs and return an iterator of ways they 
>>>>> match.
>>>>> 
>>>>> Consider cases like "?x ?p ?x" where the variables impose am additional 
>>>>> shape.
>>>>> 
>>>>> If you want variable bindings, you could build a SPARQL query or wrap up 
>>>>> some of the internal code e.g.
>>>>> 
>>>>> /** Evaluate a triple pattern */
>>>>> private static QueryIterator match(Graph source, Triple pattern) {
>>>>>ExecutionContext execContext =
>>>>>  new ExecutionContext(ARQ.getContext(), source, null, null) ;
>>>>>QueryIterator chain = QueryIterRoot.create(execContext)
>>>>>chain = new QueryIterTriplePattern(chain, pattern, execContext) ;
>>>>>return chain ;
>>>>> }
>>>>> 
>>>>>Andy
>>>>> 
>>>>> On 08/05/18 09:21, Nouwt, B. (Barry) wrote:
>>>>>> Hi everybody,
>>>>>> I’m trying to reuse Apache Jena code that parses and matches triples. 
>>>>>> I’m currently looking at the SSE class’s parseTriple() method. This 
>>>>>> seems to fit my purpose for parsing a string representation of a triple 
>>>>>> into a triple object. I also noticed the following Javadoc on the 
>>>>>> Node.maches(Node) method:
>>>>>> Answer true iff this node accepts the other one as a match.
>>>>>> The default is an equality te

[GitHub] jena issue #421: Escape @xerces.internal annotations

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/421
  
Okay, either way works for me. This is already done, so easier!


---

Re: Old code.

[ajs6f]

https://jena.apache.org/download/maven.html shows jena-maven-tools with 
description "Maven Plugins, including schemagen", and as far as I can see, 
schemagen is the _only_ tool being made available in -maven-tools.

So, eyeball and -maven-tools are candidates for a "holding pen"? (From which of 
course things could be moved back to the released space if they get some love 
and attention.)

ajs6f

> On May 12, 2018, at 5:32 AM, Andy Seaborne <a...@apache.org> wrote:
> 
> schemagen is in jena-cmds
> 
> jena-maven-tools is a maven plugin.
> 
> (all this is in the codebase : "find" is your friend!)
> 
>   Andy
> 
> On 12/05/18 09:38, Claude Warren wrote:
>> Jena maven tools.  that does not include the schema gen does it or is that
>> a different maven tool?
>> On Sat, May 12, 2018 at 12:29 AM, Andy Seaborne <a...@apache.org> wrote:
>>> 
>>> 
>>> On 11/05/18 20:07, ajs6f wrote:
>>> 
>>>> Not to hijack the thread,
>>>> 
>>> 
>>> (new thread? or at least change the subject?)
>>> 
>>> but I think a natural question is: do we have any other such examples
>>>> (well-defined portions of code that aren't under any real maintenance and
>>>> that we are not releasing)?
>>>> 
>>> 
>>> All the code is available for inspection as is the documentation so it
>>> would be useful to check that.
>>> 
>>> 
>>> One I know of:
>>> 
>>> jena-maven-tools/ isn't currently being built because it it breaks with
>>> the latest Apache parent (not the first time this has happened).
>>> 
>>> Andy
>>> 

[GitHub] jena issue #421: Escape @xerces.internal annotations

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/421
  
I've never seen `@xerces.internal` before, but it seems to denote types 
internal to Xerces, right? My one question is: do we need to keep it on all 
types or can we stick it on a package comment? (Since we're not planning on 
maintaining a separate "branch" of Xerces as a whole.)


---

Re: parsing and matching triples

[ajs6f]

> On May 9, 2018, at 3:31 AM, Dave Reynolds <dave.e.reyno...@gmail.com> wrote:
> 
> On 08/05/18 16:55, ajs6f wrote:
>> Forking a thread off to dev@:
>> Do we have a global policy about where null is accepted as a wildcard? I 
>> know it works in at least some places...
>> I would love to (over an appropriate period of time and with lots of 
>> warnings and deprecation and so forth) stop letting it be a wildcard and 
>> require code to use the actual wildcard objects.
> 
> -1
> 
> We have a huge amount of code that assumes null-as-wildcard as provided for 
> in the public RDF API and I doubt we're alone. There would no chance of 
> porting all that through a normal deprecation cycle.

Fair enough, Dave. Just to be clear, are you saying as much as that new methods 
in the public API should always accept null-as-wildcard, or just that we 
oughtn't be changing extant methods now?

ajs6f

> Dave
> 
>>> On May 8, 2018, at 11:51 AM, Andy Seaborne <a...@apache.org> wrote:
>>> 
>>> Barry,
>>> 
>>> As a general concept "matching" happens at different levels.
>>> 
>>> Triple.match corresponds to the matching done by Graph.find - RDF terms 
>>> (URI, bnode, literal) match exactly, and Node.ANY is a wildcard.
>>> 
>>> Triple t1 = Triple.ANY;
>>> Triple t2 = SSE.parseTriple("(:s :p :o)");
>>> t1.matches(t2) -> true
>>> t2.matches(t1) -> false
>>> 
>>> Variables are a concept for SPARQL - and matches usefully need to return 
>>> which variable matched which RDF Term.
>>> 
>>> Triple patterns match against graphs and return an iterator of ways they 
>>> match.
>>> 
>>> Consider cases like "?x ?p ?x" where the variables impose am additional 
>>> shape.
>>> 
>>> If you want variable bindings, you could build a SPARQL query or wrap up 
>>> some of the internal code e.g.
>>> 
>>> /** Evaluate a triple pattern */
>>> private static QueryIterator match(Graph source, Triple pattern) {
>>>ExecutionContext execContext =
>>>  new ExecutionContext(ARQ.getContext(), source, null, null) ;
>>>QueryIterator chain = QueryIterRoot.create(execContext)
>>>chain = new QueryIterTriplePattern(chain, pattern, execContext) ;
>>>return chain ;
>>> }
>>> 
>>>Andy
>>> 
>>> On 08/05/18 09:21, Nouwt, B. (Barry) wrote:
>>>> Hi everybody,
>>>> I’m trying to reuse Apache Jena code that parses and matches triples. I’m 
>>>> currently looking at the SSE class’s parseTriple() method. This seems to 
>>>> fit my purpose for parsing a string representation of a triple into a 
>>>> triple object. I also noticed the following Javadoc on the 
>>>> Node.maches(Node) method:
>>>> Answer true iff this node accepts the other one as a match.
>>>> The default is an equality test; it is over-ridden in subclasses to
>>>> provide the appropriate semantics for literals, ANY, and variables.
>>>> Since this is exactly what I’m looking for, I’ve tried to match two 
>>>> triples using the matches() method, but it does not seem to work:
>>>> Triple t1 = SSE.parseTriple("(?s ?p ?o)");
>>>> Triple t2 = SSE.parseTriple("(test:subject test:predicate test:object)", 
>>>> pm);
>>>> t1.matches(t2)
>>>> The final statement returns false, while I would expect it to return true. 
>>>> Either, I’m missing something (which is completely realistic ), or I 
>>>> should use some other method to match two triples in the way described 
>>>> above.
>>>> Any help is appreciated!
>>>> Regards, Barry
>>>> This message may contain information that is not intended for you. If you 
>>>> are not the addressee or if this message was sent to you by mistake, you 
>>>> are requested to inform the sender and delete the message. TNO accepts no 
>>>> liability for the content of this e-mail, for the manner in which you use 
>>>> it and for damage of any kind resulting from the risks inherent to the 
>>>> electronic transmission of messages.

Re: parsing and matching triples

[ajs6f]

Unfortunately, Andy, e.g. the Javadocs for DatasetGraph::delete and ::deleteAny 
are just

/** Delete a quad */
public void delete(Node g, Node s, Node p, Node o) ;

/** Delete any quads matching the pattern */
public void deleteAny(Node g, Node s, Node p, Node o) ;

So maybe part of this clears up with just a few more comments. I'll try to get 
a PR in for that.

The semantics for matching still seem a bit odd. E.g. in Node_ANY I find

@Override
public boolean matches( Node other )
{ return other != null; }

which is obviously not unworkable (we all use it every day) but does seem to 
distinguish between null and Node_ANY. The Javadocs for match() are:

/**
Answer true iff this node accepts the other one as a match.
The default is an equality test; it is over-ridden in subclasses to
provide the appropriate semantics for literals, ANY, and variables.

@param other a node to test for matching
@return true iff this node accepts the other as a match
*/

which doesn't discuss null.

ajs6f

> On May 9, 2018, at 8:23 AM, Andy Seaborne <a...@apache.org> wrote:
> 
> 
> 
> On 08/05/18 16:55, ajs6f wrote:
> 
>> Do we have a global policy about where null is accepted as a wildcard? I 
>> know it works in at least some places...
> 
> > I know it works in at least some places...
> 
> Some operations are matches (Graph.find, Graph.contains, Graph.remove; 
> DatasetGraph similarly) and the javadoc should be clear on this.
> 
> Others such as add/delete are for concrete items - no wildcards.
> 
>Andy
> 

Re: Jena Eyeball [was: JENA-1541]

[ajs6f]

Not to hijack the thread, but I think a natural question is: do we have any 
other such examples (well-defined portions of code that aren't under any real 
maintenance and that we are not releasing)?

ajs6f

> On May 11, 2018, at 9:10 AM, Chris Dollin <chris.dol...@epimorphics.com> 
> wrote:
> 
> (Sorry for previous empty reply, hit wrong button)
> 
> I'm the original developer of Eyeball. It is several years since any work
> has been done on it, and I suspect that it is now obsolete, that is,
> that there other tools with equivalent functionality (though I don't
> know what they are).
> 
> Putting it somewhere designated "Attic" with a red label saying
> "use at your own risk" seems reasonable.
> 
> Chris
> 
> On 11 May 2018 at 13:57, Chris Dollin <chris.dol...@epimorphics.com> wrote:
> 
>> 
>> 
>> On 11 May 2018 at 13:11, Andy Seaborne <a...@apache.org> wrote:
>> 
>>> Edward found Eyeball via:
>>> 
>>> https://jena.apache.org/documentation/tools/eyeball-getting-started.html
>>> 
>>> Eyeball is mentioned:
>>> 
>>> documentation/tools/index.mdtext
>>> documentation/tools/eyeball-guide.mdtext
>>> documentation/tools/eyeball-getting-started.mdtext
>>> documentation/tools/eyeball-manual.mdtext
>>> 
>>> The tools page has schemagen and eyeball on it.
>>> 
>>> eyeball-getting-started says "file a JIRA".
>>> 
>>> What should we do?
>>> If it is not released, do we retire it from the documentation?
>>> 
>>>Andy
>>> 
>>> 
>>> On 04/05/18 11:39, Edward (JIRA) wrote:
>>> 
>>>> Edward created JENA-1541:
>>>> 
>>>> 
>>>>  Summary: Jena Eyeball - ant test fails with TestCase Error
>>>>  Key: JENA-1541
>>>>  URL: https://issues.apache.org/jira/browse/JENA-1541
>>>>  Project: Apache Jena
>>>>   Issue Type: Question
>>>>   Components: Eyeball, Jena
>>>> Reporter: Edward
>>>> 
>>>> 
>>>> Hello,
>>>> 
>>>> I'm trying to get Jena Eyeball running. I wanted to do the Tutorial on
>>>> [here|[https://jena.apache.org/documentation/tools/eyeball-g
>>>> etting-started.html],] but it failed with following error:
>>>> {code:java}
>>>> [junit] Testcase: warning(junit.framework.TestSuite$1):FAILED
>>>> [junit] Class com.hp.hpl.jena.eyeball.inspec
>>>> tors.test.TestMoreOwlSyntaxInspector has no public constructor
>>>> TestCase(String name) or TestCase()
>>>> [junit] junit.framework.AssertionFailedError: Class
>>>> com.hp.hpl.jena.eyeball.inspectors.test.TestMoreOwlSyntaxInspector has
>>>> no public constructor TestCase(String name) or TestCase()
>>>> [junit]
>>>> [junit]
>>>> 
>>>> BUILD FAILED
>>>> /**/Downloads/eyeball-2.3/build.xml:146: Test
>>>> com.hp.hpl.jena.eyeball.inspectors.test.TestMoreOwlSyntaxInspector
>>>> failed
>>>> {code}
>>>> The tutorial says if the ant test doesn't pass, I should file a Jira
>>>> Issue. So that's what I am doing.
>>>> Help would be much appreciated!
>>>> 
>>>> 
>>>> Greetings,
>>>> 
>>>> Edward
>>>> 
>>>> 
>>>> 
>>>> --
>>>> This message was sent by Atlassian JIRA
>>>> (v7.6.3#76005)
>>>> 
>>>> 
>> 
>> 
>> --
>> "What I don't understand is this ..."   Trevor Chaplin, /The Beiderbeck
>> Affair/
>> 
>> Epimorphics Ltd, http://www.epimorphics.com
>> Registered address: Court Lodge, 105 High Street, Portishead, Bristol BS20
>> 6PT
>> Epimorphics Ltd. is a limited company registered in England (number
>> 7016688)
>> 
> 
> 
> 
> -- 
> "What I don't understand is this ..."   Trevor Chaplin, /The Beiderbeck
> Affair/
> 
> Epimorphics Ltd, http://www.epimorphics.com
> Registered address: Court Lodge, 105 High Street, Portishead, Bristol BS20
> 6PT
> Epimorphics Ltd. is a limited company registered in England (number 7016688)

Re: Jena Eyeball [was: JENA-1541]

[ajs6f]

I can see two options: retire it from the docs, or set up a special doc area 
with lots of warnings on it for 
"unmaintained/experimental/use-at-your-own-risk/no releases" stuff.

We've talked about what to do with non-core stuff before, but this seems a bit 
different...

ajs6f

> On May 11, 2018, at 8:11 AM, Andy Seaborne <a...@apache.org> wrote:
> 
> Edward found Eyeball via:
> 
> https://jena.apache.org/documentation/tools/eyeball-getting-started.html
> 
> Eyeball is mentioned:
> 
> documentation/tools/index.mdtext
> documentation/tools/eyeball-guide.mdtext
> documentation/tools/eyeball-getting-started.mdtext
> documentation/tools/eyeball-manual.mdtext
> 
> The tools page has schemagen and eyeball on it.
> 
> eyeball-getting-started says "file a JIRA".
> 
> What should we do?
> If it is not released, do we retire it from the documentation?
> 
>Andy
> 
> 
> On 04/05/18 11:39, Edward (JIRA) wrote:
>> Edward created JENA-1541:
>> 
>>  Summary: Jena Eyeball - ant test fails with TestCase Error
>>  Key: JENA-1541
>>  URL: https://issues.apache.org/jira/browse/JENA-1541
>>  Project: Apache Jena
>>   Issue Type: Question
>>   Components: Eyeball, Jena
>> Reporter: Edward
>> Hello,
>> I'm trying to get Jena Eyeball running. I wanted to do the Tutorial on 
>> [here|[https://jena.apache.org/documentation/tools/eyeball-getting-started.html],]
>>  but it failed with following error:
>> {code:java}
>> [junit] Testcase: warning(junit.framework.TestSuite$1):FAILED
>> [junit] Class 
>> com.hp.hpl.jena.eyeball.inspectors.test.TestMoreOwlSyntaxInspector has no 
>> public constructor TestCase(String name) or TestCase()
>> [junit] junit.framework.AssertionFailedError: Class 
>> com.hp.hpl.jena.eyeball.inspectors.test.TestMoreOwlSyntaxInspector has no 
>> public constructor TestCase(String name) or TestCase()
>> [junit]
>> [junit]
>> BUILD FAILED
>> /**/Downloads/eyeball-2.3/build.xml:146: Test 
>> com.hp.hpl.jena.eyeball.inspectors.test.TestMoreOwlSyntaxInspector failed
>> {code}
>> The tutorial says if the ant test doesn't pass, I should file a Jira Issue. 
>> So that's what I am doing.
>> Help would be much appreciated!
>>  Greetings,
>> Edward
>> --
>> This message was sent by Atlassian JIRA
>> (v7.6.3#76005)

Re: parsing and matching triples

[ajs6f]

Forking a thread off to dev@:

Do we have a global policy about where null is accepted as a wildcard? I know 
it works in at least some places...

I would love to (over an appropriate period of time and with lots of warnings 
and deprecation and so forth) stop letting it be a wildcard and require code to 
use the actual wildcard objects.

ajs6f

> On May 8, 2018, at 11:51 AM, Andy Seaborne <a...@apache.org> wrote:
> 
> Barry,
> 
> As a general concept "matching" happens at different levels.
> 
> Triple.match corresponds to the matching done by Graph.find - RDF terms (URI, 
> bnode, literal) match exactly, and Node.ANY is a wildcard.
> 
> Triple t1 = Triple.ANY;
> Triple t2 = SSE.parseTriple("(:s :p :o)");
> t1.matches(t2) -> true
> t2.matches(t1) -> false
> 
> Variables are a concept for SPARQL - and matches usefully need to return 
> which variable matched which RDF Term.
> 
> Triple patterns match against graphs and return an iterator of ways they 
> match.
> 
> Consider cases like "?x ?p ?x" where the variables impose am additional shape.
> 
> If you want variable bindings, you could build a SPARQL query or wrap up some 
> of the internal code e.g.
> 
> /** Evaluate a triple pattern */
> private static QueryIterator match(Graph source, Triple pattern) {
>ExecutionContext execContext =
>  new ExecutionContext(ARQ.getContext(), source, null, null) ;
>QueryIterator chain = QueryIterRoot.create(execContext)
>chain = new QueryIterTriplePattern(chain, pattern, execContext) ;
>return chain ;
> }
> 
>Andy
> 
> On 08/05/18 09:21, Nouwt, B. (Barry) wrote:
>> Hi everybody,
>> I’m trying to reuse Apache Jena code that parses and matches triples. I’m 
>> currently looking at the SSE class’s parseTriple() method. This seems to fit 
>> my purpose for parsing a string representation of a triple into a triple 
>> object. I also noticed the following Javadoc on the Node.maches(Node) method:
>> Answer true iff this node accepts the other one as a match.
>> The default is an equality test; it is over-ridden in subclasses to
>> provide the appropriate semantics for literals, ANY, and variables.
>> Since this is exactly what I’m looking for, I’ve tried to match two triples 
>> using the matches() method, but it does not seem to work:
>> Triple t1 = SSE.parseTriple("(?s ?p ?o)");
>> Triple t2 = SSE.parseTriple("(test:subject test:predicate test:object)", pm);
>> t1.matches(t2)
>> The final statement returns false, while I would expect it to return true. 
>> Either, I’m missing something (which is completely realistic ), or I should 
>> use some other method to match two triples in the way described above.
>> Any help is appreciated!
>> Regards, Barry
>> This message may contain information that is not intended for you. If you 
>> are not the addressee or if this message was sent to you by mistake, you are 
>> requested to inform the sender and delete the message. TNO accepts no 
>> liability for the content of this e-mail, for the manner in which you use it 
>> and for damage of any kind resulting from the risks inherent to the 
>> electronic transmission of messages.

[GitHub] jena pull request #416: JENA-1545: ParserProfileWrapper and public checkTrup...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/416#discussion_r186517509
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/riot/system/ParserProfile.java ---
@@ -49,6 +49,12 @@
 /** Create a triple */
 public Triple createTriple(Node subject, Node predicate, Node object, 
long line, long col);
 
+/** Check a triple */
--- End diff --

Might be nice to include a Javadoc note declaring what to do for bad 
tuples. Presumably `throw` something, but is there a specific type to throw 
that would be best?


---

[GitHub] jena pull request #415: JENA-1543: QueryEngineHttp support for JSON queries.

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/415#discussion_r186387602
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/engine/http/QueryEngineHTTP.java 
---
@@ -538,13 +539,29 @@ else if ( 
actualContentType.equals(WebContent.contentTypeJSON))
 @Override
 public JsonArray execJson()
 {
-throw new NotImplemented("JSON queries not implemented for remote 
calls") ;
+checkNotClosed();
+HttpQuery httpQuery = makeHttpQuery();
+httpQuery.setAccept(WebContent.contentTypeJSON);
+InputStream in = httpQuery.exec();
+JsonValue v = JSON.parseAny(in);
+if ( ! v.isArray() )
+{}
+return v.getAsArray();
 }
 
 @Override
 public Iterator execJsonItems()
 {
-throw new NotImplemented("JSON queries not implemented for remote 
calls") ;
+// Non-streaming.
+// TODO Integrate with the JSON parser to stream the results. 
+JsonArray array = execJson().getAsArray();
+List x = new ArrayList<>(array.size());
+array.forEach(elt->{
+if ( ! elt.isObject()) 
--- End diff --

Another way:
```
return array.stream().peek(e -> if (!elt.isObject()) throw etc. )
 .map(JsonValue::getAsObject)
 .iterator();
```


---

[GitHub] jena issue #299: Turtle Star

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/299
  
+1 on keeping the "default" check as-is, but making the choice of calling 
it / changing it available via a wrapper.


---

Re: CMS diff: SPARQL S-Expressions (or "SPARQL Syntax Expressions")

[ajs6f]

Committed, thank you!

ajs6f

> On May 3, 2018, at 11:36 PM, Evgeniy Tsvigun <anonym...@apache.org> wrote:
> 
> Clone URL (Committers only):
> https://cms.apache.org/redirect?new=anonymous;action=diff;uri=http://jena.apache.org/documentation%2Fnotes%2Fsse.mdtext
> 
> Evgeniy Tsvigun
> 
> Index: trunk/content/documentation/notes/sse.mdtext
> ===
> --- trunk/content/documentation/notes/sse.mdtext  (revision 1830787)
> +++ trunk/content/documentation/notes/sse.mdtext  (working copy)
> @@ -58,7 +58,7 @@
> 
> A stack can be expressed as a list. And because we want to express the
> structure, and not express the operations on the structures, data
> -structures that operational meaning don't enter the picture. There are
> +structures with operational meaning don't enter the picture. There are
> no operations, no *push*, *pop* or *peek*.
> 
> Note that this is to *express* a data structure, not *encode* or
> 

Re: new branch for JPMS?

[ajs6f]

I'm not sure how this new branch is supposed to work. If it "just contains 
modules compliant with the JPMS and that is tested for JDK9+" but the current 
(Maven) modules don't have a one-to-one correspondence with usable JPMS 
modules, where are the "modules" for this new branch coming from?

Are you suggesting decomposing the entire package structure of the codebase and 
recomposing it in a branch? On what basis would packages be grouped?

ajs6f

> On Apr 26, 2018, at 2:58 PM, Christopher Johnson <chjohnso...@gmail.com> 
> wrote:
> 
> In reference to a recent PR (https://github.com/apache/jena/pull/401), I am
> seeking input on a process to move forward towards JDK9+ (and JPMS) support
> in Jena.  As consumers of jena apis, the applications that I am developing
> will require JPMS support going forward, so this is an important issue for
> my use case.
> 
> What I have noticed is that there will be breaking changes required to
> structure the code in a way that meets the JPMS "non-interference"
> specification.[1]  A specific problem occurs when a module does not contain
> a single root package (which is a typical condition in several jena
> modules).  This results in an "ambiguous module reference" error.  To
> resolve this ambiguity may require considerable refactoring (e.g. moving
> org.apache.jena.riot to org.apache.jena.arq.riot).  I surmise that it is
> not possible to make these changes and maintain compatibility for existing
> consumers
> 
> What I would like to propose is a new branch that just contains modules
> compliant with the JPMS and that is tested for JDK9+.  Artifacts produced
> from this branch could be distinguished from master artifacts with an
> appropriate appendix.
> 
> Thank you for your consideration,
> Christopher Johnson
> Scientific Associate
> Universitätsbibliothek Leipzig
> 
> [1] http://openjdk.java.net/projects/jigsaw/spec/reqs/#non-interference

[GitHub] jena issue #401: [JENA-1524] org.apache.jena.system is split by org.apache.j...

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/401
  
This, to my mind, is now well into the realm of 4.0-sized changes.


---

[GitHub] jena issue #390: Bugfix Tutorial 6: Missing "/" at the end of an URI resulte...

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/390
  
@kinow Yep, those Commons threads are exactly what brought it back to my 
mind. Okay, I suggest that we leave this particular issue as @kinow suggests 
(Jira ticket and go from there) and I will (sometime today or tomorrow) restart 
those threads on dev@ about GitBox, etc.


---

Re: Question about Jena Indexing

[ajs6f]

Hello, Anuj--

I may be missing something about your question, but wouldn't Model.add(Model) 
[1] work here?

ajs6f

[1] 
https://jena.apache.org/documentation/javadoc/jena/org/apache/jena/rdf/model/Model.html#add-org.apache.jena.rdf.model.Model-

> On Apr 24, 2018, at 7:14 AM, anuj kumar <anuj.gandh...@gmail.com> wrote:
> 
> Hi,
> I have a scenario, where I have an instance of a Model Object that contains
> multiple Statements.
> Now I want to index the statements in this Model in Jena Text ES indexing
> implementation.
> 
> *Question is*: How can i do that. :)
> 
> Until now I was indexing data by reading the data from a file into a model
> instance using RDFDataMgr which automatically binds the read with the index
> lifecycle. This was done like this:
> 
>> dataset.begin(ReadWrite.WRITE) ;
>> try {
>>Model m = dataset.getDefaultModel() ;
>>RDFDataMgr.read(m, file) ;
>>dataset.commit() ;
>> } finally { dataset.end() ; }
>> 
>> 
>> But now, instead of reading the triples from a file (or Inputstream), i
> want to  read the triples from a Model Instance.
> 
> Can someone help me with this please.
> 
> Thanks,
> Anuj Kumar
> 
> -- 
> *Anuj Kumar*

[GitHub] jena pull request #406: JENA-1532 | Added support for escaping special chara...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/406#discussion_r183508856
  
--- Diff: 
jena-text-es/src/main/java/org/apache/jena/query/text/es/TextIndexES.java ---
@@ -422,6 +422,27 @@ public EntityDefinition getDocDef() {
 }
 
 private String parse(String fieldName, String qs, String lang) {
+//Escape special characters if any in the query string
+qs = qs.replaceAll("\\:", ":")
--- End diff --

No need to change anything-- if you are confident from the impl side, let's 
go forward!


---

[GitHub] jena pull request #406: JENA-1532 | Added support for escaping special chara...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/406#discussion_r183494640
  
--- Diff: 
jena-text-es/src/main/java/org/apache/jena/query/text/es/TextIndexES.java ---
@@ -422,6 +422,27 @@ public EntityDefinition getDocDef() {
 }
 
 private String parse(String fieldName, String qs, String lang) {
+//Escape special characters if any in the query string
+qs = qs.replaceAll("\\:", ":")
--- End diff --

This may be a question of style, but might this read a bit better as a 
constant `Pattern`? (I don't know enough to think very usefully about the 
performance implications, but I would think a `Pattern` would be at _least_ as 
performant.)


---

[GitHub] jena issue #390: Bugfix Tutorial 6: Missing "/" at the end of an URI resulte...

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/390
  
Not to hijack the thread, but do we also want to open a ticket to get all 
of the relevant (tutorial) files into Git alongside the main codebase? Or 
should we just subsume that into the larger project of getting the site into 
Git?


---

Re: Build failed in Jenkins: Jena_Development_Test #2911

[ajs6f]

This seems a bit weird-- not being able to make directories for checkout?

Have we seen this before? 

ajs6f

> On Apr 21, 2018, at 12:49 PM, Apache Jenkins Server 
> <jenk...@builds.apache.org> wrote:
> 
> See 
> <https://builds.apache.org/job/Jena_Development_Test/2911/display/redirect>
> 
> --
> Started by an SCM change
> [EnvInject] - Loading node environment variables.
> Building remotely on H24 (ubuntu xenial) in workspace 
> <https://builds.apache.org/job/Jena_Development_Test/ws/>
> java.io.IOException: Failed to mkdirs: 
> <https://builds.apache.org/job/Jena_Development_Test/ws/>
>   at hudson.FilePath.mkdirs(FilePath.java:1170)
>   at hudson.model.AbstractProject.checkout(AbstractProject.java:1200)
>   at 
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>   at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>   at 
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>   at hudson.model.Run.execute(Run.java:1724)
>   at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
>   at hudson.model.ResourceController.execute(ResourceController.java:97)
>   at hudson.model.Executor.run(Executor.java:429)
> Retrying after 10 seconds
> java.io.IOException: Failed to mkdirs: 
> <https://builds.apache.org/job/Jena_Development_Test/ws/>
>   at hudson.FilePath.mkdirs(FilePath.java:1170)
>   at hudson.model.AbstractProject.checkout(AbstractProject.java:1200)
>   at 
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>   at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>   at 
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>   at hudson.model.Run.execute(Run.java:1724)
>   at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
>   at hudson.model.ResourceController.execute(ResourceController.java:97)
>   at hudson.model.Executor.run(Executor.java:429)
> Retrying after 10 seconds
> java.io.IOException: Failed to mkdirs: 
> <https://builds.apache.org/job/Jena_Development_Test/ws/>
>   at hudson.FilePath.mkdirs(FilePath.java:1170)
>   at hudson.model.AbstractProject.checkout(AbstractProject.java:1200)
>   at 
> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
>   at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
>   at 
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
>   at hudson.model.Run.execute(Run.java:1724)
>   at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
>   at hudson.model.ResourceController.execute(ResourceController.java:97)
>   at hudson.model.Executor.run(Executor.java:429)

[GitHub] jena pull request #404: JENA-1528: Expose HTTP client socketTimeout from Jen...

[ajs6f]

GitHub user ajs6f opened a pull request:

https://github.com/apache/jena/pull/404

JENA-1528: Expose HTTP client socketTimeout from Jena



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ajs6f/jena JENA-1528

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/404.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #404


commit d609c13ea33007c5d657e43c5b5c1230978a883f
Author: ajs6f <ajs6f@...>
Date:   2018-04-21T14:12:56Z

JENA-1528: Expose HTTP client socketTimeout from Jena




---

[GitHub] jena issue #401: [JENA-1524] org.apache.jena.system is split by org.apache.j...

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/401
  
That sounds like a set of changes (e.g. module renaming) much better suited 
for 4.0.


---

[GitHub] jena pull request #401: [JENA-1524] org.apache.jena.system is split by org.a...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/401#discussion_r182101777
  
--- Diff: 
apache-jena-osgi/jena-osgi/src/main/resources/META-INF/services/org.apache.jena.system.JenaSubsystemLifecycle
 ---
@@ -1,4 +1,4 @@
-org.apache.jena.system.InitJenaCore
+org.apache.jena.sys.InitJenaCore
--- End diff --

+1 to honesty. I hear tell it's the best policy :) In all seriousness, the 
alternative seems to me to build up debt.


---

[GitHub] jena pull request #396: JENA-1520: tdb2.tdbstats: cmd and fix for rdf:type

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/396#discussion_r181458378
  
--- Diff: 
jena-db/jena-tdb2/src/main/java/org/apache/jena/tdb2/solver/stats/StatsCollectorNodeId.java
 ---
@@ -18,39 +18,39 @@
 
 package org.apache.jena.tdb2.solver.stats;
 
-import java.util.HashMap ;
-import java.util.Map ;
+import java.util.HashMap;
+import java.util.Map;
 
-import org.apache.jena.graph.Node ;
-import org.apache.jena.sparql.graph.NodeConst ;
+import org.apache.jena.graph.Node;
+import org.apache.jena.sparql.graph.NodeConst;
 import org.apache.jena.tdb2.store.NodeId;
 import org.apache.jena.tdb2.store.nodetable.NodeTable;
 
 /** Statistics collector, aggregates based on NodeId */
-public class StatsCollectorNodeId extends StatsCollectorBase
-{
-private NodeTable nodeTable ;
-
-public StatsCollectorNodeId(NodeTable nodeTable)
-{
-super(findRDFType(nodeTable)) ;
-this.nodeTable = nodeTable ;
+public class StatsCollectorNodeId extends StatsCollectorBase {
+private NodeTable nodeTable;
+
+public StatsCollectorNodeId(NodeTable nodeTable) {
+super(findRDFType(nodeTable));
+this.nodeTable = nodeTable;
 }
-
-private static NodeId findRDFType(NodeTable nodeTable2)
-{
-return nodeTable2.getAllocateNodeId(NodeConst.nodeRDFType) ;
+
+private static NodeId findRDFType(NodeTable nodeTable) {
+// It may not exist.
+NodeId nodeId = nodeTable.getNodeIdForNode(NodeConst.nodeRDFType);
+if ( NodeId.isDoesNotExist(nodeId) )
--- End diff --

Jut for my own education-- am I right in understanding this to be the fix 
for the "no type triples in the data" problem?


---

[GitHub] jena pull request #398: JENA-1523: Allow internal variables names in variabl...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/398#discussion_r181455157
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/riot/tokens/TokenizerText.java ---
@@ -739,16 +739,21 @@ private String readWordSub(boolean 
leadingDigitAllowed, boolean leadingSignAllow
 return readCharsAnd(leadingDigitAllowed, leadingSignAllowed, 
extraCharsWord, false);
 }
 
-static private char[] extraCharsVar = new char[]{'_', '.', '-', '?', 
'@', '+'};
+// This array adds the other characters that can occurs in an internal 
variable name.
+// Variable a creeated with SPARQL-illegal syntax to encsure they do 
not clash with
--- End diff --

typos, can be line-changed to:
```
Variables are created with SPARQL-illegal syntax to ensure that they do not 
clash with
```


---

Re: Old tickets.

[ajs6f]

> Yet open tickets do in some ways suggest they may happen sometime which isn't 
> the case.

Agree 100%.

I'm pretty happy closing tickets with a status that indicates just what you are 
saying; "This is so old that we can no longer effectively work it. If you 
disagree, please open a new up-to-date ticket. " Maybe we can have a closed 
ticket status like "OBSOLETE" or something like that?

Adam

> On Apr 13, 2018, at 11:58 AM, Andy Seaborne  wrote:
> 
> The number of unresolved tickets has climbed a bit recently so I cheated and 
> went and cleaned up some old ones to keep the count down.  The batch today 
> were over 4 years old (arbitrary choice) and look to be done, superseded or 
> in some way no longer relevant.
> 
> I thought they were all clear-cut but do reopen them if you see ticket 
> differently.
> 
> Generally, what to do about old tickets?
> 
> Some are still relevant, some are addressed elsewhere, some have drifted to 
> the point of being difficult to understand.  Where an old ticket that isn't 
> getting any interest (there are at least 5 SDB tickets), I don't see that 
> having it open serves much purpose; it isn't a promise to do anything about 
> it. If new information comes along, it is likely in a new ticket. Yet open 
> tickets do in some ways suggest they may happen sometime which isn't the case.
> 
> Thoughts?
> 
>Andy
> 

Re: Contribution proposal for Jena: support of a datatype for quantity values

[ajs6f]

We're (well, Andy is) working on 3.7.0 now. We've been trying to maintain a 
6-month or so release cadence, so you've hit a really good time to begin this 
work. That having been said, I don't think anyone would say that we are 
especially stringent about it, so I wouldn't worry too much about the timing 
myself.

ajs6f

> On Apr 6, 2018, at 9:36 AM, Maxime Lefrançois <maxime.lefranc...@emse.fr> 
> wrote:
> 
> Well,
> 
> I think I have a pretty clear idea how I would do this. We would end up
> using a registery like for custom functions or datatypes.
> That registry would contain an ordered list of SPARQL operator handlers,
> pre-filled by one for handling XSD datatypes.
> 
> I am currently requesting the right to fill the Apache individual
> contributor license agreement.
> 
> What would be the timeline if we wanted this shipped in the next release?
> 
> Best,
> Maxime
> 
> Le mar. 3 avr. 2018 à 15:30, ajs6f <aj...@apache.org> a écrit :
> 
>> I agree. I can imagine plenty of use cases for such a powerful pair of
>> extension points.
>> 
>> Maxime, how can we help you attack that work? Is there a design that is
>> already clear to you? Are there any blockers we can help remove?
>> 
>> ajs6f
>> 
>>> On Mar 28, 2018, at 5:08 AM, Rob Vesse <rve...@dotnetrdf.org> wrote:
>>> 
>>> I think work towards Option 2 would be the most valuable to the community
>>> 
>>> 
>>> 
>>> The SPARQL specification allows for the overloading of any
>> operator/expression where the spec currently defines the evaluation to be
>> an error so extending operators is a natural and valid extension point to
>> provide
>>> 
>>> 
>>> 
>>> The Terms of Use for UCUM would probably need us to obtain a licensing
>> assessment from Apache Legal as it is a non-standard OSS license even if
>> the code that implements it is under BSD (which is fine from an Apache
>> perspective).  Therefore having a well defined extension mechanism and then
>> having UCUM support live outside Apache Jena that as an extension
>> implementation maintained by yourself would be the easiest approach
>>> 
>>> 
>>> 
>>> Rob
>>> 
>>> 
>>> 
>>> From: Maxime Lefrançois <maxime.lefranc...@emse.fr>
>>> Reply-To: <dev@jena.apache.org>
>>> Date: Wednesday, 28 March 2018 at 09:29
>>> To: <dev@jena.apache.org>
>>> Subject: Re: Contribution proposal for Jena: support of a datatype for
>> quantity values
>>> 
>>> 
>>> 
>>> Dear all,
>>> 
>>> 
>>> 
>>> Happy to see you are interested the UCUM datatypes !
>>> 
>>> 
>>> 
>>> Ok so let's dive in the technical details.
>>> 
>>> 
>>> 
>>> # Compare Jena 3.6.0 and Jena 3.6.0-ucum
>>> 
>>> 
>>> 
>>> 
>> https://github.com/apache/jena/compare/master...OpenSensingCity:jena-3.6.0-ucum
>>> 
>>> 
>>> 
>>> # Modules, dependencies, licences
>>> 
>>> 
>>> 
>>> Two modules forked so far: jena-core and jena-arq.
>>> 
>>> One dependency added to jena-core (after a minor change I made today):
>>> 
>>> 
>>> 
>>> systems.uom:systems-ucum-java8:0.7.2
>>> 
>>> -> BSD license of systems-uom,
>>> 
>>>and license of UCUM http://unitsofmeasure.org/trac/wiki/TermsOfUse
>>> 
>>> 
>>> 
>>> --> this use implementation of JSR 363 indeed - Units of Measurement API
>>> 
>>> (see attached for the transitive dependencies, all from
>> https://github.com/unitsofmeasurement )
>>> 
>>> 
>>> 
>>> # External module ?
>>> 
>>> 
>>> 
>>> I would have been happy to develop a separate extension of Jena for the
>> UCUM datatypes.
>>> 
>>> One of the main reasons why this is not possible was pointed out by Andy:
>>> 
>>> I had to add a new value space VSPACE_QUANTITY to overload the SPARQL
>> operators '<>=' and arithmetic functions '+-*/'.
>>> 
>>> 
>>> 
>>> Indeed, there are two parts: the necessary extensions for operators, and
>> the units themselves.
>>> 
>>> 
>>> 
>>> We could choose some other unit system than UCUM, but UCUM is very
>> comprehensive and has different implementations in different programming
>> languages. It would be possible to impleme

Re: [VOTE] Apache Jena 3.7.0 RC2

[ajs6f]

I just had a successful build with:

mvn clean install -Dmaven.repo.local=/tmp/repo -Dmaven.artifact.threads=10

I find those system properties useful in situations like this (when Claude is 
seeing weirdness in the build and it seems to be related to artifact 
management). The first uses a clean empty Maven repo [1], the second configures 
Maven to download artifacts with 10 threads (instead of the default 5) which 
makes filling that repo a good bit faster [2].

I'll vote today or tomorrow once I have a chance to do some checks for signing, 
checksums, etc.

ajs6f

[1] 
https://maven.apache.org/guides/mini/guide-configuring-maven.html#Configuring_your_Local_Repository

I'm not sure why the form I give above in the system property works, because 
it's not quite in alignment with the settings.xml form, but it does work.

[2] 
https://maven.apache.org/guides/mini/guide-configuring-maven.html#Configuring_Parallel_Artifact_Resolution

> On Apr 6, 2018, at 8:31 AM, Claude Warren <cla...@xenei.com> wrote:
> 
> +1
> 
> I have the same build issue but it is workable and only seems to affect me.
> 
> On Thu, Apr 5, 2018 at 6:29 PM, Andy Seaborne <a...@apache.org> wrote:
> 
>> 
>> Please vote to approve this release:
>>> 
>>>[ ] +1 Approve the release
>>>[ ]  0 Don't care
>>>[ ] -1 Don't release, because ...
>>> 
>> 
>> +1
>> 
>> 
>> 
>>> This vote will be open until at least
>>> 
>>>2018-04-08 19:00 UTC
>>> 
>> 
> 
> 
> -- 
> I like: Like Like - The likeliest place on the web
> <http://like-like.xenei.com>
> LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Contribution proposal for Jena: support of a datatype for quantity values

[ajs6f]

I agree. I can imagine plenty of use cases for such a powerful pair of 
extension points.

Maxime, how can we help you attack that work? Is there a design that is already 
clear to you? Are there any blockers we can help remove?

ajs6f

> On Mar 28, 2018, at 5:08 AM, Rob Vesse <rve...@dotnetrdf.org> wrote:
> 
> I think work towards Option 2 would be the most valuable to the community
> 
> 
> 
> The SPARQL specification allows for the overloading of any 
> operator/expression where the spec currently defines the evaluation to be an 
> error so extending operators is a natural and valid extension point to provide
> 
> 
> 
> The Terms of Use for UCUM would probably need us to obtain a licensing 
> assessment from Apache Legal as it is a non-standard OSS license even if the 
> code that implements it is under BSD (which is fine from an Apache 
> perspective).  Therefore having a well defined extension mechanism and then 
> having UCUM support live outside Apache Jena that as an extension 
> implementation maintained by yourself would be the easiest approach
> 
> 
> 
> Rob
> 
> 
> 
> From: Maxime Lefrançois <maxime.lefranc...@emse.fr>
> Reply-To: <dev@jena.apache.org>
> Date: Wednesday, 28 March 2018 at 09:29
> To: <dev@jena.apache.org>
> Subject: Re: Contribution proposal for Jena: support of a datatype for 
> quantity values
> 
> 
> 
> Dear all,
> 
> 
> 
> Happy to see you are interested the UCUM datatypes !
> 
> 
> 
> Ok so let's dive in the technical details. 
> 
> 
> 
> # Compare Jena 3.6.0 and Jena 3.6.0-ucum
> 
> 
> 
> https://github.com/apache/jena/compare/master...OpenSensingCity:jena-3.6.0-ucum
>  
> 
> 
> 
> # Modules, dependencies, licences
> 
> 
> 
> Two modules forked so far: jena-core and jena-arq.
> 
> One dependency added to jena-core (after a minor change I made today):
> 
> 
> 
> systems.uom:systems-ucum-java8:0.7.2
> 
> -> BSD license of systems-uom,
> 
> and license of UCUM http://unitsofmeasure.org/trac/wiki/TermsOfUse 
> 
> 
> 
> --> this use implementation of JSR 363 indeed - Units of Measurement API
> 
> (see attached for the transitive dependencies, all from 
> https://github.com/unitsofmeasurement )
> 
> 
> 
> # External module ?
> 
> 
> 
> I would have been happy to develop a separate extension of Jena for the UCUM 
> datatypes.
> 
> One of the main reasons why this is not possible was pointed out by Andy:
> 
> I had to add a new value space VSPACE_QUANTITY to overload the SPARQL 
> operators '<>=' and arithmetic functions '+-*/'. 
> 
> 
> 
> Indeed, there are two parts: the necessary extensions for operators, and the 
> units themselves. 
> 
> 
> 
> We could choose some other unit system than UCUM, but UCUM is very 
> comprehensive and has different implementations in different programming 
> languages. It would be possible to implement UCUM datatypes in other 
> RDF-SPARQL engines.
> 
> 
> 
> # possible directions 
> 
> 
> 
> I see three main possible directions of work there: 
> 
> 
> 
> 1. work on the proposal as and potentially integrate it completely
> 
> 2. work on jena-core and jena-arq to make the definition of new datatypes and 
> the overloading of operators as easy as the definition of new custom 
> functions --> so that I can easily implement UCUM datatypes as an extension 
> (and not a fork)
> 
> 3. add VSPACE_QUANTITY value space and NodeValueQuantity in jena-arq, and 
> externalize the support for the UCUM systems of unit in an external module
> 
> 
> 
> Best,
> 
> Maxime
> 
> 
> 
> Le mar. 27 mars 2018 à 17:16, Andy Seaborne <a...@apache.org> a écrit :
> 
> Extending the operators for SPARQL is a new value space VSPACE_QUANTITY.
> 
> See (comparison):
> 
> https://github.com/OpenSensingCity/jena-ucum/blob/jena-3.6.0-ucum/jena-arq/src/main/java/org/apache/jena/sparql/expr/NodeValue.java#L566
> 
> and (multiply)
> 
> https://github.com/OpenSensingCity/jena-ucum/blob/jena-3.6.0-ucum/jena-arq/src/main/java/org/apache/jena/sparql/expr/nodevalue/NodeValueOps.java#L283
> 
> with a new NodeValueQuantity for javax.measure.Quantity
> 
> I'm seeing this a "one dimensional units" - a quantity and a unit.
> 
> Even then, there are two part - the necessary extensions for operators
> and the units themselves to allow for other unit systems (?).
> 
> There are new dependencies in jena-arq and jena-core.
> 
> http://unitsofmeasurement.github.io/
> JSR 363 - Units of Measurement API
> BSD-license
> 
> and an old version of something is on central:
> 

Re: [VOTE] Apache Jena 3.7.0 RC1

[ajs6f]

> Please vote to approve this release:
> 
>   [ ] +1 Approve the release

+1

>   [ ]  0 Don't care
>   [ ] -1 Don't release, because ...

> + does everything work on OS X?

Yes.

> + are the GPG signatures fine?

Yes.

> + is there a source archive?

Yes.

> + can the source archive really be built?

Yes.

ajs6f

> On Mar 29, 2018, at 2:28 PM, Andy Seaborne <a...@apache.org> wrote:
> 
> Hi,
> 
> Here is a vote on a release of Jena 3.7.0.
> 
> This is the first proposed candidate for a 3.7.0 release.
> 
> There are process changes.
> 
> Deadline:
> 
>   2018-04-01 22:00 UTC
> 
> April 1st!
> 
>  Process Changes
> 
> 1/
> MD5 files are being discouraged because MD5 is not secure.  Projects are now 
> asked to not publish md5.
> 
> There are no md5 files in the proposed dist/jena area - files on Apache 
> hardware.
> 
> There are sha1 and sha512 checksums.
> * The sha512 is in Linux sha512sum checkable format.
> * The sha1 is whatever maven generated and is the same as will go to maven 
> central.
> 
> Having the sha1 ties the dist/jena artifacts to maven central (as does the 
> .asc).
> 
> There are md5 and sha1 in the proposes maven repo staging area for sending to 
> maven central. That part of maven is hardwired to md5/sha1 still.
> 
> There's a script to setup the sha512.
> 
> 2/
> To establish the proof chain for signed artifacts in /dist/project/, I have 
> been asked to try out the new META files.
> 
> https://checker.apache.org/doc/README.html#ch-meta
> 
> There are two files
> 
> /dist/jena/META
> /dist/jena/META.asc
> 
> META says who signs what, and is itself signed by the PMC chair.
> 
>  Release changes
> 
> 55 JIRA:
> https://s.apache.org/jena-3.7.0-jira
> 
> == Significant Changes
> 
> ** Java9: Building and running on a Java9 platform is supported
> 
> JENA-1461 - Allow ARQ custom functions to be written in JavaScript
> 
> JENA-1389 - Return `this` rather than `void` from Dataset (API change)
> JENA-1495 - Return Model from PrefixMapping methods (API change)
> 
> JENA-1458, JENA-1483 - Transaction Promotion
> 
> JENA-1453 - Lucene indexes using a graph field are smaller
> 
> JENA-1490 - Working with Blank Nodes with Fuseki
> 
> == Upgrades to libraries (runtime dependencies):
> 
> No dependency changes.
> 
>  Release Vote
> 
> Everyone, not just committers, is invited to test and vote.
> Please download and test the proposed release.
> 
> Proposed dist/ area:
>  https://dist.apache.org/repos/dist/dev/jena/
> 
> Keys:
>  https://svn.apache.org/repos/asf/jena/dist/KEYS
> 
> Staging repository:
>  https://repository.apache.org/content/repositories/orgapachejena-1022/
> 
> Git commit (browser URL):
> https://git1-us-west.apache.org/repos/asf?p=jena.git;a=commit;h=d4e7063e
> 
> Git Commit Hash:
>  d4e7063e7a6db8ce77699bd0388e1a1bd6816626
> 
> Git Commit Tag:
> jena-3.7.0-rc1
> 
> Please vote to approve this release:
> 
>   [ ] +1 Approve the release
>   [ ]  0 Don't care
>   [ ] -1 Don't release, because ...
> 
> This vote will be open until at least
> 
>   2018-04-01 22:00 UTC
> 
> If you expect to check the release but the time limit does not work
> for you, please email within the schedule above with an expected time
> and we can extend the vote period.
> 
> Thanks,
> 
> Andy
> 
> Checking needed:
> 
> + does everything work on Linux?
> + does everything work on MS Windows?
> + does everything work on OS X?
> + are the GPG signatures fine?
> + are the checksums correct?
> + is there a source archive?
> 
> + can the source archive really be built?
> (NB This requires a "mvn install" first time)
> + is there a correct LICENSE and NOTICE file in each artifact
> (both source and binary artifacts)?
> + does the NOTICE file contain all necessary attributions?
> + have any licenses of dependencies changed due to upgrades?
>  if so have LICENSE and NOTICE been upgraded appropriately?
> + does the tag/commit in the SCM contain reproducible sources?

Re: Build 3.7.0

[ajs6f]

I see the logic here, but mightn't it be good to add some config for this to 
override it in certain settings?

ajs6f

> On Mar 29, 2018, at 1:06 PM, Rob Vesse <rve...@dotnetrdf.org> wrote:
> 
> Yeah it's a security think
> 
> In production it is recommended not to include the Server header because it 
> discloses the software you are running making you more easily targeted if 
> vulnerabilities are known in specific servers
> 
> Since most people will use stable releases for their production environments 
> this is the default behaviour when not using a development i.e. SNAPSHOT build
> 
> Rob
> 
> On 29/03/2018, 17:45, "Claude Warren" <cla...@xenei.com> wrote:
> 
>Fuseki was only adding Server header when SNAPSHOT was in the version?
> 
>On Thu, Mar 29, 2018 at 4:40 PM, Andy Seaborne <a...@apache.org> wrote:
> 
>> Security !!!
>> 
>> The difference is that the version did not have "SNAPSHOT" in it
>> ...
>> which means Fuseki does not add a "Server:" header in responses
>> ...
>> and the "is it Fuseki?" test failed.
>> 
>>Andy
>> 
>> 
>> On 29/03/18 14:13, Andy Seaborne wrote:
>> 
>>> Could be last resort.
>>> 
>>> In the release plugin, maven calls maven recursively.
>>> 
>>> And the integration tests are 5 minutes into the build after several
>>> modules.
>>> 
>>> Andy
>>> 
>>> On 29/03/18 14:01, Claude Warren wrote:
>>> 
>>>> A quick look at the maven script show that there is a MAVEN_DEBUG_OPTS
>>>> environment variable that it uses.  You could cause the mvn startup to
>>>> stop
>>>> and wait for the eclipse debugger to attach to it.  Not sure what you
>>>> would
>>>> be stopping on though after that.
>>>> 
>>>> Claude
>>>> 
>>>> On Thu, Mar 29, 2018 at 1:51 PM, Andy Seaborne <a...@apache.org> wrote:
>>>> 
>>>> 
>>>>> 
>>>>> On 29/03/18 13:41, Claude Warren wrote:
>>>>> 
>>>>> just to double check, do you start with a clean?
>>>>>> 
>>>>>> 
>>>>> mvn clean? Yes.
>>>>> 
>>>>> Also started with a completely empty maven repo this morning.
>>>>> 
>>>>> And I've rebooted the machine :-)
>>>>> 
>>>>> Might be an issue with an older/newer version appearing somewhere.
>>>>> 
>>>>>> 
>>>>>> 
>>>>> The release plugin checks for SNAPSHOTS.
>>>>> 
>>>>> So the dry/real difference is version numbers have been advanced.
>>>>> And I can't run this in the Eclipse debugger.
>>>>> 
>>>>> Going old school with System.err.println.  Bad memories.
>>>>> 
>>>>> Andy
>>>>> 
>>>>> 
>>>>> On Thu, Mar 29, 2018 at 1:01 PM, Andy Seaborne <a...@apache.org> wrote:
>>>>> 
>>>>>> 
>>>>>> I've started to do the release.
>>>>>> 
>>>>>>> 
>>>>>>> However, I've encountered a test failure that only happens with
>>>>>>> "release:prepare". The -DdryRun=true is fine, and it has never
>>>>>>> occurred
>>>>>>> on
>>>>>>> Jenkins.
>>>>>>> 
>>>>>>> What's more, it's an NPE and the line reported can't NPE
>>>>>>> 
>>>>>>> line 57, TestRDFConnectionFusekiBinary.java which is :
>>>>>>> 
>>>>>>> assertTrue(FusekiLib.isFuseki(dsURL));
>>>>>>> 
>>>>>>> no object access!
>>>>>>> 
>>>>>>> As it only when running the release for real, the only way to
>>>>>>> investigate
>>>>>>> is changing master and the release plugin may demand that changes are
>>>>>>> pushed to origin so there may be some commit-foo happening.
>>>>>>> 
>>>>>>>  Andy
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
> 
> 
>-- 
>I like: Like Like - The likeliest place on the web
><http://like-like.xenei.com>
>LinkedIn: http://www.linkedin.com/in/claudewarren
> 
> 
> 
> 
> 

Re: Contribution proposal for Jena: support of a datatype for quantity values

[ajs6f]

Bruno raises an interesting question-- would this contribution have any effect 
(or should it) on jena-spatial? Would it be either necessary or if not, 
appropriate to integrate there? (I'm particularly interested in this because it 
might help decide between core and an extension.)


ajs6f

> On Mar 26, 2018, at 5:40 PM, Bruno P. Kinoshita <ki...@apache.org> wrote:
> 
> Hi Maxime,
> Don't know whether it would be best as part of jena core or in an extension, 
> but sounds very interesting! Will let others comment on this.
> At work, one item in my backlog is to replace jscience by jsr363 - Units of 
> Measurement  
> |  
> |   
> |   
> |   ||
> 
>   |
> 
>  |
> |  
> |   |  
> Units of Measurement
> 
> Units of Measurement provides a set of APIs and services for handling units 
> and quantities.
>  |   |
> 
>  |
> 
>  |
> 
> 
> We use it for weather forecast and GIS, with things like wind speed, rain 
> amount, etc.
> I think another GIS library that we use did the switch as well (some OGC lib 
> I think).
> Perhaps it would be nice to consider taking a look at their api for 
> compatibility with other systems.
> CheersBruno
> 
> Sent from Yahoo Mail on Android 
> 
>  On Tue, 27 Mar 2018 at 2:07, Maxime Lefrançois<maxime.lefranc...@emse.fr> 
> wrote:   Dear all,
> 
> I am Associate Professor at MINES Saint-Étienne, France, working on
> Semantic Web and Linked Data. I'd like to let you know about our
> project *Custom
> Datatypes for Quantity Values*[1], that leverages the Unified Code of Units
> of Measures, a code system intended to include all units of measures being
> contemporarily used in international science, engineering, and business.
> Using our UCUM Datatypes, one can encode and query quantity values in a
> lightweight manner:
> 
> PREFIX cdt: <http://w3id.org/lindt/custom_datatypes#>
> PREFIX ex: <http://example.org/>
> 
> SELECT ?value1 ?value2 ?result
> WHERE{
>   VALUES ( ?value1 ?value2 ) {
> ( "1.0 m/s"^^cdt:speed "2 s"^^cdt:time )
>   }
>   BIND( ?value1 * ?value2 AS ?result )
> }
> 
> Results in
> 
> --
> | value1  | value2  | result  |
> ==
> | "1.0 m/s"^^cdt:speed | "2 s"^^cdt:time  | "2.0 m"^^cdt:length  |
> 
> See our demonstration online [2].
> It uses *a fork of Jena where we implemented UCUM datatypes* [3] (in
> jena-core and jena-arq, with several unit tests) our implementation uses
> the recent JSR 385, Units of Measurement API 2.0, and the UCUM extension
> [4].
> 
> This is not the first project I develop into/using Jena.
> - I forked it to Supporting Arbitrary Custom Datatypes in RDF and SPARQL
> fetching some Javascript definition at the URI of the datatype [5]
> - I develop SPARQL-Generate, an extension of SPARQL implemented on ARQ to
> generate RDF from web documents in XML, JSON, CSV, HTML, CBOR, and plain
> text with regular expressions  [6]
> 
> 
> If you agree we me that supporting UCUM datatypes would be a nice addition
> to Apache Jena and a nice contribution to the Semantic Web community, I
> would be willing to help to integrate our contribution to other modules
> (with jena-tdb, ... ), and help maintaining it in the future.
> 
> Best regards,
> Maxime Lefrançois,
> Associate Professor, MINES Saint-Étienne
> 
> [1] - http://w3id.org/lindt/custom_datatypes#
> [2] - http://w3id.org/lindt/playground.html?example=05-Multiply
> [3] - http://w3id.org/lindt/custom_datatypes#implementation
> [4] -
> https://github.com/unitsofmeasurement/uom-systems/tree/master/ucum-java8
> [5] - https://ci.mines-stetienne.fr/lindt/spec.html
> [6] - https://ci.mines-stetienne.fr/sparql-generate/  

Re: Contribution proposal for Jena: support of a datatype for quantity values

[ajs6f]

ajs6f

> On Mar 26, 2018, at 5:40 PM, Bruno P. Kinoshita <ki...@apache.org> wrote:
> 
> Hi Maxime,
> Don't know whether it would be best as part of jena core or in an extension, 
> but sounds very interesting! Will let others comment on this.
> At work, one item in my backlog is to replace jscience by jsr363 - Units of 
> Measurement  
> |  
> |   
> |   
> |   ||
> 
>   |
> 
>  |
> |  
> |   |  
> Units of Measurement
> 
> Units of Measurement provides a set of APIs and services for handling units 
> and quantities.
>  |   |
> 
>  |
> 
>  |
> 
> 
> We use it for weather forecast and GIS, with things like wind speed, rain 
> amount, etc.
> I think another GIS library that we use did the switch as well (some OGC lib 
> I think).
> Perhaps it would be nice to consider taking a look at their api for 
> compatibility with other systems.
> CheersBruno
> 
> Sent from Yahoo Mail on Android 
> 
>  On Tue, 27 Mar 2018 at 2:07, Maxime Lefrançois<maxime.lefranc...@emse.fr> 
> wrote:   Dear all,
> 
> I am Associate Professor at MINES Saint-Étienne, France, working on
> Semantic Web and Linked Data. I'd like to let you know about our
> project *Custom
> Datatypes for Quantity Values*[1], that leverages the Unified Code of Units
> of Measures, a code system intended to include all units of measures being
> contemporarily used in international science, engineering, and business.
> Using our UCUM Datatypes, one can encode and query quantity values in a
> lightweight manner:
> 
> PREFIX cdt: <http://w3id.org/lindt/custom_datatypes#>
> PREFIX ex: <http://example.org/>
> 
> SELECT ?value1 ?value2 ?result
> WHERE{
>   VALUES ( ?value1 ?value2 ) {
> ( "1.0 m/s"^^cdt:speed "2 s"^^cdt:time )
>   }
>   BIND( ?value1 * ?value2 AS ?result )
> }
> 
> Results in
> 
> --
> | value1  | value2  | result  |
> ==
> | "1.0 m/s"^^cdt:speed | "2 s"^^cdt:time  | "2.0 m"^^cdt:length  |
> 
> See our demonstration online [2].
> It uses *a fork of Jena where we implemented UCUM datatypes* [3] (in
> jena-core and jena-arq, with several unit tests) our implementation uses
> the recent JSR 385, Units of Measurement API 2.0, and the UCUM extension
> [4].
> 
> This is not the first project I develop into/using Jena.
> - I forked it to Supporting Arbitrary Custom Datatypes in RDF and SPARQL
> fetching some Javascript definition at the URI of the datatype [5]
> - I develop SPARQL-Generate, an extension of SPARQL implemented on ARQ to
> generate RDF from web documents in XML, JSON, CSV, HTML, CBOR, and plain
> text with regular expressions  [6]
> 
> 
> If you agree we me that supporting UCUM datatypes would be a nice addition
> to Apache Jena and a nice contribution to the Semantic Web community, I
> would be willing to help to integrate our contribution to other modules
> (with jena-tdb, ... ), and help maintaining it in the future.
> 
> Best regards,
> Maxime Lefrançois,
> Associate Professor, MINES Saint-Étienne
> 
> [1] - http://w3id.org/lindt/custom_datatypes#
> [2] - http://w3id.org/lindt/playground.html?example=05-Multiply
> [3] - http://w3id.org/lindt/custom_datatypes#implementation
> [4] -
> https://github.com/unitsofmeasurement/uom-systems/tree/master/ucum-java8
> [5] - https://ci.mines-stetienne.fr/lindt/spec.html
> [6] - https://ci.mines-stetienne.fr/sparql-generate/  

[GitHub] jena pull request #114: JENA-632: Generate JSON from SPARQL directly

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/114#discussion_r175918594
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/engine/JsonIterator.java ---
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.engine;
+
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+
+import org.apache.jena.atlas.json.JsonObject;
+import org.apache.jena.atlas.json.JsonValue;
+import org.apache.jena.atlas.lib.Closeable;
+
+import org.apache.jena.graph.Node;
+import org.apache.jena.sparql.core.Var;
+import org.apache.jena.sparql.engine.binding.Binding;
+import org.apache.jena.sparql.lib.RDFTerm2Json;
+
+/** A JSON iterator for JsonObject's, that wraps a QueryIterator, and a 
list
+ * of result variables.
+ */
+public class JsonIterator implements Iterator
+{
--- End diff --

I've been planning to offer a low-interruption plan for some large-scale 
reformatting later this spring, so no harm in leaving this.


---

[GitHub] jena issue #385: defined filters and tokenizers for ConfigurableAnalyzer

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/385
  
To be clear, what is here at Github is essentially a read-only mirror. 
What's at Apache is the real deal. We've talked about reversing that mirroring 
relationship (so that we could merge PRs here at Github) but there are other 
consequences to so doing, mostly involving integrations with other services 
like Jira.


---

[GitHub] jena issue #385: defined filters and tokenizers for ConfigurableAnalyzer

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/385
  
Hey, @xristy, you want to push instead to the `master` branch at Apache:

https://git-wip-us.apache.org/repos/asf/jena.git


---

Re: git.apache.org issue?

[ajs6f]

I had no trouble doing that checkout a few moments ago, but I am on $job 
networking with very good throughput and I work only a few miles from Ashburn, 
Virginia (more data centers than you can shake a stick at). I was getting 
between 10 and 12 MiB/s. The total transfer was 352.82 MiB, so maybe you could 
try downloading a large file of about that size from somewhere else in Apache 
infrastructure to see if it's your connection to Apache.

Otherwise, you can always checkout the Github mirror, which is the same code: 
g...@github.com:apache/jena.git

 
ajs6f

> On Mar 19, 2018, at 7:10 AM, Claude Warren <cla...@xenei.com> wrote:
> 
> I have tried to execute
> 
> git clone git://git.apache.org/jena.git
> 
> four times and each time I get:
> 
> Cloning into 'jena'...
> remote: Counting objects: 349618, done.
> remote: Compressing objects: 100% (64022/64022), done.
> fatal: The remote end hung up unexpectedly23.70 MiB | 275.00 KiB/s
> fatal: early EOF
> fatal: index-pack failed
> 
> and no jena directory created.
> 
> I have plenty of space so I am wondering if there is an issue on the apache
> server.  Though perhaps  the issue is with my ISP.
> 
> Claude
> 
> 
> 
> -- 
> I like: Like Like - The likeliest place on the web
> <http://like-like.xenei.com>
> LinkedIn: http://www.linkedin.com/in/claudewarren

Re: number of git clones?

[ajs6f]

This is a great bunch of advice from Rob. The only thing I want to add to it is 
that I was recently hepped to a really useful tool that Github has created:

https://github.com/github/hub

It's a drop-in replacement for `git`, so all of Rob's notes below (and anything 
else you find to work with git) will keep working, no change. But it offers 
some additional commands for working with Github-based projects.

For example, if I wanted to take a look at Chris' recent PR for JENA-1506, I 
can just type:

git checkout https://github.com/apache/jena/pull/385

(because I have `hub` aliased to `git`) and hub handles the bookkeeping 
(fetching, pulling, assigning branch names, etc.) to give me:

➜  jena git:(master) git checkout https://github.com/apache/jena/pull/385
remote: Counting objects: 67, done.
remote: Compressing objects: 100% (35/35), done.
remote: Total 67 (delta 18), reused 56 (delta 8), pack-reused 1
Unpacking objects: 100% (67/67), done.
From git://github.com/BuddhistDigitalResourceCenter/jena
 * [new branch]JENA-1506-PR -> 
BuddhistDigitalResourceCenter/JENA-1506-PR
Branch BuddhistDigitalResourceCenter-JENA-1506-PR set up to track remote branch 
JENA-1506-PR from BuddhistDigitalResourceCenter.
Switched to a new branch 'BuddhistDigitalResourceCenter-JENA-1506-PR'
➜  jena git:(BuddhistDigitalResourceCenter-JENA-1506-PR)

and with one command and knowing nothing about where Chris' branch is coming 
from, I'm ready to go, to examine or merge the PR. (Of course, I'm not going to 
merge anything, just an example!)

hub provides some other useful tools, for forking on Github and sending PRs and 
so forth. Well-worth the price of admission!

ajs6f

> On Mar 19, 2018, at 6:44 AM, Rob Vesse <rve...@dotnetrdf.org> wrote:
> 
> You can have multiple remote repositories defined in your local working copy. 
>  Here's the list of my defined remotes on my Jena working copy:
> 
>> git remote -vv
> afs   https://github.com/afs/jena.git (fetch)
> afs   https://github.com/afs/jena.git (push)
> originhttps://git-wip-us.apache.org/repos/asf/jena.git (fetch)
> originhttps://git-wip-us.apache.org/repos/asf/jena.git (push)
> rvessessh://g...@github.com/rvesse/jena.git (fetch)
> rvessessh://g...@github.com/rvesse/jena.git (push)
> 
> You can use git remote add   to add additional remotes and various 
> other git remote commands to manipulate these
> 
> When doing certain remote operations - pull/push/fetch - then you just need 
> to be more explicit about where to push/pull from.  For example to push a new 
> dev branch to my github clone:
> 
>> git push -u rvesse 
> 
> Where  is the local branch name you are wanting to push
> 
> Or to pull a specific branch:
> 
>> git pull  
> 
> This tries to pull updates for the given branch based on its locally set 
> upstream branch, git branch -vv shows details of your local branches and 
> their corresponding remotes:
> 
>> git branch -vv
>  JENA-1405   5937830 [rvesse/JENA-1405] Use better logging approach 
> (JENA-1405)
>  JENA-1407   9e2b4ce [rvesse/JENA-1407: ahead 1] Change degree of test 
> parallelism (JENA-1407)
>  JENA-1434   8df3702 [rvesse/JENA-1434] Minor fixes to new code 
> (JENA-1434)
>  JENA-1486   0eeb40d [rvesse/JENA-1486] Make sure to use datasets 
> transactionally (JENA-1486)
>  JENA-1497   44683c4 [rvesse/JENA-1497] Rename test cases and fix 
> warnings (JENA-1497)
>  JENA-50777a69f1 [origin/JENA-507: gone] Merge branch 'master' into 
> JENA-507
>  JENA-541cbc2dc2 Initial work on custom headers for SPARQL Updates 
> (JENA-541)
>  JENA-8933ab3138 More experimentation with splitting BZip2 inputs
>  jena-db-testing 10441d7 [afs/jena-db-testing] Check .delete() in beforeTest
>  jena2   9bc5aa9 [origin/jena2] Yet more fixes and test cases for 
> assignment inlining (JENA-780)
> * master  2e732b4 [origin/master: behind 2] Merge branch 'JENA-1497'
>  obfuscate   08bdbc3 [rvesse/obfuscate] Add naive obfuscation provider
> 
> Or:
> 
>> git fetch --all
>> git pull /
> 
> And merging a specific remote branch:
> 
>> git fetch --all
>> git merge /
> 
> Similarly to checkout a specific remote branch locally:
> 
>> git fetch --all
>> git checkout -B  --track /
> 
> You just need to be a little careful about doing pull/push without being 
> explicit about the target remote as otherwise you'll go to/from the default 
> (usually origin) which may not always be desired
> 
> Rob
> 
> On 19/03/2018, 10:31, "Claude Warren" <cla...@xenei.com> wrote:
> 
>What process are you using to keep your working copy of Jena (assuming you
>have a clone on github) from the true master copy of

[GitHub] jena pull request #384: JENA-1158: Make the union graph a graphView(Quad.uni...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/384#discussion_r174826544
  
--- Diff: jena-arq/src/main/java/org/apache/jena/sparql/core/GraphView.java 
---
@@ -146,7 +148,7 @@ private static Node graphNode(Node gn) {
 public void performAdd( Triple t ) { 
 Node g = graphNode(gn) ;
 if ( Quad.isUnionGraph(g) )
-throw new GraphViewException("Can't update the default union 
graph of a dataset") ; 
+throw new AddDeniedException("Can't update the union graph of 
a dataset") ; 
--- End diff --

Holy moly, I had no idea we had these. II'm going to remember these, they 
are way more specific than `UnsupportedOperationException`.


---

[GitHub] jena pull request #384: JENA-1158: Make the union graph a graphView(Quad.uni...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/384#discussion_r174825806
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/DatasetGraphBase.java ---
@@ -69,9 +69,11 @@ public boolean containsGraph(Node graphNode) {
 @Override
 public Graph getUnionGraph() {
 // Implementations are encouraged to implement an efficent
-// {@code DatasetGraphBase.findQuadsInUnionGraph} or
+// named graph for Quad.unionGraph, and this operation that
--- End diff --

Does this belong in a Javadoc, like maybe `@apiNote`?


---

[GitHub] jena issue #379: Refactor tests; don't test for GregorianYear< 0

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/379
  
Ok, `testDateTime_9` LGTM and if the others are just cut-and-paste, it 
should all be good.


---

[GitHub] jena pull request #377: JENA-1503: Bug in XSDDateTime toString

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/377#discussion_r173498951
  
--- Diff: 
jena-core/src/test/java/org/apache/jena/graph/test/TestTypedLiterals.java ---
@@ -989,7 +991,17 @@ public void testXSDanyURI() {
 Node node2 = NodeFactory.createLiteral("http://example/;, 
XSDDatatype.XSDstring) ;
 assertFalse(node1.sameValueAs(node2)) ;
 }
-
+
+/**
+ * Test a user error report concerning date/time literals from 
JENA-1503
+ */
+public void testDateTimeBug3() {
+final RDFDatatype XSDdateTime = 
TypeMapper.getInstance().getSafeTypeByName(XSD.dateTime.getURI());
--- End diff --

Hey, let's model good behavior! (Says the father of two toddlers.) I'll add 
a commit right now.


---

Re: Towards Jena 3.7.0

[ajs6f]

> On Mar 9, 2018, at 10:23 AM, Andy Seaborne <a...@apache.org> wrote:
> 
> Status?:
> * PR#368 "Return Model from PrefixMapping methods" JENA-1495

Just merged, I will close the ticket.

> * JENA-1499 : PR done, JIRA open - is there work still in progress?

My mistake-- I just closed the ticket.

ajs6f

[GitHub] jena pull request #377: JENA-1503: Bug in XSDDateTime toString

[ajs6f]

GitHub user ajs6f opened a pull request:

https://github.com/apache/jena/pull/377

JENA-1503: Bug in XSDDateTime toString



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ajs6f/jena JENA-1503

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/377.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #377


commit bd1ee9ed2a0aebc055f8b280e31fe087686934c8
Author: ajs6f <ajs6f@...>
Date:   2018-03-09T15:19:28Z

JENA-1503: Bug in XSDDateTime toString




---

Re: Possible fuseki/memory store bug

[ajs6f]

I've got a PR in now for 1499 (TIM remembers graph names even after they are 
empty):

https://github.com/apache/jena/pull/374

Whether or not that's the whole story here, that's a different question!

ajs6f

> On Mar 7, 2018, at 12:23 PM, Andy Seaborne <a...@apache.org> wrote:
> 
> 
> 
> On 07/03/18 16:51, Dave Reynolds wrote:
>> On 07/03/18 16:29, Andy Seaborne wrote:
>>> JENA-1499 may have knock on effects.
>>> 
>>> Adding a quad and deleting a quad and still listing the graph name would be 
>>> OK but "contains graph" returns false in TIM and true in general. That 
>>> might make a difference - I haven't traced that part of the code.
>> Understood.
>>> As for the :3030, ":" is one of the nasty characters in URIs but the Fuseki 
>>> log looks like it is %-ed correctly.
>>> 
>>> https://gist.github.com/afs/c5623911293dd19da7f6cc1b2cf64c17
>>> 
>>> is an updated script to use curl and not Jena tools.
>>> 
>>> There is variable to set the graph name.
>>> 
>>> Does this script illustrate the effect?
>> Yes with one correction. Line 18 should be:
>> G5X="$(wwwenc $G5)"
>> rather than:
>> G5X="$(wwwenc $G)"
> 
> My bad - gist corrected.
> 
>> With that correction that this illustrates the effect for me.
> 
> I always get:
> 
> ---
> | s   | p  | o  | G   |
> ===
> | :r4 | rdf:type   | :Resource  | :r4 |
> | :r4 | rdfs:label | "r 4 modified" | :r4 |
> ---
> 
> 3.4.0, 3.6.0 (I do delete run/ each start-up or use the 3.6.0 basic server to 
> ensure no funny stuff).
> 
> This is bizarre. No blank nodes either.
> 
> openjdk version "1.8.0_151"
> 
> 
>Andy
> 
>> Dave
>>>  Andy
>>> 
>>> 
>>> On 07/03/18 14:14, Dave Reynolds wrote:
>>>> On 07/03/18 12:59, Andy Seaborne wrote:
>>>>> 
>>>>> 
>>>>> On 06/03/18 22:43, Dave Reynolds wrote:
>>>>>> Hi Andy,
>>>>>> 
>>>>>> Thanks for confirming you seeing something similar, glad I wasn't 
>>>>>> hallucinating!
>>>>> 
>>>>> Not sure any more that I am :-|
>>>>> 
>>>>> There are ghost graphs in a TIM dataset after deletion, JENA-1499, but 
>>>>> I'm not seeing an empty store and do see the added triples.
>>>> 
>>>> OK so this just got weirder. I just tried the script from your gist and 
>>>> that works for me. Whereas my own was still failing. The only difference 
>>>> is that in your initial s-put you have a slightly different graph name 
>>>> (includes the port).
>>>> 
>>>> So for me on 3.4.0 and 3.6.0 if I run a trimmed version of your test [1], 
>>>> with the same update.ttl [2] and U1.ru [3] scripts. I see correct results:
>>>> 
>>>> == Step 1a
>>>> 
>>>> == Step 1b
>>>> <http://localhost:3030/graph5> {
>>>>  <http://localhost/test/r4>
>>>>  a   <http://localhost/test/Resource> ;
>>>>  <http://www.w3.org/2000/01/rdf-schema#label>
>>>>  "r 4" .
>>>> }
>>>> 
>>>> == Step 2a
>>>> 
>>>> <http://localhost/test/r4> {
>>>>  <http://localhost/test/r4>
>>>>  a   <http://localhost/test/Resource> ;
>>>>  <http://www.w3.org/2000/01/rdf-schema#label>
>>>>  "r 4 modified" .
>>>> }
>>>> 
>>>> <http://localhost:3030/graph5> {
>>>> }
>>>> 
>>>> == Step 3a
>>>> --
>>>>  
>>>> | s  | p | o   
>>>>  | G  |
>>>> ==
>>>>  
>>>> | <http://localhost/test/r4> | 
>>>> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> | 
>>&

[GitHub] jena pull request #374: JENA-1499: Pruning TIM indexes on tuple deletion

[ajs6f]

GitHub user ajs6f opened a pull request:

https://github.com/apache/jena/pull/374

JENA-1499: Pruning TIM indexes on tuple deletion



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ajs6f/jena JENA-1499

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/374.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #374


commit d78be3b006a83f4e9e3a83cd76c0c79ee549040c
Author: ajs6f <ajs6f@...>
Date:   2018-03-07T17:25:05Z

JENA-1499: Pruning TIM indexes on tuple deletion




---

Re: [jira] [Commented] (JENA-1499) The TIM dataset retains a memory of named graphs after deleting all quads.

[ajs6f]

+1!

ajs6f

> On Mar 7, 2018, at 11:29 AM, Claude Warren <cla...@xenei.com> wrote:
> 
> This looks like a prime candidate for a contract test (as soon as we agree
> on what the action should be)
> 
> Claude
> 
> On Wed, Mar 7, 2018 at 4:26 PM, A. Soroka (JIRA) <j...@apache.org> wrote:
> 
>> 
>>[ https://issues.apache.org/jira/browse/JENA-1499?page=
>> com.atlassian.jira.plugin.system.issuetabpanels:comment-
>> tabpanel=16389757#comment-16389757 ]
>> 
>> A. Soroka commented on JENA-1499:
>> -
>> 
>> [~andy.seaborne], 
>> {{org.apache.jena.sparql.core.mem.QuadTableForm.GSPO.listGraphNodes()}}
>> is just taking the keys of the GSPO index and returning a stream of them.
>> That's why (see above) if there are mappings fro{{m }}{{Node => Node =>
>> Set}} with that {{Set}} empty, we're still seeing names of graphs
>> appear at that exact method.
>> 
>>> The TIM dataset retains a memory of named graphs after deleting all
>> quads.
>>> 
>> --
>>> 
>>>Key: JENA-1499
>>>URL: https://issues.apache.org/jira/browse/JENA-1499
>>>Project: Apache Jena
>>> Issue Type: Bug
>>>   Affects Versions: Jena 3.6.0
>>>   Reporter: Andy Seaborne
>>>   Priority: Major
>>> 
>>> Illustration:
>>> {noformat}
>>>DatasetGraph dsg = DatasetGraphFactory.createTxnMem();
>>>Quad q = SSE.parseQuad("(:g :s :p :o)");
>>>dsg.add(q);
>>>dsg.delete(q);
>>>Iter.print(dsg.listGraphNodes());
>>> {noformat}
>>> prints {{http://example/g}}.
>> 
>> 
>> 
>> --
>> This message was sent by Atlassian JIRA
>> (v7.6.3#76005)
>> 
> 
> 
> 
> -- 
> I like: Like Like - The likeliest place on the web
> <http://like-like.xenei.com>
> LinkedIn: http://www.linkedin.com/in/claudewarren

[GitHub] jena pull request #373: JENA-1454: Fix to handle text output format.

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/373#discussion_r172649617
  
--- Diff: 
jena-fuseki2/jena-fuseki-core/src/main/java/org/apache/jena/fuseki/servlets/ResponseResultSet.java
 ---
@@ -131,26 +130,30 @@ public static void doResponseResultSet(HttpAction 
action, ResultSet resultSet, P
 contentType = contentTypeTextPlain ;
 
 // Some kind of general dispatch is neater but there are quite a 
few special cases.
+// text/plain is special because there is no ResultSetWriter for 
it (yet). 
+// Text plain is special because of the formatting by prologue.
+// text/plain is not a registered result set language. 
 //
 // JSON is special because of ?callback
-// 
+//
 // XML is special because of
 // (1) charset is a feature of XML, not the response 
 // (2) ?stylesheet=
 //
 // Thrift is special because
 // (1) charset is meaningless
 // (2) there is no boolean result form.
-//
-// Text plain is special because of the formatting by prologue.
-
+
+if ( Objects.equals(serializationType, contentTypeTextPlain) ) {
--- End diff --

Oh, I see-- this was cut from down below. NM.


---

[GitHub] jena pull request #373: JENA-1454: Fix to handle text output format.

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/373#discussion_r172649522
  
--- Diff: 
jena-fuseki2/jena-fuseki-core/src/main/java/org/apache/jena/fuseki/servlets/ResponseResultSet.java
 ---
@@ -131,26 +130,30 @@ public static void doResponseResultSet(HttpAction 
action, ResultSet resultSet, P
 contentType = contentTypeTextPlain ;
 
 // Some kind of general dispatch is neater but there are quite a 
few special cases.
+// text/plain is special because there is no ResultSetWriter for 
it (yet). 
+// Text plain is special because of the formatting by prologue.
+// text/plain is not a registered result set language. 
 //
 // JSON is special because of ?callback
-// 
+//
 // XML is special because of
 // (1) charset is a feature of XML, not the response 
 // (2) ?stylesheet=
 //
 // Thrift is special because
 // (1) charset is meaningless
 // (2) there is no boolean result form.
-//
-// Text plain is special because of the formatting by prologue.
-
+
+if ( Objects.equals(serializationType, contentTypeTextPlain) ) {
--- End diff --

Just out of curiosity-- why `Objects.equals` here instead of 
`contentTypeTextPlain.equals` or `serializationType.equals`?


---

[GitHub] jena pull request #372: Reformatting + two RiotChars functions.

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/372#discussion_r172582451
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/riot/lang/BlankNodeAllocatorGlobal.java 
---
@@ -18,41 +18,39 @@
 
 package org.apache.jena.riot.lang;
 
-import java.util.HashMap ;
-import java.util.Map ;
+import java.util.HashMap;
+import java.util.Map;
 
-import org.apache.jena.graph.Node ;
-import org.apache.jena.graph.NodeFactory ;
+import org.apache.jena.graph.Node;
+import org.apache.jena.graph.NodeFactory;
 
-/** Allocate blank nodes by creating a randomly generated blank node.
- *  This allocator has arbitrary sized state. 
+/**
+ * Allocate blank nodes by creating a randomly generated blank node.
+ * This allocator has arbitrary sized internal state needed to record
+ * the label to node mapping.
  */
 
 public class BlankNodeAllocatorGlobal implements BlankNodeAllocator
 {
-Map<String, Node> map = new HashMap<>() ;
+Map<String, Node> map = new HashMap<>();
 
 public BlankNodeAllocatorGlobal()  {}
 
 @Override
-public void reset() { map.clear() ; }
+public void reset() { map.clear(); }
 
 @Override
-public Node alloc(String label)
-{
-Node b = map.get(label) ;
-if ( b == null )
-{
-b = create() ;
-map.put(label, b) ;
+public Node alloc(String label) {
+Node b = map.get(label);
--- End diff --

Maybe just `return map.computeIfAbsent(x->create())`?


---

[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/114
  
Hey, @kinow , is this ready for rebase and review? I can try rebasing it 
into a separate PR, or you can if you have time... then (hopefully) we can get 
it in!


---

[GitHub] jena issue #353: Refactoring complex methods of classes FBRuleInfGraph and L...

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/353
  
I simply don't know enough about our inferencing gear to comment on 
this… @der, can you say anything?


---

[GitHub] jena pull request #367: JENA-1391: Add methods for ModelCollectors to API in...

[ajs6f]

GitHub user ajs6f reopened a pull request:

https://github.com/apache/jena/pull/367

JENA-1391: Add methods for ModelCollectors to API in ModelUtils



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ajs6f/jena JENA-1391b

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/367.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #367


commit 81cd7a1c1bf362413986dced749d750abdc035c5
Author: ajs6f <ajs6f@...>
Date:   2018-03-01T14:13:35Z

JENA-1391: Add methods for ModelCollectors to API in ModelUtils




---

[GitHub] jena pull request #367: JENA-1391: Add methods for ModelCollectors to API in...

[ajs6f]

Github user ajs6f closed the pull request at:

https://github.com/apache/jena/pull/367


---

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172276168
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map<Object, TransactionHandler> handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map<Object, TransactionHandler> buildHandlerSet(Graph 
primary, Collection graphs) {
+Map<Object, TransactionHandler> handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map<Object, TransactionHandler> 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported() )
+return;
+Object key = calcKey(graph);
+if ( th.transactionsSupported()

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172272325
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map<Object, TransactionHandler> handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map<Object, TransactionHandler> buildHandlerSet(Graph 
primary, Collection graphs) {
+Map<Object, TransactionHandler> handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map<Object, TransactionHandler> 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported() )
+return;
+Object key = calcKey(graph);
+if ( th.transactionsSupported()

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172252346
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map<Object, TransactionHandler> handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map<Object, TransactionHandler> buildHandlerSet(Graph 
primary, Collection graphs) {
+Map<Object, TransactionHandler> handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map<Object, TransactionHandler> 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported() )
+return;
+Object key = calcKey(graph);
+if ( th.transactionsSupported()

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172254605
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
--- End diff --

typo missing `)`


---

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172251788
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
--- End diff --

`onld` => `old`


---

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172254303
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map<Object, TransactionHandler> handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map<Object, TransactionHandler> buildHandlerSet(Graph 
primary, Collection graphs) {
+Map<Object, TransactionHandler> handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map<Object, TransactionHandler> 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported() )
+return;
+Object key = calcKey(graph);
+if ( th.transactionsSupported()

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172252112
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map<Object, TransactionHandler> handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map<Object, TransactionHandler> buildHandlerSet(Graph 
primary, Collection graphs) {
+Map<Object, TransactionHandler> handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map<Object, TransactionHandler> 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported() )
+return;
+Object key = calcKey(graph);
+if ( th.transactionsSupported()

[GitHub] jena issue #369: JENA-1492: Pass down transactions from general datasets

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/369
  
Okay. `::wrap` vs. `::create` is not what I am referring to as 
reformatting. It's stuff like 
[this](https://github.com/apache/jena/pull/369/files#diff-27cb8b74e951d2872a91a48066930e2d)
 or 
[this](https://github.com/apache/jena/pull/369/files#diff-9d27f4f9095b5d190b6fe8977b6c393b).

From what I understand, this all makes sense. Datasets that are "collected" 
from multiple graphs can, at best, pass the transaction through to their 
graphs, those that are in control of their own data can retain the 
responsibility for transactions themselves. That does seem to be the least 
surprising behavior and the one that veers towards "dataset as the fundamental 
unit" which we have been working towards.



---

[GitHub] jena issue #368: JENA-1495 Return Model from PrefixMapping methods.

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/368
  
This looks basically good to me: with an additional commit addressing 
https://github.com/apache/jena/pull/368#pullrequestreview-101006841, I'll be 
happy to merge this.


---

[GitHub] jena issue #369: JENA-1492: Pass down transactions from general datasets

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/369
  
There is some new `Assembler` stuff and reformatting, so I'm not totally 
clear what the thrust of this PR is: is it to ensure that `Model`s wrapping 
other `Model`s (e.g. for inferencing) correctly pass transaction control calls 
into their wrapped models (which should pass them into their underlying 
`Dataset`)?


---

[GitHub] jena issue #368: JENA-1495 Return Model from PrefixMapping methods.

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/368
  
Urp, just realized that `Model extends PrefixMapping`, so this is fine wrt 
compatibility. Sorry for the confusion!


---

[GitHub] jena issue #368: JENA-1495 Return Model from PrefixMapping methods.

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/368
  
This seems like a useful change, but it is not back-compatible, so in 
theory we should hold it for a major version release.


---

[GitHub] jena pull request #368: JENA-1495 Return Model from PrefixMapping methods.

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/368#discussion_r172049845
  
--- Diff: jena-core/src/main/java/org/apache/jena/rdf/model/Model.java ---
@@ -1051,4 +1051,12 @@ Remove all the statements matching (s, p, o) from 
this model.
 Answer true iff .close() has been called on this Model.
 */
 public boolean isClosed();
+
+// Override return type for methods inherited from PrefixMapping
--- End diff --

We should use `@Override` for explicitness on these changes.


---

[GitHub] jena pull request #367: JENA-1391: Add methods for ModelCollectors to API in...

[ajs6f]

GitHub user ajs6f opened a pull request:

https://github.com/apache/jena/pull/367

JENA-1391: Add methods for ModelCollectors to API in ModelUtils



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ajs6f/jena JENA-1391b

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/367.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #367


commit b74d3775f180c1077669ba7084eab3e8ef363cbe
Author: ajs6f <ajs6f@...>
Date:   2018-03-01T14:13:35Z

JENA-1391: Add methods for ModelCollectors to API in ModelUtils




---

[GitHub] jena pull request #363: JENA-1491: Migrate FindBugs to StopBugs

[ajs6f]

GitHub user ajs6f opened a pull request:

https://github.com/apache/jena/pull/363

JENA-1491: Migrate FindBugs to StopBugs



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ajs6f/jena JENA-1491

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/363.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #363


commit 8cc26701f842584c16d2120f2e0013e24482c1cc
Author: ajs6f <ajs6f@...>
Date:   2018-02-16T21:31:25Z

JENA-1491: Migrate FindBugs to StopBugs




---

[GitHub] jena pull request #361: Small improvements

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/361#discussion_r168644094
  
--- Diff: jena-tdb/src/main/java/org/apache/jena/tdb/solver/BindingTDB.java 
---
@@ -120,6 +121,8 @@ public Node get1(Var var)
 if ( id == null )
 return null ; 
 n = nodeTable.getNodeForNodeId(id) ;
+if ( n == null )
+throw new TDBException("No node in NodeTable for NodeId 
"+id);
--- End diff --

Right-o, sounds like it's much better to leave it be.


---

[GitHub] jena pull request #361: Small improvements

[ajs6f]

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/361#discussion_r168523058
  
--- Diff: jena-tdb/src/main/java/org/apache/jena/tdb/solver/BindingTDB.java 
---
@@ -120,6 +121,8 @@ public Node get1(Var var)
 if ( id == null )
 return null ; 
 n = nodeTable.getNodeForNodeId(id) ;
+if ( n == null )
+throw new TDBException("No node in NodeTable for NodeId 
"+id);
--- End diff --

Do we know all/most of the circumstances that might cause this? Is it 
basically an interrupted write? If so, it might be worth including some error 
logging with that hint, but if there are more than one or two potential causes, 
never mind. I mention it because getting from "a missing entry in the 
`NodeTable`" to "that was the aftereffect of a failed write" is going to bring 
some users to the list when they might be able to figure out what happened with 
a simple hint.


---

[GitHub] jena pull request #362: JENA-1389: Return 'this' rather than 'void' from Dat...

[ajs6f]

GitHub user ajs6f opened a pull request:

https://github.com/apache/jena/pull/362

JENA-1389: Return 'this' rather than 'void' from Dataset



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ajs6f/jena JENA-1389

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/362.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #362


commit d21aba27b73d3340e595f22b2ae58f7019b455aa
Author: ajs6f <ajs6f@...>
Date:   2018-02-15T15:47:37Z

JENA-1389: Return 'this' rather than 'void' from Dataset




---

[GitHub] jena issue #359: Fixes for transactions after JENA-1458

[ajs6f]

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/359
  
I meant code review. (I have no problem believing that I overlooked them, 
just surprised that everyone did.)


---

Re: [DRAFT] Apache Jena and Java9

[ajs6f]

LGTM, with one suggestion: when discussing point [4] modules, maybe use the 
terms "JPMS" or Jigsaw or "new Java Platform Module System" or something to be 
very explicit. Some folks pay closer attention than others to "the state of the 
Java" and it may not be obvious to some what we mean .

ajs6f

> On Feb 11, 2018, at 1:39 PM, Andy Seaborne <a...@apache.org> wrote:
> 
> I thought that some text about the status of Jena and Java9 would be useful 
> to users@ to state the current state. This could go in the 3.7.0 release 
> notes.
> 
>Andy
> 
> DRAFT:: For discussion
> 
> -
> Java8 is the current LTS version of Java and receives security updates. The 
> next LTS is Java11. [Sched]
> 
> [Sched] http://www.oracle.com/technetwork/java/eol-135779.html
> 
> There are different cases for running with Java9:
> 
> [1] Run on a Java9 JVM
> [2] Build using Java9 JDK, output Java8 classfiles.
> [3] Java9 required - language features and runtime library
> [4] Java9 modules
> 
> 
> [1] at Jena 3.7.0, running with a Java9 platform (except as noted below) is 
> supported.
> 
> [2] at Jena 3.7.0, running maven using Java9 to produce Java8 bytecode is 
> supported (expect as noted).  We now run a Jenkins job daily to check this 
> for the main jars.
> 
> [3] The project has no current plans to require a Java9 language runtime. The 
> language requirement is still Java8.
> 
> [4] Jena jars can work as automatic modules but Jena itself does not provide 
> a "modules" version. Proper migration needs all the dependencies to be 
> modules, and also has implications on the upstream. Contribution and 
> discussions are welcome!
> 
> For discussion and background see
> 
> [A] http://blog.joda.org/2017/05/java-se-9-jpms-automatic-modules.html
> [B] http://blog.joda.org/2017/04/java-se-9-jpms-module-naming.html
> 
> 
> And note: Java 9 is obsolete March 2018, as soon as java10 comes out.
> http://openjdk.java.net/projects/jdk/10/#Schedule
> 
> Notes:
> 
> jena-elephas depends on Hadoop that depends on jdk.tools that is not 
> available in Java9.