from:"Paul A. Jackson"

RE: io and graphson-v3

2017-09-06 Thread Paul A. Jackson

This v3 looks like a <3 heart where the top broke off.

Sad.


-Original Message-
From: Stephen Mallette [mailto:spmalle...@gmail.com] 
Sent: Wednesday, September 06, 2017 1:45 PM
To: dev@tinkerpop.apache.org
Subject: Re: io and graphson-v3

never mind - i see what's messed up

On Wed, Sep 6, 2017 at 1:43 PM, Stephen Mallette 
wrote:

> Which tests in particular are failing for SerializationTest? all of them?
>
> On Wed, Sep 6, 2017 at 1:35 PM, pieter gmail 
> wrote:
>
>> Hi,
>>
>> Pulled TINKERPOP-1767 branch, changed SqlgGraph's io method and ran 
>> the tests.
>>
>> All the io tests are passing.
>> Only SerializationTest fails for the same reason. It too needs the 
>> version specified. I did that locally and then all tests passes.
>>
>> Thanks
>> Pieter
>>
>> On 06/09/2017 18:09, Stephen Mallette wrote:
>>
>>> Pieter, I created this issue:
>>>
>>> https://issues.apache.org/jira/browse/TINKERPOP-1767
>>>
>>> and made an effort to try to figure a way to fix it:
>>>
>>> https://github.com/apache/tinkerpop/tree/TINKERPOP-1767
>>>
>>> Note the change to TinkerGraph and its io() method. I suppose you 
>>> could do something similar to get the right registry in play? could 
>>> you have a look and see if what i did helps? if that works then i'll 
>>> issue a PR and we can get it reviewed/merged.
>>>
>>>
>>> On Tue, Sep 5, 2017 at 12:10 PM, pieter gmail 
>>> 
>>> wrote:
>>>
>>> Ok, at present there is only one SimpleModule, the default. I can 
>>> make it
 v2 or v3 but not both.

 Lets say I make the SimpleModule support V2.
 Then when calling IoEdgeTest for

  {"graphson-v3", true, true,
  (Function) g -> g.io 
 (IoCore.graphson()).reader 
 ().mapper(g.io(GraphSONIo.build(GraphSONVersion.V3_0)).
 mapper().create()).create(),
  (Function) g -> g.io 
 (IoCore.graphson()).writer 
 ().mapper(g.io(GraphSONIo.build(GraphSONVersion.V3_0)).
 mapper().create()).create()},

 then the deserializers that run are for V2,

  static class SchemaTableIdJacksonDeserializerV2d0 extends 
 AbstractObjectDeserializer {
  SchemaTableIdJacksonDeserializerV2d0() {
  super(SchemaTable.class);
  }

  @Override
  public SchemaTable createObject(final Map data) {
  return SchemaTable.of((String)data.get("schema"), 
 (String) data.get("table"));
  }
  }

 when createObject fires the map data has V3 elements in it like 
 @type and @value whilst its expecting "schema" and "table"

 If we make the SimpleModule support V3 then graphson-v2 will fail.

 {"graphson-v2", false, false,
 (Function) g -> g.io 
 (IoCore.graphson()).reader 
 ().mapper(g.io(GraphSONIo.build(GraphSONVersion.V2_0)).
 mapper().typeInfo(TypeInfo.NO_TYPES).create()).create(),
 (Function) g -> g.io 
 (IoCore.graphson()).writer 
 ().mapper(g.io(GraphSONIo.build(GraphSONVersion.V2_0)).

 mapper().typeInfo(TypeInfo.NO_TYPES).create()).create()},

 Now the deserializers are for V3.

  static class SchemaTableJacksonDeserializerV3d0 extends 
 StdDeserializer {
  public SchemaTableJacksonDeserializerV3d0() {
  super(RecordId.class);
  }

  @Override
  public SchemaTable deserialize(final JsonParser 
 jsonParser, final DeserializationContext deserializationContext) 
 throws IOException, JsonProcessingException {
  final Map data = 
 deserializationContext.readValue(jsonParser,
 Map.class);
  return SchemaTable.of((String)data.get("schema"), 
 (String) data.get("table"));
  }

  @Override
  public boolean isCachable() {
  return true;
  }
  }

 This does not fire at all. Eventually I get a detached edge with an 
 id that is a map. It never deserialized.

 So basically it only works if the SimpleModule version, i.e.
 serialize/deserialize code matches up with the version.

 Sqlg serializes RecordId to,

  static class RecordIdJacksonSerializerV3d0 extends 
 StdScalarSerializer {
  public RecordIdJacksonSerializerV3d0() {
  super(RecordId.class);
  }
  @Override
  public void serialize(final RecordId recordId, final 
 JsonGenerator jsonGenerator, final SerializerProvider 
 serializerProvider)
  throws IOException, JsonGenerationException {
  final Map m = new HashMap<>();
  m.put("schemaTable", recordId.getSchemaTable());

RE: Requirements for listing as a vendor

2017-05-05 Thread Paul A. Jackson

Hi Robert,

Sorry for the delayed response. Here's the verbiage we would like to use.

Pitney Bowes Spectrum Data Hub 
Module<http://www.pitneybowes.com/us/customer-information-management/data-integration-management/spectrum-data-hub-module.html>
 – Uses Gremlin OLTP to query Neo4j-powered MDM graph DB.
Do you have room for a logo? If another size is needed, I can provide one.
[http://bizwire.tekgroup.com/media/95/20150114105005209pb_hor_4cp_grd_pos.jpg]<http://bizwire.tekgroup.com/media/95/20150114105005209pb_hor_4cp_grd_pos.jpg>

Thanks,
-Paul


From: Robert Dale [mailto:robd...@gmail.com]
Sent: Thursday, April 20, 2017 3:38 PM
To: dev@tinkerpop.apache.org; Paul A. Jackson <paul.jack...@pb.com>
Subject: Re: Requirements for listing as a vendor


Paul,

We have come up with a format for the Powered By listing.

It can be "Company Product" or just "Product name" link and then some optional 
summary text.  The max length of the whole line is 100 chars.

For example,

Pitney Bowes Spectrum Data Hub 
Module<http://www.pitneybowes.com/us/customer-information-management/data-integration-management/spectrum-data-hub-module.html>
 - uses Gremlin in some short way
or
Spectrum Data Hub 
Module<http://www.pitneybowes.com/us/customer-information-management/data-integration-management/spectrum-data-hub-module.html>
 - uses Gremlin in much bigger and longer and uncut ways


Let us know what you would like on there.

--
Robert Dale

RE: [DISCUSS] ASF Board Draft Report - April 2017

2017-04-04 Thread Paul A. Jackson

Is this a forum for the "Powered By" concept?

Thanks,
-Paul

-Original Message-
From: Stephen Mallette [mailto:spmalle...@gmail.com]
Sent: Tuesday, April 04, 2017 1:42 PM
To: dev@tinkerpop.apache.org
Subject: [DISCUSS] ASF Board Draft Report - April 2017

Here's a draft of the board report for April. Please let me know if you think 
there's anything else we might add. Note that there is one outstanding issue 
that might make it on here related to PMC membership - we'll see if that shakes 
out before this needs to be submitted to the board.

## Description:
Apache TinkerPop is a graph computing framework for both graph databases
(OLTP) and graph analytic systems (OLAP).

## Activity:
The TinkerPop PMC voted positively to add a new committer, but that person 
declined when invited, preferring instead to continue to contribute as they 
have been.

TinkerPop released versions 3.1.6 and 3.2.5. New development has started on  
the follow-on releases to those versions and work for the new major line of 
3.3.0 continues.

The wider TinkerPop community saw some growth with the annoucement of the 
release of ChronoGraph[1], a versioned graph database that implements the 
TinkerPop interfaces. Also of note, JanusGraph[2] (formerly Titan) is under 
development at the Linux Foundation. JanusGraph continues in the steps of Titan 
in also implementing the TinkerPop interfaces.

There were also a number of talks given related to TinkerPop and the Gremlin 
language. Jason Plurad, a PMC member, gave one such talk[3].

## Issues:
There are no issues requiring board attention at this time.

## Releases:
- 3.1.6 (February 8, 2017)
- 3.2.4 (February 8, 2017)

## PMC/Committer:

- Last PMC addition was Jason Plurad - August 2016
- Last committer addition was Robert Dale - October 2016

## Links

[1] https://github.com/MartinHaeusler/chronos/
[2] http://janusgraph.org/
[3] https://www.youtube.com/watch?v=7kTqqvBzL60

RE: Requirements for listing as a vendor

2017-03-17 Thread Paul A. Jackson

This sounds great.

At the same time, though, I was looking at other providers and judging from 
architecture diagrams (eg. 
https://grakn.ai/pages/documentation/the-fundamentals/grakn-architecture.html) 
some look similar to ours.

It may be the case that our web site is directed more towards business types 
than developers, is available through direct sales only, etc, rather than 
architectural differences?

-Paul


-Original Message-
From: Marko Rodriguez [mailto:okramma...@gmail.com]
Sent: Thursday, March 16, 2017 4:13 PM
To: dev@tinkerpop.apache.org
Subject: Re: Requirements for listing as a vendor

I say we add a “Powered By” section to the website and list “leveraging 
systems” as I wouldn’t call this a “TinkerPop Provider” like I would a more 
“low level” graph database system….

If we open up “Powered By” it would allow us to show people the types of 
products, projects, nick nacks and doo dads that are out there leveraging 
TinkerPizzzop.

Marko.


> On Mar 16, 2017, at 1:56 PM, Paul A. Jackson <paul.jack...@pb.com> wrote:
>
> I agree that the last bullet is not met. We bundle the Tinkerpop stack, so 
> the idea of version compatibility doesn't make sense. Instead, we offer a 
> platform that builds an ecosystem around the graph (ETL, fuzzy matching, rich 
> analysis client, etc). One feature is graph query. The users can supply their 
> own Gremlin queries, or they can use our graphical query builder, which 
> generates Gremlin.
>
> Anywho, what do you think? I concur we do not fit in the provider bucket. But 
> we have been around for 5 years. The offering is pretty mature at this point.
>
> -Paul
>
>
>
> -Original Message-
> From: Robert Dale [mailto:robd...@gmail.com]
> Sent: Thursday, March 16, 2017 1:50 PM
> To: dev@tinkerpop.apache.org
> Subject: Re: Requirements for listing as a vendor
>
> You should start here:  http://tinkerpop.apache.org/policy.html
>
> It's not obvious to me that the last 3 bullets are met.
>
> Robert Dale
>
> On Thu, Mar 16, 2017 at 12:56 PM, Paul A. Jackson
> <paul.jack...@pb.com>
> wrote:
>
>> Hi,
>>
>> What are the requirements for being lists as a Gremlin provider? We
>> (Pitney Bowes) have an integrated product offering called Spectrum
>> Data Hub (you'll also see it listed as part of the Spectrum MDM
>> solution). The product is its own application server and we embed
>> Gremlin on top of embedded Neo4j using our own "blueprints" adapter.
>> So, access to the graph is from our own apps, rather that via a Gremlin 
>> console or server.
>>
>> I don't know if this makes us a special case or not. Can you let us
>> know if we qualify and/or if there's anything we would need to do so?
>>
>> Here's a link to the product literature:
>> http://www.pitneybowes.com/us/
>> customer-information-management/data-integration-
>> management/spectrum-data-hub-module.html
>>
>> Thanks,
>> -Paul
>>
>> 
>>
>>
>
> 
>

RE: Requirements for listing as a vendor

2017-03-16 Thread Paul A. Jackson

I agree that the last bullet is not met. We bundle the Tinkerpop stack, so the 
idea of version compatibility doesn't make sense. Instead, we offer a platform 
that builds an ecosystem around the graph (ETL, fuzzy matching, rich analysis 
client, etc). One feature is graph query. The users can supply their own 
Gremlin queries, or they can use our graphical query builder, which generates 
Gremlin.

Anywho, what do you think? I concur we do not fit in the provider bucket. But 
we have been around for 5 years. The offering is pretty mature at this point.

-Paul

-Original Message-
From: Robert Dale [mailto:robd...@gmail.com]
Sent: Thursday, March 16, 2017 1:50 PM
To: dev@tinkerpop.apache.org
Subject: Re: Requirements for listing as a vendor

You should start here:  http://tinkerpop.apache.org/policy.html

It's not obvious to me that the last 3 bullets are met.

Robert Dale

On Thu, Mar 16, 2017 at 12:56 PM, Paul A. Jackson <paul.jack...@pb.com>
wrote:

> Hi,
>
> What are the requirements for being lists as a Gremlin provider? We
> (Pitney Bowes) have an integrated product offering called Spectrum
> Data Hub (you'll also see it listed as part of the Spectrum MDM
> solution). The product is its own application server and we embed
> Gremlin on top of embedded Neo4j using our own "blueprints" adapter.
> So, access to the graph is from our own apps, rather that via a Gremlin 
> console or server.
>
> I don't know if this makes us a special case or not. Can you let us
> know if we qualify and/or if there's anything we would need to do so?
>
> Here's a link to the product literature:
> http://www.pitneybowes.com/us/
> customer-information-management/data-integration-
> management/spectrum-data-hub-module.html
>
> Thanks,
> -Paul
>
> 
>
>

Requirements for listing as a vendor

2017-03-16 Thread Paul A. Jackson

Hi,

What are the requirements for being lists as a Gremlin provider? We (Pitney 
Bowes) have an integrated product offering called Spectrum Data Hub (you'll 
also see it listed as part of the Spectrum MDM solution). The product is its 
own application server and we embed Gremlin on top of embedded Neo4j using our 
own "blueprints" adapter. So, access to the graph is from our own apps, rather 
that via a Gremlin console or server.

I don't know if this makes us a special case or not. Can you let us know if we 
qualify and/or if there's anything we would need to do so?

Here's a link to the product literature: 
http://www.pitneybowes.com/us/customer-information-management/data-integration-management/spectrum-data-hub-module.html

Thanks,
-Paul

RE: [GitHub] tinkerpop issue #567: TINKERPOP-1644 Improve script compilation syncronisati...

2017-03-07 Thread Paul A. Jackson

This code is using the double-checked locking idiom, which does not work in 
Java:
   Class clazz = getScriptClassFromMap(script);
   if (clazz != null) {
   return clazz;
   }
   synchronized (this) {
   clazz = getScriptClassFromMap(script);
   if (clazz != null) {
   return clazz;
   }

See https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html

I don't know the details well enough to judge whether the failure modes will be 
harmful in this case. I think it's best avoided or replaced with one of the 
suggestions in the link above.

-Paul

-Original Message-
From: spmallette [mailto:g...@git.apache.org]
Sent: Tuesday, March 07, 2017 6:25 AM
To: dev@tinkerpop.apache.org
Subject: [GitHub] tinkerpop issue #567: TINKERPOP-1644 Improve script 
compilation syncronisati...

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/567

I will clean up a few code formatting things on merge, but this looks good 
to me. All tests pass with `docker/build.sh -t -n -i`

VOTE +1


---
If your project is set up for it, you can reply to this email and have your 
reply appear on GitHub as well. If your project does not have this feature 
enabled and wishes so, or if the feature is enabled but not working, please 
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with 
INFRA.
---

RE: [DISCUSS] P.not and __.not

2017-02-24 Thread Paul A. Jackson

If you'll forgive "thinking out loud," is there any way to create a wrapper-not 
that implements both interfaces and just works?

-Original Message-
From: Stephen Mallette [mailto:spmalle...@gmail.com]
Sent: Friday, February 24, 2017 2:35 PM
To: dev@tinkerpop.apache.org
Subject: [DISCUSS] P.not and __.not

P.not and __.not unfortunately tangle with each other when using static 
imports. I think we allowed the console to dictate to us that P.not is the 
lucky one that gets to be used without its qualifying prefix. I'm not sure 
there was any conscious decision to do it that way. Indeed, I think i would 
prefer getting __.not over P.not. I also think that the behavior is sort of 
random that we get P.not rather than __.not (for reasons I can go into in more 
detail if anyone cares).

Anyway, I'd like to resolve this issue in 3.3.0. I think that immediately, I 
can introduce the breaking change to the console that explicitly imports __.not 
rather than P.not - this will remove randomness. In the longer term we can 
deprecate P.not and either drop it all together or rename it. I'm not sure how 
strongly folks feel about usage of P.not so I guess I'll just open it up for 
discussion as to what the long term fix should be here.

If we don't develop any real consensus here for the longer term option I will 
just create an issue in JIRA and it can be dealt with later. I'm mostly 
interested in getting a short-term solution in place to solve some problems I'm 
facing right now.

Thanks,

Stephen

RE: [VOTE] TinkerPop 3.2.4 Release

2017-02-10 Thread Paul A. Jackson

I'm looking for some insight for when 3.2.4 will release. It looks like voting 
closes tomorrow. Is that on schedule, and if so what's a typical ETA for 
binaries to be posted?

I don't know if my vote counts, but FTW!

VOTE: +1

Thanks,
-Paul


-Original Message-
From: Stephen Mallette [mailto:spmalle...@gmail.com]
Sent: Friday, February 10, 2017 7:03 AM
To: dev@tinkerpop.apache.org
Subject: Re: [VOTE] TinkerPop 3.2.4 Release

validate-distribution.sh was good for me - thanks for doing all the work on 
this one Jason. nice job

VOTE: +1

On Fri, Feb 10, 2017 at 1:22 AM, Ted Wilmes  wrote:

> Release and docs looks good to me.
>
> Validating source distribution
>
> * downloading Apache TinkerPop 3.2.4 (apache-tinkerpop-3.2.4-src.zip)...
> OK
> * validating signatures and checksums ...
>   * PGP signature ... OK
>   * MD5 checksum ... OK
>   * SHA1 checksum ... OK
> * unzipping Apache TinkerPop 3.2.4 ... OK
> * building project ... OK
>
>
> VOTE: +1
>
> Thanks,
> Ted
>
> On Thu, Feb 9, 2017 at 12:32 PM, pieter-gmail
> 
> wrote:
>
> > Ok, however I did test on the 3.2.4-SNAPSHOT immediately after
> > Jason's email on the 2/2/2017 and those changes were not there.
> > They are there now but there was a SNAPSHOT release on the
> > 08/02/2017 so things changed.
> > Anyhow that might just be some SNAPSHOT confusion thing.
> >
> > Next time I'll pull the code and build it manually to make sure.
> >
> > Thanks
> > Pieter
> >
> >
> >
> > On 09/02/2017 22:27, Marko Rodriguez wrote:
> > > Hi,
> > >
> > >> The significant change is
> > >> ffe1b4c Marko A. Rodriguez  on 2016/11/15
> > >> at
> > 12:44 AM
> > > Ah yea. Thats different from what I thought you had issue with —
> > has-containers and arrays/collections handling.
> > >
> > >> I don't have a problem with the changes just that is quite a
> > >> refactor
> on
> > >> my side as it changes the structure of the HasSteps and its
> > >> HasContainers. I had made some assumptions around how HasSteps
> > >> and HasContainers look when optimizing.
> > > Yea, it “folds left” now so you don’t have to walk over has()-chains.
> > >
> > >> The change is quite old but somehow it was not present in the
> > >> 3.2.4-SNAPSHOT I tested on it when the code freeze announcement
> > >> was made. Not sure what happened there but alas its not giving me
> > >> enough time to get things working again.
> > > I would say don’t wait till releases to test Sqlg. In principle,
> > > you
> > should VOTE on every PR by building and testing the changes against Sqlg.
> > That is where you can make a huge contribution.
> > >
> > >> So I am not voting negative just requesting a weekend, if
> > >> possible, to get through the refactor.
> > > Again, PR awareness, not release awareness.
> > >
> > > Marko.
> > >
> > > http://markorodriguez.com
> > >
> > >
> > >> Cheers
> > >> Pieter
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> "added TraversalHelper.addHasContainer() on 2016/11/15
> > >>
> > >> On 09/02/2017 22:11, Stephen Mallette wrote:
> > >>> Unless I'm missing something, HasStep hasn't changed in 4 months
> > >>> and HasContainer hasn't changed in 3 months. The only update
> > >>> that went in
> > after
> > >>> 2/2 that I can think of that would have any bearing for graph
> > providers who
> > >>> tested before/after that date would be the AutoCloseable stuff:
> > >>>
> > >>> https://github.com/apache/tinkerpop/pull/548
> > >>>
> > >>> Your problems aren't related to that are they?  Can you provide
> > >>> some synopsis of where the problems lie?
> > >>>
> > >>>
> > >>>
> > >>> On Thu, Feb 9, 2017 at 2:47 PM, pieter-gmail <
> pieter.mar...@gmail.com>
> > >>> wrote:
> > >>>
> >  Hi,
> > 
> >  Some issues regarding the release process.
> > 
> >  On the 2/2/2017 the 3.2.4-SNAPSHOT was released.
> >  I then started testing and found almost no issues.
> > 
> >  However yesterday when the VOTE mail came I found many issues
> >  on
> > 3.2.4.
> >  To understand the confusion I tested again on 3.2.4-SNAPSHOT
> >  and
> found
> >  the same new issues.
> >  I then checked the 3.2.4-SNAPSHOT timestamp and it changed to
> > 08/02/2017
> > 
> >  Not sure what happened there as I can not, nor is it worth it,
> >  check
> > an
> >  old SNAPSHOT version's binary.
> >  The relevant code changes are quite old (December 2016) but it
> >  may
> > have
> >  been done on a separate branch and rebasing may loose the
> >  merging information. Not sure about this though.
> > 
> >  This only leaves the 72 hours to catch up.
> > 
> >  Even though it is a minor release there has been significant
> >  changes
> > to
> >  HasStep and HasContainer which breaks the heart (mine at least)
> >  of implementors optimization code.
> > 
> >  Basically to get any value from a vote I for one will need more
> time.
> > 
> >  So far I have

RE: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator

2017-01-27 Thread Paul A. Jackson

I am git illiterate.
I committed to my repo.
It looks like that's all I have to do.
Travis build tells me to hang tight.
It knows the commit is related to an open pull request
and your repo knows to pull it in automatically?
That's cool, if true.
-Paul

-Original Message-
From: Paul A. Jackson [mailto:paul.jack...@pb.com]
Sent: Friday, January 27, 2017 2:24 PM
To: dev@tinkerpop.apache.org
Subject: RE: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced 
CloseableIterator

Hi Stephen,

I think there are two parts to this, and I may have responded to the wrong part.

I can easily and cleanly make those subclasses implement AutoCloseable and 
remove that from FlatMapStep.

Implementing AutoCloseable is needed for the second half of the problem, where 
a traversal is closed before it is read to completion. This is easily delegated 
to the subclasses.

There's the first half of the problem dealing with closing iterators as 
FlatMapStep finishes with them, but before it loses its reference to them. This 
is very hard to remove from FlatMapStep without duplicating all the code in the 
subclasses.

If you consent I'd like to update the pull request that removes AutoCloseable 
from FlatMapStep but keeps the closing of iterators on the fly in FlatMapStep.

I'd can also move that try/catch stuff to a helper method (in 
CloseableInterface?). That keeps FlatMapStep very clean.

Thoughts?

public abstract class FlatMapStep<S, E> extends AbstractStep<S, E> {

private Traverser.Admin head = null;
protected Iterator iterator = EmptyIterator.instance(); // Made protected

public FlatMapStep(final Traversal.Admin traversal) {
super(traversal);
}

@Override
protected Traverser.Admin processNextStart() {
while (true) {
if (this.iterator.hasNext()) {
return this.head.split(this.iterator.next(), this);
} else {
this.head = this.starts.next();
closeIterator();// Added
this.iterator = this.flatMap(this.head);
}
}
}

protected abstract Iterator flatMap(final Traverser.Admin traverser);

@Override
public void reset() {
super.reset();
closeIterator();// Added
this.iterator = EmptyIterator.instance();
}

void closeIterator() {
CloseableIterator.closeIterator(this.iterator);
}
}

Thanks,
-Paul

-Original Message-
From: Paul A. Jackson [mailto:paul.jack...@pb.com]
Sent: Friday, January 27, 2017 1:55 PM
To: dev@tinkerpop.apache.org
Subject: RE: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced 
CloseableIterator

I can do this, and I can see why you are suggesting it, but I didn't originally 
because it means overriding all the methods from FlatMapStep into these 
subclasses.

The way it's currently designed all the subclasses need to do is implement 
flatMap(). FlatMapStep has private access to iterator and head. It knows when 
it is done with an iterator, that when it's done with an iterator that it 
should replace it with an EmptyIterator, that if it's not done it needs to call 
head.split with the next element, etc.

To implement what you suggest, correct me if I'm wrong, all that private 
knowledge needs to be duplicated in the subclasses. If feels like a bad design 
because if any of those behaviors change in FlatMapStep we need to remember to 
go into the subclasses and mirror the changes there.

Maybe I should move the try/catch stuff to a helper method so that all 
FlatMapStep needs to do is execute one line when it's done with an iterator?

Should I submit parallel pull request so you can compare/contrast?

-Paul

-Original Message-
From: spmallette [mailto:g...@git.apache.org]
Sent: Friday, January 27, 2017 1:23 PM
To: dev@tinkerpop.apache.org
Subject: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced 
CloseableIterator

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/548

I'm pretty sure @okram was affirming that `VertexStep`, `EdgeVertexStep` 
and `PropertiesStep` should implement `AutoCloseable` (I don't think that there 
are others) rather than a blanket change of just applying it to `FlatMapStep`. 
As the logic is re-used, perhaps you could do an `ElementStep` that extends the 
`FlatMapStep` and implements `AutoCloseable`. Then have those three steps 
extend `ElementStep` which would contain the close logic that you have.


---
If your project is set up for it, you can reply to this email and have your 
reply appear on GitHub as well. If your project does not have this feature 
enabled and wishes so, or if the feature is enabled but not working, please 
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with 
INFRA.
---

RE: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator

2017-01-27 Thread Paul A. Jackson

Hi Stephen,

I think there are two parts to this, and I may have responded to the wrong part.

I can easily and cleanly make those subclasses implement AutoCloseable and 
remove that from FlatMapStep.

Implementing AutoCloseable is needed for the second half of the problem, where 
a traversal is closed before it is read to completion. This is easily delegated 
to the subclasses.

There's the first half of the problem dealing with closing iterators as 
FlatMapStep finishes with them, but before it loses its reference to them. This 
is very hard to remove from FlatMapStep without duplicating all the code in the 
subclasses.

If you consent I'd like to update the pull request that removes AutoCloseable 
from FlatMapStep but keeps the closing of iterators on the fly in FlatMapStep.

I'd can also move that try/catch stuff to a helper method (in 
CloseableInterface?). That keeps FlatMapStep very clean.

Thoughts?

public abstract class FlatMapStep<S, E> extends AbstractStep<S, E> {

private Traverser.Admin head = null;
protected Iterator iterator = EmptyIterator.instance(); // Made protected

public FlatMapStep(final Traversal.Admin traversal) {
super(traversal);
}

@Override
protected Traverser.Admin processNextStart() {
while (true) {
if (this.iterator.hasNext()) {
return this.head.split(this.iterator.next(), this);
} else {
this.head = this.starts.next();
closeIterator();// Added
this.iterator = this.flatMap(this.head);
}
}
}

protected abstract Iterator flatMap(final Traverser.Admin traverser);

@Override
public void reset() {
super.reset();
closeIterator();// Added
this.iterator = EmptyIterator.instance();
}

void closeIterator() {
CloseableIterator.closeIterator(this.iterator);
}
}

Thanks,
-Paul

-Original Message-
From: Paul A. Jackson [mailto:paul.jack...@pb.com]
Sent: Friday, January 27, 2017 1:55 PM
To: dev@tinkerpop.apache.org
Subject: RE: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced 
CloseableIterator

I can do this, and I can see why you are suggesting it, but I didn't originally 
because it means overriding all the methods from FlatMapStep into these 
subclasses.

The way it's currently designed all the subclasses need to do is implement 
flatMap(). FlatMapStep has private access to iterator and head. It knows when 
it is done with an iterator, that when it's done with an iterator that it 
should replace it with an EmptyIterator, that if it's not done it needs to call 
head.split with the next element, etc.

To implement what you suggest, correct me if I'm wrong, all that private 
knowledge needs to be duplicated in the subclasses. If feels like a bad design 
because if any of those behaviors change in FlatMapStep we need to remember to 
go into the subclasses and mirror the changes there.

Maybe I should move the try/catch stuff to a helper method so that all 
FlatMapStep needs to do is execute one line when it's done with an iterator?

Should I submit parallel pull request so you can compare/contrast?

-Paul

-Original Message-
From: spmallette [mailto:g...@git.apache.org]
Sent: Friday, January 27, 2017 1:23 PM
To: dev@tinkerpop.apache.org
Subject: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced 
CloseableIterator

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/548

I'm pretty sure @okram was affirming that `VertexStep`, `EdgeVertexStep` 
and `PropertiesStep` should implement `AutoCloseable` (I don't think that there 
are others) rather than a blanket change of just applying it to `FlatMapStep`. 
As the logic is re-used, perhaps you could do an `ElementStep` that extends the 
`FlatMapStep` and implements `AutoCloseable`. Then have those three steps 
extend `ElementStep` which would contain the close logic that you have.


---
If your project is set up for it, you can reply to this email and have your 
reply appear on GitHub as well. If your project does not have this feature 
enabled and wishes so, or if the feature is enabled but not working, please 
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with 
INFRA.
---

RE: [DISCUSS] Release 3.2.4 and 3.1.6

2017-01-26 Thread Paul A. Jackson

OK, that's done, as you can no doubt see. Hoping this can make it into 3.2.4.

Thanks.

-Paul

-Original Message-
From: Stephen Mallette [mailto:spmalle...@gmail.com]
Sent: Wednesday, January 25, 2017 6:13 PM
To: dev@tinkerpop.apache.org
Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6

sure - please put that in the prefix of the PR title so it links back to that 
original jira ticket

On Wed, Jan 25, 2017 at 5:17 PM, Paul A. Jackson <paul.jack...@pb.com>
wrote:

> OK, low-risk PR coming for OLTP part. Should I reuse TINKERPOP-1589?
>
> Thanks,
> -Paul
>
> -Original Message-
> From: Stephen Mallette [mailto:spmalle...@gmail.com]
> Sent: Wednesday, January 25, 2017 5:07 PM
> To: dev@tinkerpop.apache.org
> Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6
>
> Being as close as we are to our code freeze/test week, I'd say that
> big, complex or otherwise risky changes probably won't collect too
> many +1 reviews at this point. If the OLTP improvement is
> small/concise (low risk), it could be considered for inclusion in 3.2.4.
>
> On Wed, Jan 25, 2017 at 5:00 PM, Paul A. Jackson <paul.jack...@pb.com>
> wrote:
>
> > I have something that fixes OLTP. I haven't worked with OLAP and it
> > looks like the changes for this will be extensive, touching
> > IteratorUtils and so on.
> >
> > Would you be interested in a PR for just the OLTP part?
> >
> >
> > -Original Message-
> > From: Stephen Mallette [mailto:spmalle...@gmail.com]
> > Sent: Wednesday, January 25, 2017 3:58 PM
> > To: dev@tinkerpop.apache.org
> > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6
> >
> > in his case, it should go to tp32.
> >
> > On Wed, Jan 25, 2017 at 3:56 PM, Paul A. Jackson
> > <paul.jack...@pb.com>
> > wrote:
> >
> > > For what branch should a pull request be submitted?
> > >
> > >
> > > -Original Message-
> > > From: Stephen Mallette [mailto:spmalle...@gmail.com]
> > > Sent: Wednesday, January 25, 2017 3:41 PM
> > > To: dev@tinkerpop.apache.org
> > > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6
> > >
> > > I went with the most obvious implementation place for
> CloseableIterator.
> > > If you see other spots where you could make an argument that it
> > > would make sense to add it then feel free to offer a pull request
> > > and we can get it reviewed. I didn't look into your VertexStep
> > > suggestion too deeply, but a quick review seems to have me
> > > thinking that it would make
> > sense to do that.
> > > Basically anywhere that a step interacts with the structure API
> > > seems like it would be a candidate for CloseableIterator to be
> > > implemented as it is in GraphStep.
> > >
> > > On Wed, Jan 25, 2017 at 3:30 PM, Paul A. Jackson
> > > <paul.jack...@pb.com>
> > > wrote:
> > >
> > > > So, I modified my code to work with CloseableIterator. I was
> > > > hoping this would be honored in more places than it is.
> > > >
> > > > Where it does work is if the user of a traversal calls
> > > > traversal.close() all the steps will get closed, including the
> > > > typically
> > > first GraphStep.
> > > > GraphStep in turn checks whether the iterator that was provided
> > > > by iteratorSupplier implements CloseableIterator and if so,
> > > > closes it, and this is good.
> > > >
> > > > What I was hoping, in addition, though, was when
> > > > VertexStep.flatMap() (or anything else) calls Vertex.vertices()
> > > > or
> > > > Vertex.edges() that before it finishes with the iterator it also
> > > > make the same check for CloseableIterator and call close().
> > > >
> > > > Thoughts?
> > > >
> > > > -Paul
> > > >
> > > > -Original Message-
> > > > From: Paul A. Jackson [mailto:paul.jack...@pb.com]
> > > > Sent: Tuesday, January 24, 2017 3:01 PM
> > > > To: dev@tinkerpop.apache.org
> > > > Subject: RE: [DISCUSS] Release 3.2.4 and 3.1.6
> > > >
> > > > Great. I'll try it out.
> > > >
> > > > -Paul
> > > >
> > > >
> > > > -Original Message-
> > > > From: Stephen Mallette [mailto:spmalle...@gmail.com]
> > > > Sent: Tuesday, January 24, 2017 2:54 PM
> > > > To: dev@tinkerpop.apache.org
> > > > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6
> > > >
&

RE: [DISCUSS] Release 3.2.4 and 3.1.6

2017-01-25 Thread Paul A. Jackson

OK, low-risk PR coming for OLTP part. Should I reuse TINKERPOP-1589?

Thanks,
-Paul

-Original Message-
From: Stephen Mallette [mailto:spmalle...@gmail.com]
Sent: Wednesday, January 25, 2017 5:07 PM
To: dev@tinkerpop.apache.org
Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6

Being as close as we are to our code freeze/test week, I'd say that big, 
complex or otherwise risky changes probably won't collect too many +1 reviews 
at this point. If the OLTP improvement is small/concise (low risk), it could be 
considered for inclusion in 3.2.4.

On Wed, Jan 25, 2017 at 5:00 PM, Paul A. Jackson <paul.jack...@pb.com>
wrote:

> I have something that fixes OLTP. I haven't worked with OLAP and it
> looks like the changes for this will be extensive, touching
> IteratorUtils and so on.
>
> Would you be interested in a PR for just the OLTP part?
>
>
> -Original Message-
> From: Stephen Mallette [mailto:spmalle...@gmail.com]
> Sent: Wednesday, January 25, 2017 3:58 PM
> To: dev@tinkerpop.apache.org
> Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6
>
> in his case, it should go to tp32.
>
> On Wed, Jan 25, 2017 at 3:56 PM, Paul A. Jackson <paul.jack...@pb.com>
> wrote:
>
> > For what branch should a pull request be submitted?
> >
> >
> > -Original Message-
> > From: Stephen Mallette [mailto:spmalle...@gmail.com]
> > Sent: Wednesday, January 25, 2017 3:41 PM
> > To: dev@tinkerpop.apache.org
> > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6
> >
> > I went with the most obvious implementation place for CloseableIterator.
> > If you see other spots where you could make an argument that it
> > would make sense to add it then feel free to offer a pull request
> > and we can get it reviewed. I didn't look into your VertexStep
> > suggestion too deeply, but a quick review seems to have me thinking
> > that it would make
> sense to do that.
> > Basically anywhere that a step interacts with the structure API
> > seems like it would be a candidate for CloseableIterator to be
> > implemented as it is in GraphStep.
> >
> > On Wed, Jan 25, 2017 at 3:30 PM, Paul A. Jackson
> > <paul.jack...@pb.com>
> > wrote:
> >
> > > So, I modified my code to work with CloseableIterator. I was
> > > hoping this would be honored in more places than it is.
> > >
> > > Where it does work is if the user of a traversal calls
> > > traversal.close() all the steps will get closed, including the
> > > typically
> > first GraphStep.
> > > GraphStep in turn checks whether the iterator that was provided by
> > > iteratorSupplier implements CloseableIterator and if so, closes
> > > it, and this is good.
> > >
> > > What I was hoping, in addition, though, was when
> > > VertexStep.flatMap() (or anything else) calls Vertex.vertices() or
> > > Vertex.edges() that before it finishes with the iterator it also
> > > make the same check for CloseableIterator and call close().
> > >
> > > Thoughts?
> > >
> > > -Paul
> > >
> > > -Original Message-
> > > From: Paul A. Jackson [mailto:paul.jack...@pb.com]
> > > Sent: Tuesday, January 24, 2017 3:01 PM
> > > To: dev@tinkerpop.apache.org
> > > Subject: RE: [DISCUSS] Release 3.2.4 and 3.1.6
> > >
> > > Great. I'll try it out.
> > >
> > > -Paul
> > >
> > >
> > > -Original Message-
> > > From: Stephen Mallette [mailto:spmalle...@gmail.com]
> > > Sent: Tuesday, January 24, 2017 2:54 PM
> > > To: dev@tinkerpop.apache.org
> > > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6
> > >
> > > no - it's in 3.2.4 and merged forward to 3.3.0:
> > >
> > > https://github.com/apache/tinkerpop/blob/e3889bf2401b42c3afbc85eab
> > > c2 fb c ebf2588974/gremlin-core/src/main/java/org/apache/
> > > tinkerpop/gremlin/structure/util/CloseableIterator.java
> > >
> > > On Tue, Jan 24, 2017 at 2:52 PM, Paul A. Jackson
> > > <paul.jack...@pb.com>
> > > wrote:
> > >
> > > > Is CloseableIterator only in the 3.3 branch?
> > > >
> > > > -Paul
> > > >
> > > >
> > > > -Original Message-
> > > > From: Stephen Mallette [mailto:spmalle...@gmail.com]
> > > > Sent: Monday, January 23, 2017 9:58 AM
> > > > To: dev@tinkerpop.apache.org
> > > > Subject: [DISCUSS] Release 3.2.4 and 3.1.6
> > > >
> > > > It's been a while since we've had a release (Octob

RE: [DISCUSS] Release 3.2.4 and 3.1.6

2017-01-25 Thread Paul A. Jackson

I have something that fixes OLTP. I haven't worked with OLAP and it looks like 
the changes for this will be extensive, touching IteratorUtils and so on.

Would you be interested in a PR for just the OLTP part?


-Original Message-
From: Stephen Mallette [mailto:spmalle...@gmail.com]
Sent: Wednesday, January 25, 2017 3:58 PM
To: dev@tinkerpop.apache.org
Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6

in his case, it should go to tp32.

On Wed, Jan 25, 2017 at 3:56 PM, Paul A. Jackson <paul.jack...@pb.com>
wrote:

> For what branch should a pull request be submitted?
>
>
> -Original Message-
> From: Stephen Mallette [mailto:spmalle...@gmail.com]
> Sent: Wednesday, January 25, 2017 3:41 PM
> To: dev@tinkerpop.apache.org
> Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6
>
> I went with the most obvious implementation place for CloseableIterator.
> If you see other spots where you could make an argument that it would
> make sense to add it then feel free to offer a pull request and we can
> get it reviewed. I didn't look into your VertexStep suggestion too
> deeply, but a quick review seems to have me thinking that it would make sense 
> to do that.
> Basically anywhere that a step interacts with the structure API seems
> like it would be a candidate for CloseableIterator to be implemented
> as it is in GraphStep.
>
> On Wed, Jan 25, 2017 at 3:30 PM, Paul A. Jackson <paul.jack...@pb.com>
> wrote:
>
> > So, I modified my code to work with CloseableIterator. I was hoping
> > this would be honored in more places than it is.
> >
> > Where it does work is if the user of a traversal calls
> > traversal.close() all the steps will get closed, including the
> > typically
> first GraphStep.
> > GraphStep in turn checks whether the iterator that was provided by
> > iteratorSupplier implements CloseableIterator and if so, closes it,
> > and this is good.
> >
> > What I was hoping, in addition, though, was when
> > VertexStep.flatMap() (or anything else) calls Vertex.vertices() or
> > Vertex.edges() that before it finishes with the iterator it also
> > make the same check for CloseableIterator and call close().
> >
> > Thoughts?
> >
> > -Paul
> >
> > -Original Message-
> > From: Paul A. Jackson [mailto:paul.jack...@pb.com]
> > Sent: Tuesday, January 24, 2017 3:01 PM
> > To: dev@tinkerpop.apache.org
> > Subject: RE: [DISCUSS] Release 3.2.4 and 3.1.6
> >
> > Great. I'll try it out.
> >
> > -Paul
> >
> >
> > -Original Message-
> > From: Stephen Mallette [mailto:spmalle...@gmail.com]
> > Sent: Tuesday, January 24, 2017 2:54 PM
> > To: dev@tinkerpop.apache.org
> > Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6
> >
> > no - it's in 3.2.4 and merged forward to 3.3.0:
> >
> > https://github.com/apache/tinkerpop/blob/e3889bf2401b42c3afbc85eabc2
> > fb c ebf2588974/gremlin-core/src/main/java/org/apache/
> > tinkerpop/gremlin/structure/util/CloseableIterator.java
> >
> > On Tue, Jan 24, 2017 at 2:52 PM, Paul A. Jackson
> > <paul.jack...@pb.com>
> > wrote:
> >
> > > Is CloseableIterator only in the 3.3 branch?
> > >
> > > -Paul
> > >
> > >
> > > -Original Message-
> > > From: Stephen Mallette [mailto:spmalle...@gmail.com]
> > > Sent: Monday, January 23, 2017 9:58 AM
> > > To: dev@tinkerpop.apache.org
> > > Subject: [DISCUSS] Release 3.2.4 and 3.1.6
> > >
> > > It's been a while since we've had a release (October 2016) and
> > > given the importance of the recent critical security fix from
> > > Groovy 2.4.8 I think it's worth getting some new versions out
> > > there. I'm not sure what everyone is working on or has concerns
> > > about, but after the PRs that are out there for tp32 get merged
> > > (especially https://github.com/apache/
> > > tinkerpop/pull/541)
> > > I don't really have anything else critical for those versions.
> > > Please call out any issues that might be important for this
> > > release on this
> > thread.
> > >
> > > I don't think we should worry about doing a milestone release of
> > > 3.3.0
> > yet.
> > > I'd like to see some more change go into that branch before we do
> > > that, but if others feel differently and would like to offer an
> > > argument I'd be open to the idea.
> > >
> > > I propose we focus on a release of 3.1.6 and 3.2.4 in two weeks
> > > time with the code freeze going into place at end of day friday of
> > > this week (January 27, 2017). If there are no objections in the
> > > next three days (Thursday, January 26, 2017, 10:00am), let's
> > > assume lazy consensus and move forward with that plan.
> > >
> > > Assuming we do move forward with a release, are there any
> > > volunteers for release manager?
> > >
> > > 
> > >
> > >
> >
> > 
> >
> >
> > 
> >
> >
>
> 
>
>

RE: [DISCUSS] Release 3.2.4 and 3.1.6

2017-01-25 Thread Paul A. Jackson

For what branch should a pull request be submitted?


-Original Message-
From: Stephen Mallette [mailto:spmalle...@gmail.com]
Sent: Wednesday, January 25, 2017 3:41 PM
To: dev@tinkerpop.apache.org
Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6

I went with the most obvious implementation place for CloseableIterator. If you 
see other spots where you could make an argument that it would make sense to 
add it then feel free to offer a pull request and we can get it reviewed. I 
didn't look into your VertexStep suggestion too deeply, but a quick review 
seems to have me thinking that it would make sense to do that.
Basically anywhere that a step interacts with the structure API seems like it 
would be a candidate for CloseableIterator to be implemented as it is in 
GraphStep.

On Wed, Jan 25, 2017 at 3:30 PM, Paul A. Jackson <paul.jack...@pb.com>
wrote:

> So, I modified my code to work with CloseableIterator. I was hoping
> this would be honored in more places than it is.
>
> Where it does work is if the user of a traversal calls
> traversal.close() all the steps will get closed, including the typically 
> first GraphStep.
> GraphStep in turn checks whether the iterator that was provided by
> iteratorSupplier implements CloseableIterator and if so, closes it,
> and this is good.
>
> What I was hoping, in addition, though, was when VertexStep.flatMap()
> (or anything else) calls Vertex.vertices() or Vertex.edges() that
> before it finishes with the iterator it also make the same check for
> CloseableIterator and call close().
>
> Thoughts?
>
> -Paul
>
> -Original Message-
> From: Paul A. Jackson [mailto:paul.jack...@pb.com]
> Sent: Tuesday, January 24, 2017 3:01 PM
> To: dev@tinkerpop.apache.org
> Subject: RE: [DISCUSS] Release 3.2.4 and 3.1.6
>
> Great. I'll try it out.
>
> -Paul
>
>
> -Original Message-
> From: Stephen Mallette [mailto:spmalle...@gmail.com]
> Sent: Tuesday, January 24, 2017 2:54 PM
> To: dev@tinkerpop.apache.org
> Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6
>
> no - it's in 3.2.4 and merged forward to 3.3.0:
>
> https://github.com/apache/tinkerpop/blob/e3889bf2401b42c3afbc85eabc2fb
> c ebf2588974/gremlin-core/src/main/java/org/apache/
> tinkerpop/gremlin/structure/util/CloseableIterator.java
>
> On Tue, Jan 24, 2017 at 2:52 PM, Paul A. Jackson <paul.jack...@pb.com>
> wrote:
>
> > Is CloseableIterator only in the 3.3 branch?
> >
> > -Paul
> >
> >
> > -Original Message-
> > From: Stephen Mallette [mailto:spmalle...@gmail.com]
> > Sent: Monday, January 23, 2017 9:58 AM
> > To: dev@tinkerpop.apache.org
> > Subject: [DISCUSS] Release 3.2.4 and 3.1.6
> >
> > It's been a while since we've had a release (October 2016) and given
> > the importance of the recent critical security fix from Groovy 2.4.8
> > I think it's worth getting some new versions out there. I'm not sure
> > what everyone is working on or has concerns about, but after the PRs
> > that are out there for tp32 get merged (especially
> > https://github.com/apache/
> > tinkerpop/pull/541)
> > I don't really have anything else critical for those versions.
> > Please call out any issues that might be important for this release
> > on this
> thread.
> >
> > I don't think we should worry about doing a milestone release of
> > 3.3.0
> yet.
> > I'd like to see some more change go into that branch before we do
> > that, but if others feel differently and would like to offer an
> > argument I'd be open to the idea.
> >
> > I propose we focus on a release of 3.1.6 and 3.2.4 in two weeks time
> > with the code freeze going into place at end of day friday of this
> > week (January 27, 2017). If there are no objections in the next
> > three days (Thursday, January 26, 2017, 10:00am), let's assume lazy
> > consensus and move forward with that plan.
> >
> > Assuming we do move forward with a release, are there any volunteers
> > for release manager?
> >
> > 
> >
> >
>
> 
>
>
> 
>
>

RE: [DISCUSS] Release 3.2.4 and 3.1.6

2017-01-25 Thread Paul A. Jackson

So, I modified my code to work with CloseableIterator. I was hoping this would 
be honored in more places than it is.

Where it does work is if the user of a traversal calls traversal.close() all 
the steps will get closed, including the typically first GraphStep. GraphStep 
in turn checks whether the iterator that was provided by iteratorSupplier 
implements CloseableIterator and if so, closes it, and this is good.

What I was hoping, in addition, though, was when VertexStep.flatMap() (or 
anything else) calls Vertex.vertices() or Vertex.edges() that before it 
finishes with the iterator it also make the same check for CloseableIterator 
and call close().

Thoughts?

-Paul

-Original Message-
From: Paul A. Jackson [mailto:paul.jack...@pb.com]
Sent: Tuesday, January 24, 2017 3:01 PM
To: dev@tinkerpop.apache.org
Subject: RE: [DISCUSS] Release 3.2.4 and 3.1.6

Great. I'll try it out.

-Paul


-Original Message-
From: Stephen Mallette [mailto:spmalle...@gmail.com]
Sent: Tuesday, January 24, 2017 2:54 PM
To: dev@tinkerpop.apache.org
Subject: Re: [DISCUSS] Release 3.2.4 and 3.1.6

no - it's in 3.2.4 and merged forward to 3.3.0:

https://github.com/apache/tinkerpop/blob/e3889bf2401b42c3afbc85eabc2fbcebf2588974/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/util/CloseableIterator.java

On Tue, Jan 24, 2017 at 2:52 PM, Paul A. Jackson <paul.jack...@pb.com>
wrote:

> Is CloseableIterator only in the 3.3 branch?
>
> -Paul
>
>
> -Original Message-
> From: Stephen Mallette [mailto:spmalle...@gmail.com]
> Sent: Monday, January 23, 2017 9:58 AM
> To: dev@tinkerpop.apache.org
> Subject: [DISCUSS] Release 3.2.4 and 3.1.6
>
> It's been a while since we've had a release (October 2016) and given
> the importance of the recent critical security fix from Groovy 2.4.8 I
> think it's worth getting some new versions out there. I'm not sure
> what everyone is working on or has concerns about, but after the PRs
> that are out there for tp32 get merged (especially
> https://github.com/apache/
> tinkerpop/pull/541)
> I don't really have anything else critical for those versions. Please
> call out any issues that might be important for this release on this thread.
>
> I don't think we should worry about doing a milestone release of 3.3.0 yet.
> I'd like to see some more change go into that branch before we do
> that, but if others feel differently and would like to offer an
> argument I'd be open to the idea.
>
> I propose we focus on a release of 3.1.6 and 3.2.4 in two weeks time
> with the code freeze going into place at end of day friday of this
> week (January 27, 2017). If there are no objections in the next three
> days (Thursday, January 26, 2017, 10:00am), let's assume lazy
> consensus and move forward with that plan.
>
> Assuming we do move forward with a release, are there any volunteers
> for release manager?
>
> 
>
>

RE: [DISCUSS] Release 3.2.4 and 3.1.6

2017-01-24 Thread Paul A. Jackson

Is CloseableIterator only in the 3.3 branch?

-Paul

-Original Message-
From: Stephen Mallette [mailto:spmalle...@gmail.com]
Sent: Monday, January 23, 2017 9:58 AM
To: dev@tinkerpop.apache.org
Subject: [DISCUSS] Release 3.2.4 and 3.1.6

It's been a while since we've had a release (October 2016) and given the 
importance of the recent critical security fix from Groovy 2.4.8 I think it's 
worth getting some new versions out there. I'm not sure what everyone is 
working on or has concerns about, but after the PRs that are out there for tp32 
get merged (especially https://github.com/apache/tinkerpop/pull/541)
I don't really have anything else critical for those versions. Please call out 
any issues that might be important for this release on this thread.

I don't think we should worry about doing a milestone release of 3.3.0 yet.
I'd like to see some more change go into that branch before we do that, but if 
others feel differently and would like to offer an argument I'd be open to the 
idea.

I propose we focus on a release of 3.1.6 and 3.2.4 in two weeks time with the 
code freeze going into place at end of day friday of this week (January 27, 
2017). If there are no objections in the next three days (Thursday, January 26, 
2017, 10:00am), let's assume lazy consensus and move forward with that plan.

Assuming we do move forward with a release, are there any volunteers for 
release manager?

RE: [TinkerPop] gremlin-x

2016-12-06 Thread Paul A. Jackson

I have 2 cents to throw in FWIW.

We use Neo4j and support Gremlin as a query language.

Sometimes it makes sense to preload properties when you load a vertex if you 
are in a concurrent environment. Say, you find a vertex via query lookup, if 
you read the actual property lazily it's possible for that property to change 
or disappear (Neo4j has READ_COMMITTED isolation level). But, if you are in a 
context where others cannot write you can benefit from loading properties 
lazily.

For us, this knowledge and the related decisions can all be done below the 
Gremlin layer.

-Paul

-Original Message-
From: pieter-gmail [mailto:pieter.mar...@gmail.com]
Sent: Tuesday, December 06, 2016 10:43 AM
To: gremlin-us...@googlegroups.com
Cc: dev@tinkerpop.apache.org
Subject: Re: [TinkerPop] gremlin-x

Hi,

Now that we have established our disagreement lets continue...

"You prefer your model to be that of the underlying graph (following that 
logic, you would use Hibernate to map to Table objects?) and I prefer using 
application domain models."

When working with TinkerPop I certainly prefer that its model is that of the 
underlying meta model i.e. having the property graph model as a first class 
construct. For modeling Cat/Dog/Person/Organization/Molecule... I also, like 
you use application models as an abstraction on top of TinkerPop. Hibernate on 
top off TinkerPop as opposed to JDBC if you will.

"You prefer your query to return the underlying graph model and I prefer it to 
return any data."

True, however I am totally fine with g.V().this().that().values(...); for the 
any data query.

"You prefer your query to always return all properties and I prefer it to 
always return only selected properties."

This is partially true and indeed the default that I and I'd say the TinkerPop 
test suites have assumed.
There are countless gremlins that return GraphTraversal that 
happily iterate the traversal accessing its properties assuming that they are 
right there and not another db hit. It has bitten me though with some fat 
vertices making it not such a sensible default anymore. And when loading 
vertices just to create edges, then loading the properties are somewhat silly.

"You prefer your objects to be proxies to the underlying datastore (I think 
this blurs the lines between being a graph provider and gremlin
client) and I prefer my objects to be detached with load/store being explicit."

True. My programming model assumes this to be the case and again so does the 
TinkerPop test suite since day one. I'd argue it makes for eloquent code. No 
need to worry about attaching and detaching and reattaching with transaction 
semantics getting confused. You read and write within the same transaction 
boundaries with the Vertex/Edge being bound to the transaction. Very similar to 
you previous mail where you suggested binding the interaction to a connection, 
except for the object we bind to to is the transaction. Pretty much a one to 
one between transaction and connection for most ACID databases.

"In the end, it sounds like you want gremlin to be an object-graph mapper in 
the graph model and I prefer a layered approach where gremlin is a simple query 
language of which an object-graph mapper, in any domain model, could be built 
on top (like so many other query languages)."

Not so true. Object Graph Mappers will use TinkerPop as the graph layer to 
model their domain on top off. TinkerPop responsibility is to model its own 
model. The property graph model. One can think of Vertex/Edge/Property as a 
baby hardcoded meta ORM but maybe its not a useful analogy.

All that said I think we are not all that far apart as the way I read it our 
primary disagreement is the properties.

Lets take GraphTraversal.
If its a ReferencedVertex then every property access will be another round 
trip. This seems unacceptable to me.
If it returns a Vertex with all its properties pre-loaded then its also 
unacceptable. (Even if its the current default that I assume)

So perhaps what Marko suggested

g.withDetachment(…)
  - Detach.reference // just ids
  - Detach.reduced // ids, labels, keys, values (basically, what you
need for toString())
  - Detach.rich // property data included
  - Detached.full // edge data included (basically, StarGraph)

will makes us both happy?

Cheers
Pieter

On 05/12/2016 17:06, Robert Dale wrote:
> Clearly we have different use cases.
>
> You prefer your model to be that of the underlying graph (following
> that logic, you would use Hibernate to map to Table objects?) and I
> prefer using application domain models.
>
> You prefer your query to return the underlying graph model and I
> prefer it to return any data.
>
> You prefer your query to always return all properties and I prefer it
> to always return only selected properties.
>
> You prefer your objects to be proxies to the underlying datastore (I
> think this blurs the lines between being a

RE: io and graphson-v3

RE: Requirements for listing as a vendor

RE: [DISCUSS] ASF Board Draft Report - April 2017

RE: Requirements for listing as a vendor

RE: Requirements for listing as a vendor

Requirements for listing as a vendor

RE: [GitHub] tinkerpop issue #567: TINKERPOP-1644 Improve script compilation syncronisati...

RE: [DISCUSS] P.not and __.not

RE: [VOTE] TinkerPop 3.2.4 Release

RE: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator

RE: [GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator

RE: [DISCUSS] Release 3.2.4 and 3.1.6

RE: [DISCUSS] Release 3.2.4 and 3.1.6

RE: [DISCUSS] Release 3.2.4 and 3.1.6

RE: [DISCUSS] Release 3.2.4 and 3.1.6

RE: [DISCUSS] Release 3.2.4 and 3.1.6

RE: [DISCUSS] Release 3.2.4 and 3.1.6

RE: [TinkerPop] gremlin-x

18 matches

Site Navigation

Mail list logo

Footer information