Re: [DISCUSS] Thinking on Release

[Stephen Mallette]

Thanks for the update Ted. We may need an extra day or two to clear up some
other discussion on some of the other PRs - i don't think we can start a
code freeze on Monday as I'd proposed.

On Fri, Jul 1, 2016 at 6:49 PM, Ted Wilmes  wrote:

> Hi everyone,
> I had some time this week and made further progress on 1254.  Still working
> out some kinks but I'll see how far I can get over the weekend.  Have a
> good 4th.
>
> --Ted
>
> On Tue, Jun 28, 2016 at 11:50 AM, Marko Rodriguez 
> wrote:
>
> > Hi,
> >
> > Ted — I think we should get that work into the next release. Thus, if you
> > need more time (reasonable amount), then I say we delay accordingly.
> >
> > NOTE: The Gremlin-Python stuff will not get into the next release. It has
> > gotten really complex/powerful and is currently 100+ commits ahead of
> > master/ ! :) .. As such, given the gargantuan undertaking, we will save
> > this work for a future release.
> >
> > Thanks,
> > Marko.
> >
> > http://markorodriguez.com
> >
> >
> >
> > > On Jun 28, 2016, at 10:36 AM, Ted Wilmes  wrote:
> > >
> > > I would like to get TinkerPop-1254 in before code freeze.  Shooting for
> > > finishing it up by end of this week but I'll drop a note if it looks
> > like I
> > > won't make it.
> > >
> > > --Ted
> > >
> > > On Tue, Jun 28, 2016 at 5:35 AM, Stephen Mallette <
> spmalle...@gmail.com>
> > > wrote:
> > >
> > >> It's end of June - time to start firming up for release. We have a
> > number
> > >> of outstanding PRs that need votes/merge. Perhaps we try to get those
> > all
> > >> in this week and begin code freeze next Monday (7/4)?
> > >>
> > >> I think we'll have to postpone the GLV work with gremlin-python for
> this
> > >> release. There's just too much left to do to get that in "right". Are
> > there
> > >> any other open issues of importance?
> > >>
> > >> On Mon, Jun 6, 2016 at 9:18 AM, Marko Rodriguez  >
> > >> wrote:
> > >>
> > >>> Yes, end of June-ish is best for me as I have few things on my plate
> > the
> > >>> first half of this month.
> > >>>
> > >>> Thanks,
> > >>> Marko.
> > >>>
> > >>> http://markorodriguez.com
> > >>>
> > >>>
> >  On Jun 6, 2016, at 4:46 AM, Stephen Mallette 
> > >>> wrote:
> > 
> >  We didn't really discuss a date for release on this thread. I was
> > >>> thinking
> >  that we could start looking at the week of July 4th as the target
> week
> > >>> for
> >  VOTE and nail down a date as we get closer.
> > 
> >  On Wed, May 25, 2016 at 2:42 PM, Jason Plurad 
> > >> wrote:
> > 
> > > I'd think from a TinkerPop branding perspective, it probably helps
> to
> > >>> have
> > > the name in there. It's Apache TinkerPop, not Apache Gremlin.
> > >
> > > I just took a quick look on a mirror, and some other Apache
> projects
> > > (Spark, Kafka, HBase, NiFi, Pig, Zookeeper) don't even include
> apache
> > >> in
> > > their distributables, so maybe we can just do:
> > >
> > > tinkerpop-gremlin-console-x.y.z.zip
> > > tinkerpop-gremlin-server-x.y.z.zip
> > >
> > >
> > > On Wed, May 25, 2016 at 11:03 AM Stephen Mallette <
> > >> spmalle...@gmail.com
> > 
> > > wrote:
> > >
> > >> jason i think that was a suggestion to conform more to standard
> > >> apache
> > >> releases from someone in incubator. if it was mandatory we would
> > have
> > >> burned for that too many times to count at this point. i'm good to
> > >>> change
> > >> it if everyone else is. what do we want them to be?
> > >>
> > >> apache-tinkerpop-console-x.y.z.zip
> > >> apache-tinkerpop-server-x.y.z.zip
> > >>
> > >> or the full business:
> > >>
> > >> apache-tinkerpop-gremlin-console-x.y.z.zip
> > >> apache-tinkerpop-gremlin-server-x.y.z.zip
> > >>
> > >> i guess we lost "-incubating" now so the latter doesn't look so
> bad
> > >> to
> > >>> me
> > >> anymore.
> > >>
> > >>
> > >> On Wed, May 25, 2016 at 10:56 AM, Marko Rodriguez <
> > >>> okramma...@gmail.com>
> > >> wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> Yes, an imminent release is good. There are 2 severe bug fixes in
> > > master/
> > >>> (3.2.1) that I would like to get out there. 3.2.0 had lots of
> > >> internal
> > >>> changes to OLAP and I paid the price by incurring bugs. :|
> > >>>
> >  Somebody had mentioned that our distributables are supposed to
> be
> > > named
> >  apache-tinkerpop*.zip instead of apache-gremlin*.zip. Maybe
> that's
> >  something that should be done along with this release.
> > >>>
> > >>> There is really no such thing as "tinkerpop" besides the source
> > code
> > >> which
> > >>> is distributed as apache-tinkerpop-*.zip. The two other
> > >> distributions
> > > are
> > >>> gremlin-console and gremlin-server and I think 

[jira] [Commented] (TINKERPOP-1341) UnshadedKryoAdapter fails to deserialize StarGraph when SparkConf sets spark.rdd.compress=true whereas GryoSerializer works

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359760#comment-15359760
 ] 

ASF GitHub Bot commented on TINKERPOP-1341:
---

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/353
  
VOTE: +1


> UnshadedKryoAdapter fails to deserialize StarGraph when SparkConf sets 
> spark.rdd.compress=true whereas GryoSerializer works
> ---
>
> Key: TINKERPOP-1341
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1341
> Project: TinkerPop
>  Issue Type: Bug
>  Components: io
>Affects Versions: 3.2.1, 3.3.0
>Reporter: Dylan Bethune-Waddell
>Priority: Minor
>
> When trying to bulk load a large dataset into Titan I was running into OOM 
> errors and decided to try tweaking some spark configuration settings - 
> although I am having trouble bulk loading with the new 
> GryoRegistrator/UnshadedKryo serialization shim stuff in master whereby a few 
> hundred tasks into the edge loading stage (stage 5) exceptions are thrown 
> complaining about the need to explicitly register CompactBuffer[].class with 
> Kryo, this approach with spark.rdd.compress=true fails a few hundred tasks 
> into the vertex loading stage (stage 1) of BulkLoaderVertexProgram. 
> GryoSerializer instead of KryoSerializer with GryoRegistrator does not fail 
> and successfully loads the data with this compression flag flipped on whereas 
> before I would just get OOM errors until eventually the job was set back so 
> far that it just failed. So it would seem it is desirable in some instances 
> to use this setting, and the new Serialization stuff seems to break it. Could 
> be a Spark upstream issue based on this open JIRA ticket 
> (https://issues.apache.org/jira/browse/SPARK-3630). Here is the exception 
> that is thrown with the middle bits cut out:
> com.esotericsoftware.kryo.KryoException: java.io.IOException: PARSING_ERROR(2)
> at com.esotericsoftware.kryo.io.Input.fill(Input.java:142)
> at com.esotericsoftware.kryo.io.Input.require(Input.java:169)
> at com.esotericsoftware.kryo.io.Input.readLong_slow(Input.java:715)
> at com.esotericsoftware.kryo.io.Input.readLong(Input.java:665)
> at 
> com.esotericsoftware.kryo.serializers.DefaultSerializers$LongSerializer.read(DefaultSerializers.java:113)
> at 
> com.esotericsoftware.kryo.serializers.DefaultSerializers$LongSerializer.read(DefaultSerializers.java:103)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:729)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.kryoshim.unshaded.UnshadedKryoAdapter.readClassAndObject(UnshadedKryoAdapter.java:48)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.kryoshim.unshaded.UnshadedKryoAdapter.readClassAndObject(UnshadedKryoAdapter.java:30)
> at 
> org.apache.tinkerpop.gremlin.structure.util.star.StarGraphSerializer.readEdges(StarGraphSerializer.java:134)
> at 
> org.apache.tinkerpop.gremlin.structure.util.star.StarGraphSerializer.read(StarGraphSerializer.java:91)
> at 
> org.apache.tinkerpop.gremlin.structure.util.star.StarGraphSerializer.read(StarGraphSerializer.java:45)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.kryoshim.unshaded.UnshadedSerializerAdapter.read(UnshadedSerializerAdapter.java:55)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:626)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.kryoshim.unshaded.UnshadedKryoAdapter.readObject(UnshadedKryoAdapter.java:42)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.kryoshim.unshaded.UnshadedKryoAdapter.readObject(UnshadedKryoAdapter.java:30)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.VertexWritableSerializer.read(VertexWritableSerializer.java:46)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.VertexWritableSerializer.read(VertexWritableSerializer.java:36)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.kryoshim.unshaded.UnshadedSerializerAdapter.read(UnshadedSerializerAdapter.java:55)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:729)
> at 
> org.apache.spark.serializer.KryoDeserializationStream.readObject(KryoSerializer.scala:228)
>  and so on 
> .
> Caused by: java.io.IOException: PARSING_ERROR(2)
> at org.xerial.snappy.SnappyNative.throw_error(SnappyNative.java:84)
> at org.xerial.snappy.SnappyNative.uncompressedLength(Native Method)
> at org.xerial.snappy.Snappy.uncompressedLength(Snappy.java:594)
> 

[GitHub] tinkerpop issue #353: TINKERPOP-1341 Add missing classes to GryoRegistrator

[dkuppitz]

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/353
  
VOTE: +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359621#comment-15359621
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
imo, I think that if we document GraphSON this way, there should not be too 
much confusion as to how the type system works. 


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop issue #351: TINKERPOP-1274: GraphSON 2.0.

[spmallette]

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
imo, I think that if we document GraphSON this way, there should not be too 
much confusion as to how the type system works. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359614#comment-15359614
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user newkek commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
@spmallette yes, maybe some code will help explain.

```
ObjectMapper mapper = 
GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create().createMapper();
Map map = new HashMap<>();
map.put("helo", 2);


String s = mapper.writeValueAsString(map);

// prints 's = {"helo":2}'
System.out.println("s = " + s);

Map read = mapper.readValue(s, Map.class);

// prints 'read.get("helo") = class java.lang.Integer'
System.out.println("read.get(\"helo\") = " + 
read.get("helo").getClass());

Map map3 = new HashMap<>();
map3.put("helo", 2L);


String s2 = mapper.writeValueAsString(map3);

// prints 's2 = {"helo":[{"@class":"Long"},2]}'
System.out.println("s2 = " + s2);

Map read2 = mapper.readValue(s2, Map.class);

// prints 'read2.get("helo") = class java.lang.Long'
System.out.println("read2.get(\"helo\") = " + 
read2.get("helo").getClass());
```

> number with decimal and no type = Java Float

Almost, by default decimals are Double, not Floats.


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop issue #351: TINKERPOP-1274: GraphSON 2.0.

[newkek]

Github user newkek commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
@spmallette yes, maybe some code will help explain.

```
ObjectMapper mapper = 
GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create().createMapper();
Map map = new HashMap<>();
map.put("helo", 2);


String s = mapper.writeValueAsString(map);

// prints 's = {"helo":2}'
System.out.println("s = " + s);

Map read = mapper.readValue(s, Map.class);

// prints 'read.get("helo") = class java.lang.Integer'
System.out.println("read.get(\"helo\") = " + 
read.get("helo").getClass());

Map map3 = new HashMap<>();
map3.put("helo", 2L);


String s2 = mapper.writeValueAsString(map3);

// prints 's2 = {"helo":[{"@class":"Long"},2]}'
System.out.println("s2 = " + s2);

Map read2 = mapper.readValue(s2, Map.class);

// prints 'read2.get("helo") = class java.lang.Long'
System.out.println("read2.get(\"helo\") = " + 
read2.get("helo").getClass());
```

> number with decimal and no type = Java Float

Almost, by default decimals are Double, not Floats.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop issue #351: TINKERPOP-1274: GraphSON 2.0.

[spmallette]

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
If i understand right: Serializing integer without a type tag would result 
in integer. You would have to specify "Long" as the type for it to be 
interpreted as a Long in java. This is generally just a problem for numerics so 
the conversion is:

* number with no decimal and no type = Java integer
* number with decimal and no type = Java Float
* all other numerics will have a java type present (simple name)

@newkek is that an appropriate description of what's happening in 
conversion?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359580#comment-15359580
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
If i understand right: Serializing integer without a type tag would result 
in integer. You would have to specify "Long" as the type for it to be 
interpreted as a Long in java. This is generally just a problem for numerics so 
the conversion is:

* number with no decimal and no type = Java integer
* number with decimal and no type = Java Float
* all other numerics will have a java type present (simple name)

@newkek is that an appropriate description of what's happening in 
conversion?


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop issue #351: TINKERPOP-1274: GraphSON 2.0.

[banjiewen]

Github user banjiewen commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
> To be very explicit, in JSON everything is Doubles and Longs

According to [RFC 7159][1], all numerics (i.e., both integral and decimal) 
in JSON are of a single type, Numeric, with implementation-specific ranges. In 
any case would "assuming Integer" result in payloads that don't round-trip to 
the same types? For example, if I serialize an Integer without a type tag, will 
subsequent deserializations (via, say, Jackson) return a Long?

[1]: https://tools.ietf.org/html/rfc7159#section-6


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359364#comment-15359364
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user newkek commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
> I thought that java integer would be implied.

Yes with the current PR serializing an Integer will result in no type 
added. Serializing a Long or a Short will result in an explicit type added. To 
be very explicit, in JSON everything is Doubles and Longs, it does that because 
it's the bigger container format that contains the others less precise ones. 
Jackson however, considers that when an Integer is to be serialized, there is 
no need for an explicit type because the precision will be kept since for JSON 
it's in a Long. When it comes to serializing a Long, still no precision loss 
but the format explicitly indicates to the Deserializer that what has been 
serialized initially was a Long. So everything is as defined as if the Integer 
was typed, because by default the reader assumes everything's an Integer, and 
if not there will be a type to specify what it is. However we have save a large 
payload size by not typing the integer. Same concept for Float VS Double, for 
JSON everything is a Double, if a value is a Float, it will be typed, if not, 
by default it's a Float.

So as I said in the description I think the outcome of the mail 
conversation was to not type Simple values. Mostly because we're not sure if 
this is going to be useful or not. However, adding those types in the future, 
for Simple values, can be very easily done and I've left detailed comments in 
the `TypeSerializer` on how to add them when we deem it necessary. Also, adding 
them would not break existing code since the format would be the same, it's 
just that _every_ value would follow the format for typed values. Since there's 
no possibility to mix-up for the Numeric values as explained above with the 
Shorts and Longs and etc... I still think we should wait somebody explicitly 
requires it.

> Perhaps {"@class":"java.lang.Integer", "value": 1} is a better option.

As I see it for how I implemented the TypeDeserializer, it acts as a meta 
deserializer that will read the raw text JSON sequentially, so there's no 
chance there can be a mixup in the order, it does not deserialize the whole 
structure and creates a `List>` object. Same for the TypeSerializer, it 
does not create a Java `List` object to write the type, but it writes directly 
in JSON "write a start_array token, write a start_map token, write a property 
name, write a text field (the type name), write a end_array token, etc" so same 
thing, there is no chance there can be a mix up in the order.
I don't know what it could look like for parsers of other languages, but it 
would seem like doing something else than that would be quite inefficient in 
terms of performance because it would that for every typed value you would 
instantiate a `new List>` just to read a simple value. Does that makes 
sense ?


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop issue #351: TINKERPOP-1274: GraphSON 2.0.

[newkek]

Github user newkek commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
> I thought that java integer would be implied.

Yes with the current PR serializing an Integer will result in no type 
added. Serializing a Long or a Short will result in an explicit type added. To 
be very explicit, in JSON everything is Doubles and Longs, it does that because 
it's the bigger container format that contains the others less precise ones. 
Jackson however, considers that when an Integer is to be serialized, there is 
no need for an explicit type because the precision will be kept since for JSON 
it's in a Long. When it comes to serializing a Long, still no precision loss 
but the format explicitly indicates to the Deserializer that what has been 
serialized initially was a Long. So everything is as defined as if the Integer 
was typed, because by default the reader assumes everything's an Integer, and 
if not there will be a type to specify what it is. However we have save a large 
payload size by not typing the integer. Same concept for Float VS Double, for 
JSON everything is a Double, if a value is a Float, it will be t
 yped, if not, by default it's a Float.

So as I said in the description I think the outcome of the mail 
conversation was to not type Simple values. Mostly because we're not sure if 
this is going to be useful or not. However, adding those types in the future, 
for Simple values, can be very easily done and I've left detailed comments in 
the `TypeSerializer` on how to add them when we deem it necessary. Also, adding 
them would not break existing code since the format would be the same, it's 
just that _every_ value would follow the format for typed values. Since there's 
no possibility to mix-up for the Numeric values as explained above with the 
Shorts and Longs and etc... I still think we should wait somebody explicitly 
requires it.

> Perhaps {"@class":"java.lang.Integer", "value": 1} is a better option.

As I see it for how I implemented the TypeDeserializer, it acts as a meta 
deserializer that will read the raw text JSON sequentially, so there's no 
chance there can be a mixup in the order, it does not deserialize the whole 
structure and creates a `List>` object. Same for the TypeSerializer, it 
does not create a Java `List` object to write the type, but it writes directly 
in JSON "write a start_array token, write a start_map token, write a property 
name, write a text field (the type name), write a end_array token, etc" so same 
thing, there is no chance there can be a mix up in the order.
I don't know what it could look like for parsers of other languages, but it 
would seem like doing something else than that would be quite inefficient in 
terms of performance because it would that for every typed value you would 
instantiate a `new List>` just to read a simple value. Does that makes 
sense ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359292#comment-15359292
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
I thought that java integer would be implied. I believe that if we wrote a 
java short for example it would type to a "Short" then you would know what kind 
of number it is. 

We purposely went away from the fully qualified class name which really 
doesn't mean much to non-JVM languages in favor of the more brief and less 
scary simple name - I thought we had decided that a smaller byte size for the 
network outweighed the downside of being less specific about the type. no?

> I would also like to point out that this format 
[{"@class":"java.lang.Integer"}, 1] can be a pain in systems that do not 
necessarily order lists.

I seem to remember that we discussed that before but don't remember the 
outcome - @newkek do you remember what was said?


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop issue #351: TINKERPOP-1274: GraphSON 2.0.

[spmallette]

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
I thought that java integer would be implied. I believe that if we wrote a 
java short for example it would type to a "Short" then you would know what kind 
of number it is. 

We purposely went away from the fully qualified class name which really 
doesn't mean much to non-JVM languages in favor of the more brief and less 
scary simple name - I thought we had decided that a smaller byte size for the 
network outweighed the downside of being less specific about the type. no?

> I would also like to point out that this format 
[{"@class":"java.lang.Integer"}, 1] can be a pain in systems that do not 
necessarily order lists.

I seem to remember that we discussed that before but don't remember the 
outcome - @newkek do you remember what was said?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[newkek]

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69327810
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

So to sum up on that samples issue : 
- the grateful-dead V1 samples have changed because for some reason, some 
of the `inE` of some vertices were not written in the same order. I'm almost 
sure the fix here has nothing to do with that order change, so I pushed the 
changed ones. It also doesn't concern the `normalize` option of the GraphSON 
mapper, since the `inE` are generally not ordered. So, quite a mystery but I 
definitely don't that's something introduced by this branch.
- The `data/` folder now has the new `-v2d0` and `-v2d0-typed` graphs, but 
not the `normalized` ones.
- The `gremlin-test/src/main/ressources/etc...` has all the new v2 graphs 
and the normalized ones.
- I don't know what's up with the `sample.kryo` but there's near to 
0.1% chances it's related to that branch. But I pushed the change though.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359255#comment-15359255
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user PommeVerte commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
Hey guys,

I've been super busy lately but I definitely plan on diving deep into this 
PR over the weekend. One quick remark though.

1. Even the JSON supported types are not prone to lossiness in multi 
language settings. They should also be typed.
2. In a multi language setting, having type names without their java 
classes is not helpful. 

I can illustrate both of these points with the following JSON : `{"id":1}`
The client assumes `id` is `int` but what exactly is an int? is it `16bit`, 
`32bit`, or `64bit`? Languages will have their own definition here. Actually 
some languages will even have different values of `int` depending on how they 
were compiled. 
Changing it to `{"id":[{"@class":"Integer"}, 1]}` is not helpful in this 
case either. However the following is explicit and is something you can work 
with: `{"id":[{"@class":"java.lang.Integer"}, 1]}`. It's immediately 
identifiable and well documented. The client knows this is a 32bit Int and can 
work accordingly. Without this you would have to go through documentation or 
code to figure out what you were dealing with. 

In conclusion:

1. Thinking about it some more it's possible that Integer is the only 
special case that would need typing in the json supported types. I'll give it 
some more though. We could possibly have a "verbose" option for those who 
require typing of everything.
2. Type names should refer to the java class. This also seems to make sense 
when dealing with custom objects.


PS: I would also like to point out that this format 
`[{"@class":"java.lang.Integer"}, 1]` can be a pain in systems that do not 
necessarily order lists. With these systems you need to check that your list 
has two elements, that one is a map, and that the map contains a `@class` key. 
Costly operation.
Perhaps `{"@class":"java.lang.Integer", "value": 1}` is a better option. 


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop issue #351: TINKERPOP-1274: GraphSON 2.0.

[PommeVerte]

Github user PommeVerte commented on the issue:

https://github.com/apache/tinkerpop/pull/351
  
Hey guys,

I've been super busy lately but I definitely plan on diving deep into this 
PR over the weekend. One quick remark though.

1. Even the JSON supported types are not prone to lossiness in multi 
language settings. They should also be typed.
2. In a multi language setting, having type names without their java 
classes is not helpful. 

I can illustrate both of these points with the following JSON : `{"id":1}`
The client assumes `id` is `int` but what exactly is an int? is it `16bit`, 
`32bit`, or `64bit`? Languages will have their own definition here. Actually 
some languages will even have different values of `int` depending on how they 
were compiled. 
Changing it to `{"id":[{"@class":"Integer"}, 1]}` is not helpful in this 
case either. However the following is explicit and is something you can work 
with: `{"id":[{"@class":"java.lang.Integer"}, 1]}`. It's immediately 
identifiable and well documented. The client knows this is a 32bit Int and can 
work accordingly. Without this you would have to go through documentation or 
code to figure out what you were dealing with. 

In conclusion:

1. Thinking about it some more it's possible that Integer is the only 
special case that would need typing in the json supported types. I'll give it 
some more though. We could possibly have a "verbose" option for those who 
require typing of everything.
2. Type names should refer to the java class. This also seems to make sense 
when dealing with custom objects.


PS: I would also like to point out that this format 
`[{"@class":"java.lang.Integer"}, 1]` can be a pain in systems that do not 
necessarily order lists. With these systems you need to check that your list 
has two elements, that one is a map, and that the map contains a `@class` key. 
Costly operation.
Perhaps `{"@class":"java.lang.Integer", "value": 1}` is a better option. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359121#comment-15359121
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69315539
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

If you re-run the tests after you generate and they all pass, i think we 
should probably be safe.


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[spmallette]

Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69315539
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

If you re-run the tests after you generate and they all pass, i think we 
should probably be safe.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[newkek]

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69314893
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

I'll try to find the differences for the grateful-dead in json but I'm not 
sure I'd be able to tell concerning the diff for the `sample.kryo` as I'm not 
fluent in bianry (yet) :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop issue #352: TINKERPOP-1352 Fixed logic in ConnectionPool that was ...

[spmallette]

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/352
  
nice @robertdale - thanks for taking the time to review. always a big help 
when folks look over the PRs and provide feedback.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1352) Connection Pool doesn't always grow

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359064#comment-15359064
 ] 

ASF GitHub Bot commented on TINKERPOP-1352:
---

Github user robertdale commented on the issue:

https://github.com/apache/tinkerpop/pull/352
  
5 consecutive passes


> Connection Pool doesn't always grow 
> 
>
> Key: TINKERPOP-1352
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1352
> Project: TinkerPop
>  Issue Type: Bug
>  Components: driver
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Assignee: stephen mallette
>Priority: Critical
> Fix For: 3.1.3, 3.2.1
>
>
> With certain configurations (that aren't terribly intuitive) of the driver, 
> the connection pool won't grow properly - this includes the default 
> configuration. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[spmallette]

Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69308786
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

yes - you should push the new ones. not sure what changed on the existing 
ones. can you tell what the changes are? if it is just the order of keys 
changing or the order of vertices or something like that then i wouldn't bother 
to push it, but if it's something else i would wonder if there is something 
wrong somewhere because we should be 100% backward compatible with GraphSON 
1.0..nothing should have changed in that sense.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1341) UnshadedKryoAdapter fails to deserialize StarGraph when SparkConf sets spark.rdd.compress=true whereas GryoSerializer works

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358859#comment-15358859
 ] 

ASF GitHub Bot commented on TINKERPOP-1341:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/353
  
VOTE +1


> UnshadedKryoAdapter fails to deserialize StarGraph when SparkConf sets 
> spark.rdd.compress=true whereas GryoSerializer works
> ---
>
> Key: TINKERPOP-1341
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1341
> Project: TinkerPop
>  Issue Type: Bug
>  Components: io
>Affects Versions: 3.2.1, 3.3.0
>Reporter: Dylan Bethune-Waddell
>Priority: Minor
>
> When trying to bulk load a large dataset into Titan I was running into OOM 
> errors and decided to try tweaking some spark configuration settings - 
> although I am having trouble bulk loading with the new 
> GryoRegistrator/UnshadedKryo serialization shim stuff in master whereby a few 
> hundred tasks into the edge loading stage (stage 5) exceptions are thrown 
> complaining about the need to explicitly register CompactBuffer[].class with 
> Kryo, this approach with spark.rdd.compress=true fails a few hundred tasks 
> into the vertex loading stage (stage 1) of BulkLoaderVertexProgram. 
> GryoSerializer instead of KryoSerializer with GryoRegistrator does not fail 
> and successfully loads the data with this compression flag flipped on whereas 
> before I would just get OOM errors until eventually the job was set back so 
> far that it just failed. So it would seem it is desirable in some instances 
> to use this setting, and the new Serialization stuff seems to break it. Could 
> be a Spark upstream issue based on this open JIRA ticket 
> (https://issues.apache.org/jira/browse/SPARK-3630). Here is the exception 
> that is thrown with the middle bits cut out:
> com.esotericsoftware.kryo.KryoException: java.io.IOException: PARSING_ERROR(2)
> at com.esotericsoftware.kryo.io.Input.fill(Input.java:142)
> at com.esotericsoftware.kryo.io.Input.require(Input.java:169)
> at com.esotericsoftware.kryo.io.Input.readLong_slow(Input.java:715)
> at com.esotericsoftware.kryo.io.Input.readLong(Input.java:665)
> at 
> com.esotericsoftware.kryo.serializers.DefaultSerializers$LongSerializer.read(DefaultSerializers.java:113)
> at 
> com.esotericsoftware.kryo.serializers.DefaultSerializers$LongSerializer.read(DefaultSerializers.java:103)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:729)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.kryoshim.unshaded.UnshadedKryoAdapter.readClassAndObject(UnshadedKryoAdapter.java:48)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.kryoshim.unshaded.UnshadedKryoAdapter.readClassAndObject(UnshadedKryoAdapter.java:30)
> at 
> org.apache.tinkerpop.gremlin.structure.util.star.StarGraphSerializer.readEdges(StarGraphSerializer.java:134)
> at 
> org.apache.tinkerpop.gremlin.structure.util.star.StarGraphSerializer.read(StarGraphSerializer.java:91)
> at 
> org.apache.tinkerpop.gremlin.structure.util.star.StarGraphSerializer.read(StarGraphSerializer.java:45)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.kryoshim.unshaded.UnshadedSerializerAdapter.read(UnshadedSerializerAdapter.java:55)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:626)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.kryoshim.unshaded.UnshadedKryoAdapter.readObject(UnshadedKryoAdapter.java:42)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.kryoshim.unshaded.UnshadedKryoAdapter.readObject(UnshadedKryoAdapter.java:30)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.VertexWritableSerializer.read(VertexWritableSerializer.java:46)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.VertexWritableSerializer.read(VertexWritableSerializer.java:36)
> at 
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.kryoshim.unshaded.UnshadedSerializerAdapter.read(UnshadedSerializerAdapter.java:55)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:729)
> at 
> org.apache.spark.serializer.KryoDeserializationStream.readObject(KryoSerializer.scala:228)
>  and so on 
> .
> Caused by: java.io.IOException: PARSING_ERROR(2)
> at org.xerial.snappy.SnappyNative.throw_error(SnappyNative.java:84)
> at org.xerial.snappy.SnappyNative.uncompressedLength(Native Method)
> at org.xerial.snappy.Snappy.uncompressedLength(Snappy.java:594)
>

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358854#comment-15358854
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69288856
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

So, since now the build creates the new v2d0 graphs, and it seems like the 
other ones are pushed in the repo, should I push the new ones too? Also, it 
seems that the already pushed ones have modifications. Should I push all of as 
well ? Here's the diff : 
https://gist.github.com/newkek/4e0488268f9bb78e0d3597821ae6b357


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[newkek]

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69288856
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

So, since now the build creates the new v2d0 graphs, and it seems like the 
other ones are pushed in the repo, should I push the new ones too? Also, it 
seems that the already pushed ones have modifications. Should I push all of as 
well ? Here's the diff : 
https://gist.github.com/newkek/4e0488268f9bb78e0d3597821ae6b357


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358847#comment-15358847
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69288319
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

Apparently the TestHelper behaviour has changed in commit da2eb7e, but the 
pom wasn't changed with regards to that change.

Ok I'll change the pom as explained earlier


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[newkek]

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69288319
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

Apparently the TestHelper behaviour has changed in commit da2eb7e, but the 
pom wasn't changed with regards to that change.

Ok I'll change the pom as explained earlier


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358840#comment-15358840
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69287612
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

hmm - strange that never was a problem before. something must have changed 
(we haven't run with -Dio in a long long time) feel free to fix it in this 
branch. thanks


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[spmallette]

Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69287612
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

hmm - strange that never was a problem before. something must have changed 
(we haven't run with -Dio in a long long time) feel free to fix it in this 
branch. thanks


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop issue #352: TINKERPOP-1352 Fixed logic in ConnectionPool that was ...

[spmallette]

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/352
  
Just pushed what I think is the final fixes for this issue. Note that this 
branch also contains the fix for 

https://issues.apache.org/jira/browse/TINKERPOP-1351

Please feel free to test/review/vote at this point. Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1352) Connection Pool doesn't always grow

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358838#comment-15358838
 ] 

ASF GitHub Bot commented on TINKERPOP-1352:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/352
  
Just pushed what I think is the final fixes for this issue. Note that this 
branch also contains the fix for 

https://issues.apache.org/jira/browse/TINKERPOP-1351

Please feel free to test/review/vote at this point. Thanks.


> Connection Pool doesn't always grow 
> 
>
> Key: TINKERPOP-1352
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1352
> Project: TinkerPop
>  Issue Type: Bug
>  Components: driver
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Assignee: stephen mallette
>Priority: Critical
> Fix For: 3.1.3, 3.2.1
>
>
> With certain configurations (that aren't terribly intuitive) of the driver, 
> the connection pool won't grow properly - this includes the default 
> configuration. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[newkek]

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69283289
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

This is also broken on master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358784#comment-15358784
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69283048
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

Ah, looking at the pom, I'm getting this warning during the build : 

```
[INFO] --- maven-resources-plugin:2.6:copy-resources 
(copy-graphson-from-tmp-to-resources) @ tinkergraph-gremlin ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 
/Users/kevingallardo/Documents/workspace/newkek-incubator-tinkerpop/tinkergraph-gremlin/target/tinkerpop-io
```
Seems like the pom is going to search for the ressources in 
`${project.build.directory}/tinkerpop-io` and 
`${project.build.directory}` is defined as 
`${basedir}/target`.
But the `tempPath` in IoDataGenerationTest` is `tempPath = 
TestHelper.makeTestDataPath(TinkerGraphTest.class, "tinkerpop-io").getPath() + 
File.separator;`

If I change the pom to search in the right directory : 
`${project.build.directory}/test-case-data/TinkerGraphTest/tinkerpop-io`,
 it works.


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[newkek]

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69283048
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

Ah, looking at the pom, I'm getting this warning during the build : 

```
[INFO] --- maven-resources-plugin:2.6:copy-resources 
(copy-graphson-from-tmp-to-resources) @ tinkergraph-gremlin ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 
/Users/kevingallardo/Documents/workspace/newkek-incubator-tinkerpop/tinkergraph-gremlin/target/tinkerpop-io
```
Seems like the pom is going to search for the ressources in 
`${project.build.directory}/tinkerpop-io` and 
`${project.build.directory}` is defined as 
`${basedir}/target`.
But the `tempPath` in IoDataGenerationTest` is `tempPath = 
TestHelper.makeTestDataPath(TinkerGraphTest.class, "tinkerpop-io").getPath() + 
File.separator;`

If I change the pom to search in the right directory : 
`${project.build.directory}/test-case-data/TinkerGraphTest/tinkerpop-io`,
 it works.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[spmallette]

Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69279953
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

When you run with `-Dio` it's suppose to copy the files to all the right 
places. See the tinkergraph-gremlin pom.xmlnot sure why that's not 
happening.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358731#comment-15358731
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69276332
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

Oh I think I see what you mean, I don't see them in the root's `data/` dir, 
I don't know why


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[newkek]

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69276332
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

Oh I think I see what you mean, I don't see them in the root's `data/` dir, 
I don't know why


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358728#comment-15358728
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69276176
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

https://gist.github.com/newkek/643c907e54e93ab36594295281c3e4e6


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[newkek]

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69276176
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

https://gist.github.com/newkek/643c907e54e93ab36594295281c3e4e6


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358725#comment-15358725
 ] 

ASF GitHub Bot commented on TINKERPOP-1274:
---

Github user newkek commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69275891
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

Where are they supposed to be located ? I see them in the 
`tinkergraph-gremlin/target/test-case-data/TinkerGraphTest/tinkerpop-io/`. 
Should they be somewhere else?


> GraphSON Version 2.0
> 
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for 
> non-JVM languages that consume it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1351) Number of connections going beyond the pool max size

[Ramzi Oueslati (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358700#comment-15358700
 ] 

Ramzi Oueslati commented on TINKERPOP-1351:
---

OK I will

> Number of connections going beyond the pool max size
> 
>
> Key: TINKERPOP-1351
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1351
> Project: TinkerPop
>  Issue Type: Bug
>  Components: driver
>Affects Versions: 3.1.2-incubating
> Environment: RESTful web service using gremlin driver to send request 
> to a Gremlin Server
>Reporter: Ramzi Oueslati
>Assignee: stephen mallette
> Fix For: 3.1.3, 3.2.1
>
>
> When the gremlin driver is used with an important number of concurrent 
> requests, sockets are opened far beyond the max pool size.
> At some point, the connections are destroyed, the pool is empty and then the 
> borrowConnection process goes through :
> {code:java}
> if (connections.isEmpty()) {
> logger.debug("Tried to borrow connection but the pool was empty 
> for {} - scheduling pool creation and waiting for connection", host);
> for (int i = 0; i < minPoolSize; i++) {
> scheduledForCreation.incrementAndGet();
> newConnection();
> }
> return waitForConnection(timeout, unit);
> }
> {code}
> If many connections are borrowed at the same time then this code will 
> schedule as many connections for creation.
> I added a check :
> {code:java}
> for (int i = 0; i < minPoolSize; i++) {
> if (scheduledForCreation.get() < minPoolSize) {
> scheduledForCreation.incrementAndGet();
> logger.debug("borrowConnection: [inc] 
> scheduledForCreation=" + scheduledForCreation.get());
> newConnection();
> }
> }
> {code}
> It seems to solve the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (TINKERPOP-1351) Number of connections going beyond the pool max size

[Ramzi Oueslati (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358700#comment-15358700
 ] 

Ramzi Oueslati edited comment on TINKERPOP-1351 at 7/1/16 9:29 AM:
---

OK I will
Thank you Stephen


was (Author: ramzi):
OK I will

> Number of connections going beyond the pool max size
> 
>
> Key: TINKERPOP-1351
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1351
> Project: TinkerPop
>  Issue Type: Bug
>  Components: driver
>Affects Versions: 3.1.2-incubating
> Environment: RESTful web service using gremlin driver to send request 
> to a Gremlin Server
>Reporter: Ramzi Oueslati
>Assignee: stephen mallette
> Fix For: 3.1.3, 3.2.1
>
>
> When the gremlin driver is used with an important number of concurrent 
> requests, sockets are opened far beyond the max pool size.
> At some point, the connections are destroyed, the pool is empty and then the 
> borrowConnection process goes through :
> {code:java}
> if (connections.isEmpty()) {
> logger.debug("Tried to borrow connection but the pool was empty 
> for {} - scheduling pool creation and waiting for connection", host);
> for (int i = 0; i < minPoolSize; i++) {
> scheduledForCreation.incrementAndGet();
> newConnection();
> }
> return waitForConnection(timeout, unit);
> }
> {code}
> If many connections are borrowed at the same time then this code will 
> schedule as many connections for creation.
> I added a check :
> {code:java}
> for (int i = 0; i < minPoolSize; i++) {
> if (scheduledForCreation.get() < minPoolSize) {
> scheduledForCreation.incrementAndGet();
> logger.debug("borrowConnection: [inc] 
> scheduledForCreation=" + scheduledForCreation.get());
> newConnection();
> }
> }
> {code}
> It seems to solve the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop pull request #351: TINKERPOP-1274: GraphSON 2.0.

[spmallette]

Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/351#discussion_r69272477
  
--- Diff: 
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
 ---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
 
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
 g);
 os4.close();
 }
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream = 
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
 {
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
 g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath + 
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
 readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check 
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when 
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --

did you run 

```text
cd tinkergraph-gremlin
mvn clean install -Dio
```

I don't see the newly generated data files in the various data directories.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---