Re: [DISCUSS] 3.1.x code line

[Dylan Millikin]

+1 from me as well. Sounds good.

On Thu, Jul 14, 2016 at 2:14 PM, Ted Wilmes  wrote:

> It does for me too, +1.
>
> --Ted
>
> On Wed, Jul 13, 2016 at 3:44 PM, Hadrian Zbarcea 
> wrote:
>
> > +1. It does to me.
> > Hadrian
> >
> >
> > On 07/13/2016 04:29 PM, Stephen Mallette wrote:
> >
> >> Since we don't really follow semantic versioning for releases, I thought
> >> we
> >> should discuss the 3.1.x code line. We've been steadily adding features
> >> right up to our current 3.1.3 release which we will vote on shortly. I
> >> think it's pretty awesome that we've managed to maintain that older line
> >> of
> >> code for as long as we have and I think we've evolved it in a very
> >> sensible
> >> way.
> >>
> >> I think we should continue to maintain 3.1.x after the 3.1.3 release,
> >> which
> >> would mean a 3.1.4 release at some point, but we should strictly limit
> the
> >> changes there to bug fixes and not do any "new features" on that line of
> >> code (i.e the tp31 branch). As it stands, I don't see any open "bugs"
> for
> >> the 3.1.3 in JIRA so as of right now, we wouldn't have much planned for
> >> 3.1.4.
> >>
> >> Does that make sense for everyone?
> >>
> >>
>

Re: [VOTE] TinkerPop 3.1.3 Release

[Daniel Kuppitz]

$ bin/validate-distribution.sh 3.1.3

Validating binary distributions

* downloading Apache Gremlin Console
(apache-gremlin-console-3.1.3-bin.zip)... OK
* validating signatures and checksums ...
  * PGP signature ... OK
  * MD5 checksum ... OK
  * SHA1 chacksum ... OK
* unzipping Apache Gremlin Console ... OK
* validating Apache Gremlin Console's docs ... OK
* validating Apache Gremlin Console's binaries ... OK
* validating Apache Gremlin Console's legal files ...
  * LICENSE ... OK
  * NOTICE ... OK
* validating Apache Gremlin Console's plugin directory ... OK
* validating Apache Gremlin Console's lib directory ... OK
* testing script evaluation ... OK

* downloading Apache Gremlin Server
(apache-gremlin-server-3.1.3-bin.zip)... OK
* validating signatures and checksums ...
  * PGP signature ... OK
  * MD5 checksum ... OK
  * SHA1 chacksum ... OK
* unzipping Apache Gremlin Server ... OK
* validating Apache Gremlin Server's docs ... OK
* validating Apache Gremlin Server's binaries ... OK
* validating Apache Gremlin Server's legal files ...
  * LICENSE ... OK
  * NOTICE ... OK
* validating Apache Gremlin Server's plugin directory ... OK
* validating Apache Gremlin Server's lib directory ... OK

Validating source distribution

* downloading Apache Tinkerpop 3.1.3 (apache-tinkerpop-3.1.3-src.zip)... OK
* validating signatures and checksums ...
  * PGP signature ... OK
  * MD5 checksum ... OK
  * SHA1 chacksum ... OK
* unzipping Apache Tinkerpop 3.1.3 ... OK
OK

Looks good. I will do a few more manual tests tomorrow, but for now...

VOTE: +1

Oh, and I remember that I already mentioned it last time: We should fix
those typos in the script (chacksum .> checksum, Tinkerpop -> TinkerPop)

Cheers,
Daniel


On Mon, Jul 18, 2016 at 10:32 PM, Stephen Mallette 
wrote:

> Hello,
>
> We are happy to announce that TinkerPop 3.1.3 is ready for release - note
> the lack of "-incubating" everywhere.  :)
>
> The release artifacts can be found at this location:
> https://dist.apache.org/repos/dist/dev/tinkerpop/3.1.3/
>
> The source distribution is provided by:
> apache-tinkerpop-3.1.3-src.zip
>
> Two binary distributions are provided for user convenience:
> apache-gremlin-console-3.1.3-bin.zip
> apache-gremlin-server-3.1.3-bin.zip
>
> The GPG key used to sign the release artifacts is available at:
> https://dist.apache.org/repos/dist/dev/tinkerpop/KEYS
>
> The online docs can be found here:
> http://tinkerpop.apache.org/docs/3.1.3/reference/ (user docs)
> http://tinkerpop.apache.org/docs/3.1.3/upgrade/ (upgrade docs)
> http://tinkerpop.apache.org/javadocs/3.1.3/core/ (core javadoc)
> http://tinkerpop.apache.org/javadocs/3.1.3/full/ (full javadoc)
>
> The tag in Apache Git can be found here:
>
> https://git-wip-us.apache.org/repos/asf?p=tinkerpop.git;a=tag;h=0f960ff0823246176343468b746b6e3ac2ade23c
>
> The release notes are available here:
>
> https://github.com/apache/tinkerpop/blob/3.1.3/CHANGELOG.asciidoc#tinkerpop-313-release-date-july-18-2016
>
> The [VOTE] will be open for the next 72 hours --- closing Thursday (July
> 21, 2016) at 4:30 pm EST.
>
> My vote is +1.
>
> Thank you very much,
> Stephen
>

Re: Code Freeze 3.1.3/3.2.1

[Stephen Mallette]

Here's the revised validate-distribution.sh file:

https://gist.github.com/spmallette/8a1296a50320fe0e1b350fc6caef372e

Please use that to test out the release (if you use that approach - it
really is a super easy way to checkout the distribution if you'd like to
participate in the vote and don't have a ton of time to look at everything
manually).  I'll commit these changes to the repo after release is all
settled.

On Mon, Jul 18, 2016 at 4:00 PM, Stephen Mallette 
wrote:

> Having some trouble with release. Some of the trouble is a bit out of my
> control - like a bank of thunderstorms flickering my internet connection.
> Some of the trouble was self-created, like trying to rename the
> distributions - which I've abandoned doing for this release (we'll do it
> for the next one now that I see all the problems with doing that). And now
> i'm stuck with validate-distribution.sh which doesn't seem to want to
> behave. I think I have it figured out now with a patch or two. I'll post
> that as a gist later for everyone to use as a replacement for the one in
> git or perhaps make the commit to master before i start working on 3.2.1 -
> I don't plan to go back through the release process for 3.1.3 to make
> another commit on tp31 at this point. Anyway, as soon as this
> validate-distribution.sh run finishes successfully, you can expect a VOTE
> thread for 3.1.3.
>
> If I can get 3.2.1 out as well tonight I will but it might be better for
> me to wait until morning because I don't want to make any dumb mistakes at
> this point and it's always good to have a sharp mind when doing this stuff.
> So, worst case, the 3.2.1 VOTE should start tomorrow morning. Sorry for the
> delay
>
> Thanks,
>
> Stephen
>
>
>
> On Sat, Jul 16, 2016 at 8:57 AM, Marko Rodriguez 
> wrote:
>
>> Thanks again for handling the release process.
>>
>> I appreciate nearly nothing in this world except for the effort you put
>> into the release process.
>>
>> Thank you,
>> Marko.
>>
>> http://markorodriguez.com
>>
>>
>>
>> > On Jul 15, 2016, at 4:39 PM, Stephen Mallette 
>> wrote:
>> >
>> > Everything looks good to do a vote for 3.1.3 and 3.1.2 on Monday. I'm
>> going
>> > to publish the final SNAPSHOT docs over the weekend - nothing has
>> changed
>> > of significance for the artifacts SNAPSHOTS to warrant republishing.
>> >
>> > On Wed, Jul 13, 2016 at 7:08 AM, Stephen Mallette > >
>> > wrote:
>> >
>> >> I've run full integration tests on tp31 and master and both passed. As
>> a
>> >> result, I've published 3.2.1-SNAPSHOT and 3.1.3-SNAPSHOT to the Apache
>> >> Snapshots repo. I hope developers of graph databases, drivers, etc.
>> can get
>> >> a moment to try it out and report any problems.
>> >>
>> >>
>> >> On Tue, Jul 12, 2016 at 9:52 AM, Marko Rodriguez > >
>> >> wrote:
>> >>
>> >>> Hello,
>> >>>
>>  Code freeze essentially begins today. We cleared up all the remaining
>>  issues that were lingering. Please focus on testing and documentation
>> >>> for
>>  the next few days with a vote for release on Monday July 18th 2016.
>> >>>
>> >>>
>> >>> Thank you Stephen.
>> >>>
>> >>> As I was saying in HipChat, while we have merged the
>> >>> PathRetractionStrategy work (
>> >>> https://issues.apache.org/jira/browse/TINKERPOP-1254 <
>> >>> https://issues.apache.org/jira/browse/TINKERPOP-1254>), I’m
>> continually
>> >>> benchmarking and testing scenarios and am finding numerous little
>> nick nack
>> >>> bugs and obvious optimizations. I’m going to continue to CTR to
>> master/
>> >>> these updates and note that each evening I’m doing local integration
>> >>> testing to ensure that no one finds a hiccup (if any) that were
>> caused by
>> >>> this “nick nack” work I’m doing.
>> >>>
>> >>> Again, thank you Stephen for managing the release process (as always).
>> >>>
>> >>> Marko.
>> >>>
>> >>> http://markorodriguez.com
>> >>>
>> >>>
>> >>
>>
>>
>

[VOTE] TinkerPop 3.1.3 Release

[Stephen Mallette]

Hello,

We are happy to announce that TinkerPop 3.1.3 is ready for release - note
the lack of "-incubating" everywhere.  :)

The release artifacts can be found at this location:
https://dist.apache.org/repos/dist/dev/tinkerpop/3.1.3/

The source distribution is provided by:
apache-tinkerpop-3.1.3-src.zip

Two binary distributions are provided for user convenience:
apache-gremlin-console-3.1.3-bin.zip
apache-gremlin-server-3.1.3-bin.zip

The GPG key used to sign the release artifacts is available at:
https://dist.apache.org/repos/dist/dev/tinkerpop/KEYS

The online docs can be found here:
http://tinkerpop.apache.org/docs/3.1.3/reference/ (user docs)
http://tinkerpop.apache.org/docs/3.1.3/upgrade/ (upgrade docs)
http://tinkerpop.apache.org/javadocs/3.1.3/core/ (core javadoc)
http://tinkerpop.apache.org/javadocs/3.1.3/full/ (full javadoc)

The tag in Apache Git can be found here:
https://git-wip-us.apache.org/repos/asf?p=tinkerpop.git;a=tag;h=0f960ff0823246176343468b746b6e3ac2ade23c

The release notes are available here:
https://github.com/apache/tinkerpop/blob/3.1.3/CHANGELOG.asciidoc#tinkerpop-313-release-date-july-18-2016

The [VOTE] will be open for the next 72 hours --- closing Thursday (July
21, 2016) at 4:30 pm EST.

My vote is +1.

Thank you very much,
Stephen

Re: Code Freeze 3.1.3/3.2.1

[Stephen Mallette]

Having some trouble with release. Some of the trouble is a bit out of my
control - like a bank of thunderstorms flickering my internet connection.
Some of the trouble was self-created, like trying to rename the
distributions - which I've abandoned doing for this release (we'll do it
for the next one now that I see all the problems with doing that). And now
i'm stuck with validate-distribution.sh which doesn't seem to want to
behave. I think I have it figured out now with a patch or two. I'll post
that as a gist later for everyone to use as a replacement for the one in
git or perhaps make the commit to master before i start working on 3.2.1 -
I don't plan to go back through the release process for 3.1.3 to make
another commit on tp31 at this point. Anyway, as soon as this
validate-distribution.sh run finishes successfully, you can expect a VOTE
thread for 3.1.3.

If I can get 3.2.1 out as well tonight I will but it might be better for me
to wait until morning because I don't want to make any dumb mistakes at
this point and it's always good to have a sharp mind when doing this stuff.
So, worst case, the 3.2.1 VOTE should start tomorrow morning. Sorry for the
delay

Thanks,

Stephen



On Sat, Jul 16, 2016 at 8:57 AM, Marko Rodriguez 
wrote:

> Thanks again for handling the release process.
>
> I appreciate nearly nothing in this world except for the effort you put
> into the release process.
>
> Thank you,
> Marko.
>
> http://markorodriguez.com
>
>
>
> > On Jul 15, 2016, at 4:39 PM, Stephen Mallette 
> wrote:
> >
> > Everything looks good to do a vote for 3.1.3 and 3.1.2 on Monday. I'm
> going
> > to publish the final SNAPSHOT docs over the weekend - nothing has changed
> > of significance for the artifacts SNAPSHOTS to warrant republishing.
> >
> > On Wed, Jul 13, 2016 at 7:08 AM, Stephen Mallette 
> > wrote:
> >
> >> I've run full integration tests on tp31 and master and both passed. As a
> >> result, I've published 3.2.1-SNAPSHOT and 3.1.3-SNAPSHOT to the Apache
> >> Snapshots repo. I hope developers of graph databases, drivers, etc. can
> get
> >> a moment to try it out and report any problems.
> >>
> >>
> >> On Tue, Jul 12, 2016 at 9:52 AM, Marko Rodriguez 
> >> wrote:
> >>
> >>> Hello,
> >>>
>  Code freeze essentially begins today. We cleared up all the remaining
>  issues that were lingering. Please focus on testing and documentation
> >>> for
>  the next few days with a vote for release on Monday July 18th 2016.
> >>>
> >>>
> >>> Thank you Stephen.
> >>>
> >>> As I was saying in HipChat, while we have merged the
> >>> PathRetractionStrategy work (
> >>> https://issues.apache.org/jira/browse/TINKERPOP-1254 <
> >>> https://issues.apache.org/jira/browse/TINKERPOP-1254>), I’m
> continually
> >>> benchmarking and testing scenarios and am finding numerous little nick
> nack
> >>> bugs and obvious optimizations. I’m going to continue to CTR to master/
> >>> these updates and note that each evening I’m doing local integration
> >>> testing to ensure that no one finds a hiccup (if any) that were caused
> by
> >>> this “nick nack” work I’m doing.
> >>>
> >>> Again, thank you Stephen for managing the release process (as always).
> >>>
> >>> Marko.
> >>>
> >>> http://markorodriguez.com
> >>>
> >>>
> >>
>
>

[jira] [Created] (TINKERPOP-1376) Rename TinkerPop artifacts

[stephen mallette (JIRA)]

stephen mallette created TINKERPOP-1376:
---

 Summary: Rename TinkerPop artifacts
 Key: TINKERPOP-1376
 URL: https://issues.apache.org/jira/browse/TINKERPOP-1376
 Project: TinkerPop
  Issue Type: Improvement
  Components: build-release
Affects Versions: 3.1.3
Reporter: stephen mallette
Priority: Minor
 Fix For: 3.1.4


Rename files to include "apache-tinkerpop-" as a prefix. This is a bit more 
complicated than just renaming because it affects:

* Assembly files
* validate-distribution.sh
* release documentation (when copying files to svn and such)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (TINKERPOP-1340) docs do not state at what version an API was introduced (or deprecated)

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette updated TINKERPOP-1340:

Fix Version/s: (was: 3.1.3)
   3.1.4

> docs do not state at what version an API was introduced (or deprecated)
> ---
>
> Key: TINKERPOP-1340
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1340
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 3.2.0-incubating, 3.1.2-incubating
>Reporter: Jason Plurad
>Priority: Trivial
>  Labels: easyfix, newbie
> Fix For: 3.2.1, 3.1.4
>
>
> An all-to-common problem for new developers coming to the TinkerPop is 
> figuring out which APIs work against the graph they are using. Since 
> TinkerPop is ahead of many graph implementation out there (Titan/TP 3.0.1, 
> Stardog/TP 3.0.2, Blazegraph/TP 3.1.0 -- kudos to Sqlg and Unipop for TP 
> 3.2.0!), it's really important that developers know which APIs in the current 
> docs are valid. Ideally the developers would go directly to that version of 
> the TP docs, but that doesn't always happen thanks to Google.
> I propose doc updates on each Graph Traversal Step in the [reference 
> docs|http://tinkerpop.apache.org/docs/current/reference/#graph-traversal-steps]
>  and in the javadocs (on 
> [__.java|http://tinkerpop.apache.org/javadocs/current/full/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/__.html]
>  and on specific step implementations, i.e. 
> [AddEdgeStep.java|http://tinkerpop.apache.org/javadocs/current/full/org/apache/tinkerpop/gremlin/process/traversal/step/map/AddEdgeStep.html]).
> It should state the specific version (3.y.z) the API was introduced or 
> deprecated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (TINKERPOP-1342) Allow setting scriptEvaluationTimeout in driver

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette updated TINKERPOP-1342:

Fix Version/s: (was: 3.1.3)
   3.1.4

> Allow setting scriptEvaluationTimeout in driver
> ---
>
> Key: TINKERPOP-1342
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1342
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: driver
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Assignee: stephen mallette
> Fix For: 3.1.4
>
>
> Allow this setting to be applied in the driver somehow so as to give the user 
> the ability to override the server setting in the request.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (TINKERPOP-1287) StarGraph has an overdose of Stream usage.

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette updated TINKERPOP-1287:

Fix Version/s: (was: 3.1.3)
   3.1.4

> StarGraph has an overdose of Stream usage.
> --
>
> Key: TINKERPOP-1287
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1287
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: hadoop, structure
>Affects Versions: 3.2.0-incubating, 3.1.2-incubating
>Reporter: Marko A. Rodriguez
>Assignee: Ted Wilmes
> Fix For: 3.2.1, 3.1.4
>
> Attachments: stage0.svg, stage1.svg, stage2.svg
>
>
> {{StarGraph}} is loaded with {{Stream}}-usage. Gutting streams from 
> TinkerGraph made it much faster. It would be good if we did the same thing 
> for {{StarGraph}}.
> This can go into tp31/ and upmerge to master/.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (TINKERPOP-980) Add a service script or daemon mode in the distribution

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette updated TINKERPOP-980:
---
Fix Version/s: (was: 3.1.3)
   3.1.4

> Add a service script or daemon mode in the distribution
> ---
>
> Key: TINKERPOP-980
> URL: https://issues.apache.org/jira/browse/TINKERPOP-980
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: server
>Affects Versions: 3.0.2-incubating
>Reporter: Jeremy Hanna
>Assignee: Dylan Millikin
>Priority: Minor
> Fix For: 3.1.4
>
>
> Based on this discussion, it looks like there was an example from [~dkuppitz] 
> on how to create a gremlin server service on linux:
> https://groups.google.com/forum/#!msg/gremlin-users/uA48IQ3YJcw/4KnUKIS8HI4J
> Here is a link to the gist for the service:
> https://gist.github.com/dkuppitz/20bda51e3465a612cd9b
> I think it would be great to include this or a way to daemonize the server 
> into the tinkerpop distribution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1375) Possible ByteBuf leak for certain transactional scenarios

[Robert Dale (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382098#comment-15382098
 ] 

Robert Dale commented on TINKERPOP-1375:


Recommend updating to latest Netty 4.0.39.Final.  Tinkerpop is at 4.0.34.

4.0.37 fixes a "leak" and other memory issues.  It also addresses security 
issue "CVE-2016-4970 and can lead to a DOS"

Release Notes:
http://netty.io/news/2016/03/21/4-0-35-Final.html
http://netty.io/news/2016/04/04/4-0-36-Final.html
http://netty.io/news/2016/06/07/4-0-37-Final.html
http://netty.io/news/2016/07/01/4-0-38-Final-4-1-2-Final.html
http://netty.io/news/2016/07/15/4-0-39-Final-4-1-3-Final.html

> Possible ByteBuf leak for certain transactional scenarios
> -
>
> Key: TINKERPOP-1375
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1375
> Project: TinkerPop
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.2.0-incubating
>Reporter: stephen mallette
>Assignee: stephen mallette
>
> Not sure how to recreate this but certain transactional scenarios in sessions 
> seem to generate a standard Netty "LEAK" log message. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [DISCUSS] New IO format for GLVs/Gremlin Server

[Robert Dale]

On Mon, Jul 18, 2016 at 6:29 AM, gallardo.kev...@gmail.com
 wrote:
>
> On 2016-07-15 21:32 (+0100), Robert Dale  wrote:
>> Responding to Marko and Kevin...
[...]
>>
>> Kevin wrote:
>> >> Correct, these types weren't relevant... I only wanted to show you the 
>> >> format...
>> > However, I don't manage to understand the structure behind the format you 
>> > suggest, and I don't manage to establish a clear explicit representation 
>> > in my mind, regarding the example you provided in the TP-1274 PR. Could 
>> > you please give an example of how you would imagine the serialized JSON of 
>> > :
>> > - an example list of typed values, like List
>> > - an example list of typed and untyped values, like a list with UUIDs and 
>> > booleans
>> > - an example map of typed and untyped values
>> >
>> > How would you define that format in a general way ? Like what I did when 
>> > saying
>> > "- untyped : value
>> > - typed : {"@type", "typeName", "value" : value}"
>> >
>> > Just trying your point better.
>> > Also what are the downsides you see with the format suggested above ?
>>
>> The original format was in a list. I must have missed where you
>> accepted this format. In any case, like I originally stated, if you
>> want strong-typing, then _everything_ must be an _object_.
>>
>> Here's an example of non-typed:
>> https://gist.github.com/robertdale/02931f5633be55a59c13bca3b0e58655
>> - native json only
>>
>> Here's strongly typed:
>> https://gist.github.com/robertdale/6c074b165a72efee701e26f851f8b68a
>> - set (as an object), list (as an object), mixed-type lists, etc
>>
>
> OK, glad to see your revised version of the format is the exact same I 
> defined initially. I think we're on the same page here now. Except one thing, 
> it seems like the type information for vertex is not consistent with the 
> rest, if as you say if "everything is an object", then it would be like this 
> : https://gist.github.com/newkek/2d748dc59029f01af18b2a0e80494a31 .
> However, strong typing does not necessarily mean to me that there needs to be 
> a type metadata if the type is already properly handled by JSON. I.e. I don't 
> see the necessity to add type information for data like boolean. There is no 
> ambiguity possible.

Vertex (etc) is already an object so no it doesn't need to be nested
inside another object. The "type, value" pattern is primarily for
scalars but can also be used to differentiate collections - sets,
lists, arrays, etc. I got carried away with typing on boolean.  I
think the last item I disagree on is having a default type for
integers. I think they should all be typed. Otherwise, I agree we're
on the same page.

[...]
> No, in DSE Graph, the schema has to be defined upfront and does not depend on 
> the first element inserted. But I'm not the best person to talk about that 
> and I'm not sure this is the right place..

Specifically, "developer" mode allows this. "Production" mode requires schema.

-- 
Robert Dale

Re: [DISCUSS] New IO format for GLVs/Gremlin Server

[gallardo.kev...@gmail.com]

On 2016-07-15 21:32 (+0100), Robert Dale  wrote: 
> Responding to Marko and Kevin...
> 
> Marko wrote:
> > SIDENOTE: This serves as a foundation for when we move to GraphSON 2.0. In 
> > terms of numbers, I think, unfortunately, we have to stick with int32, 
> > int64, float, double, etc. given graph database providers and their type 
> > systems. Its not about the Gremlin traversal API, its more about provider 
> > schemas. has(“someNumber”,12L) vs. has(“someNumber”,12).
> 
> I call the above behavior a bug or a peculiarity of Titan; it clings
> to a java object idiom. On the other hand, DSE graph exhibits expected
> behavior (as does IBM Graph, Neo4j.)  I know of no other query
> language that behaves like this - e.g. SQL, CassandraQL, JPQL, JOOQ
> (the gremlin of sql).  Typically the underlying driver/provider does
> the "right" thing (or doesn't).  Again, take UUID in gremlin, I can
> pass a string.  The underlying driver seems to convert it to UUID, I
> don't have to provide an UUID object.  This seems inconsistent.
> Either it's doing strong typing or not.  Which is it??
> 
> IMO, the query language should be abstracted from the storage schema.
> And I think this is where we have the impedance mismatch in this
> thread.  What gremlin is really acting like in addition to query
> language is an Object Graph Mapper (like an ORM).  It's playing two
> roles. So I'm also arguing that it should have a single
> responsibility. Yes, I've said this before. But maybe it changes
> things too drastically.  Maybe there are aspects of gremlin that
> actually require strong typing. I don't know. I haven't run into them.
> On to the next item...
> 
> Kevin wrote:
> >> Correct, these types weren't relevant... I only wanted to show you the 
> >> format...
> > However, I don't manage to understand the structure behind the format you 
> > suggest, and I don't manage to establish a clear explicit representation in 
> > my mind, regarding the example you provided in the TP-1274 PR. Could you 
> > please give an example of how you would imagine the serialized JSON of :
> > - an example list of typed values, like List
> > - an example list of typed and untyped values, like a list with UUIDs and 
> > booleans
> > - an example map of typed and untyped values
> >
> > How would you define that format in a general way ? Like what I did when 
> > saying
> > "- untyped : value
> > - typed : {"@type", "typeName", "value" : value}"
> >
> > Just trying your point better.
> > Also what are the downsides you see with the format suggested above ?
> 
> The original format was in a list. I must have missed where you
> accepted this format. In any case, like I originally stated, if you
> want strong-typing, then _everything_ must be an _object_.
> 
> Here's an example of non-typed:
> https://gist.github.com/robertdale/02931f5633be55a59c13bca3b0e58655
> - native json only
> 
> Here's strongly typed:
> https://gist.github.com/robertdale/6c074b165a72efee701e26f851f8b68a
> - set (as an object), list (as an object), mixed-type lists, etc
> 

OK, glad to see your revised version of the format is the exact same I defined 
initially. I think we're on the same page here now. Except one thing, it seems 
like the type information for vertex is not consistent with the rest, if as you 
say if "everything is an object", then it would be like this : 
https://gist.github.com/newkek/2d748dc59029f01af18b2a0e80494a31 .
However, strong typing does not necessarily mean to me that there needs to be a 
type metadata if the type is already properly handled by JSON. I.e. I don't see 
the necessity to add type information for data like boolean. There is no 
ambiguity possible.

> Let me add that while there's no strict definition of schemaless, it
> was not necessarily intended to include having mixed data types for a
> single field. This is a really bad idea. Experts warn against this.
> Most NoSQL databases don't even support this. You will probably die if
> you use it. The default behavior for DSE graph, IBM graph, and even
> Titan is to create the schema based on the first type inserted.  It
> will complain if any subsequent type is different.

No, in DSE Graph, the schema has to be defined upfront and does not depend on 
the first element inserted. But I'm not the best person to talk about that and 
I'm not sure this is the right place..

However concerning mix typed/non-typed I am not concerned about what the Graph 
provider would do but more about what the protocol can handle and hence I am in 
favour of having a protocol that can handle as much as possible in a consistent 
way, for example collections of typed and non typed values, as it is possible 
in a TinkerGraph. Which means, a VertexProperty can be a list of Strings and 
UUIDs, one doesn't need type, the other does.

> 
> Also, schemaless doesn't mean without any schema. While not having to
> define a schema up-front during a quickstart or early development
> makes life