Re: [DISCUSS] TinkerPop 4 HTTP API Design
I've recently spent some more time thinking about the response format and I
think the results should be inside a list called data again. This
essentially turns it back into the GraphSONv3 ResponseMessage. So "result"
would be an object that contains an array called "data". It might be useful
to do this just in case there is some metadata that needs to be added to
the response.
On Tue, May 7, 2024 at 1:59 PM Ken Hu wrote:
> Another update is that for GraphSONv4, the result field will no longer be
> null if there are no results instead it will become an empty array. This
> means that the new ResponseResult always expects a List rather than
> just an Object.
>
> Are there any concerns about changing it from being null to an empty array?
>
> On Wed, Apr 24, 2024 at 1:30 PM Ken Hu wrote:
>
>> An update regarding the API, the materializeProperties option is being
>> added back which should be set to either "all" or "tokens". The "all"
>> option returns all the properties while the "tokens" option returns only
>> the id and label. So the request syntax currently looks like:
>>
>> -- Request Syntax --
>>
>> POST /gremlin HTTP/1.1
>> Accept: <>
>> Content-Type: <>
>> Gremlin-Hints: <>
>>
>> {
>> "gremlin": "string",
>> "timeoutMs": number,
>> "materializeProperties": "string",
>> "bindings": object,
>> "g": "string"
>> "language" : "string"
>> }
>>
>> Additionally, there seems to be a difference between options needed to
>> change server behavior (timeoutMs, materializeProperties) and options
>> needed to change the behavior of the Traversal. Currently, we allow these
>> server options to be set using the with() options. I think it makes sense
>> for these to only be set in the HTTP request body as it isn't information
>> needed by the traversal. This leads to a better separation of options that
>> affect both all traversals (embedded and remote) and options that are
>> server/remote only. This means that we would drop the GremlinScriptChecker
>> from the server that is used for parsing through a script for specific
>> with() options that the server needs. However, a user may need to set
>> specific options per query so we could add a GLV-only step like request().
>> The request() step wouldn't be part of the language and would be something
>> that only the GLVs would recognize and would cause them to add those
>> options to the HTTP request body. It should be noted that request() is a
>> working idea, but for now, the most important point is that there will be a
>> way to set these server options per request from the GLVs.
>>
>> On Wed, Apr 10, 2024 at 12:45 PM Ken Hu wrote:
>>
>>> Hi,
>>>
>>> The following is a reference of what the upcoming HTTP API will look
>>> like for the /gremlin resource. Some smaller items are probably subject to
>>> change.
>>>
>>> There are three differences between the 4.x HTTP API and the 3.x HTTP
>>> API that should be noted. First, the server always generates the requestId
>>> and returns it to the client. Second, the format of the request body has
>>> changed as the OpProcessors have been removed so the request fields have
>>> been flattened. Third, because the results are now streamed back to the
>>> user with chunked transfer encoding, there are certain errors that can
>>> occur after the initial 200 OK has been sent. This means that the user must
>>> check either the status object in the response body or the trailers to see
>>> if there are any errors.
>>>
>>> The format below uses a bit of pseudo-JSON to help represent request and
>>> response bodies. The actual format of the request and response bodies will
>>> be determined by the serializers defined via the "Accept" and
>>> "Content-Type" headers. As a result, a generic type definition in this
>>> document like "number" could translate to a "long" for a serializer that
>>> supports types like GraphBinary.
>>>
>>>
>>> -- Request Syntax --
>>>
>>> POST /gremlin HTTP/1.1
>>> Accept: <>
>>> Content-Type: <>
>>> Gremlin-Hints: <>
>>>
>>> {
>>> "gremlin": "string",
>>> "timeoutMs": number,
>>> "bindings": object,
>>&
Re: [DISCUSS] Release of a Milestone Build for TinkerPop 4.0
Looking through the master-http branch, it seems like transactions aren't ready yet. It might be best to disable tx() and other similar functions just so users aren't confused in that case. So maybe just throw an exception in those cases? On Fri, Oct 4, 2024 at 5:51 PM Yang Xia wrote: > Hello everyone, > > As we get closer towards the end of the year, a lot of changes have been > implemented for 4.0 in the master-http branch. While there is still a lot > to do before we are ready for a full 4.0 release, it would be good to > release a milestone build of what is done so far to gather feedback. > > This milestone would roughly offer the opportunity to try Gremlin Server > HTTP with the revised GraphSON and GraphBinary serialization formats for > this version, using Java and Python drivers. These drivers will initially > continue to support GraphSON as it can help with debugging (given its human > readable format) and offers wider serialization options for users in these > early days, though, as mentioned in previous posts, we should expect to > just have GraphBinary support for drivers in the future for simplicity. > > Also, as this is a “milestone” release, we should consider it an official > one as opposed to a SNAPSHOT or dev release candidate. Please respond if > you have any thoughts on this. If there is no feedback by October 11th, I > will assume a lazy consensus in favour of such a milestone release. > > Cheers, > > Yang >
Re: [DISCUSS] Proposed updates to GraphBinaryV4 and GraphSONV4
Just a quick follow-up, I'm planning to make a small change that I thought you all might want to be aware of but it's probably small enough to not need its own thread. In GraphSON, edge had a "inVLabel", "outVLabel", "inV", and "outV" to refer to the vertices in both directions. However, this isn't the same as the format for a vertex which is an object that contains an "id" and "label". I think it makes more sense to modify "inV" and "outV" to be reference vertices that contain a "id" and a "label" without "properties". I've made a PR for that at https://github.com/apache/tinkerpop/pull/2810 On Thu, Sep 12, 2024 at 1:42 PM Yang Xia wrote: > Thanks Ken! Given consensus on this discussion, I've began implementing > some of the changes in Java and Python to align the GraphBinary serializers > with the spec, starting with removing old serializers and updating > DateTime: https://github.com/apache/tinkerpop/pull/2776. Reviews and > comments are welcome. > > Cheers, > > Yang > > On 2024/08/28 04:01:51 Ken Hu wrote: > > Here are some other updates that I think should be made for > GraphBinaryV4. > > > > There are steps that can return a distinct ordering in Maps like > > order(local). However, these aren't necessarily deserialized in GLVs as > > Map-types that preserve this ordering. I think that we should add a > > value_flag to the Map type in GraphBinary that signals that the > > deserialized version of this Map should maintain the ordering. Similar to > > the BulkSet change, we can use a value_flag with the value 0x02 to denote > > ordering. The value itself would remain unchanged, but the order of the > > items in that list should be the insertion/iteration order of the > > deserialized Map. > > > > All values that the Date type can represent are covered by OffsetDateTime > > so I think it makes sense to remove even the Date type. Date is also > > outdated in Java and its use isn't recommended. > > > > The current GraphBinary standard states that properties for elements like > > Vertex, Edge and VertexProperty should be set to null. The reasoning for > > this was that properties were never returned in remote traversals. > However, > > this changed starting in 3.7, and properties can now be returned. > > Properties should instead be a List of Properties and if there are no > > properties then an empty list should be returned. > > > > Labels for elements are currently serialized as String for both > GraphBinary > > and GraphSON. However, we may want to support multi-labels or no-label > > scenarios in the future. To prevent needing a breaking change if this > were > > to happen, I propose that "label" become a List of String. > > > > In the new HTTP API, a ReponseMessage is returned per HTTP chunk and the > > HTTP response could end with a trailer. The GLVs may need some way to > > distinguish between chunks and trailers which could be accomplished > using a > > one byte buffer in between the last ResponseMessage and the trailing > > header. The GLVs may need information to recreate a chunk as some HTTP > > clients transparently dechunk the body. A 4-byte header that indicates > the > > size of ResponseMessage could be added to help GLVs recognize chunk > > boundaries which simplifies deserialization. > > > > On Thu, Aug 22, 2024 at 2:34 PM Ken Hu wrote: > > > > > Hi All, > > > > > > I've noticed that there are some serializers/deserializers that exist > for > > > GraphBinary and GraphSON that may not serve a real purpose and I'm > > > proposing that we remove them in V4. In general, I think we should move > > > away from providing serializers for all types in the Java Standard > Library, > > > as there are many types that other GLVs simply won't support. The > types in > > > GraphSONv3/GraphBinaryV3 that I would remove include: > > > > > > Barrier, Cardinality, Column, DT, Merge, Operator, Order, Pick, Pop, > > > Scope, T > > > > > > Because the GLVs will now use GremlinLang (instead of Bytecode), these > > > values will be translated to Strings so they don't need their own > specific > > > serializers. T and Direction are the two exceptions because they are > > > returned in the valueMap() and elementMap() steps, although it's likely > > > they will be replaced in the future as well. For similar reasons, P and > > > TextP should also be removed since they will also be Strin
[jira] [Created] (TINKERPOP-3114) Update connection pool handling in Gremlin Python
Ken Hu created TINKERPOP-3114:
-
Summary: Update connection pool handling in Gremlin Python
Key: TINKERPOP-3114
URL: https://issues.apache.org/jira/browse/TINKERPOP-3114
Project: TinkerPop
Issue Type: Improvement
Components: python
Affects Versions: 3.7.2, 3.6.7
Reporter: Ken Hu
A Discord user (e8l) was mentioning problems they were seeing with the
connection pool in Gremlin Python. The pool can't determine whether or not a
connection is healthy and therefore can't remove any problematic connections
from the pool. This can lead to cases where the pool fills up with unusable
connections which leads to the driver being unresponsive as it waits for an
available connection.
As reported by user e8l on Discord:
{quote}I am struggling to avoid problems after a connection error occur.
And now, I suspect it might be led by something bug of gremlinpython...
Are these bugs? Or just I use it wrongly?
Please let me know.
Case 1: Script is hanged up when all pooled connections are consumed?
When I specify wrong url to simulate network error,
gremlinpython might consume connections and do not return them into the pool.
So, below script is hanged up after all pooled connections are consumed.
Python Script: see case1.py
The Output: see case1-output.txt
The result is changed when I specify different value to pool_size argument.
My expectation is that error messages are shown in 9 times and the script ends.
Case 2: Manual transaction is never rolled back(closed)
Same as case 1, manual transaction is never ended.
So, I cannot recover the error.
Python Script: see case2.py
The Output: see case2-output.py
My expectation is that this script is end after trying 9 times and all trials
are failed.
Case 3: Once a connection error occurred, pooled connections are broken
After I stopped TinkerPop server(JanusGraph) temporary,
some pooled connections are broken and will not be recovered.
Python Script: see case3.py
The Output: see case3-output.txt
My expectation is that connections are refreshed if they are not available when
get them from the pool.
{quote}
case1:
{code:python}
from gremlin_python.driver.serializer import GraphSONMessageSerializer
from gremlin_python.process.anonymous_traversal import traversal
from gremlin_python.driver.driver_remote_connection import
DriverRemoteConnection
from gremlin_python.driver.aiohttp.transport import AiohttpTransportconnection
= DriverRemoteConnection(
'ws://localhost:18182/gremlin', # this is wrong url
'g',
pool_size=4,
message_serializer=GraphSONMessageSerializer(),
transport_factory=lambda: AiohttpTransport())
g = traversal().with_remote(connection)for count in range(1,10):
try:
print(f'{count=}')
result = g.V().to_list()
print(f'{result=}')
except Exception as e:
print(e)connection.close(){code}
case1-output:
{code:java}
count=1
Cannot connect to host localhost:18182 ssl:default [Connect call failed
('127.0.0.1', 18182)]
count=2
Cannot connect to host localhost:18182 ssl:default [Connect call failed
('127.0.0.1', 18182)]
count=3
Cannot connect to host localhost:18182 ssl:default [Connect call failed
('127.0.0.1', 18182)]
count=4
Cannot connect to host localhost:18182 ssl:default [Connect call failed
('127.0.0.1', 18182)]
count=5
# the script is hanged here and we need to send signal(Ctrl+C) to abort this.
{code}
case2:
{code:python}
from gremlin_python.driver.serializer import GraphSONMessageSerializer
from gremlin_python.process.anonymous_traversal import traversal
from gremlin_python.driver.driver_remote_connection import
DriverRemoteConnection
from gremlin_python.driver.aiohttp.transport import AiohttpTransportconnection
= DriverRemoteConnection(
'ws://localhost:18182/gremlin', # this is wrong url
'g',
message_serializer=GraphSONMessageSerializer(),
transport_factory=lambda: AiohttpTransport())
g = traversal().with_remote(connection)for count in range(1,10):
try:
print(f'{count=}')
tx = g.tx()
gtx = tx.begin()
result = gtx.add_v().next()
tx.commit()
print(f'{result=}')
except Exception as e:
print(e)
if tx.is_open():
print('rollback')
try:
tx.rollback()
except Exception as re:
print(re)
finally:
if tx.is_open():
print('close transaction')
tx.close()connection.close() {code}
case2-output:
{code:python}
count=1
Cannot connect to host localhost:18182 ssl:default [Connect call failed
('127.0.0.1', 18182)]
rollback
# the script is hanged here and we need to send signal(Ctrl+C) to abort this.
# note that we have to send signals twice due to tx.close() in finally block.
# tx.close() is also hanged, so we need to do that once more. {co
Re: [Discuss] October Release for 3.6.8 and 3.7.3
I agree, it's probably time to get a release out. I remember there were users asking a little while ago when the next release would happen so I think there are those in the community that are eager for a release. Thanks for stepping up to manage this. On Thu, Oct 3, 2024 at 6:22 PM Cole Greer wrote: > Hi everyone, > > Based on the work that has accumulated in 3.6-dev and 3.7-dev, as well as > the > time that has passed since our last release, I’d like to propose that we > conduct a > maintenance release in both branches by the end of the month. Both releases > would primarily consist of bug and security fixes. > > I would like to suggest that we start code freeze in the 3.6 and 3.7 > branches on > October 16th, which would set us up to have the release artifacts published > around October 28th. > > I will be looking into resolving all current open JIRAs with a priority of > Blocker > prior to the start of code freeze (TINKERPOP-3050, TINKERPOP-3051, > TINKERPOP-3052, TINKERPOP-3053, TINKERPOP-3076, TINKERPOP-3077, > TINKERPOP-3086, and TINKERPOP-2700). I’m able to step in as release manager > as well. > > Please reach out here if you are working on any additional changes which > you > would like to have included in this release. > > Thanks, > Cole >
[DISCUSS] Common HTTP options
Hi All, In the past there have been complaints about the compatibility amongst the GLVs. One of the complaints is about the differences in connection pooling options. I think that some of these options should be standardized across the GLVs in the next major version (which uses HTTP as its protocol). Here are the HTTP-specific options that I was thinking about: TlsConfiguration - Settings for TLS which should be: Enable - True/False Default: False Version - SSL3/TLS1.1/TLS1.2/TLS1.3/Auto Default Auto CertificateStore - path to .pem files that contain the CAs to use for verifying the server - Default "" Interceptor - Adds to a list of HTTP interceptors which will be executed in the order they were added. The actual structure of the interceptor will be language-dependent but will need to give access to both request header and request body. Authentication/authorization mechanisms should make use of this. - Default empty list MaxConnPerEndpoint - Maximum number of open TCP connections per endpoint. This acts as a cap on the number of connections in the pool. What happens if this limit is reached will be language dependent. - Default 128 UserAgent - Adds the user-provided string to the custom part of the userAgent. - Default "" MaxResponseContentLength - A setting for the maximum size of the entire response. Prevents accidental errors where the client can't actually handle a response of that size; that is, the user probably accidentally made an infinite traversal. - Default 2147483647 (max integer) GLVs will need to have their own specific options as the underlying HTTP client used may have some options that users will need access to. But, I think that GLV-specific options should be kept to a minimum. Any thoughts on this? Thanks, Ken
Re: [DISCUSS] Proposed updates to GraphBinaryV4 and GraphSONV4
Here are some other updates that I think should be made for GraphBinaryV4. There are steps that can return a distinct ordering in Maps like order(local). However, these aren't necessarily deserialized in GLVs as Map-types that preserve this ordering. I think that we should add a value_flag to the Map type in GraphBinary that signals that the deserialized version of this Map should maintain the ordering. Similar to the BulkSet change, we can use a value_flag with the value 0x02 to denote ordering. The value itself would remain unchanged, but the order of the items in that list should be the insertion/iteration order of the deserialized Map. All values that the Date type can represent are covered by OffsetDateTime so I think it makes sense to remove even the Date type. Date is also outdated in Java and its use isn't recommended. The current GraphBinary standard states that properties for elements like Vertex, Edge and VertexProperty should be set to null. The reasoning for this was that properties were never returned in remote traversals. However, this changed starting in 3.7, and properties can now be returned. Properties should instead be a List of Properties and if there are no properties then an empty list should be returned. Labels for elements are currently serialized as String for both GraphBinary and GraphSON. However, we may want to support multi-labels or no-label scenarios in the future. To prevent needing a breaking change if this were to happen, I propose that "label" become a List of String. In the new HTTP API, a ReponseMessage is returned per HTTP chunk and the HTTP response could end with a trailer. The GLVs may need some way to distinguish between chunks and trailers which could be accomplished using a one byte buffer in between the last ResponseMessage and the trailing header. The GLVs may need information to recreate a chunk as some HTTP clients transparently dechunk the body. A 4-byte header that indicates the size of ResponseMessage could be added to help GLVs recognize chunk boundaries which simplifies deserialization. On Thu, Aug 22, 2024 at 2:34 PM Ken Hu wrote: > Hi All, > > I've noticed that there are some serializers/deserializers that exist for > GraphBinary and GraphSON that may not serve a real purpose and I'm > proposing that we remove them in V4. In general, I think we should move > away from providing serializers for all types in the Java Standard Library, > as there are many types that other GLVs simply won't support. The types in > GraphSONv3/GraphBinaryV3 that I would remove include: > > Barrier, Cardinality, Column, DT, Merge, Operator, Order, Pick, Pop, > Scope, T > > Because the GLVs will now use GremlinLang (instead of Bytecode), these > values will be translated to Strings so they don't need their own specific > serializers. T and Direction are the two exceptions because they are > returned in the valueMap() and elementMap() steps, although it's likely > they will be replaced in the future as well. For similar reasons, P and > TextP should also be removed since they will also be Strings in GremlinLang. > > Period, Timestamp, Instant, ZonedDateTime, OffsetTime, LocalDateTime, > LocalDate, LocalTime, MonthDay, YearMonth, Year, ZoneOffset > > These are from the java.time package but are all variants of > OffsetDateTime and Date. I don't think it makes sense to provide > serializers for these Java-specific types that are mostly unsupported by > the other GLVs and can be represented by either an OffsetDateTime and/or > Date, so I think we should only have OffsetDateTime and Date. > > BulkSet and Traverser are two optimizations that exist to increase > performance but that the user shouldn't be able to interact with. I'm > suggesting that we remove these serializers, however, to keep the > performance optimizations, I would suggest that we add a flag to the List > type so that it can achieve the same sort of bulking that those types > provided. GraphSON would simply serialize these as arrays without a bulk so > it would be the expanded BulkSet. GraphBinary would have an additional > value_flag for List with the value 0x02 to denote bulking. This would mean > the item_i is a fully qualified item followed by a Long value that is the > bulk. > > Class and InetAddress are highly Java-specific and can't be used by the > other GLVs. Binding and Bytecode have both been removed so it doesn't make > sense to keep their serializers around. Metrics, TraversalMetrics can be > represented as a list of Strings so they shouldn't require their own > serializers. > > The last type that I think should be removed is Custom. The most common > complaint that I've seen from end-users regarding custom types is that they > lack serializers in their GLV of choic
[jira] [Commented] (TINKERPOP-3107) Ability to run Gremlin Server with only gremlin-lang
[ https://issues.apache.org/jira/browse/TINKERPOP-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876301#comment-17876301 ] Ken Hu commented on TINKERPOP-3107: --- With the move to making GremlinLangScriptEngine default in 4.x, it makes sense for something like this to happen in that branch. You marked the affects version as 3.7.x, so just wanted to confirm that you think its valuable to see a change like this in 3.7.x as well. > Ability to run Gremlin Server with only gremlin-lang > > > Key: TINKERPOP-3107 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3107 > Project: TinkerPop > Issue Type: Improvement > Components: server >Affects Versions: 3.7.2 >Reporter: Valentyn Kahamlyk >Priority: Major > > Currently, GraphTraversalSource creation is done via > `ScriptFileGremlinPlugin` and groovy script. > Disabling the plugin leads to > `The traversal source [g] for alias [g] is not configured on the server`. > It would be nice to configure `GraphTraversalSource` without using groovy. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TINKERPOP-3040) Remote Console won't send queries that cause exceptions locally
[
https://issues.apache.org/jira/browse/TINKERPOP-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876184#comment-17876184
]
Ken Hu commented on TINKERPOP-3040:
---
It's been a while since I've taken a look at this part of the code myself so I
can't quite remember the details.
We depend on the local evaluation to tell us if the multiline script is
fully-formed and complete so that it is ready to be sent to server. The way the
parser is setup now in Groovy 4 may require that evaluation to happen twice per
line. In the past, there were alternative parsers for Groovy but I didn't look
to see if alternatives were available in Groovy 4. That is possibly one thing
to check on.
If you are looking for a way forward for this issue, the easiest thing to try
would be to add a "catch(RuntimeException)". Of course, this wouldn't always
work since it might actually be a real error. But, it might be enough to
satisfy your use case.
> Remote Console won't send queries that cause exceptions locally
> ---
>
> Key: TINKERPOP-3040
> URL: https://issues.apache.org/jira/browse/TINKERPOP-3040
> Project: TinkerPop
> Issue Type: Bug
> Components: console
>Affects Versions: 3.7.0
>Reporter: Ken Hu
>Priority: Critical
>
> From: https://groups.google.com/g/gremlin-users/c/U2gIxmRcrI8/m/x12banhdAAAJ
> It appears that if an exception occurs when the remote console is executing a
> command locally then it won't send that query to the server.
>
> Example:
> {code:java}
> \,,,/
> (o o)
> -oOOo-(3)-oOOo-
> plugin activated: tinkerpop.server
> plugin activated: tinkerpop.utilities
> plugin activated: tinkerpop.tinkergraph
> plugin activated: janusgraph.imports
> gremlin> :remote connect tinkerpop.server conf/remote.yaml
> ==>Configured localhost/127.0.0.1:45940
> gremlin> :remote console
> ==>All scripts will now be sent to Gremlin Server -
> [localhost/127.0.0.1:45940] - type ':remote console' to return to local mode
> gremlin> JanusGraphFactory.getGraphNames()
> Gremlin Server must be configured to use the JanusGraphManager.
> Type ':help' or ':h' for help.
> Display stack trace? [yN]y
> java.lang.NullPointerException: Gremlin Server must be configured to use the
> JanusGraphManager.
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:921)
> at
> org.janusgraph.core.JanusGraphFactory.getGraphNames(JanusGraphFactory.java:187)
> at
> org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
> at groovysh_evaluate.run(groovysh_evaluate:3)
> at
> org.codehaus.groovy.vmplugin.v8.IndyInterface.selectMethod(IndyInterface.java:355)
> at
> org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
> at
> org.apache.groovy.groovysh.Interpreter.evaluate(Interpreter.groovy:82)
> at
> org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
> at
> org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:118)
> at
> org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
> at org.apache.groovy.groovysh.Shell.leftShift(Shell.groovy:121)
> at
> org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
> at org.apache.groovy.groovysh.ShellRunner.work(ShellRunner.groovy:93)
> at
> org.apache.groovy.groovysh.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124)
> at
> org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
> at org.apache.groovy.groovysh.ShellRunner.run(ShellRunner.groovy:57)
> at
> org.apache.groovy.groovysh.InteractiveShellRunner.run(InteractiveShellRunner.groovy:96)
> at
> org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
> at
> org.apache.tinkerpop.gremlin.console.Console.(Console.groovy:175)
> at
> org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
> at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:574)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[DISCUSS] Proposed updates to GraphBinaryV4 and GraphSONV4
Hi All, I've noticed that there are some serializers/deserializers that exist for GraphBinary and GraphSON that may not serve a real purpose and I'm proposing that we remove them in V4. In general, I think we should move away from providing serializers for all types in the Java Standard Library, as there are many types that other GLVs simply won't support. The types in GraphSONv3/GraphBinaryV3 that I would remove include: Barrier, Cardinality, Column, DT, Merge, Operator, Order, Pick, Pop, Scope, T Because the GLVs will now use GremlinLang (instead of Bytecode), these values will be translated to Strings so they don't need their own specific serializers. T and Direction are the two exceptions because they are returned in the valueMap() and elementMap() steps, although it's likely they will be replaced in the future as well. For similar reasons, P and TextP should also be removed since they will also be Strings in GremlinLang. Period, Timestamp, Instant, ZonedDateTime, OffsetTime, LocalDateTime, LocalDate, LocalTime, MonthDay, YearMonth, Year, ZoneOffset These are from the java.time package but are all variants of OffsetDateTime and Date. I don't think it makes sense to provide serializers for these Java-specific types that are mostly unsupported by the other GLVs and can be represented by either an OffsetDateTime and/or Date, so I think we should only have OffsetDateTime and Date. BulkSet and Traverser are two optimizations that exist to increase performance but that the user shouldn't be able to interact with. I'm suggesting that we remove these serializers, however, to keep the performance optimizations, I would suggest that we add a flag to the List type so that it can achieve the same sort of bulking that those types provided. GraphSON would simply serialize these as arrays without a bulk so it would be the expanded BulkSet. GraphBinary would have an additional value_flag for List with the value 0x02 to denote bulking. This would mean the item_i is a fully qualified item followed by a Long value that is the bulk. Class and InetAddress are highly Java-specific and can't be used by the other GLVs. Binding and Bytecode have both been removed so it doesn't make sense to keep their serializers around. Metrics, TraversalMetrics can be represented as a list of Strings so they shouldn't require their own serializers. The last type that I think should be removed is Custom. The most common complaint that I've seen from end-users regarding custom types is that they lack serializers in their GLV of choice. Custom serializers require extra effort from providers as it is they that need to maintain and publish them. Users often don't realize that they can't just use the GLV without adding the custom serializer from their provider which can lead to a frustrating initial experience. So, instead, to provide a more seamless user experience I'm proposing we replace Custom with a Provider Defined Type (PDT). Actually, two PDTs, one for primitive types and one for composite types. These PDTs will also serve as a mechanism to allow reading of values whose serializer has been removed in V4. This would allow a provider that supports one of those Java-specific types to still return the value as a PDT. The PrimitivePdt will contain two fields: a String for the type name and a String for the actual value. The CompositePdt will contain two fields: a String for the type name and a Map that contains the fields of the composite type (most likely a class). PDTs will come with default serializers and deserializers. On the user-side, the values will be deserialized into either PrimitivePdt or CompositePdt. In order to write a PDT to the server, the user would have to look up the required fields for that specific type name. This information would likely need to exist in the provider's documentation. On the server, there will be an API (probably attached to the Graph interface) that will allow the provider to add types that should be able to be serialized/deserialized from PDT. In order to achieve this, there will need to be a TinkerPopCustomType interface that has two methods: serialize() and deserialize(). Then, any type that implements TinkerPopCustomType could automatically be serialized/deserialized with the provided PDT serializers for both GraphSONv4 and GraphBinaryV4. Does anyone have any comments or concerns about these proposed changes? Thanks, Ken
HTTP vs WS performance evaluation
Hi All, I just wanted to share some performance evaluations that I did between the old 3.7 Java driver/server (WebSockets) and the new driver/server (HTTP). The comparison uses the ProfilingApplication tool located in the gremlin-driver package that has been updated with latency testing. For more details please check out https://github.com/apache/tinkerpop/pull/2680 which has the complete data. I'll share just a few points here. For a latency tests that returned ~30million vertices, the HTTP driver took 54.89s while the WebSocket driver took 53.48s. For a throughput tests that issued 1 queries, the HTTP driver processed 30063 queries/sec, while the WebSocket driver processed 30488 queries/sec. So, performance wise, the new HTTP implementation is within a few percentage points of the 3.7.x WebSocket implementation.
[jira] [Created] (TINKERPOP-3104) Make unfold() on Maps consistent between embedded and remote use
Ken Hu created TINKERPOP-3104: - Summary: Make unfold() on Maps consistent between embedded and remote use Key: TINKERPOP-3104 URL: https://issues.apache.org/jira/browse/TINKERPOP-3104 Project: TinkerPop Issue Type: Improvement Components: process Reporter: Ken Hu The result of an unfolding a Map is different when run in embedded mode and when run remotely. In the embedded case, you'll get a Map.Entry, in the remote case, you'll get a size one Map. It would be better to make this consistent. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TINKERPOP-2363) GraphBinary and numerics for Python
[
https://issues.apache.org/jira/browse/TINKERPOP-2363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu updated TINKERPOP-2363:
--
Priority: Critical (was: Major)
> GraphBinary and numerics for Python
> ---
>
> Key: TINKERPOP-2363
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2363
> Project: TinkerPop
> Issue Type: Improvement
> Components: python
>Affects Versions: 3.4.6
>Reporter: Stephen Mallette
>Priority: Critical
>
> After TINKERPOP-2360 we've started down the path of coercing python numerics
> to Java type ranges. GraphBinary doesn't do that and therefore switching
> serializers from GraphSON to GraphBinary will have the potential to result in
> error if the python numeric overflows the Java integer space. In addition, we
> don't support {{BigInteger}} serialization apparently so that remains a
> problem. This ticket should just sort out GraphBinary and numbers all at once.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2363) GraphBinary and numerics for Python
[
https://issues.apache.org/jira/browse/TINKERPOP-2363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17873291#comment-17873291
]
Ken Hu commented on TINKERPOP-2363:
---
This opens a wider discussion about how we want to handle coercion for drivers
in languages that have a single numeric type. Maybe we should do something
similar as what was done in TINKERPOP-2360 but extend it to the rest of the
numeric types.
In any case, since GraphBinary is now the default (and in the future maybe the
only) serializer for the GLVs, I'm going to increase the priority of this
ticket.
> GraphBinary and numerics for Python
> ---
>
> Key: TINKERPOP-2363
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2363
> Project: TinkerPop
> Issue Type: Improvement
> Components: python
>Affects Versions: 3.4.6
>Reporter: Stephen Mallette
>Priority: Major
>
> After TINKERPOP-2360 we've started down the path of coercing python numerics
> to Java type ranges. GraphBinary doesn't do that and therefore switching
> serializers from GraphSON to GraphBinary will have the potential to result in
> error if the python numeric overflows the Java integer space. In addition, we
> don't support {{BigInteger}} serialization apparently so that remains a
> problem. This ticket should just sort out GraphBinary and numbers all at once.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (TINKERPOP-2848) Deserialization of integer based datatypes lose the type definition
[ https://issues.apache.org/jira/browse/TINKERPOP-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu updated TINKERPOP-2848: -- Priority: Major (was: Minor) > Deserialization of integer based datatypes lose the type definition > --- > > Key: TINKERPOP-2848 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2848 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.5.4 >Reporter: Yang Xia >Priority: Major > > As proposed by this PR > [[https://github.com/apache/tinkerpop/pull/1848,|https://github.com/apache/tinkerpop/pull/1848] > currently in python, deserializing of integer based datatypes don't preserve > the type definition from server. > Some discussion may be needed around the impact of updating this > deserialization. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2848) Deserialization of integer based datatypes lose the type definition
[ https://issues.apache.org/jira/browse/TINKERPOP-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17873289#comment-17873289 ] Ken Hu commented on TINKERPOP-2848: --- This affects the ability to "round trip" numeric types with gremlin-python. A number will be stored as Python's internal numeric representation and it will then attempt to serialize it back as a 32-bit integer. This can lead to a result that is too big or too small. > Deserialization of integer based datatypes lose the type definition > --- > > Key: TINKERPOP-2848 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2848 > Project: TinkerPop > Issue Type: Improvement > Components: python >Affects Versions: 3.5.4 >Reporter: Yang Xia >Priority: Minor > > As proposed by this PR > [[https://github.com/apache/tinkerpop/pull/1848,|https://github.com/apache/tinkerpop/pull/1848] > currently in python, deserializing of integer based datatypes don't preserve > the type definition from server. > Some discussion may be needed around the impact of updating this > deserialization. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (TINKERPOP-3103) Element comparison that includes properties
Ken Hu created TINKERPOP-3103: - Summary: Element comparison that includes properties Key: TINKERPOP-3103 URL: https://issues.apache.org/jira/browse/TINKERPOP-3103 Project: TinkerPop Issue Type: Improvement Components: dotnet, driver, go, javascript, python Affects Versions: 3.7.1 Reporter: Ken Hu Starting in 3.7.x, properties can now be returned for Element types like Vertex and Edge. However, in some GLVs, comparison for Elements are still based only on Id (and maybe Label). This will probably need to get updated to reflect the fact that properties can be returned in queries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TINKERPOP-3100) Traversal.Admin.lock() has excessive recursion
[
https://issues.apache.org/jira/browse/TINKERPOP-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17870662#comment-17870662
]
Ken Hu commented on TINKERPOP-3100:
---
I ran a similar example on x86 and noticed a comparable performance loss.
Because this is a performance issue, I increased the priority to critical so
that it hopefully gets addressed in the next release.
> Traversal.Admin.lock() has excessive recursion
> --
>
> Key: TINKERPOP-3100
> URL: https://issues.apache.org/jira/browse/TINKERPOP-3100
> Project: TinkerPop
> Issue Type: Bug
> Components: console, process
>Affects Versions: 3.7.2
> Environment: M1 Macbook, 16GB RAM
>Reporter: Cameron Fiander
>Priority: Critical
> Labels: performance
>
> Hi, we are currently upgrading to TinkerPop 3.7, and noticed that some of our
> larger queries were taking much longer than expected to applyStrategies /
> lock.[ I've created an example project with a contrived complex traversal
> that reproduces the
> issue|https://github.com/camfiander-sonrai/TinkerPopLockPerformanceRepro].
> I noticed that many of the children steps were getting locked more than once,
> which shouldn't be necessary. I think the problem is
> [here|https://github.com/apache/tinkerpop/blob/3.7.2/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/util/DefaultTraversal.java#L333].
> {{{}applyTraversalRecursively{}}}, called from the traversal root, should
> apply the {{lock()}} to all children, grandchildren, and so on. But each
> child also calls {{{}applyTraversalRecursively{}}}, meaning the number of
> times a step is locked scales with the depth of the traversal.
> {code:java}
> # TinkerPop 3.5.3
> Applied strategies in PT3.695S
> # TinkerPop 3.7.2
> Applied strategies in PT15.079S{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (TINKERPOP-3100) Traversal.Admin.lock() has excessive recursion
[
https://issues.apache.org/jira/browse/TINKERPOP-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu updated TINKERPOP-3100:
--
Priority: Critical (was: Major)
> Traversal.Admin.lock() has excessive recursion
> --
>
> Key: TINKERPOP-3100
> URL: https://issues.apache.org/jira/browse/TINKERPOP-3100
> Project: TinkerPop
> Issue Type: Bug
> Components: console, process
>Affects Versions: 3.7.2
> Environment: M1 Macbook, 16GB RAM
>Reporter: Cameron Fiander
>Priority: Critical
> Labels: performance
>
> Hi, we are currently upgrading to TinkerPop 3.7, and noticed that some of our
> larger queries were taking much longer than expected to applyStrategies /
> lock.[ I've created an example project with a contrived complex traversal
> that reproduces the
> issue|https://github.com/camfiander-sonrai/TinkerPopLockPerformanceRepro].
> I noticed that many of the children steps were getting locked more than once,
> which shouldn't be necessary. I think the problem is
> [here|https://github.com/apache/tinkerpop/blob/3.7.2/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/util/DefaultTraversal.java#L333].
> {{{}applyTraversalRecursively{}}}, called from the traversal root, should
> apply the {{lock()}} to all children, grandchildren, and so on. But each
> child also calls {{{}applyTraversalRecursively{}}}, meaning the number of
> times a step is locked scales with the depth of the traversal.
> {code:java}
> # TinkerPop 3.5.3
> Applied strategies in PT3.695S
> # TinkerPop 3.7.2
> Applied strategies in PT15.079S{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TINKERPOP-3097) Request for Mocking Capabilities in gremlin-go for Easier Unit Testing
[ https://issues.apache.org/jira/browse/TINKERPOP-3097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17870642#comment-17870642 ] Ken Hu commented on TINKERPOP-3097: --- Do you mind clarifying what you would like to see with a small example? Something that shows why unit testing is difficult right now with gremlin-go. Are you asking for an easier way to mock the result/response? > Request for Mocking Capabilities in gremlin-go for Easier Unit Testing > -- > > Key: TINKERPOP-3097 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3097 > Project: TinkerPop > Issue Type: Improvement > Components: go >Affects Versions: 3.7.2 >Reporter: thomas-dg >Priority: Major > > I have been using the gremlin-go library and found that it currently lacks > built-in mocking capabilities. This absence makes it challenging to perform > unit testing effectively. > As a workaround, I have defined an additional interface layer between the > caller and gremlin-go, which allows me to unit test the business logic of the > caller. However, this is not the most efficient solution. > It would be significantly more convenient if the gremlin-go library could > provide native support for mocking. This feature would streamline the testing > process and improve the library's usability. > I look forward to your response and further discussion on this matter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Benchmarking Framework
Hey Lyndon, This is a very interesting idea. You mentioned throughput testing but how does this compare to other graph testing that use specific generated datasets and specific queries? Asked another way, what kind of queries are you using to test in this system? Regards, Ken On Tue, Jul 9, 2024 at 2:00 PM Lyndon Bauto wrote: > Hi devs, > > I've been working on a benchmarking framework for tinkerpop, specifically > the Java driver. > > The idea is to have a benchmarking framework that a TinkerPop user can > target their instance of gremlin-server with (can be any provider) and what > this will allow them to do is fix some of their configs of the driver while > having others as variables. The framework will then run through a bunch of > different settings, recording latency and throughput. > > The output of the benchmarking framework would be guidance for the user of > the Java driver for optimal configuration for both latency and throughput, > that they can then use to optimize their workload outside the framework. > > A provider could also use this to manually adjust > gremlinPool/threadPoolWorkers/etc and run the framework under different > settings to optimize throughput and latency there as well. > > The benchmark is built on JMH and is build into a docker container so it is > very easy to use. The configs are passed at runtime, so a user would just > call a docker build then docker run script, with the configs setup in the > docker config. > > We could also add other benchmarks at any scale to the framework that allow > benchmark publishing from providers who wish to participate. > > Anyone have any thoughts on this? > > Cheers, > Lyndon > -- > > *Lyndon Bauto* > *Senior Software Engineer* > *Aerospike, Inc.* > www.aerospike.com > lba...@aerospike.com >
[jira] [Closed] (TINKERPOP-3093) optimization of readmap function
[ https://issues.apache.org/jira/browse/TINKERPOP-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu closed TINKERPOP-3093. - Resolution: Fixed > optimization of readmap function > > > Key: TINKERPOP-3093 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3093 > Project: TinkerPop > Issue Type: Bug > Components: go >Affects Versions: 3.7.2 >Reporter: thomas-dg >Priority: Minor > Fix For: 4.0.0, 3.7.3 > > > There are two optimization points for the readmap function: > # Map key cannot be an incomparable type, otherwise it will cause a panic, > but the function does not distinguish > # When the key is an incomparable type, adjusted to fmt.Sprint(k) to avoid > unreadable output -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TINKERPOP-3093) optimization of readmap function
[ https://issues.apache.org/jira/browse/TINKERPOP-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu updated TINKERPOP-3093: -- Fix Version/s: 4.0.0 > optimization of readmap function > > > Key: TINKERPOP-3093 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3093 > Project: TinkerPop > Issue Type: Bug > Components: go >Affects Versions: 3.7.2 >Reporter: thomas-dg >Priority: Minor > Fix For: 4.0.0, 3.7.3 > > > There are two optimization points for the readmap function: > # Map key cannot be an incomparable type, otherwise it will cause a panic, > but the function does not distinguish > # When the key is an incomparable type, adjusted to fmt.Sprint(k) to avoid > unreadable output -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TINKERPOP-3093) optimization of readmap function
[ https://issues.apache.org/jira/browse/TINKERPOP-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu updated TINKERPOP-3093: -- Issue Type: Bug (was: Improvement) > optimization of readmap function > > > Key: TINKERPOP-3093 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3093 > Project: TinkerPop > Issue Type: Bug > Components: go >Affects Versions: 3.7.2 >Reporter: thomas-dg >Priority: Minor > Fix For: 3.7.3 > > > There are two optimization points for the readmap function: > # Map key cannot be an incomparable type, otherwise it will cause a panic, > but the function does not distinguish > # When the key is an incomparable type, adjusted to fmt.Sprint(k) to avoid > unreadable output -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TINKERPOP-3093) optimization of readmap function
[ https://issues.apache.org/jira/browse/TINKERPOP-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17863403#comment-17863403 ] Ken Hu commented on TINKERPOP-3093: --- https://github.com/apache/tinkerpop/pull/2673 > optimization of readmap function > > > Key: TINKERPOP-3093 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3093 > Project: TinkerPop > Issue Type: Bug > Components: go >Affects Versions: 3.7.2 >Reporter: thomas-dg >Priority: Minor > Fix For: 3.7.3 > > > There are two optimization points for the readmap function: > # Map key cannot be an incomparable type, otherwise it will cause a panic, > but the function does not distinguish > # When the key is an incomparable type, adjusted to fmt.Sprint(k) to avoid > unreadable output -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TINKERPOP-3093) optimization of readmap function
[ https://issues.apache.org/jira/browse/TINKERPOP-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu updated TINKERPOP-3093: -- Fix Version/s: 3.7.3 > optimization of readmap function > > > Key: TINKERPOP-3093 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3093 > Project: TinkerPop > Issue Type: Improvement > Components: go >Affects Versions: 3.7.2 >Reporter: thomas-dg >Priority: Minor > Fix For: 3.7.3 > > > There are two optimization points for the readmap function: > # Map key cannot be an incomparable type, otherwise it will cause a panic, > but the function does not distinguish > # When the key is an incomparable type, adjusted to fmt.Sprint(k) to avoid > unreadable output -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TINKERPOP-3093) optimization of readmap function
[ https://issues.apache.org/jira/browse/TINKERPOP-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu updated TINKERPOP-3093: -- Affects Version/s: 3.7.2 > optimization of readmap function > > > Key: TINKERPOP-3093 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3093 > Project: TinkerPop > Issue Type: Improvement > Components: go >Affects Versions: 3.7.2 >Reporter: thomas-dg >Priority: Minor > > There are two optimization points for the readmap function: > # Map key cannot be an incomparable type, otherwise it will cause a panic, > but the function does not distinguish > # When the key is an incomparable type, adjusted to fmt.Sprint(k) to avoid > unreadable output -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TINKERPOP-3093) optimization of readmap function
[ https://issues.apache.org/jira/browse/TINKERPOP-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu updated TINKERPOP-3093: -- Component/s: go > optimization of readmap function > > > Key: TINKERPOP-3093 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3093 > Project: TinkerPop > Issue Type: Improvement > Components: go >Reporter: thomas-dg >Priority: Minor > > There are two optimization points for the readmap function: > # Map key cannot be an incomparable type, otherwise it will cause a panic, > but the function does not distinguish > # When the key is an incomparable type, adjusted to fmt.Sprint(k) to avoid > unreadable output -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (TINKERPOP-3092) Gremlin Standard Test Suite fails on Java 9+ with a module system error
[
https://issues.apache.org/jira/browse/TINKERPOP-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu closed TINKERPOP-3092.
-
Resolution: Information Provided
> Gremlin Standard Test Suite fails on Java 9+ with a module system error
> ---
>
> Key: TINKERPOP-3092
> URL: https://issues.apache.org/jira/browse/TINKERPOP-3092
> Project: TinkerPop
> Issue Type: Bug
> Components: test-suite
>Reporter: Martin Häusler
>Priority: Major
>
> I'm taking a second shot at upgrading my graph implementation from Java 8 to
> the current stable Java version (21). However, a lot (922) of TinkerPop tests
> which previously succeeded fail on the new Java version, because Kryo
> attempts to access a private field in AtomicLong which should have been left
> alone.
> The stack trace is attached below; apart from the specific test case, the
> error is identical for all of them). Please also note that this stack trace
> is entirely coming from the "org.apache.tinkerpop" namespace, there's not a
> single class from my implementation in there.
>
> Any advice on what I could do? Do I really have to resort to JVM flags to
> open the module just to make the test suite run? Or am I missing something
> else?
>
> {code:java}
> java.lang.IllegalArgumentException: Unable to create serializer
> "org.apache.tinkerpop.shaded.kryo.serializers.FieldSerializer" for class:
> java.util.concurrent.atomic.AtomicLong at
> org.apache.tinkerpop.shaded.kryo.factories.ReflectionSerializerFactory.makeSerializer(ReflectionSerializerFactory.java:67)
>at
> org.apache.tinkerpop.shaded.kryo.factories.ReflectionSerializerFactory.makeSerializer(ReflectionSerializerFactory.java:45)
>at
> org.apache.tinkerpop.shaded.kryo.Kryo.newDefaultSerializer(Kryo.java:380)
> at org.apache.tinkerpop.shaded.kryo.Kryo.getDefaultSerializer(Kryo.java:364)
> at
> org.apache.tinkerpop.gremlin.structure.io.gryo.GryoTypeReg.registerWith(GryoTypeReg.java:122)
> at
> org.apache.tinkerpop.gremlin.structure.io.gryo.GryoMapper.createMapper(GryoMapper.java:101)
> at
> org.apache.tinkerpop.gremlin.structure.io.gryo.GryoMapper.createMapper(GryoMapper.java:75)
>at
> org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader.(GryoReader.java:71)
> at
> org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader.(GryoReader.java:64)
> at
> org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader$Builder.create(GryoReader.java:302)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.IoStep.constructReader(IoStep.java:156)
>at
> org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.IoStep.read(IoStep.java:132)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.IoStep.processNextStart(IoStep.java:110)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:150)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:55)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep.processNextStart(FilterStep.java:40)
>at
> org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:135)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:40)
> at
> org.apache.tinkerpop.gremlin.process.traversal.Traversal.iterate(Traversal.java:210)
> at
> org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal.iterate(GraphTraversal.java:3327)
> at
> org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal$Admin.iterate(GraphTraversal.java:202)
>at
> org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal.iterate(DefaultGraphTraversal.java:48)
> at
> org.apache.tinkerpop.gremlin.AbstractGraphProvider.readIntoGraph(AbstractGraphProvider.java:153)
> at
> org.apache.tinkerpop.gremlin.AbstractGraphProvider.loadGraphData(AbstractGraphProvider.java:93)
> at
> org.apache.tinkerpop.gremlin.GraphManager$ManagedGraphProvider.loadGraphData(GraphManager.java:168)
> at
> org.apache.tinkerpop.gremlin.AbstractGremlinTest.setup(AbstractGremlinTest.java:116)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580) at
>
[jira] [Commented] (TINKERPOP-3092) Gremlin Standard Test Suite fails on Java 9+ with a module system error
[
https://issues.apache.org/jira/browse/TINKERPOP-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17863360#comment-17863360
]
Ken Hu commented on TINKERPOP-3092:
---
For now you'll have to depend on JVM flags. I believe the only way we can
support this without relying on the flags is to implement Modules. See
TINKERPOP-3019 and
[https://lists.apache.org/thread/vkvqcxn6t2lsgpfym7v3g3qvksrh3cwc] for
discussion around that topic.
> Gremlin Standard Test Suite fails on Java 9+ with a module system error
> ---
>
> Key: TINKERPOP-3092
> URL: https://issues.apache.org/jira/browse/TINKERPOP-3092
> Project: TinkerPop
> Issue Type: Bug
> Components: test-suite
>Reporter: Martin Häusler
>Priority: Major
>
> I'm taking a second shot at upgrading my graph implementation from Java 8 to
> the current stable Java version (21). However, a lot (922) of TinkerPop tests
> which previously succeeded fail on the new Java version, because Kryo
> attempts to access a private field in AtomicLong which should have been left
> alone.
> The stack trace is attached below; apart from the specific test case, the
> error is identical for all of them). Please also note that this stack trace
> is entirely coming from the "org.apache.tinkerpop" namespace, there's not a
> single class from my implementation in there.
>
> Any advice on what I could do? Do I really have to resort to JVM flags to
> open the module just to make the test suite run? Or am I missing something
> else?
>
> {code:java}
> java.lang.IllegalArgumentException: Unable to create serializer
> "org.apache.tinkerpop.shaded.kryo.serializers.FieldSerializer" for class:
> java.util.concurrent.atomic.AtomicLong at
> org.apache.tinkerpop.shaded.kryo.factories.ReflectionSerializerFactory.makeSerializer(ReflectionSerializerFactory.java:67)
>at
> org.apache.tinkerpop.shaded.kryo.factories.ReflectionSerializerFactory.makeSerializer(ReflectionSerializerFactory.java:45)
>at
> org.apache.tinkerpop.shaded.kryo.Kryo.newDefaultSerializer(Kryo.java:380)
> at org.apache.tinkerpop.shaded.kryo.Kryo.getDefaultSerializer(Kryo.java:364)
> at
> org.apache.tinkerpop.gremlin.structure.io.gryo.GryoTypeReg.registerWith(GryoTypeReg.java:122)
> at
> org.apache.tinkerpop.gremlin.structure.io.gryo.GryoMapper.createMapper(GryoMapper.java:101)
> at
> org.apache.tinkerpop.gremlin.structure.io.gryo.GryoMapper.createMapper(GryoMapper.java:75)
>at
> org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader.(GryoReader.java:71)
> at
> org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader.(GryoReader.java:64)
> at
> org.apache.tinkerpop.gremlin.structure.io.gryo.GryoReader$Builder.create(GryoReader.java:302)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.IoStep.constructReader(IoStep.java:156)
>at
> org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.IoStep.read(IoStep.java:132)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.IoStep.processNextStart(IoStep.java:110)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:150)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:55)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep.processNextStart(FilterStep.java:40)
>at
> org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:135)
> at
> org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:40)
> at
> org.apache.tinkerpop.gremlin.process.traversal.Traversal.iterate(Traversal.java:210)
> at
> org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal.iterate(GraphTraversal.java:3327)
> at
> org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal$Admin.iterate(GraphTraversal.java:202)
>at
> org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal.iterate(DefaultGraphTraversal.java:48)
> at
> org.apache.tinkerpop.gremlin.AbstractGraphProvider.readIntoGraph(AbstractGraphProvider.java:153)
> at
> org.apache.tinkerpop.gremlin.AbstractGraphProvider.loadGraphData(AbstractGraphProvider.java:93)
> at
> org.apache.tinkerpop.gremlin.GraphManager$ManagedGraphProvider.loadGraphData(GraphManager.java:168)
> at
&
[jira] [Closed] (TINKERPOP-3081) When using authentication, evaluationTimeout is ignored
[
https://issues.apache.org/jira/browse/TINKERPOP-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu closed TINKERPOP-3081.
-
Resolution: Fixed
> When using authentication, evaluationTimeout is ignored
> ---
>
> Key: TINKERPOP-3081
> URL: https://issues.apache.org/jira/browse/TINKERPOP-3081
> Project: TinkerPop
> Issue Type: Bug
>Affects Versions: 3.6.7
>Reporter: Lyndon Bauto
>Assignee: Lyndon Bauto
>Priority: Major
> Fix For: 3.6.8, 3.7.3
>
>
> Using g.with("evaluationTimeout"). ignored evaluationTimeout when
> authentication is enabled.
>
> Relevant discord thread:
> https://discord.com/channels/838910279550238720/1245462911728615629/1245462911728615629
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (TINKERPOP-3081) When using authentication, evaluationTimeout is ignored
[
https://issues.apache.org/jira/browse/TINKERPOP-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu updated TINKERPOP-3081:
--
Fix Version/s: 3.6.8
3.7.3
> When using authentication, evaluationTimeout is ignored
> ---
>
> Key: TINKERPOP-3081
> URL: https://issues.apache.org/jira/browse/TINKERPOP-3081
> Project: TinkerPop
> Issue Type: Bug
>Affects Versions: 3.6.7
>Reporter: Lyndon Bauto
>Assignee: Lyndon Bauto
>Priority: Major
> Fix For: 3.6.8, 3.7.3
>
>
> Using g.with("evaluationTimeout"). ignored evaluationTimeout when
> authentication is enabled.
>
> Relevant discord thread:
> https://discord.com/channels/838910279550238720/1245462911728615629/1245462911728615629
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (TINKERPOP-3081) When using authentication, evaluationTimeout is ignored
[
https://issues.apache.org/jira/browse/TINKERPOP-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu updated TINKERPOP-3081:
--
Affects Version/s: 3.6.7
> When using authentication, evaluationTimeout is ignored
> ---
>
> Key: TINKERPOP-3081
> URL: https://issues.apache.org/jira/browse/TINKERPOP-3081
> Project: TinkerPop
> Issue Type: Bug
>Affects Versions: 3.6.7
>Reporter: Lyndon Bauto
>Assignee: Lyndon Bauto
>Priority: Major
>
> Using g.with("evaluationTimeout"). ignored evaluationTimeout when
> authentication is enabled.
>
> Relevant discord thread:
> https://discord.com/channels/838910279550238720/1245462911728615629/1245462911728615629
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (TINKERPOP-3094) Add custom handler for WebSocket Close frames
Ken Hu created TINKERPOP-3094: - Summary: Add custom handler for WebSocket Close frames Key: TINKERPOP-3094 URL: https://issues.apache.org/jira/browse/TINKERPOP-3094 Project: TinkerPop Issue Type: Improvement Components: driver, server Affects Versions: 3.7.2 Reporter: Ken Hu Currently, the Close frame is only used to close the connection. There are situations where it would be useful to know what caused the connection to close, particularly if it was caused by an error. A straightforward way to do this would be to include the reason for connection closing in the body of the Close frame. Then each GLV could use that information to determine if special handling is required. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] Add PR Template to GitHub
I've made a first attempt at something like this. https://github.com/apache/tinkerpop/pull/2664 On Tue, Jun 11, 2024 at 2:24 PM Ken Hu wrote: > Hi Cole, > > I've definitely found that new contributors are often confused on which > branch to target so I think this is an issue that needs to be solved. I'm > not sure if PR template is the right solution to this problem, but I think > it's a reasonable first step so I'm all for trying it. > > The other item that people often leave out is the CHANGELOG entry and > sometimes UPGRADE documentation. There is also sometimes confusion on where > to add the entry to those documents so I think adding some information > about that would be useful as well. I think it's one of those > inconveniences to contributors as sometimes the only review comment they > get is something like "needs CHANGELOG entry". > > Regards, > Ken > > On Tue, Jun 11, 2024 at 2:10 PM Cole Greer > wrote: > >> Hey everyone, >> >> A topic that has come up a few times in the past is adding a PR template >> to our repo. I would like to kick off an official discussion before adding >> one to the repo. >> >> My suggestion is to use the Spark PR template ( >> https://github.com/apache/spark/blob/v3.5.1/.github/PULL_REQUEST_TEMPLATE) >> as a reference and make some small updates to link to our docs and better >> fit our process. The main TP specific details I would like to add is >> guidance about which branch the PR should target, as well as a reminder to >> properly reference any related JIRAs or devlist posts. >> >> I don’t expect we would be strict with template adherence, but I think it >> would help guide newer contributors in the right direction. Let me know any >> thoughts you have, and if there are other instructions which should be >> included in the template. My plan is to draft up a template including any >> feedback here sometime this week. >> >> Thanks, >> Cole >> >
Re: [DISCUSS] Java GLV client simplification in TP4.
I agree with removing this. I've seen users complain about this method not
working properly, but I find it very unexpected that they are using it in
the first place. At the very least it should be made package-private as it
seems to me to be an internal implementation, rather than something users
should have access to.
On Mon, Jun 24, 2024 at 4:12 PM Valentin Kagamlyk <
valentin.kagam...@gmail.com> wrote:
> Hi all,
>
> Currently, it is possible to send a Traversal from the Java GLV using
> Client. For example,
>
> final Graph graph = TinkerGraph.open();
> final GraphTraversalSource g = graph.traversal();
> final Client aliased = client.alias("gmodern");
> final ResultSet resultSetUnrolled = aliased.submit(g.V());
>
> The problem is that this is equivalent to submitting a bytecode which will
> be removed in TP4.
>
> I'm thinking of removing this confusing function. The Client is designed to
> send string Gremlin queries, but for Traversals, DriverRemoteConnection
> should be used instead.
>
> If you want to keep `Client.submit(Traversal)`, then do not hesitate to say
> in the next 72 hours.
>
> Regards,
> Valentyn
>
Re: [DISCUSS] Add PR Template to GitHub
Hi Cole, I've definitely found that new contributors are often confused on which branch to target so I think this is an issue that needs to be solved. I'm not sure if PR template is the right solution to this problem, but I think it's a reasonable first step so I'm all for trying it. The other item that people often leave out is the CHANGELOG entry and sometimes UPGRADE documentation. There is also sometimes confusion on where to add the entry to those documents so I think adding some information about that would be useful as well. I think it's one of those inconveniences to contributors as sometimes the only review comment they get is something like "needs CHANGELOG entry". Regards, Ken On Tue, Jun 11, 2024 at 2:10 PM Cole Greer wrote: > Hey everyone, > > A topic that has come up a few times in the past is adding a PR template > to our repo. I would like to kick off an official discussion before adding > one to the repo. > > My suggestion is to use the Spark PR template ( > https://github.com/apache/spark/blob/v3.5.1/.github/PULL_REQUEST_TEMPLATE) > as a reference and make some small updates to link to our docs and better > fit our process. The main TP specific details I would like to add is > guidance about which branch the PR should target, as well as a reminder to > properly reference any related JIRAs or devlist posts. > > I don’t expect we would be strict with template adherence, but I think it > would help guide newer contributors in the right direction. Let me know any > thoughts you have, and if there are other instructions which should be > included in the template. My plan is to draft up a template including any > feedback here sometime this week. > > Thanks, > Cole >
[jira] [Closed] (TINKERPOP-3082) Tinkerpop hardcoded the Spark AppName
[ https://issues.apache.org/jira/browse/TINKERPOP-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu closed TINKERPOP-3082. - Resolution: Fixed > Tinkerpop hardcoded the Spark AppName > - > > Key: TINKERPOP-3082 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3082 > Project: TinkerPop > Issue Type: Improvement >Affects Versions: 3.7.2 >Reporter: Redriver >Priority: Major > Fix For: 4.0.0, 3.7.3 > > Attachments: Screenshot 2024-06-03 at 16.14.17.png > > > When I run gremlin through SparkGraphComputer, the SparkUI always displays > the Spark App as "[Apache TinkerPop's > Spark-Gremlin|http://hdc49-mcc10-01-0510-3405-002-tess0097.stratus.rno.ebay.com:4040/]"; > (see > https://github.com/apache/tinkerpop/blob/master/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/structure/Spark.java#L51) > which confused users if I run multiple gremlin with SparkGraphComputer. > It should allow override the Spark App name if user wants to. If user does > not specify the Spark app name, we can provide the default one. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TINKERPOP-3082) Tinkerpop hardcoded the Spark AppName
[ https://issues.apache.org/jira/browse/TINKERPOP-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu updated TINKERPOP-3082: -- Fix Version/s: 4.0.0 3.7.3 > Tinkerpop hardcoded the Spark AppName > - > > Key: TINKERPOP-3082 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3082 > Project: TinkerPop > Issue Type: Improvement >Affects Versions: 3.7.2 >Reporter: Redriver >Priority: Major > Fix For: 4.0.0, 3.7.3 > > Attachments: Screenshot 2024-06-03 at 16.14.17.png > > > When I run gremlin through SparkGraphComputer, the SparkUI always displays > the Spark App as "[Apache TinkerPop's > Spark-Gremlin|http://hdc49-mcc10-01-0510-3405-002-tess0097.stratus.rno.ebay.com:4040/]"; > (see > https://github.com/apache/tinkerpop/blob/master/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/structure/Spark.java#L51) > which confused users if I run multiple gremlin with SparkGraphComputer. > It should allow override the Spark App name if user wants to. If user does > not specify the Spark app name, we can provide the default one. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] Replace WebSockets with HTTP 1.1
One other thing to note with the removal of Sessions is that there will no longer be any ability to send Groovy scripts that depend on one another. For example: "x = 1 + 1" "y = x + 1" won't work anymore because the x variable won't be saved to a Session's bindings. The impact of this change should be reduced because we will also be switching to gremlin-lang by default which never had this capability. This mostly affects users that want to use gremlin-groovy to send remote commands to the server. On Wed, Apr 3, 2024 at 5:01 PM Ken Hu wrote: > One thing to add to this is that SASL is difficult to support with > HTTP/1.1 so it will likely need to be dropped. This means that only Basic > Auth will be supported with HTTP/1.1. Support for SASL can be added back > when we move to HTTP/2. However, I'm not sure how widely SASL was used. If > there aren't any complaints from users when it is temporarily removed for > HTTP/1.1 then maybe we should just consider dropping support for it > altogether and not adding it back. > > On Mon, Mar 4, 2024 at 3:02 PM Ken Hu wrote: > >> I believe that I've answered most questions about this and there doesn't >> seem to have been any objections. Therefore, I'm going to assume consensus >> has been reached and that we will move forward with HTTP only in the next >> major release. >> >> As part of the work to move to HTTP, I would like to simplify some other >> parts of the server that I believe aren't being used. This will help to >> simplify some of the options and arguments used in the HTTP API. >> Specifically, I would like to remove support for OpProcessors, Sessions, >> and the UnifiedChannelizer. OpProcessor is an interesting concept to allow >> providers to add and modify server functionality but it doesn't seem like >> it was ever used (other than by the default ones provided). Sessions are >> tied to WebSocket connections and the threaded nature of the current >> Transaction API. However, the Transaction API will get reworked and >> WebSockets is being removed which makes the Session concept obsolete. The >> UnifiedChannelizer, I believe, was supposed to simplify Sessions but since >> sessions don't make much sense anymore, it probably doesn't make sense to >> keep the UnifiedChannelizer around either. >> >> Is anyone aware of any implementations that are actively making use of >> these? If not, I think it makes sense to remove or replace them in the next >> major version as well. >> >> On Thu, Feb 29, 2024 at 12:52 PM Ken Hu wrote: >> >>> Thanks for the responses so far. >>> >>> I think we're still open to having multiple protocols as long as there >>> is some form of commitment to maintaining it and there is demand for it >>> within the community. This would mean that we could add grpc but someone >>> would have to be willing to step up to maintain it. Also, I view HTTP/1.1 >>> as a starting point for the transition from WebSockets to HTTP. The >>> features you mentioned that would be beneficial are part of HTTP/2 which, I >>> think, is where we probably want to head longer term. >>> >>> From an end user perspective, there should hopefully be little to no >>> change at all. The underlying protocol used to communicate with the server >>> is mostly transparent to them when they use the GLVs. The only real >>> difference is that they will probably have to adjust some of their >>> connection options which is something users have raised issues with before >>> and this would improve that situation. The users that potentially will get >>> affected the most are those that use a third party driver, they may have to >>> continue using 3.x unless those drivers are updated. For providers, this >>> change shouldn't affect them as long as they use the Gremlin Server (which >>> from what I can tell includes most providers). In the end, I believe this >>> is a change that may cause some minor differences for users, but will >>> mostly help with the continued maintenance of the project. >>> >>> On Wed, Feb 28, 2024 at 4:38 PM Josh Perryman >>> wrote: >>> >>>> I understand the cost/complexity argument. But do we have any sense of >>>> the >>>> value / feature side of things? What's the implementation look like >>>> among >>>> providers? Is it exclusively HTTP or do some implement both, or just >>>> Websockets? From there, what about actual usage among users? Do the >>>> implementers see any p
[DISCUSS] gremlin-shaded issues
Hi All, I was just wondering how everyone felt about gremlin-shaded. I'd be curious to know if you have encountered any issues or pain points with using it. For me, I dislike the fact that there is friction in using it with IntelliJ (it's sometimes not recognized as a library when built). Although it provides a way to solve dependency issues, I feel as though there are other ways to solve this. Does anyone find gremlin-shaded particularly useful? Would anyone be opposed to removing gremlin-shaded? Thanks, Ken
Re: [DISCUSS] TinkerPop 4 HTTP API Design
Another update is that for GraphSONv4, the result field will no longer be
null if there are no results instead it will become an empty array. This
means that the new ResponseResult always expects a List rather than
just an Object.
Are there any concerns about changing it from being null to an empty array?
On Wed, Apr 24, 2024 at 1:30 PM Ken Hu wrote:
> An update regarding the API, the materializeProperties option is being
> added back which should be set to either "all" or "tokens". The "all"
> option returns all the properties while the "tokens" option returns only
> the id and label. So the request syntax currently looks like:
>
> -- Request Syntax --
>
> POST /gremlin HTTP/1.1
> Accept: <>
> Content-Type: <>
> Gremlin-Hints: <>
>
> {
> "gremlin": "string",
> "timeoutMs": number,
> "materializeProperties": "string",
> "bindings": object,
> "g": "string"
> "language" : "string"
> }
>
> Additionally, there seems to be a difference between options needed to
> change server behavior (timeoutMs, materializeProperties) and options
> needed to change the behavior of the Traversal. Currently, we allow these
> server options to be set using the with() options. I think it makes sense
> for these to only be set in the HTTP request body as it isn't information
> needed by the traversal. This leads to a better separation of options that
> affect both all traversals (embedded and remote) and options that are
> server/remote only. This means that we would drop the GremlinScriptChecker
> from the server that is used for parsing through a script for specific
> with() options that the server needs. However, a user may need to set
> specific options per query so we could add a GLV-only step like request().
> The request() step wouldn't be part of the language and would be something
> that only the GLVs would recognize and would cause them to add those
> options to the HTTP request body. It should be noted that request() is a
> working idea, but for now, the most important point is that there will be a
> way to set these server options per request from the GLVs.
>
> On Wed, Apr 10, 2024 at 12:45 PM Ken Hu wrote:
>
>> Hi,
>>
>> The following is a reference of what the upcoming HTTP API will look like
>> for the /gremlin resource. Some smaller items are probably subject to
>> change.
>>
>> There are three differences between the 4.x HTTP API and the 3.x HTTP API
>> that should be noted. First, the server always generates the requestId and
>> returns it to the client. Second, the format of the request body has
>> changed as the OpProcessors have been removed so the request fields have
>> been flattened. Third, because the results are now streamed back to the
>> user with chunked transfer encoding, there are certain errors that can
>> occur after the initial 200 OK has been sent. This means that the user must
>> check either the status object in the response body or the trailers to see
>> if there are any errors.
>>
>> The format below uses a bit of pseudo-JSON to help represent request and
>> response bodies. The actual format of the request and response bodies will
>> be determined by the serializers defined via the "Accept" and
>> "Content-Type" headers. As a result, a generic type definition in this
>> document like "number" could translate to a "long" for a serializer that
>> supports types like GraphBinary.
>>
>>
>> -- Request Syntax --
>>
>> POST /gremlin HTTP/1.1
>> Accept: <>
>> Content-Type: <>
>> Gremlin-Hints: <>
>>
>> {
>> "gremlin": "string",
>> "timeoutMs": number,
>> "bindings": object,
>> "g": "string"
>> "language" : "string"
>> }
>>
>>
>> -- Headers --
>>
>> Gremlin-Hints - When provided this header is a semi-colon separated list
>> of key/value
>> pair metadata that could be helpful to the server in
>> processing a
>> particular request in some way.
>> Required: No
>>
>> -- Header Options --
>>
>> mimetype - the serialization accepted for the response.
>> Required: No. Defaults to
>> "application/vnd.gremlin-v4.0+json;types=false"
>>
>> application/vnd.gremlin-v4.0+json;types=false
>> GraphSON without embedded types
[DISCUSS] Revert GraphManager Notification Changes For HTTP
Hi All, In TINKERPOP-2806 (https://issues.apache.org/jira/browse/TINKERPOP-2806) there were some methods added to GraphManager to allow providers to have more insight into when execution occurs in the server and the outcome of that execution. The interface was based on RequestMessage having a request ID. However, for TinkerPop 4, the new RequestMessageV4 no longer contains a request ID (although one could be added). And, it probably makes sense for this interface to work with the upcoming Transaction API. I've looked around at some open source providers and it doesn't seem like this interface is being used. So for those reasons, I'm suggesting that we revert these changes in TinkerPop 4 (they will remain on 3.x) until we can have a better understanding of what sort of notification system makes sense for providers in TinkerPop 4. We can then add it back at a later time. Does anyone have concerns about reverting these changes? Thanks, Ken
Re: [DISCUSS] TinkerPop 4 HTTP API Design
An update regarding the API, the materializeProperties option is being
added back which should be set to either "all" or "tokens". The "all"
option returns all the properties while the "tokens" option returns only
the id and label. So the request syntax currently looks like:
-- Request Syntax --
POST /gremlin HTTP/1.1
Accept: <>
Content-Type: <>
Gremlin-Hints: <>
{
"gremlin": "string",
"timeoutMs": number,
"materializeProperties": "string",
"bindings": object,
"g": "string"
"language" : "string"
}
Additionally, there seems to be a difference between options needed to
change server behavior (timeoutMs, materializeProperties) and options
needed to change the behavior of the Traversal. Currently, we allow these
server options to be set using the with() options. I think it makes sense
for these to only be set in the HTTP request body as it isn't information
needed by the traversal. This leads to a better separation of options that
affect both all traversals (embedded and remote) and options that are
server/remote only. This means that we would drop the GremlinScriptChecker
from the server that is used for parsing through a script for specific
with() options that the server needs. However, a user may need to set
specific options per query so we could add a GLV-only step like request().
The request() step wouldn't be part of the language and would be something
that only the GLVs would recognize and would cause them to add those
options to the HTTP request body. It should be noted that request() is a
working idea, but for now, the most important point is that there will be a
way to set these server options per request from the GLVs.
On Wed, Apr 10, 2024 at 12:45 PM Ken Hu wrote:
> Hi,
>
> The following is a reference of what the upcoming HTTP API will look like
> for the /gremlin resource. Some smaller items are probably subject to
> change.
>
> There are three differences between the 4.x HTTP API and the 3.x HTTP API
> that should be noted. First, the server always generates the requestId and
> returns it to the client. Second, the format of the request body has
> changed as the OpProcessors have been removed so the request fields have
> been flattened. Third, because the results are now streamed back to the
> user with chunked transfer encoding, there are certain errors that can
> occur after the initial 200 OK has been sent. This means that the user must
> check either the status object in the response body or the trailers to see
> if there are any errors.
>
> The format below uses a bit of pseudo-JSON to help represent request and
> response bodies. The actual format of the request and response bodies will
> be determined by the serializers defined via the "Accept" and
> "Content-Type" headers. As a result, a generic type definition in this
> document like "number" could translate to a "long" for a serializer that
> supports types like GraphBinary.
>
>
> -- Request Syntax --
>
> POST /gremlin HTTP/1.1
> Accept: <>
> Content-Type: <>
> Gremlin-Hints: <>
>
> {
> "gremlin": "string",
> "timeoutMs": number,
> "bindings": object,
> "g": "string"
> "language" : "string"
> }
>
>
> -- Headers --
>
> Gremlin-Hints - When provided this header is a semi-colon separated list
> of key/value
> pair metadata that could be helpful to the server in
> processing a
> particular request in some way.
> Required: No
>
> -- Header Options --
>
> mimetype - the serialization accepted for the response.
> Required: No. Defaults to
> "application/vnd.gremlin-v4.0+json;types=false"
>
> application/vnd.gremlin-v4.0+json;types=false
> GraphSON without embedded types that will be easy to parse and work
> with using
> cURL and similar tools.
>
> application/vnd.gremlin-v4.0+json;types=true
> GraphSON with embedded types, mostly present for consistency, so
> that both
> configurations of GraphSON are available. Note that official
> TinkerPop drivers
> will not support this serializer.
>
> application/vnd.graphbinary-v4.0
> GraphBinary which has embedded types, mostly of use to Gremlin
> Language Drivers.
>
> hints
> mutations - Indicates if the Gremlin contains steps that can mutate
> the graph. The
> default would be "unknown". Language drivers would set
> this hint
> automatically at qu
[DISCUSS] TinkerPop 4 HTTP API Design
Hi,
The following is a reference of what the upcoming HTTP API will look like
for the /gremlin resource. Some smaller items are probably subject to
change.
There are three differences between the 4.x HTTP API and the 3.x HTTP API
that should be noted. First, the server always generates the requestId and
returns it to the client. Second, the format of the request body has
changed as the OpProcessors have been removed so the request fields have
been flattened. Third, because the results are now streamed back to the
user with chunked transfer encoding, there are certain errors that can
occur after the initial 200 OK has been sent. This means that the user must
check either the status object in the response body or the trailers to see
if there are any errors.
The format below uses a bit of pseudo-JSON to help represent request and
response bodies. The actual format of the request and response bodies will
be determined by the serializers defined via the "Accept" and
"Content-Type" headers. As a result, a generic type definition in this
document like "number" could translate to a "long" for a serializer that
supports types like GraphBinary.
-- Request Syntax --
POST /gremlin HTTP/1.1
Accept: <>
Content-Type: <>
Gremlin-Hints: <>
{
"gremlin": "string",
"timeoutMs": number,
"bindings": object,
"g": "string"
"language" : "string"
}
-- Headers --
Gremlin-Hints - When provided this header is a semi-colon separated list of
key/value
pair metadata that could be helpful to the server in
processing a
particular request in some way.
Required: No
-- Header Options --
mimetype - the serialization accepted for the response.
Required: No. Defaults to
"application/vnd.gremlin-v4.0+json;types=false"
application/vnd.gremlin-v4.0+json;types=false
GraphSON without embedded types that will be easy to parse and work
with using
cURL and similar tools.
application/vnd.gremlin-v4.0+json;types=true
GraphSON with embedded types, mostly present for consistency, so that
both
configurations of GraphSON are available. Note that official
TinkerPop drivers
will not support this serializer.
application/vnd.graphbinary-v4.0
GraphBinary which has embedded types, mostly of use to Gremlin
Language Drivers.
hints
mutations - Indicates if the Gremlin contains steps that can mutate the
graph. The
default would be "unknown". Language drivers would set this
hint
automatically at query submission by dynamically detecting
mutating
steps in the query being sent.
yes | no | unknown
-- Request Body --
gremlin - The Gremlin query to execute.
Type: String
Required: Yes.
timeoutMs - The maximum time a query is allowed to execute in milliseconds.
Type: Number
Required: No. If not present, then server default is used.
bindings - Any bindings used to execute the query. For example, the
incoming Gremlin script might
be g.V(x) where x=1 is a binding that would be applied to it.
Type: object
Required: No.
g - The name of the graph traversal source to which the query applies.
Type: String
Required: No, defaults to "g"
language - The name of the ScriptEngine to use to parse the gremlin query.
Type: String
Required: No, defaults to "gremlin-lang"
-- Response Syntax --
HTTP/1.1 200
Content-type: <>
Transfer-Encoding: chunked
Gremlin-RequestId: uuid
{
"result": list,
"status": object
}
-- Headers --
Gremlin-RequestId - A UUID to identify this request, which can be used to
help trace the request on the server in logs and such.
Type: uuid
-- Response Elements --
result - A list of objects that represent the result of the query. Single
objects are
coerced to list.
Type: list
status - The actual status of the request after its been fully executed.
Type: object
-- Example Responses --
An example "result", in this case a single graph element, looks like the
following:
// g.V().limit(1)
[
{
"id": "string"
"label": "string"
"type": "string"
"properties": object
}
]
An example "status", which comes at the very end of the response looks like
the following:
{
code: "string"
message: "string"
exception: "string"
}
A concrete example would be:
{
"code": "400"
"message": "An eval requires a gremlin argument"
"exception": "InvalidRequestException"
}
The trailers will have the following format:
Status: "number"
Exception: "string"
The status corresponds to the real status code and the exception is an
exception class returned by the server. For example,
Status: 429
Exception: TooManyRequestsException
Are there any modifications or updates that you think would be appropriate?
Thanks,
Ken
[DISCUSS] Only Support User Agent in Request Header
Hi All, I just wanted to get your thoughts regarding the support of user agent. There are multiple ways to set a user agent with the most recent one being in the header of the HTTP request ( https://tinkerpop.apache.org/docs/current/dev/provider/#_graph_driver_provider_requirements). There is an older user agent that is sent in the body of the request ( https://github.com/apache/tinkerpop/blob/master/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Client.java#L365). The former is tracked in the metrics of the server, but the latter seems to be an optional field that providers can use in custom OpProcessors. Are there any providers that make use of the old "userAgent" arg in the request message? I think it makes sense to support only the header version (for both WebSockets and HTTP) and deprecate/remove the one that is sent as an argument in the request body. They seem to serve the same purpose and having the user agent in the header is standard in more APIs. Thanks, Ken
[DISCUSS] Remove Support for Bytecode in TinkerPop 4 and Switch to Gremlin-Lang
Hi All, As we move towards the next major version of TinkerPop, it may be time for us to start thinking about removing support for bytecode and only allowing script execution. One of the purposes that bytecode served was to provide a universal way to translate a Traversal. However, with the introduction of the gremlin-lang parser this need can be fulfilled differently. Any Gremlin script can be converted into a Traversal in a uniform way which reduces the need for bytecode. Now, we are left with two systems that serve a similar purpose and it is probably time to remove one of them during a major version upgrade. One issue with bytecode is the complexity needed in the JavaTranslator to translate bytecode into a traversal. There are lots of nested loops and lots of reflection needed to hold the system together. The parser, on the hand, is a simpler system, however, it lacks some features that will need to be implemented. For example, there are some types such as UUID, Set, Edge and ByteBuffer that are missing from the gremlin-lang grammar that need to be added. To clarify, the GLVs will still allow for creating a Traversal in that particular language, however, it will turn that Traversal into a script rather than bytecode to send to the server. The server will then use one of the ScriptEngines to process the query. If we can agree towards moving to script only queries then the next step is to decide which should be the primary ScriptEngine to execute the queries. I believe promoting gremlin-lang to be the default ScriptEngine instead of gremlin-groovy is the better choice. While gremlin-groovy is extremely flexible since it can execute any arbitrary Groovy code, that flexibility comes at the cost of security as anything can be executed. Having such a flexible system also makes it difficult to move towards supporting certain features in standard steps as users will sometimes write partial implementations with their own lambdas. Are there any thoughts about removing bytecode and switching to gremlin-lang? Thanks, Ken
Re: [DISCUSS] Replace WebSockets with HTTP 1.1
One thing to add to this is that SASL is difficult to support with HTTP/1.1 so it will likely need to be dropped. This means that only Basic Auth will be supported with HTTP/1.1. Support for SASL can be added back when we move to HTTP/2. However, I'm not sure how widely SASL was used. If there aren't any complaints from users when it is temporarily removed for HTTP/1.1 then maybe we should just consider dropping support for it altogether and not adding it back. On Mon, Mar 4, 2024 at 3:02 PM Ken Hu wrote: > I believe that I've answered most questions about this and there doesn't > seem to have been any objections. Therefore, I'm going to assume consensus > has been reached and that we will move forward with HTTP only in the next > major release. > > As part of the work to move to HTTP, I would like to simplify some other > parts of the server that I believe aren't being used. This will help to > simplify some of the options and arguments used in the HTTP API. > Specifically, I would like to remove support for OpProcessors, Sessions, > and the UnifiedChannelizer. OpProcessor is an interesting concept to allow > providers to add and modify server functionality but it doesn't seem like > it was ever used (other than by the default ones provided). Sessions are > tied to WebSocket connections and the threaded nature of the current > Transaction API. However, the Transaction API will get reworked and > WebSockets is being removed which makes the Session concept obsolete. The > UnifiedChannelizer, I believe, was supposed to simplify Sessions but since > sessions don't make much sense anymore, it probably doesn't make sense to > keep the UnifiedChannelizer around either. > > Is anyone aware of any implementations that are actively making use of > these? If not, I think it makes sense to remove or replace them in the next > major version as well. > > On Thu, Feb 29, 2024 at 12:52 PM Ken Hu wrote: > >> Thanks for the responses so far. >> >> I think we're still open to having multiple protocols as long as there is >> some form of commitment to maintaining it and there is demand for it within >> the community. This would mean that we could add grpc but someone would >> have to be willing to step up to maintain it. Also, I view HTTP/1.1 as a >> starting point for the transition from WebSockets to HTTP. The features you >> mentioned that would be beneficial are part of HTTP/2 which, I think, is >> where we probably want to head longer term. >> >> From an end user perspective, there should hopefully be little to no >> change at all. The underlying protocol used to communicate with the server >> is mostly transparent to them when they use the GLVs. The only real >> difference is that they will probably have to adjust some of their >> connection options which is something users have raised issues with before >> and this would improve that situation. The users that potentially will get >> affected the most are those that use a third party driver, they may have to >> continue using 3.x unless those drivers are updated. For providers, this >> change shouldn't affect them as long as they use the Gremlin Server (which >> from what I can tell includes most providers). In the end, I believe this >> is a change that may cause some minor differences for users, but will >> mostly help with the continued maintenance of the project. >> >> On Wed, Feb 28, 2024 at 4:38 PM Josh Perryman >> wrote: >> >>> I understand the cost/complexity argument. But do we have any sense of >>> the >>> value / feature side of things? What's the implementation look like among >>> providers? Is it exclusively HTTP or do some implement both, or just >>> Websockets? From there, what about actual usage among users? Do the >>> implementers see any preference on that side? >>> >>> I think this a reasonable ask, but the way it is presented is heavily >>> biased toward developer preferences. Do we have any information on the >>> user >>> side? >>> >>> -Josh >>> >>> On Tue, Feb 27, 2024 at 4:29 PM Ken Hu wrote: >>> >>> > Hi All, >>> > >>> > I would like to start the discussion of making HTTP the only protocol >>> that >>> > Apache TinkerPop supports starting with the next major version. There >>> are >>> > several reasons I'm suggesting this. First, we already support both >>> > WebSockets and HTTP in Gremlin Server which causes a maintenance >>> burden of >>> > having to support both. Also, I ran some tests and
[jira] [Created] (TINKERPOP-3065) Implement a new HTTP API
Ken Hu created TINKERPOP-3065: - Summary: Implement a new HTTP API Key: TINKERPOP-3065 URL: https://issues.apache.org/jira/browse/TINKERPOP-3065 Project: TinkerPop Issue Type: Improvement Components: server Affects Versions: 4.0.0 Reporter: Ken Hu Replace the current WebSocket protocol with an HTTP one in version 4. This will help ease the maintenance of the server long term. Tasks includes: * Add streaming/chunking support * Create a new HTTP API that has support for more functionality than the current HTTPChannelizer * Rework the Transaction API alongside the new HTTP API * Remove underutilized features in the server such as OpProcessor and the UnifiedChannelizer. See [https://lists.apache.org/thread/vfs1j9ycb8voxwc00gdzfmlg2gghx3n1] for more details. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] Next set of maintenance releases - 3.6.7/3.7.2
Hi Yang, This sounds good to me. It matches our recent release cadence so I'm guessing that it's what users will expect as well. Regards, Ken On Tue, Mar 5, 2024 at 6:08 PM Yang Xia wrote: > Hi all, > > It has been a few months since our last release, and we've seen a fair > amount of bug fixes landing during this period[1], as well as new > contributions in review [2]. > > As we head into March, I think it's a good time to start thinking > about releasing these changes with the next set of maintenance releases for > TinkerPop 3.6.7 and 3.7.2. > > I'd like to suggest for us to target code freeze at the end of March, i.e. > the week of March 25, with a release in early April. > > Any thoughts on this? > > Cheers, > > Yang > > [1] > > https://github.com/apache/tinkerpop/blob/3.6-dev/CHANGELOG.asciidoc#tinkerpop-367-not-officially-released-yet > [2] https://github.com/apache/tinkerpop/pull/2506 >
Driver and Server Compatibility Testing
Hi All, Currently, it is difficult for a user to tell if upgrading their graph database to a newer version will cause their application to break. We don't know which versions of the driver will work with which versions of TinkerGraph running in the Gremlin Server. This question gets even more complicated when you need to consider other providers. This sort of driver and server compatibility testing is missing from Apache TinkerPop. To solve this problem, I've been working on a system that will allow testing for these scenarios. A very early version of it can be seen here ( https://github.com/apache/tinkerpop/commit/c1e1bfdbce09cab08d08a53a53886a2d645f4274). Right now, it is only capable of running the non-Java GLVs feature tests against different versions of TinkerGraph. However, the vision for this system in the future would be for it to be able to run a customizable set of feature tests, on any GLV, against any provider. I'm curious if there are any providers that have experienced similar questions from their users about which versions of the GLVs they can use against different versions of their server. Thanks, Ken
Re: [DISCUSS] Replace WebSockets with HTTP 1.1
I believe that I've answered most questions about this and there doesn't seem to have been any objections. Therefore, I'm going to assume consensus has been reached and that we will move forward with HTTP only in the next major release. As part of the work to move to HTTP, I would like to simplify some other parts of the server that I believe aren't being used. This will help to simplify some of the options and arguments used in the HTTP API. Specifically, I would like to remove support for OpProcessors, Sessions, and the UnifiedChannelizer. OpProcessor is an interesting concept to allow providers to add and modify server functionality but it doesn't seem like it was ever used (other than by the default ones provided). Sessions are tied to WebSocket connections and the threaded nature of the current Transaction API. However, the Transaction API will get reworked and WebSockets is being removed which makes the Session concept obsolete. The UnifiedChannelizer, I believe, was supposed to simplify Sessions but since sessions don't make much sense anymore, it probably doesn't make sense to keep the UnifiedChannelizer around either. Is anyone aware of any implementations that are actively making use of these? If not, I think it makes sense to remove or replace them in the next major version as well. On Thu, Feb 29, 2024 at 12:52 PM Ken Hu wrote: > Thanks for the responses so far. > > I think we're still open to having multiple protocols as long as there is > some form of commitment to maintaining it and there is demand for it within > the community. This would mean that we could add grpc but someone would > have to be willing to step up to maintain it. Also, I view HTTP/1.1 as a > starting point for the transition from WebSockets to HTTP. The features you > mentioned that would be beneficial are part of HTTP/2 which, I think, is > where we probably want to head longer term. > > From an end user perspective, there should hopefully be little to no > change at all. The underlying protocol used to communicate with the server > is mostly transparent to them when they use the GLVs. The only real > difference is that they will probably have to adjust some of their > connection options which is something users have raised issues with before > and this would improve that situation. The users that potentially will get > affected the most are those that use a third party driver, they may have to > continue using 3.x unless those drivers are updated. For providers, this > change shouldn't affect them as long as they use the Gremlin Server (which > from what I can tell includes most providers). In the end, I believe this > is a change that may cause some minor differences for users, but will > mostly help with the continued maintenance of the project. > > On Wed, Feb 28, 2024 at 4:38 PM Josh Perryman > wrote: > >> I understand the cost/complexity argument. But do we have any sense of the >> value / feature side of things? What's the implementation look like among >> providers? Is it exclusively HTTP or do some implement both, or just >> Websockets? From there, what about actual usage among users? Do the >> implementers see any preference on that side? >> >> I think this a reasonable ask, but the way it is presented is heavily >> biased toward developer preferences. Do we have any information on the >> user >> side? >> >> -Josh >> >> On Tue, Feb 27, 2024 at 4:29 PM Ken Hu wrote: >> >> > Hi All, >> > >> > I would like to start the discussion of making HTTP the only protocol >> that >> > Apache TinkerPop supports starting with the next major version. There >> are >> > several reasons I'm suggesting this. First, we already support both >> > WebSockets and HTTP in Gremlin Server which causes a maintenance burden >> of >> > having to support both. Also, I ran some tests and a streaming version >> of >> > HTTP using chunked transfer encoding performs just as well as streaming >> via >> > WebSockets so there won't be a performance loss in terms of latency. >> > Second, HTTP is significantly more popular so it will be easier for >> other >> > third party clients to connect to. Third, this is an opportunity to >> rework >> > the remote Transaction API which in its current form causes unnecessary >> > complexity in the server. This will also serve as a good opportunity to >> > make the connection options more similar among all the GLVs. >> > >> > I've experimented with replacing WebSockets with HTTP (both 1.1 and 2) >> and >> > noticed similar performance in most cases. For HTTP/1.1, there was some >
Re: [DISCUSS] Replace WebSockets with HTTP 1.1
Thanks for the responses so far. I think we're still open to having multiple protocols as long as there is some form of commitment to maintaining it and there is demand for it within the community. This would mean that we could add grpc but someone would have to be willing to step up to maintain it. Also, I view HTTP/1.1 as a starting point for the transition from WebSockets to HTTP. The features you mentioned that would be beneficial are part of HTTP/2 which, I think, is where we probably want to head longer term. >From an end user perspective, there should hopefully be little to no change at all. The underlying protocol used to communicate with the server is mostly transparent to them when they use the GLVs. The only real difference is that they will probably have to adjust some of their connection options which is something users have raised issues with before and this would improve that situation. The users that potentially will get affected the most are those that use a third party driver, they may have to continue using 3.x unless those drivers are updated. For providers, this change shouldn't affect them as long as they use the Gremlin Server (which from what I can tell includes most providers). In the end, I believe this is a change that may cause some minor differences for users, but will mostly help with the continued maintenance of the project. On Wed, Feb 28, 2024 at 4:38 PM Josh Perryman wrote: > I understand the cost/complexity argument. But do we have any sense of the > value / feature side of things? What's the implementation look like among > providers? Is it exclusively HTTP or do some implement both, or just > Websockets? From there, what about actual usage among users? Do the > implementers see any preference on that side? > > I think this a reasonable ask, but the way it is presented is heavily > biased toward developer preferences. Do we have any information on the user > side? > > -Josh > > On Tue, Feb 27, 2024 at 4:29 PM Ken Hu wrote: > > > Hi All, > > > > I would like to start the discussion of making HTTP the only protocol > that > > Apache TinkerPop supports starting with the next major version. There are > > several reasons I'm suggesting this. First, we already support both > > WebSockets and HTTP in Gremlin Server which causes a maintenance burden > of > > having to support both. Also, I ran some tests and a streaming version of > > HTTP using chunked transfer encoding performs just as well as streaming > via > > WebSockets so there won't be a performance loss in terms of latency. > > Second, HTTP is significantly more popular so it will be easier for other > > third party clients to connect to. Third, this is an opportunity to > rework > > the remote Transaction API which in its current form causes unnecessary > > complexity in the server. This will also serve as a good opportunity to > > make the connection options more similar among all the GLVs. > > > > I've experimented with replacing WebSockets with HTTP (both 1.1 and 2) > and > > noticed similar performance in most cases. For HTTP/1.1, there was some > > slowdown in cases where thousands of queries with few results were issued > > per second as lots of HTTP connections needed to be opened, but I feel > like > > this isn't a practical use case. My recommendation for now is to go with > > HTTP/1.1 as it has all the features we need and it has mostly been > > implemented already on the server side. > > > > To ease the transition, I'm suggesting that we maintain WebSockets for > the > > foreseeable future in the 3.x line while the 4.x line is moved to use > HTTP > > exclusively. This will also give more time for us to build out support > for > > transactions over HTTP which is the only missing feature that is > supported > > when using WebSockets. > > > > I'd like to hear any thoughts regarding this. > > > > Thanks, > > Ken > > >
[DISCUSS] Replace WebSockets with HTTP 1.1
Hi All, I would like to start the discussion of making HTTP the only protocol that Apache TinkerPop supports starting with the next major version. There are several reasons I'm suggesting this. First, we already support both WebSockets and HTTP in Gremlin Server which causes a maintenance burden of having to support both. Also, I ran some tests and a streaming version of HTTP using chunked transfer encoding performs just as well as streaming via WebSockets so there won't be a performance loss in terms of latency. Second, HTTP is significantly more popular so it will be easier for other third party clients to connect to. Third, this is an opportunity to rework the remote Transaction API which in its current form causes unnecessary complexity in the server. This will also serve as a good opportunity to make the connection options more similar among all the GLVs. I've experimented with replacing WebSockets with HTTP (both 1.1 and 2) and noticed similar performance in most cases. For HTTP/1.1, there was some slowdown in cases where thousands of queries with few results were issued per second as lots of HTTP connections needed to be opened, but I feel like this isn't a practical use case. My recommendation for now is to go with HTTP/1.1 as it has all the features we need and it has mostly been implemented already on the server side. To ease the transition, I'm suggesting that we maintain WebSockets for the foreseeable future in the 3.x line while the 4.x line is moved to use HTTP exclusively. This will also give more time for us to build out support for transactions over HTTP which is the only missing feature that is supported when using WebSockets. I'd like to hear any thoughts regarding this. Thanks, Ken
[jira] [Created] (TINKERPOP-3040) Remote Console won't send queries that cause exceptions locally
Ken Hu created TINKERPOP-3040:
-
Summary: Remote Console won't send queries that cause exceptions
locally
Key: TINKERPOP-3040
URL: https://issues.apache.org/jira/browse/TINKERPOP-3040
Project: TinkerPop
Issue Type: Bug
Components: console
Affects Versions: 3.7.0
Reporter: Ken Hu
From: https://groups.google.com/g/gremlin-users/c/U2gIxmRcrI8/m/x12banhdAAAJ
It appears that if an exception occurs when the remote console is executing a
command locally then it won't send that query to the server.
Example:
{code:java}
\,,,/
(o o)
-oOOo-(3)-oOOo-
plugin activated: tinkerpop.server
plugin activated: tinkerpop.utilities
plugin activated: tinkerpop.tinkergraph
plugin activated: janusgraph.imports
gremlin> :remote connect tinkerpop.server conf/remote.yaml
==>Configured localhost/127.0.0.1:45940
gremlin> :remote console
==>All scripts will now be sent to Gremlin Server - [localhost/127.0.0.1:45940]
- type ':remote console' to return to local mode
gremlin> JanusGraphFactory.getGraphNames()
Gremlin Server must be configured to use the JanusGraphManager.
Type ':help' or ':h' for help.
Display stack trace? [yN]y
java.lang.NullPointerException: Gremlin Server must be configured to use the
JanusGraphManager.
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:921)
at
org.janusgraph.core.JanusGraphFactory.getGraphNames(JanusGraphFactory.java:187)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at groovysh_evaluate.run(groovysh_evaluate:3)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.selectMethod(IndyInterface.java:355)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at
org.apache.groovy.groovysh.Interpreter.evaluate(Interpreter.groovy:82)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at
org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:118)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at org.apache.groovy.groovysh.Shell.leftShift(Shell.groovy:121)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at org.apache.groovy.groovysh.ShellRunner.work(ShellRunner.groovy:93)
at
org.apache.groovy.groovysh.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at org.apache.groovy.groovysh.ShellRunner.run(ShellRunner.groovy:57)
at
org.apache.groovy.groovysh.InteractiveShellRunner.run(InteractiveShellRunner.groovy:96)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at
org.apache.tinkerpop.gremlin.console.Console.(Console.groovy:175)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:574)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (TINKERPOP-3039) Java driver won't propagate with args when using aliased client directly
Ken Hu created TINKERPOP-3039:
-
Summary: Java driver won't propagate with args when using aliased
client directly
Key: TINKERPOP-3039
URL: https://issues.apache.org/jira/browse/TINKERPOP-3039
Project: TinkerPop
Issue Type: Bug
Components: driver
Affects Versions: 3.7.1, 3.6.6
Reporter: Ken Hu
The Java driver is unable to pass on options defined in the query via a .with()
configuration step when attempting to submit with the AliasClusteredClient's
submit() method.
Example:
{code:java}
Cluster cluster = Cluster.build().create();
Client client = cluster.connect().alias("ggrateful");
GraphTraversalSource g =
traversal().withRemote(DriverRemoteConnection.using(client, "ggrateful"));
GraphTraversal traversal = g.with("evaluationTimeout",
1).V().both().both().both();
try {
List results = client.submit(traversal).all().get();
} catch (Exception e) {
e.printStackTrace();
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (TINKERPOP-3038) Console plugins file can't accept empty lines
Ken Hu created TINKERPOP-3038: - Summary: Console plugins file can't accept empty lines Key: TINKERPOP-3038 URL: https://issues.apache.org/jira/browse/TINKERPOP-3038 Project: TinkerPop Issue Type: Bug Components: console Affects Versions: 3.7.1, 3.6.6 Reporter: Ken Hu As reported in https://groups.google.com/g/gremlin-users/c/U2gIxmRcrI8/m/WzX3eyhTAAAJ An empty line in the plugins.txt file will cause a NPE to occur. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] Support for using TinkerPop as a Java Module
Thanks for the response Pieter. I'll try to direct any users that are asking about JPMS to the aforementioned jira. On Mon, Dec 11, 2023 at 9:32 PM pieter wrote: > Hi, > > Yes thanks, that is reasonable. > I'll keep an eye on the jira issue. > > Thanks > Pieter > > On Mon, 2023-12-11 at 20:14 -0800, Ken Hu wrote: > > Thanks for your input Cole. > > > > My position on this hasn't changed much. I recognize that the ask to > > rename > > a package is a fairly small one, but I want to highlight that recent > > changes to package names have caused issues and I've seen reports of > > several applications breaking. It's hard to predict the kinds of > > issues > > this package renaming will cause. But, from recent experience, we can > > tell > > that there most likely will be problems. My preference is still to > > state > > that we don't provide any support for JPMS in the documentation, > > unless > > there is considerable evidence to prove that a reasonable number of > > users > > would actually adopt JPMS if it were possible to use TinkerPop as an > > automatic module. > > > > Since it isn't possible to make this breaking change until the next > > major > > version which is likely a while off, would the two of you agree to > > put this > > decision on hold until we are closer to its release? In the meantime, > > we > > can try to see how many users/providers would actually be interested > > in > > this change. We can monitor the activity on the associated Jira ( > > https://issues.apache.org/jira/browse/TINKERPOP-3019) for comments or > > watchers. I'd be willing to change my mind if I saw multiple members > > of the > > community come forth and ask for this. > > > > Do you find this to be a reasonable way to move forward on this? > > > > On Mon, Dec 11, 2023 at 5:24 PM Cole Greer > > > > wrote: > > > > > Hi Pieter, > > > > > > My thoughts on this matter have shifted back and forth a fair bit > > > over the > > > last few days but I wanted to weigh in on the discussion. As far as > > > I > > > understand, JPMS adoption remains very limited in the Java > > > community which > > > suggests that fixing the split-package issue will only benefit very > > > few > > > users. However, one of the main reasons for the slow adoption of > > > JPMS is > > > that many projects are stuck with dependencies which are not > > > modularized. > > > There is a bit of a chicken and egg problem limiting JPMS adoption > > > and I > > > wouldn’t want TinkerPop to perpetuate this unnecessarily. > > > > > > As Ken mentioned, any package renaming would be a breaking change > > > and > > > would need to wait until our next major release. Personally, I > > > don’t mind > > > the inconvenience of a package renaming during a major version > > > bump, > > > however I also have no plans to use JPMS in any projects in the > > > near > > > future. I would be curious to hear from others how they feel about > > > the > > > impact of the package change, as well as if any other projects are > > > looking > > > to utilize TinkerPop in a modularized app. > > > > > > Overall, I am generally in favour of your renaming solution > > > (perhaps > > > renaming from gremlin- > > > core/org.apache.tinkerpop.gremlin.language.grammar to > > > gremlin-core/org.apache.tinkerpop.gremlin.core.grammar), however I > > > may be > > > swayed against it if others begin to raise concerns regarding the > > > impact of > > > the change. Additionally, if we do proceed with the package > > > renaming, I > > > would view this as a one-off fix to remove a barrier for some users > > > and I > > > would be weary to claim we officially support (and will continue to > > > support) being used as automatic modules. I agree with Ken that > > > broad plan > > > to fully adopt and support JPMS is a pre-requisite for such a > > > stance. > > > > > > Regards, > > > > > > Cole > > > > > > From: pieter > > > Date: Friday, December 8, 2023 at 12:58 AM > > > To: dev@tinkerpop.apache.org > > > Subject: Re: [DISCUSS] Support for using TinkerPop as a Java Module > > > Hi, > > > > > > For a client to use TinkerPop only 2 modules are requir
Re: [DISCUSS] Support for using TinkerPop as a Java Module
Thanks for your input Cole. My position on this hasn't changed much. I recognize that the ask to rename a package is a fairly small one, but I want to highlight that recent changes to package names have caused issues and I've seen reports of several applications breaking. It's hard to predict the kinds of issues this package renaming will cause. But, from recent experience, we can tell that there most likely will be problems. My preference is still to state that we don't provide any support for JPMS in the documentation, unless there is considerable evidence to prove that a reasonable number of users would actually adopt JPMS if it were possible to use TinkerPop as an automatic module. Since it isn't possible to make this breaking change until the next major version which is likely a while off, would the two of you agree to put this decision on hold until we are closer to its release? In the meantime, we can try to see how many users/providers would actually be interested in this change. We can monitor the activity on the associated Jira ( https://issues.apache.org/jira/browse/TINKERPOP-3019) for comments or watchers. I'd be willing to change my mind if I saw multiple members of the community come forth and ask for this. Do you find this to be a reasonable way to move forward on this? On Mon, Dec 11, 2023 at 5:24 PM Cole Greer wrote: > Hi Pieter, > > My thoughts on this matter have shifted back and forth a fair bit over the > last few days but I wanted to weigh in on the discussion. As far as I > understand, JPMS adoption remains very limited in the Java community which > suggests that fixing the split-package issue will only benefit very few > users. However, one of the main reasons for the slow adoption of JPMS is > that many projects are stuck with dependencies which are not modularized. > There is a bit of a chicken and egg problem limiting JPMS adoption and I > wouldn’t want TinkerPop to perpetuate this unnecessarily. > > As Ken mentioned, any package renaming would be a breaking change and > would need to wait until our next major release. Personally, I don’t mind > the inconvenience of a package renaming during a major version bump, > however I also have no plans to use JPMS in any projects in the near > future. I would be curious to hear from others how they feel about the > impact of the package change, as well as if any other projects are looking > to utilize TinkerPop in a modularized app. > > Overall, I am generally in favour of your renaming solution (perhaps > renaming from gremlin-core/org.apache.tinkerpop.gremlin.language.grammar to > gremlin-core/org.apache.tinkerpop.gremlin.core.grammar), however I may be > swayed against it if others begin to raise concerns regarding the impact of > the change. Additionally, if we do proceed with the package renaming, I > would view this as a one-off fix to remove a barrier for some users and I > would be weary to claim we officially support (and will continue to > support) being used as automatic modules. I agree with Ken that broad plan > to fully adopt and support JPMS is a pre-requisite for such a stance. > > Regards, > > Cole > > From: pieter > Date: Friday, December 8, 2023 at 12:58 AM > To: dev@tinkerpop.apache.org > Subject: Re: [DISCUSS] Support for using TinkerPop as a Java Module > Hi, > > For a client to use TinkerPop only 2 modules are required. > > gremlin-language > gremlin-core > > I am of the opinion that these 2 modules should actually be its own > dedicated project, but that is for another day. > > Regarding these two modules, there is only one split package that > requires refactoring. > > org.apache.tinkerpop.gremlin.language.grammar > > I locally renamed > > gremlin-core/org.apache.tinkerpop.gremlin.language.grammar > > and was able two successfully build the client application with the > following module-info.java > > module-info.java > > requires gremlin.core; > requires gremlin.language; > > This now gives the client manyoptions including building native images. > > So in short, rename one package and I for one will be a happy JPMS > client. > > Thanks > Pieter > > > > > On Wed, 2023-12-06 at 11:56 -0800, Ken Hu wrote: > > Hi All, > > > > Recently, there has been some discussion around the use of > > TinkerPop's Java > > libraries as a Java Module. My proposal is to not support this for > > now. The > > reason I feel this way is due to the fact that Java 8 will be > > supported for > > the foreseeable future. There are many dependencies of TinkerPop that > > don't > > support JPMS and would become automatic modules. I'm generally not in > > favor > > of this approach s
Re: [DISCUSS] Start of Code Freeze
Hi Yang, I would like to join the Contributorcast to discuss list functions. Thanks, Ken On Tue, Dec 5, 2023 at 12:24 PM Yang Xia wrote: > Hello everyone, > > The code freeze is now lifted with TinkerPop 3.5.8, 3.6.6, and 3.7.1 > releases. > > Both 3.6-dev and 3.7-dev are now back in SNAPSHOT mode and development can > resume on those branches. Please note that 3.5.8 was the last release of > the 3.5.x line and the branch will not be opened for development. > > We are also looking to host another Contributorcast Twitch stream on these > new releases next Thursday, 11 am PDT (UTC-7). Again, this will be a casual > conversation around all the new features and updates that were included. If > you have contributed to TinkerPop 3.5.8/3.6.6/3.7.2, and you would like to > join us on the stream to chat about your contribution or anything around > these releases, feel free to reply and we'll get in contact with you! > > Cheers, > > Yang > > > On Tue, Nov 28, 2023 at 2:52 PM Yang Xia wrote: > > > Hi everyone, > > > > Just a quick follow up on the release process. We currently have 2 > binding > > votes, and are waiting for a final one for the go ahead. A couple of > folks > > who usually vote on the releases are away on vacation, so we are keeping > > the VOTE threads open until people become available to review everything > > and give the final vote. > > > > Cheers, > > > > Yang > > > > On Sat, Nov 11, 2023 at 12:29 PM Ken Hu > > wrote: > > > >> Hi Cole, > >> > >> I'd like to mention that I'm working on a PR for TINKERPOP-3013 that I > >> hope > >> will be landing soon. I'd like that fix to go into the 3.7.1 release. > >> > >> Thanks, > >> Ken > >> > >> > >> On Fri, Nov 10, 2023 at 4:29 PM Cole Greer >> .invalid> > >> wrote: > >> > >> > Hi everyone, > >> > > >> > As we approach the end of the year, I think it’s time that we get > >> started > >> > on the next set of minor releases for TinkerPop. This would include > >> 3.5.8, > >> > 3.6.6, and 3.7.1. > >> > > >> > There are a couple of open PR’s at the moment which appear to be ready > >> to > >> > merge in the next few days: > >> > > >> > > >> > * Bytecode support for HTTP requests: > >> > https://github.com/apache/tinkerpop/pull/2336 > >> > * TINKERPOP-2988 Add requestId checking to Javascript and Python > >> > drivers: https://github.com/apache/tinkerpop/pull/2329 > >> > * TINKERPOP-2830 Handle User-Agent from HTTP Requests to server: > >> > https://github.com/apache/tinkerpop/pull/2328 > >> > * TINKERPOP-2984 Replace Moq with NSubstitute: > >> > https://github.com/apache/tinkerpop/pull/2319 > >> > > >> > Additionally, there are several smaller PR’s regarding configuration > and > >> > dependency tweaks which I believe can be reviewed and voted on in the > >> next > >> > few days as well. > >> > > >> > I propose that we initiate code freeze on 3.5-dev, 3.6-dev, and > 3.7-dev > >> > today, with the intent that the PR’s mentioned above, as well as other > >> > small non-breaking fixes can still be merged in the next few days. > With > >> > this timing, the VOTE threads could be expected starting November 20, > >> and > >> > the releases would be expected within the first week of December. If > >> there > >> > are any objections to this timeline, or if there are any other desired > >> > changes before the release, please reply to this thread. > >> > > >> > I would additionally like to propose that we plan this to be the final > >> > release in the 3.5.x line, and that we leave 3.5-dev frozen > >> indefinitely. > >> > This could of course be reversed in the future if a critical need > >> arises, > >> > but unless there are any objections, I believe it’s best that we > return > >> to > >> > actively working on no more than 3 branches. > >> > > >> > Thanks, > >> > > >> > Cole Greer > >> > > >> > > >
[DISCUSS] Support for using TinkerPop as a Java Module
Hi All, Recently, there has been some discussion around the use of TinkerPop's Java libraries as a Java Module. My proposal is to not support this for now. The reason I feel this way is due to the fact that Java 8 will be supported for the foreseeable future. There are many dependencies of TinkerPop that don't support JPMS and would become automatic modules. I'm generally not in favor of this approach since I feel that it mostly defeats the purpose of the module system in the first place. I'd prefer that more of our dependencies became explicit modules first before converting our Java libraries into explicit modules. A second related question that has come up is the issue with using TinkerPop's Java libraries as automatic modules. The same package is exported in multiple projects which causes split packages to occur when attempting to use TinkerPop in this way. This will require workarounds from the user to allow TinkerPop to be used as an automatic module. Although it can be argued that this change would be simpler to do than converting the project into an explicit module, I'd still be against doing this. My preference would be to either fully support JPMS or not support it at all. Changing package names also constitutes a breaking change which I feel the majority of our users would not benefit from. Unless there is strong demand from users or someone is willing to champion this change (implement and support) then I'd prefer to not support this for now. What are your thoughts on this subject? If there is no disagreement in the next three days then I'm going to assume lazy consensus and update the documentation accordingly to show that we don't support being used as a Java Module. Thanks, Ken
[jira] [Closed] (TINKERPOP-2972) ProjectStep will produce an unexpected output when dealing with duplicate keys
[
https://issues.apache.org/jira/browse/TINKERPOP-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu closed TINKERPOP-2972.
-
Resolution: Done
> ProjectStep will produce an unexpected output when dealing with duplicate keys
> --
>
> Key: TINKERPOP-2972
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2972
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
>Affects Versions: 3.6.4
>Reporter: Jianzhong Liu
>Priority: Major
> Fix For: 3.6.6, 3.7.1
>
>
> g.V().project("a", "b", "b").by("lang").by(T.id).by("age")
> TP:
> 1\{'b': 29}
> 2\{'b': 27}
> 3\{'a': 'java', 'b': '3'}
> 4\{'b': 32}
> 5\{'a': 'java', 'b': '5'}
> 6\{'b': 35}
> TinkerPop erroneously generates an output from the final 'productive'
> by-traversal when a key is used more than once, a situation that should not
> occur. A proposed solution is to throw an exception when a redundant key is
> detected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (TINKERPOP-2972) ProjectStep will produce an unexpected output when dealing with duplicate keys
[
https://issues.apache.org/jira/browse/TINKERPOP-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu updated TINKERPOP-2972:
--
Fix Version/s: 3.6.6
3.7.1
> ProjectStep will produce an unexpected output when dealing with duplicate keys
> --
>
> Key: TINKERPOP-2972
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2972
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
>Affects Versions: 3.6.4
>Reporter: Jianzhong Liu
>Priority: Major
> Fix For: 3.6.6, 3.7.1
>
>
> g.V().project("a", "b", "b").by("lang").by(T.id).by("age")
> TP:
> 1\{'b': 29}
> 2\{'b': 27}
> 3\{'a': 'java', 'b': '3'}
> 4\{'b': 32}
> 5\{'a': 'java', 'b': '5'}
> 6\{'b': 35}
> TinkerPop erroneously generates an output from the final 'productive'
> by-traversal when a key is used more than once, a situation that should not
> occur. A proposed solution is to throw an exception when a redundant key is
> detected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (TINKERPOP-2988) Serialization error throws an Invalid OpProcessor exception when using stream() API
[
https://issues.apache.org/jira/browse/TINKERPOP-2988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu closed TINKERPOP-2988.
-
Resolution: Fixed
> Serialization error throws an Invalid OpProcessor exception when using
> stream() API
> ---
>
> Key: TINKERPOP-2988
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2988
> Project: TinkerPop
> Issue Type: Improvement
> Components: server
>Affects Versions: 3.7.0
>Reporter: Taylor Riggan
>Priority: Critical
> Fix For: 3.6.6, 3.7.1, 4.0.0
>
>
> The following throws an
> {code:java}
> Invalid OpProcessor requested [null] (499){code}
> exception. The cause is due to an invalid RequestId (an non-conformant UUID
> is being used):
> {code:java}
> resp = client.stream ("g.V()", {}, { batchSize: 1, requestId: 'foo',
> userAgent: 'bar', evaluationTimeout: 5000 });{code}
>
> The error was diagnosed using Gremlin Server 3.7.0 and seeing the following
> in the server logs:
>
> {code:java}
> [WARN] o.a.t.g.s.h.WsGremlinBinaryRequestDecoder - Serialization error while
> decoding request
> org.apache.tinkerpop.gremlin.driver.ser.SerializationException:
> org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not
> deserialize the JSON value as required. Nested exception:
> org.apache.tinkerpop.shaded.jackson.databind.exc.InvalidFormatException:
> Cannot deserialize value of type `java.util.UUID` from String "foo": UUID has
> to be represented by standard 36-char representation
> at [Source: (byte[])"{"requestId":{{{}
> {"@type":"g:UUID","@value":"foo"}
> {}}},"op":"eval","processor":"","args":{"gremlin":"g.V()","aliases":{{{}
> {"g":"g"}
> {}}},"batchSize":1,"userAgent":"bar","evaluationTimeout":5000,"bindings":{},"language":"gremlin-groovy","accept":"application/vnd.gremlin-v3.0+json"";
> line: 1, column: 41]{}}}
> at [Source: (byte[])"{"requestId":{{{}
> {"@type":"g:UUID","@value":"foo"}
> {}}},"op":"eval","processor":"","args":{"gremlin":"g.V()","aliases":{{{}
> {"g":"g"}
> {}}},"batchSize":1,"userAgent":"bar","evaluationTimeout":5000,"bindings":{},"language":"gremlin-groovy","accept":"application/vnd.gremlin-v3.0+json"";
> line: 1, column: 41] (through reference chain:
> java.util.LinkedHashMap["requestId"]){}}}
> at
> org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0.deserializeRequest(AbstractGraphSONMessageSerializerV2d0.java:124)
> at
> org.apache.tinkerpop.gremlin.server.handler.WsGremlinBinaryRequestDecoder.decode(WsGremlinBinaryRequestDecoder.java:77)
>
> {code}
>
> A better client side error message is needed in order to provide the user
> with guidance on how to fix the issue. `Invalid OpProcessor` is too
> ambiguous.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (TINKERPOP-2988) Serialization error throws an Invalid OpProcessor exception when using stream() API
[
https://issues.apache.org/jira/browse/TINKERPOP-2988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu updated TINKERPOP-2988:
--
Fix Version/s: 3.6.6
3.7.1
4.0.0
> Serialization error throws an Invalid OpProcessor exception when using
> stream() API
> ---
>
> Key: TINKERPOP-2988
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2988
> Project: TinkerPop
> Issue Type: Improvement
> Components: server
>Affects Versions: 3.7.0
>Reporter: Taylor Riggan
>Priority: Critical
> Fix For: 3.6.6, 3.7.1, 4.0.0
>
>
> The following throws an
> {code:java}
> Invalid OpProcessor requested [null] (499){code}
> exception. The cause is due to an invalid RequestId (an non-conformant UUID
> is being used):
> {code:java}
> resp = client.stream ("g.V()", {}, { batchSize: 1, requestId: 'foo',
> userAgent: 'bar', evaluationTimeout: 5000 });{code}
>
> The error was diagnosed using Gremlin Server 3.7.0 and seeing the following
> in the server logs:
>
> {code:java}
> [WARN] o.a.t.g.s.h.WsGremlinBinaryRequestDecoder - Serialization error while
> decoding request
> org.apache.tinkerpop.gremlin.driver.ser.SerializationException:
> org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not
> deserialize the JSON value as required. Nested exception:
> org.apache.tinkerpop.shaded.jackson.databind.exc.InvalidFormatException:
> Cannot deserialize value of type `java.util.UUID` from String "foo": UUID has
> to be represented by standard 36-char representation
> at [Source: (byte[])"{"requestId":{{{}
> {"@type":"g:UUID","@value":"foo"}
> {}}},"op":"eval","processor":"","args":{"gremlin":"g.V()","aliases":{{{}
> {"g":"g"}
> {}}},"batchSize":1,"userAgent":"bar","evaluationTimeout":5000,"bindings":{},"language":"gremlin-groovy","accept":"application/vnd.gremlin-v3.0+json"";
> line: 1, column: 41]{}}}
> at [Source: (byte[])"{"requestId":{{{}
> {"@type":"g:UUID","@value":"foo"}
> {}}},"op":"eval","processor":"","args":{"gremlin":"g.V()","aliases":{{{}
> {"g":"g"}
> {}}},"batchSize":1,"userAgent":"bar","evaluationTimeout":5000,"bindings":{},"language":"gremlin-groovy","accept":"application/vnd.gremlin-v3.0+json"";
> line: 1, column: 41] (through reference chain:
> java.util.LinkedHashMap["requestId"]){}}}
> at
> org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0.deserializeRequest(AbstractGraphSONMessageSerializerV2d0.java:124)
> at
> org.apache.tinkerpop.gremlin.server.handler.WsGremlinBinaryRequestDecoder.decode(WsGremlinBinaryRequestDecoder.java:77)
>
> {code}
>
> A better client side error message is needed in order to provide the user
> with guidance on how to fix the issue. `Invalid OpProcessor` is too
> ambiguous.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [DISCUSS] Start of Code Freeze
Hi Cole, I'd like to mention that I'm working on a PR for TINKERPOP-3013 that I hope will be landing soon. I'd like that fix to go into the 3.7.1 release. Thanks, Ken On Fri, Nov 10, 2023 at 4:29 PM Cole Greer wrote: > Hi everyone, > > As we approach the end of the year, I think it’s time that we get started > on the next set of minor releases for TinkerPop. This would include 3.5.8, > 3.6.6, and 3.7.1. > > There are a couple of open PR’s at the moment which appear to be ready to > merge in the next few days: > > > * Bytecode support for HTTP requests: > https://github.com/apache/tinkerpop/pull/2336 > * TINKERPOP-2988 Add requestId checking to Javascript and Python > drivers: https://github.com/apache/tinkerpop/pull/2329 > * TINKERPOP-2830 Handle User-Agent from HTTP Requests to server: > https://github.com/apache/tinkerpop/pull/2328 > * TINKERPOP-2984 Replace Moq with NSubstitute: > https://github.com/apache/tinkerpop/pull/2319 > > Additionally, there are several smaller PR’s regarding configuration and > dependency tweaks which I believe can be reviewed and voted on in the next > few days as well. > > I propose that we initiate code freeze on 3.5-dev, 3.6-dev, and 3.7-dev > today, with the intent that the PR’s mentioned above, as well as other > small non-breaking fixes can still be merged in the next few days. With > this timing, the VOTE threads could be expected starting November 20, and > the releases would be expected within the first week of December. If there > are any objections to this timeline, or if there are any other desired > changes before the release, please reply to this thread. > > I would additionally like to propose that we plan this to be the final > release in the 3.5.x line, and that we leave 3.5-dev frozen indefinitely. > This could of course be reversed in the future if a critical need arises, > but unless there are any objections, I believe it’s best that we return to > actively working on no more than 3 branches. > > Thanks, > > Cole Greer >
[jira] [Commented] (TINKERPOP-2971) having order() before fold() omits an empty list in GroupStep's value traversal
[
https://issues.apache.org/jira/browse/TINKERPOP-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784593#comment-17784593
]
Ken Hu commented on TINKERPOP-2971:
---
Might be related to OLAP. Try this against SparkGraphComputer and see what
happens.
> having order() before fold() omits an empty list in GroupStep's value
> traversal
> ---
>
> Key: TINKERPOP-2971
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2971
> Project: TinkerPop
> Issue Type: Bug
> Components: tinkergraph
>Affects Versions: 3.6.4
>Reporter: Norio Akagi
>Priority: Critical
>
> Using the modern graph,
> {code:java}
> gremlin> g.V().group().by().by(out().fold())
> ==>[v[1]:[v[3],v[2],v[4]],v[2]:[],v[3]:[],v[4]:[v[5],v[3]],v[5]:[],v[6]:[v[3]]]{code}
> {{out().fold()}} produces an empty list when there is no solution for
> {{out()}} in a value traversal of GroupStep. However if I put {{order()}}
> those empty lists are gone.
> {code:java}
> gremlin> g.V().group().by().by(out().order().fold())
> ==>[v[1]:[v[2],v[3],v[4]],v[4]:[v[3],v[5]],v[6]:[v[3]]]{code}
> I think this is not an expected behavior.
> I don't see the similar effect in by() itself
> {code:java}
> gremlin> g.V().as("a").select("a").by(out().fold())
> ==>[v[3],v[2],v[4]]
> ==>[]
> ==>[]
> ==>[v[5],v[3]]
> ==>[]
> ==>[v[3]]{code}
> {code:java}
> gremlin> g.V().as("a").select("a").by(out().order().fold())
> ==>[v[2],v[3],v[4]]
> ==>[]
> ==>[]
> ==>[v[3],v[5]]
> ==>[]
> ==>[v[3]]{code}
> So likely this is GroupStep specific issue. We need to investigate how this
> occurs and fix if needed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (TINKERPOP-3010) Move TinkerGraph specific transaction testing
[
https://issues.apache.org/jira/browse/TINKERPOP-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu closed TINKERPOP-3010.
-
Resolution: Fixed
> Move TinkerGraph specific transaction testing
> -
>
> Key: TINKERPOP-3010
> URL: https://issues.apache.org/jira/browse/TINKERPOP-3010
> Project: TinkerPop
> Issue Type: Bug
> Components: test-suite
>Affects Versions: 3.7.0
>Reporter: Ken Hu
>Priority: Critical
> Fix For: 3.7.1
>
>
> As reported by multiple providers on Discord,
> From Boxuan:
> {quote}The newly added
> TransactionMultiThreadedTest::shouldHandleAddingSameVertexInDifferentTx in
> 3.7.0 release assumes the provider supports custom vertex ID, which was not
> the case in previous releases. Assuming this was not intentional, should we
> have a feature flag for this requirement?
> The newly added
> TransactionMultiThreadedTest::shouldHandleConcurrentChangeForProperty in
> 3.7.0 release assumes the behavior of two conflicting transactions is
> TransactionException. This is totally fine, but IMO this is TinkerGraph
> specific behavior and should not apply to providers in general. For example,
> another common approach is to let one transaction wait for the other.
> {quote}
> From Pieter:
> {quote}Hi, in upgrading Sqlg to 3.7.0 the TransactionMultiThreadedTest is
> executing now.
> Quite a few of the tests are hanging, but it seems to me an implementation
> issue with the test rather than the underlying transaction semantics.
> Multi threaded tests should not themselves have one thread waiting for
> another. The tests are creating artificial conditions where one thread waits
> for another.
> Take for example
> TransactionMultiThreadedTest.shouldHandleConcurrentVertexDelete
> shouldHandleAddingPropertyWhenOtherTxDeleteEdge
> shouldHandleAddingPropertyWhenOtherTxDeleteVertex
> shouldDeleteEdgeOnCommit
> shouldHandleConcurrentChangeForProperty
> shouldHandleAddingEdgeWhenOtherTxDeleteVertex
> shouldHandleConcurrentChangeForVertexProperty
> shouldHandleConcurrentDeleteEdge
> shouldHandleAddingSameVertexInDifferentTx
> shouldThrowExceptionWhenTryToAddVertexWithUsedId
> These are the ones I had to OptOut of.
> The other minor issue is on shouldThrowExceptionWhenTryToAddVertexWithUsedId
> My code throws an UnsupportedOperationException while the test expects an
> IllegalStateException
> I did not investigate which one is correct by I suspect its
> UnsupportedOperationException
> {quote}
> These tests exhibit TinkerTransactionGraph specific behavior and should be
> moved to TinkerTransactionGraphTest. Search through all tests in
> TransactionMultiThreadedTest for possible candidates.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (TINKERPOP-3010) Move TinkerGraph specific transaction testing
[
https://issues.apache.org/jira/browse/TINKERPOP-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu updated TINKERPOP-3010:
--
Fix Version/s: 3.7.1
> Move TinkerGraph specific transaction testing
> -
>
> Key: TINKERPOP-3010
> URL: https://issues.apache.org/jira/browse/TINKERPOP-3010
> Project: TinkerPop
> Issue Type: Bug
> Components: test-suite
>Affects Versions: 3.7.0
>Reporter: Ken Hu
>Priority: Critical
> Fix For: 3.7.1
>
>
> As reported by multiple providers on Discord,
> From Boxuan:
> {quote}The newly added
> TransactionMultiThreadedTest::shouldHandleAddingSameVertexInDifferentTx in
> 3.7.0 release assumes the provider supports custom vertex ID, which was not
> the case in previous releases. Assuming this was not intentional, should we
> have a feature flag for this requirement?
> The newly added
> TransactionMultiThreadedTest::shouldHandleConcurrentChangeForProperty in
> 3.7.0 release assumes the behavior of two conflicting transactions is
> TransactionException. This is totally fine, but IMO this is TinkerGraph
> specific behavior and should not apply to providers in general. For example,
> another common approach is to let one transaction wait for the other.
> {quote}
> From Pieter:
> {quote}Hi, in upgrading Sqlg to 3.7.0 the TransactionMultiThreadedTest is
> executing now.
> Quite a few of the tests are hanging, but it seems to me an implementation
> issue with the test rather than the underlying transaction semantics.
> Multi threaded tests should not themselves have one thread waiting for
> another. The tests are creating artificial conditions where one thread waits
> for another.
> Take for example
> TransactionMultiThreadedTest.shouldHandleConcurrentVertexDelete
> shouldHandleAddingPropertyWhenOtherTxDeleteEdge
> shouldHandleAddingPropertyWhenOtherTxDeleteVertex
> shouldDeleteEdgeOnCommit
> shouldHandleConcurrentChangeForProperty
> shouldHandleAddingEdgeWhenOtherTxDeleteVertex
> shouldHandleConcurrentChangeForVertexProperty
> shouldHandleConcurrentDeleteEdge
> shouldHandleAddingSameVertexInDifferentTx
> shouldThrowExceptionWhenTryToAddVertexWithUsedId
> These are the ones I had to OptOut of.
> The other minor issue is on shouldThrowExceptionWhenTryToAddVertexWithUsedId
> My code throws an UnsupportedOperationException while the test expects an
> IllegalStateException
> I did not investigate which one is correct by I suspect its
> UnsupportedOperationException
> {quote}
> These tests exhibit TinkerTransactionGraph specific behavior and should be
> moved to TinkerTransactionGraphTest. Search through all tests in
> TransactionMultiThreadedTest for possible candidates.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (TINKERPOP-3011) Add serialization to set for gremlin-javascript
Ken Hu created TINKERPOP-3011: - Summary: Add serialization to set for gremlin-javascript Key: TINKERPOP-3011 URL: https://issues.apache.org/jira/browse/TINKERPOP-3011 Project: TinkerPop Issue Type: Improvement Components: javascript Reporter: Ken Hu gremlin-javascript used to serialize/deserialize Sets into array types. However starting with 4.0.0, support was added for deserializing Sets into javascript Sets. The ability to seralize to Sets needs to be added as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TINKERPOP-3011) Add serialization to set for gremlin-javascript
[ https://issues.apache.org/jira/browse/TINKERPOP-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu updated TINKERPOP-3011: -- Affects Version/s: 4.0.0 > Add serialization to set for gremlin-javascript > --- > > Key: TINKERPOP-3011 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3011 > Project: TinkerPop > Issue Type: Improvement > Components: javascript >Affects Versions: 4.0.0 >Reporter: Ken Hu >Priority: Major > > gremlin-javascript used to serialize/deserialize Sets into array types. > However starting with 4.0.0, support was added for deserializing Sets into > javascript Sets. The ability to seralize to Sets needs to be added as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (TINKERPOP-3010) Move TinkerGraph specific transaction testing
Ken Hu created TINKERPOP-3010:
-
Summary: Move TinkerGraph specific transaction testing
Key: TINKERPOP-3010
URL: https://issues.apache.org/jira/browse/TINKERPOP-3010
Project: TinkerPop
Issue Type: Bug
Components: test-suite
Affects Versions: 3.7.0
Reporter: Ken Hu
As reported by multiple providers on Discord,
>From Boxuan:
{quote}The newly added
TransactionMultiThreadedTest::shouldHandleAddingSameVertexInDifferentTx in
3.7.0 release assumes the provider supports custom vertex ID, which was not the
case in previous releases. Assuming this was not intentional, should we have a
feature flag for this requirement?
The newly added
TransactionMultiThreadedTest::shouldHandleConcurrentChangeForProperty in 3.7.0
release assumes the behavior of two conflicting transactions is
TransactionException. This is totally fine, but IMO this is TinkerGraph
specific behavior and should not apply to providers in general. For example,
another common approach is to let one transaction wait for the other.
{quote}
>From Pieter:
{quote}Hi, in upgrading Sqlg to 3.7.0 the TransactionMultiThreadedTest is
executing now.
Quite a few of the tests are hanging, but it seems to me an implementation
issue with the test rather than the underlying transaction semantics.
Multi threaded tests should not themselves have one thread waiting for another.
The tests are creating artificial conditions where one thread waits for another.
Take for example TransactionMultiThreadedTest.shouldHandleConcurrentVertexDelete
shouldHandleAddingPropertyWhenOtherTxDeleteEdge
shouldHandleAddingPropertyWhenOtherTxDeleteVertex
shouldDeleteEdgeOnCommit
shouldHandleConcurrentChangeForProperty
shouldHandleAddingEdgeWhenOtherTxDeleteVertex
shouldHandleConcurrentChangeForVertexProperty
shouldHandleConcurrentDeleteEdge
shouldHandleAddingSameVertexInDifferentTx
shouldThrowExceptionWhenTryToAddVertexWithUsedId
These are the ones I had to OptOut of.
The other minor issue is on shouldThrowExceptionWhenTryToAddVertexWithUsedId
My code throws an UnsupportedOperationException while the test expects an
IllegalStateException
I did not investigate which one is correct by I suspect its
UnsupportedOperationException
{quote}
These tests exhibit TinkerTransactionGraph specific behavior and should be
moved to TinkerTransactionGraphTest. Search through all tests in
TransactionMultiThreadedTest for possible candidates.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [DISCUSS] Transition to Semantic Versioning
I think it makes sense to go with semantic versioning. It seems like in the past, users have been confused about our current scheme and the expectation was for it to be more like semantic versioning. Therefore, we should probably just go ahead with what users were expecting in the first place. On Thu, Oct 12, 2023 at 12:59 PM Cole Greer wrote: > Hi Fred, > > Thanks for the suggestion, I have some concerns about CalVer mostly due to > it being a greater departure from our current versioning than a switch to > semantic versioning would be. We are already using semantic versioning if > you simply ignore the 3. If we switch to CalVer, I think the scheme which > best fits our development and support cycle would be something like > YY.MAJOR.MINOR. In this case our next major release would likely be > TinkerPop 24.1.0 (which would receive updates for roughly 2 years forming > the 24.1.x line). The 2 issues here being that we close the door on ever > having micro patch releases, and if we stick with the expected cadence of 1 > major release per year, then the MAJOR number will always be 1, there would > never be a 24.2.0 as the next major release would not land until 2025. To > fix both problems we could switch to YY.MINOR.PATCH however this scheme > would restrict us from ever doing more than 1 major release in a calendar > year. I don’t think we want to rid ourselves of that option. > > I think that semantic versioning is the most broadly understood system by > developers. Semantic versioning gives users a clear picture of the > compatibility between versions and the general scope of changes between > releases. For these reasons I would still prefer to go with a semantic > versioning scheme. > > Please let me know your thoughts on this. > > Thanks, > > Cole > > From: Fred Eisele > Date: Wednesday, September 27, 2023 at 2:56 PM > To: dev@tinkerpop.apache.org > Subject: Re: [DISCUSS] Transition to Semantic Versioning > What about CalVer? https://calver.org/ > There have been TinkerPop versions in the past which were quite different > 1, 2, 3. > I would expect a version 4 (as some proposals suggested). > It seems that this major version number is part of the product name more > than just a version. > > > On Wed, Sep 27, 2023 at 4:14 PM Cole Greer .invalid> > wrote: > > > Hi everyone, > > > > I think now is a good time to start thinking about 3.7.x development from > > master into 3.7-dev and to setup master to take breaking changes for the > > next major release. I opted to delay this after the last release to > > temporarily save some of the admin burden of 4 active branches. > > > > I think this gives a good opportunity to revisit our versioning scheme. > > The current scheme ( > > https://tinkerpop.apache.org/docs/current/dev/developer/#versioning) is > > EPOCH.MAJOR.MINOR.PATCH, where EPOCH has been fixed at 3 since 2015, and > > the PATCH digit has never been utilized. > > > > I suggest that we move to a more standard semantic versioning scheme ( > > https://semver.org/) with just MAJOR.MINOR.PATCH starting with 4.0.0. I > > would expect the MAJOR and MINOR numbers to essentially retain the same > > roles as they have now (4.x.x -> 5.0.0 would be comparable to 3.6.x -> > > 3.7.0). I don’t expect we would utilize the PATCH release often, but I > > would for it to be a formalized process for publishing partial releases > > (perhaps only updating a single component) which can hopefully skip much > of > > the overhead of a full release. We’ve previously deployed > release-candidate > > convenience binaries for some GLV’s in the past for certain users in this > > fashion, it would be good to bring this into a proper release. > > > > I believe this versioning system is more aligned with what most > developers > > are used to. Additionally, I hope that dropping the EPOCH value will > allow > > us to be a little more aggressive with changes in major releases (perhaps > > introducing a type system or schema capability). > > > > It would also be useful if we started publishing end-of-life dates for > > major releases to give users a clearer picture of expected support. I > would > > suggest as a starting point 1 year of active support (adding new > features, > > acting as the main target branch for non-breaking changes) and 2 years of > > security support (branch is kept open for critical bug fixes and security > > patches). If we plan to maintain roughly 1 major release per year, this > > remains somewhat in line with current support timelines. We would > > presumably retain the right to do additional releases for a branch beyond > > the promised support window if we ever deemed it necessary. > > > > Let me know if you have any thoughts on this change. If there is no > > feedback, I will assume a lazy consensus in favour of adopting semantic > > versioning and moving the master branch to 4.0.0-SNAPSHOT development. > > There are still some details to be worked out regarding patch releases > > under the new system
[DISCUSS] Draft ASF Board Report - October 2023
Yang is away right now so I'm attaching a copy of the draft of the board report for this quarter on her behalf. Please respond to this post if there is anything to add or edit. ## Description: Apache TinkerPop is a graph computing framework for both graph databases (OLTP) and graph analytic systems (OLAP). ## Issues: There are no issues requiring board attention. ## Project Status: Ongoing We have relatively high activity on the project, with on-going quarterly releases, daily commits and development discussions on the devlist. Our Discord channel has active engagement with a steady stream of new users and questions, as well as steady participants in our new bi-weekly TinkerPop Gathering event. The regular knowledge-sharing Twitch streams have been on-going as planned. ## Membership Data: Kelvin Lawrence has assumed the PMC Chair position starting July as per the project chair rotation setup. Community changes, past quarter: - No new PMC members. Last addition was Yang Xia on 2023-03-15. - No new committer. Last addition was Cole Greer on 2023-05-30 ## Project Activity: TinkerPop released versions 3.7.0, 3.5.7, and 3.6.5 on July 31, 2023, making it the first major version release since the 3.6.0 release in April 2022, as well as the first triple-version release. Various new features were included in this 3.7.0 major release. We have expanded our support to JDK 17 as well as Groovy 4. This release included many usability improvements in areas often requested by users, which included returning vertices and edges with their properties by default instead of only as reference elements, which often led to confusion, especially for new users. The first of the long-desired string manipulation function, concat(), is now part of Gremlin transversal language. We have also introduced transaction support in TinkerGraph, the TinkerPop reference graph implementation We are currently closing in on releasing 3.7.1 shortly, with up to 20 new steps added to the traversal language. These new steps will include further string manipulation step functions for user convenience, as well as additional date and list manipulation steps. Together with 3.7.1, we plan to also publish maintenance releases 3.6.6, and 3.5.8. ### Releases: 3.7.0 was released on 2023-07-31. 3.6.5 was released on 2023-07-31. 3.5.7 was released on 2023-07-31. ## Community Health: We again had some live events on Twitch[1] where the recordings are available on our YouTube channel[2]: * Contributorcast: Apache TinkerPop 3.7.0/3.6.5/3.5.7 Post-Release Review - Presenter: Yang Xia, Valentyn Kahamlyk, Cole Greer, and Ken Hu * TinkerPop Gathering: We have begun a regular bi-weekly event on the Discord channel starting on Aug 18, serving as a dedicated time for users, providers, and developers to meet and chat about any questions or suggestions they have. ## Links [1]: https://www.twitch.tv/apachetinkerpop [2]: https://www.youtube.com/@apachetinkerpop
[jira] [Updated] (TINKERPOP-3000) Issue with union step when using path().by()
[
https://issues.apache.org/jira/browse/TINKERPOP-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu updated TINKERPOP-3000:
--
Priority: Critical (was: Major)
> Issue with union step when using path().by()
>
>
> Key: TINKERPOP-3000
> URL: https://issues.apache.org/jira/browse/TINKERPOP-3000
> Project: TinkerPop
> Issue Type: Bug
> Components: language
>Affects Versions: 3.7.0
>Reporter: Ken Hu
>Priority: Critical
>
> The following query
> {code:java}
> g.union(__.V().out().out().path().by("name")){code}
> throws
> {code:java}
> The by("name") modulator can only be applied to a traverser that is an
> Element or a Map - it is being applied to [false] a Boolean class
> instead{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (TINKERPOP-3000) Issue with union step when using path().by()
Ken Hu created TINKERPOP-3000:
-
Summary: Issue with union step when using path().by()
Key: TINKERPOP-3000
URL: https://issues.apache.org/jira/browse/TINKERPOP-3000
Project: TinkerPop
Issue Type: Bug
Components: language
Affects Versions: 3.7.0
Reporter: Ken Hu
The following query
{code:java}
g.union(__.V().out().out().path().by("name")){code}
throws
{code:java}
The by("name") modulator can only be applied to a traverser that is an Element
or a Map - it is being applied to [false] a Boolean class instead{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (TINKERPOP-2999) 3.7.0 Remote Console Sends Incomplete Queries
[ https://issues.apache.org/jira/browse/TINKERPOP-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu updated TINKERPOP-2999: -- Description: As reported by user "pdsway" on gremlin-users. In remote console mode the 3.7.0 Gremlin Console will send incomplete queries to the server causing a parsing error. This is caused by the upgrade to Groovy 4. That version of Groovy uses the newer antlr4 parser which throws errors later in execution. So even if the parser says the the line is COMPLETE, it could still be incomplete. We need to add the same try catch to the remote console's execution to make the handling the same as groovysh. was: In remote console mode the 3.7.0 Gremlin Console will send incomplete queries to the server causing a parsing error. This is caused by the upgrade to Groovy 4. That version of Groovy uses the newer antlr4 parser which throws errors later in execution. So even if the parser says the the line is COMPLETE, it could still be incomplete. We need to add the same try catch to the remote console's execution to make the handling the same as groovysh. > 3.7.0 Remote Console Sends Incomplete Queries > - > > Key: TINKERPOP-2999 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2999 > Project: TinkerPop > Issue Type: Bug > Components: console >Affects Versions: 3.7.0 >Reporter: Ken Hu >Priority: Critical > > As reported by user "pdsway" on gremlin-users. > In remote console mode the 3.7.0 Gremlin Console will send incomplete queries > to the server causing a parsing error. > This is caused by the upgrade to Groovy 4. That version of Groovy uses the > newer antlr4 parser which throws errors later in execution. So even if the > parser says the the line is COMPLETE, it could still be incomplete. We need > to add the same try catch to the remote console's execution to make the > handling the same as groovysh. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (TINKERPOP-2999) 3.7.0 Remote Console Sends Incomplete Queries
Ken Hu created TINKERPOP-2999: - Summary: 3.7.0 Remote Console Sends Incomplete Queries Key: TINKERPOP-2999 URL: https://issues.apache.org/jira/browse/TINKERPOP-2999 Project: TinkerPop Issue Type: Bug Components: console Affects Versions: 3.7.0 Reporter: Ken Hu In remote console mode the 3.7.0 Gremlin Console will send incomplete queries to the server causing a parsing error. This is caused by the upgrade to Groovy 4. That version of Groovy uses the newer antlr4 parser which throws errors later in execution. So even if the parser says the the line is COMPLETE, it could still be incomplete. We need to add the same try catch to the remote console's execution to make the handling the same as groovysh. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] Ternary Boolean Handling in New Steps
en returning a value out to the step > level, it would have to be either true or false. Under this model we would > no longer view a tree of And/Or/Not steps as a single logic expression but > instead as a series of discrete steps. I think this may be the right path > forward although it likely requires leveling up the capabilities of > predicates in order to be more useful. > > I would be curious to know if others have any thoughts on what the right > rules are for determining when to reduce from ternary boolean logic to true > and false. > > Regards, > > Cole > > > > From: Mike Personick > Date: Wednesday, September 20, 2023 at 9:41 AM > To: dev@tinkerpop.apache.org > Subject: Re: [DISCUSS] Ternary Boolean Handling in New Steps > Early reduction is not possible because of negation. Please see the > discussion of ternary boolean semantics here: > > > https://tinkerpop.apache.org/docs/3.6.0/dev/provider/#_ternary_boolean_logics > > On Tue, Sep 19, 2023 at 12:25 PM Ken Hu > wrote: > > > Thanks for your input Pieter. > > > > I agree with a lot of what you said and I think your suggestion is > > reasonable. From what I can tell, the logic in the FilterStep is there > > because reduction points are needed for the ternary boolean system. One > of > > the ways to move logic out of this area would be to get rid of ternary > > boolean by immediately reducing any ERROR state to FALSE. > > > > On Sat, Sep 16, 2023 at 1:06 AM pieter wrote: > > > > > Hi, > > > I have not really applied my mind to issue of the semantics, but what I > > > do recall is being irritated with `GremlinValueComparator` throwing > > > `GremlinTypeErrorException`. > > > Sometimes its propogated and sometimes swallowed. Code smell!!! > > > Using exceptions as process logic right there in the heart of > > > TinkerPop's iterator logic seemed to me as a bad idea and breaks > > > providers ability to override classes. > > > A good example is the logic in FilterStep.processNextStart() where the > > > exception is being swalloed. This logic should not be here and > > > exceptions should not be used for control flow. > > > Providers expect the base filter step to filter, not conditionally > > > swallow exceptions based on a long if statement. > > > My suggestion is let the comparator do what comparators do and return a > > > int. The type issue should be handled higher up the stack. > > > > > > Regards > > > Pieter > > > On Fri, 2023-09-15 at 14:14 -0700, Ken Hu wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Cole, "However, it makes sense for this short-term decision > > > > > > > to align with our long-term direction regarding comparability > > > > > > > semantics. I > > > > > > > wouldn’t be opposed to your proposed implementation if the > > > > > > > long-term plan is to move all steps towards this immediate > > > > > > > reduction behaviour." This is sort of my thinking as well. As > > > > > > > you demonstrated in your post, there is already an > > > > > > > inconsistency with the way ternary > > > > > > > boolean is reduced which leads to different results for > > > > > > > equivalent queries. > > > > > > > This is why I would prefer to just move ahead with an > > > > > > > implementation that I > > > > > > > believe is the most consistent with the expectations of users. > > > > > > > However, you > > > > > > > have valid concerns about adding even more inconsistencies to > > > > > > > the > > > > > > > language so if others voice their concern as well then I'll > > > > > > > make it > > > > > > > behave more like AND and OR. Regards, Ken On Mon, Sep 11, 2023 > > > > > > > at 6:11 PM Cole Greer cole.gr...@improving.com.invalidwrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > &
Re: [DISCUSS] Ternary Boolean Handling in New Steps
Thanks for your input Pieter. I agree with a lot of what you said and I think your suggestion is reasonable. From what I can tell, the logic in the FilterStep is there because reduction points are needed for the ternary boolean system. One of the ways to move logic out of this area would be to get rid of ternary boolean by immediately reducing any ERROR state to FALSE. On Sat, Sep 16, 2023 at 1:06 AM pieter wrote: > Hi, > I have not really applied my mind to issue of the semantics, but what I > do recall is being irritated with `GremlinValueComparator` throwing > `GremlinTypeErrorException`. > Sometimes its propogated and sometimes swallowed. Code smell!!! > Using exceptions as process logic right there in the heart of > TinkerPop's iterator logic seemed to me as a bad idea and breaks > providers ability to override classes. > A good example is the logic in FilterStep.processNextStart() where the > exception is being swalloed. This logic should not be here and > exceptions should not be used for control flow. > Providers expect the base filter step to filter, not conditionally > swallow exceptions based on a long if statement. > My suggestion is let the comparator do what comparators do and return a > int. The type issue should be handled higher up the stack. > > Regards > Pieter > On Fri, 2023-09-15 at 14:14 -0700, Ken Hu wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Cole, "However, it makes sense for this short-term decision > > > > > to align with our long-term direction regarding comparability > > > > > semantics. I > > > > > wouldn’t be opposed to your proposed implementation if the > > > > > long-term plan is to move all steps towards this immediate > > > > > reduction behaviour." This is sort of my thinking as well. As > > > > > you demonstrated in your post, there is already an > > > > > inconsistency with the way ternary > > > > > boolean is reduced which leads to different results for > > > > > equivalent queries. > > > > > This is why I would prefer to just move ahead with an > > > > > implementation that I > > > > > believe is the most consistent with the expectations of users. > > > > > However, you > > > > > have valid concerns about adding even more inconsistencies to > > > > > the > > > > > language so if others voice their concern as well then I'll > > > > > make it > > > > > behave more like AND and OR. Regards, Ken On Mon, Sep 11, 2023 > > > > > at 6:11 PM Cole Greer cole.gr...@improving.com.invalidwrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Ken, Thanks for bringing this up, I believe topic > > > > > > > > > warrants some > > > > > > > > > further discussion. My understanding of the intent of > > > > > > > > > the > > > > > > > > > current system is that it aims to provide a consistent > > > > > > > > > and predictable set of > > > > > > > > > rules for comparisons between any datatypes. Prior to > > > > > > > > > 3.6, in general > > > > > > > > > comparisons between different types in gremlin produced > > > > > > > > > undefined behaviour (in practice this usually meant an > > > > > > > > > exception). The current system > > > > > > > > > successfully resolved much of this issue although it > > > > > > > > > has introduced certain > > > > > > > > > semantic consistency issues (see > > > > > > > >
Re: [DISCUSS] Ternary Boolean Handling in New Steps
Hi Cole,
"However, it makes sense for this short-term decision to align with our
long-term direction regarding comparability semantics. I wouldn’t be
opposed to your proposed implementation if the long-term plan is to move
all steps towards this immediate reduction behaviour."
This is sort of my thinking as well. As you demonstrated in your post,
there is already an inconsistency with the way ternary boolean is reduced
which leads to different results for equivalent queries. This is why I
would prefer to just move ahead with an implementation that I believe is
the most consistent with the expectations of users. However, you have valid
concerns about adding even more inconsistencies to the language so if
others voice their concern as well then I'll make it behave more like AND
and OR.
Regards,
Ken
On Mon, Sep 11, 2023 at 6:11 PM Cole Greer
wrote:
> Hi Ken,
>
> Thanks for bringing this up, I believe topic warrants some further
> discussion. My understanding of the intent of the current system is that it
> aims to provide a consistent and predictable set of rules for comparisons
> between any datatypes. Prior to 3.6, in general comparisons between
> different types in gremlin produced undefined behaviour (in practice this
> usually meant an exception). The current system successfully resolved much
> of this issue although it has introduced certain semantic consistency
> issues (see https://issues.apache.org/jira/browse/TINKERPOP-2940).
> Further, while the docs (
> https://tinkerpop.apache.org/docs/3.7.0/dev/provider/#_ternary_boolean_logics)
> are quite clear regarding the propagation/reduction behaviour in many
> cases, as you probe the edges it becomes muddier.
>
> Considering the following example, the docs quite clearly define the
> expected behaviour of the first traversal, but the expected behaviour is
> not clear outside of basic combinations of AND, OR, and NOT:
>
> gremlin> g.inject(1).not(is(gt("one")))
> // Produces no output
> gremlin> g.inject(1).not(union(is(gt("one")), is(eq("zero"
> ==>1 // Error is reduced to false prior to Union Step, and thus not
> propagated into the Not Step.
>
> This is a good example that we are currently in a bit of a weird place
> where some of the language semantics are formally defined in documentation,
> while the rest of the language semantics are defined by implementation. It
> currently cannot be determined if the above example is expected or a bug. I
> believe it is important that we find a resolution to this by expanding our
> formally defined semantics or changing the implementation (when a breaking
> change is permittable).
>
> As for the short-term question posed by ANY and ALL, my only concern with
> your suggestion is it would be subject to the following inconsistency
> although as shown above there is current precedent for this sort of thing.
>
> gremlin> g.inject(1).not(is(lt("one")))
> // Produces no output
> gremlin> g.inject([1]).not(any(is(lt("one"
> ==>[1]
>
> In my opinion the most neutral direction would be for ANY to behave the
> same as a chain of OR’s and for ALL to act as a chain of ANDs. However, it
> makes sense for this short-term decision to align with our long-term
> direction regarding comparability semantics. I wouldn’t be opposed to your
> proposed implementation if the long-term plan is to move all steps towards
> this immediate reduction behaviour.
>
> Thanks,
>
> Cole Greer
>
>
> From: Ken Hu
> Date: Monday, September 11, 2023 at 4:16 PM
> To: dev@tinkerpop.apache.org
> Subject: [DISCUSS] Ternary Boolean Handling in New Steps
> Hi All,
>
> Starting in version 3.6, the ternary boolean system was introduced to
> handle comparison/equality tests within Gremlin. Recently, I've been
> implementing some list functions from Proposal 3 which make heavy use of
> the GremlinValueComparator to determine if values satisfy a specific
> condition. However, I'm finding it a bit tricky to understand how I should
> handle the GremlinTypeErrorException. For any() and all(), it seems like it
> would make sense to immediately reduce any ERROR state to false as it's a
> filter step. In the case of all(), if a GremlinTypeErrorException is
> caught, it would mean there was a comparison error so the traverser should
> be removed from the stream. However, doing this seemingly clashes with the
> original intention of ternary boolean which is to allow a provider-specific
> response on how to handle an ERROR state.
>
> My current thoughts are that we should rework the ternary boolean system in
> the future to make it easier to incorporate it into new steps. One of the
> trickiest parts is that it uses unchecked exce
[DISCUSS] Ternary Boolean Handling in New Steps
Hi All, Starting in version 3.6, the ternary boolean system was introduced to handle comparison/equality tests within Gremlin. Recently, I've been implementing some list functions from Proposal 3 which make heavy use of the GremlinValueComparator to determine if values satisfy a specific condition. However, I'm finding it a bit tricky to understand how I should handle the GremlinTypeErrorException. For any() and all(), it seems like it would make sense to immediately reduce any ERROR state to false as it's a filter step. In the case of all(), if a GremlinTypeErrorException is caught, it would mean there was a comparison error so the traverser should be removed from the stream. However, doing this seemingly clashes with the original intention of ternary boolean which is to allow a provider-specific response on how to handle an ERROR state. My current thoughts are that we should rework the ternary boolean system in the future to make it easier to incorporate it into new steps. One of the trickiest parts is that it uses unchecked exceptions as a means to implement the ERROR state which can get easily missed or accidentally leaked to the user (which has happened before). For now, I'm planning to go ahead and immediately reduce ERROR states as I think that is what makes the most sense for list functions. Does anyone have any thoughts about this? Thanks, Ken
[DISCUSS] Remove the Proposed NoneStep
Hi All, I propose that we remove the none() step from Proposal 3 ( https://github.com/apache/tinkerpop/blob/master/docs/src/dev/future/proposal-3-remove-closures.asciidoc#none). After looking into several languages that support list functions (including Groovy, Python, LINQ, and Clojure), most of them don't include a "none" function. Instead, "none" functionality is achieved through not "all". This provides a sensible way to handle an empty list as "all" would return true and "any" would return false. Given that our new approach to "all" will take a predicate in as a parameter, a user could negate the predicate to achieve the same result. Users are likely already familiar with not having access to "none" function as it probably doesn't exist in their language of choice. It's also easy to add the "none" function later if we find that users expect it to be in the language. Please let me know what you think about removing the none() step. Thanks, Ken
Re: [DISCUSS] Local Scope As Default
Thanks for your input Stephen. The discussion of whether we want all(),
any(), none() to act like the IsStep or a Predicate was actually something
I had planned on mentioning in a separate DISCUSS thread, but I agree with
your reasoning for it to act like is(). For your question about what to do
if it isn't already in a List type, I was actually thinking the opposite. I
thought it might make sense to use IteratorUtils.asIterator() to force the
incoming traverser into some type of iterator. As for incorrect type, I
think the current proposal is to throw an exception in this case, although
I think it's reasonable to do the filtering you described as well.
On Mon, Aug 7, 2023 at 4:33 AM Stephen Mallette
wrote:
> I think your argument makes sense but your example made me think of
> something else:
>
> g.V().hasLabel('person').values('age').fold().where(all(gt(18)))
>
> the design of all() is to return true or false. if that's the case then
> using it in a where() will always be successful. returning true/false is
> really the job of a P so far and this would introduce something new. this
> direction could make sense if you wanted this:
>
> g.V().group().by('classroom').by(values('age').fold().all(gt(18
>
> where you were trying to set the value of true/false to each "classroom"
> being over age 18. i'm not sure that's what we want all() to be doing
> exactly. i think it's intention is more like the one you supplied and i
> think it was meant more as a specialized filtering step, specialized in
> that it worked on List sorts of types only and more like the intention of
> the examples you used, as in:
>
> gremlin> g.V().hasLabel('person').values('age').fold().all(gt(18))
> ==>[29,27,32,35]
>
> In this way, all()/any()/some() is a bit like how is() behaves in that it
> can apply a predicate to an item in the traversal stream. A good use case
> might be for dealing with results like:
>
> gremlin>
>
> g.V().both().both().group().by('name').by(outE().values('weight').fold()).unfold()
> ==>ripple=[]
> ==>peter=[0.2, 0.2, 0.2]
> ==>vadas=[]
> ==>josh=[1.0, 1.0, 1.0, 0.4, 0.4, 0.4, 1.0, 1.0, 1.0, 0.4, 0.4, 0.4, 1.0,
> 0.4]
> ==>lop=[]
> ==>marko=[0.4, 0.4, 0.4, 0.5, 0.5, 0.5, 1.0, 1.0, 1.0, 0.4, 0.5, 1.0, 0.4,
> 0.4, 0.4, 0.5, 0.5, 0.5, 1.0, 1.0, 1.0]
>
> where you dont even need to fold() the result. given the results above, i
> think folks might want to "find all key/value pairs where all weight values
> are gt(0.3)":
>
>
> g.V().both().both().group().by('name').by(outE().values('weight').fold()).unfold().
> where(select(values).all(gt(0.3)))
>
> i think the analogous form today without all() would maybe be something
> like:
>
> gremlin>
>
> g.V().both().both().group().by('name').by(outE().values('weight').fold()).unfold().
> ..1> filter(select(values).
> ..2> and(count(local).is(gt(0)),
> ..3> unfold().choose(__.is(gt(0.3)), constant(1),
> constant(0)).
> ..4> fold().
> ..5> union(sum(local), count(local)).fold().as('x').
> ..6>
> where('x',eq('x')).by(limit(local,1)).by(tail(local,1
> ==>josh=[1.0, 1.0, 1.0, 0.4, 0.4, 0.4, 1.0, 1.0, 1.0, 0.4, 0.4, 0.4, 1.0,
> 0.4]
> ==>marko=[0.4, 0.4, 0.4, 0.5, 0.5, 0.5, 1.0, 1.0, 1.0, 0.4, 0.5, 1.0, 0.4,
> 0.4, 0.4, 0.5, 0.5, 0.5, 1.0, 1.0, 1.0]
>
> so, i think all() in this form does make good sense. it seems less of a
> type of P which would imply some sort of fold(), (i.e. stream consumption -
> has('weight',all(0.3))) into it which i think would complicate other forms
> that take P. making it more like an is() that is designed to work on a List
> seems best. i also don't think all() should do too much magic and force
> folks to fold() if they don't already have a List sort of type. That said,
> I'd be curious how you define all() to behave when it doesn't get that type
> or the List type is empty. I assume it would filter in those cases since
> that fits the current design direction made in other steps going back to
> mid-3.5.x releases.
>
>
>
>
> On Fri, Aug 4, 2023 at 9:29 PM Ken Hu
> wrote:
>
> > Hi All,
> >
> > As I continue to take a further look into the list functions described in
> > Proposal 3, I noticed that they don't take in a Scope. Yet, it could have
> > been added because there are some functions that make sense as
> > ReducingBarriers. There are some instances, however, where the global
> scope
> > m
[DISCUSS] Local Scope As Default
Hi All,
As I continue to take a further look into the list functions described in
Proposal 3, I noticed that they don't take in a Scope. Yet, it could have
been added because there are some functions that make sense as
ReducingBarriers. There are some instances, however, where the global scope
makes no sense so I would propose that we implement these as stated in
Proposal 3. I just want to point out that this would likely be the first
time local scope was used as a default for a Step (that isn't unfold) and
would like to give an opportunity for someone to voice their concerns about
this.
Let's take a look at one of the examples from Proposal 3.
List Example 1 (LE1)
Given a list of people, return the list of ages if everyone’s age > 18
g.V().hasLabel('person').values('age').fold().where(all(gt(18)))
Let's assume the proposal should have included this usage for all() that
takes a predicate as a parameter. If we remove the fold() from the above
example so that the example becomes
g.V().hasLabel('person').values('age').where(all(gt(18)))
If all() were to behave like a global scope step here then it would be
pretty meaningless as the incoming traverser is not a list type. In fact
the three proposed Steps that return boolean (all, any, none) shouldn't be
used unless the incoming traverser is an iterable type. In addition, the
set operations (intersect, union, disjunct and difference) also require the
incoming traverser to be a list/array type for them to have any sort of
meaning. I think it's reasonable for the default behavior of all the
proposed list functions to be the local scope versions. The concat() string
function has already set a precedent of not taking Scope as a parameter as
it currently only supports the global scope. In this case, for list
functions, we would only support local scope. If the user needs global
scope functionality then they would just need to add a fold() to their
query.
Does anyone have objections to allowing this to become the default behavior
for these Steps?
Thanks,
Ken
Re: [DISCUSS] Is null equal to null
Hi Josh, Thanks for your input. There seems to be a push in the graph database world towards having a schema. It's likely something like this would be introduced in TinkerPop in the future. Let's assume that TinkerPop does support schemas, and therefore would have a type system, would this change your opinion on the matter? Thanks again, Ken On Wed, Aug 2, 2023 at 3:54 PM Joshua Shinavier wrote: > For what it is worth, I think the question of whether null == null is only > meaningful in the context of a specific type system, which Gremlin so far > does not provide. My personal preference is to avoid SQL-style nulls and > achieve optionality through union types (e.g. Java's Optional or Haskell's > Maybe). In the case of two lists, if you can assume that the type of the > list is list>, then you can safely treat null like > Optional.empty(), and compare it with another null of the same logical type > (int). If that is the interpretation of your two lists, then the > intersection is [1, null]. > > Josh > > > > On Tue, Aug 1, 2023 at 5:47 PM Ken Hu > wrote: > > > Hi All, > > > > As Gremlin evolves and gains more functionality, it is important that we > > establish some fundamental rules to provide consistency in results. One > > such question that we should come to agreement on is how null values are > > compared. Currently, Gremlin seems to mostly follow the comparison that > is > > used in Java where NULL == NULL returns TRUE. However, in many other > > database systems, NULL == NULL would return FALSE (or NULL). > > > > This question comes about as I'm starting to look a little deeper into > the > > proposed list functions. An example of where this is applicable is the > > INTERSECT list function. For example, if you have a list, A, which is > > [1,2,null] and a list, B, which is [1,null]. Should the result of an > > INTERSECT be [1,null] or [1]? > > > > I think it makes sense in Gremlin for us to follow the rule that most > > programming languages follow which is the former (NULL == NULL returns > > TRUE) because it feels more in line with how Gremlin was meant to be used > > (together with your code rather than as a string query). In this case the > > return value would be [1,null]. > > > > What are your thoughts on this subject? > > > > Thanks, > > Ken > > >
[DISCUSS] Is null equal to null
Hi All, As Gremlin evolves and gains more functionality, it is important that we establish some fundamental rules to provide consistency in results. One such question that we should come to agreement on is how null values are compared. Currently, Gremlin seems to mostly follow the comparison that is used in Java where NULL == NULL returns TRUE. However, in many other database systems, NULL == NULL would return FALSE (or NULL). This question comes about as I'm starting to look a little deeper into the proposed list functions. An example of where this is applicable is the INTERSECT list function. For example, if you have a list, A, which is [1,2,null] and a list, B, which is [1,null]. Should the result of an INTERSECT be [1,null] or [1]? I think it makes sense in Gremlin for us to follow the rule that most programming languages follow which is the former (NULL == NULL returns TRUE) because it feels more in line with how Gremlin was meant to be used (together with your code rather than as a string query). In this case the return value would be [1,null]. What are your thoughts on this subject? Thanks, Ken
[jira] [Closed] (TINKERPOP-2871) Borrowed count can be wildly incorrect
[ https://issues.apache.org/jira/browse/TINKERPOP-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu closed TINKERPOP-2871. - Resolution: Duplicate > Borrowed count can be wildly incorrect > -- > > Key: TINKERPOP-2871 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2871 > Project: TinkerPop > Issue Type: Bug > Components: driver >Affects Versions: 3.5.5 >Reporter: Stephen Mallette >Priority: Critical > Fix For: 3.6.0 > > > It doesn't seem like it affects the algorithm for borrowing/closing > connections in the pool but the borrowed counter can sometimes reach wildly > large numbers. it shows up in the log and is confusing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TINKERPOP-2871) Borrowed count can be wildly incorrect
[ https://issues.apache.org/jira/browse/TINKERPOP-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu updated TINKERPOP-2871: -- Fix Version/s: 3.6.0 > Borrowed count can be wildly incorrect > -- > > Key: TINKERPOP-2871 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2871 > Project: TinkerPop > Issue Type: Bug > Components: driver >Affects Versions: 3.5.5 >Reporter: Stephen Mallette >Priority: Critical > Fix For: 3.6.0 > > > It doesn't seem like it affects the algorithm for borrowing/closing > connections in the pool but the borrowed counter can sometimes reach wildly > large numbers. it shows up in the log and is confusing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2871) Borrowed count can be wildly incorrect
[
https://issues.apache.org/jira/browse/TINKERPOP-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747215#comment-17747215
]
Ken Hu commented on TINKERPOP-2871:
---
Short reproducer:
{code:java}
public static void main(String[] args) {
ExecutorService jobQueue = Executors.newFixedThreadPool(500); // (1)
Cluster cluster = Cluster.build("localhost")
.maxInProcessPerConnection(8)
.maxSimultaneousUsagePerConnection(8) // (2)
.create();
Client.ClusteredClient client = cluster.connect(); for (int i=0; i<1000;
i++) {
jobQueue.submit( () -> {
try {
client.submit("Thread.sleep(2000)");
} catch (Exception e) {
;
}
});
}
} {code}
Running this code for 3.5-dev, you can see entries like "borrowed=486
pending=486", but in 3.6-dev, the highest observed value will be "borrowed=8
pending=0". In 3.5-dev, you can "borrow" a connection more than what is
specified by "maxSimultaneousUsagePerConnection" which isn't supposed to
happen. If you look at (2) in the example, you can see that the maximum should
be 8. This borrowed count is affected by what is set in (1) as a larger
threadpool will allow the count to go higher. I suspect the borrow count isn't
actually incorrect, rather in 3.5-dev, you can borrow a connection many more
times than you should be able to.
This behavior was fixed in TINKERPOP-2486 by
https://github.com/apache/tinkerpop/pull/1465. Seeing as this has already been
fixed in 3.6-dev, we can probably just close off this ticket as we previously
decided not to backport this change to 3.5-dev.
> Borrowed count can be wildly incorrect
> --
>
> Key: TINKERPOP-2871
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2871
> Project: TinkerPop
> Issue Type: Bug
> Components: driver
>Affects Versions: 3.5.5
>Reporter: Stephen Mallette
>Priority: Critical
>
> It doesn't seem like it affects the algorithm for borrowing/closing
> connections in the pool but the borrowed counter can sometimes reach wildly
> large numbers. it shows up in the log and is confusing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (TINKERPOP-2953) Static import for __.values() overriden by Column.values()
[
https://issues.apache.org/jira/browse/TINKERPOP-2953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu closed TINKERPOP-2953.
-
Resolution: Fixed
> Static import for __.values() overriden by Column.values()
> --
>
> Key: TINKERPOP-2953
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2953
> Project: TinkerPop
> Issue Type: Bug
> Components: groovy
>Affects Versions: 3.6.4
>Reporter: Stephen Mallette
>Assignee: Ken Hu
>Priority: Blocker
> Fix For: 3.7.0, 3.5.7, 3.6.5
>
>
> this has always been a problem:
> {code}
> gremlin> :> values()
> ==>keys
> ==>values
> {code}
> but now after latest groovy upgrade in 3.6.3 we also get:
> {code}
> gremlin> :> values("a")
> No signature of method:
> org.apache.tinkerpop.gremlin.structure.Column$2.call() is applicable for
> argument types: (String) values: [a]
> Possible solutions: wait(), name(), any(), wait(long),
> tap(groovy.lang.Closure), each(groovy.lang.Closure)
> Type ':help' or ':h' for help.
> Display stack trace? [yN]
> {code}
> Recommended workaround would be to prefer use of the explicit {{__}} or
> {{Column}} when there is potential for naming conflicts when using Groovy.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (TINKERPOP-2953) Static import for __.values() overriden by Column.values()
[
https://issues.apache.org/jira/browse/TINKERPOP-2953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ken Hu updated TINKERPOP-2953:
--
Fix Version/s: 3.7.0
3.5.7
3.6.5
> Static import for __.values() overriden by Column.values()
> --
>
> Key: TINKERPOP-2953
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2953
> Project: TinkerPop
> Issue Type: Bug
> Components: groovy
>Affects Versions: 3.6.4
>Reporter: Stephen Mallette
>Assignee: Ken Hu
>Priority: Blocker
> Fix For: 3.7.0, 3.5.7, 3.6.5
>
>
> this has always been a problem:
> {code}
> gremlin> :> values()
> ==>keys
> ==>values
> {code}
> but now after latest groovy upgrade in 3.6.3 we also get:
> {code}
> gremlin> :> values("a")
> No signature of method:
> org.apache.tinkerpop.gremlin.structure.Column$2.call() is applicable for
> argument types: (String) values: [a]
> Possible solutions: wait(), name(), any(), wait(long),
> tap(groovy.lang.Closure), each(groovy.lang.Closure)
> Type ':help' or ':h' for help.
> Display stack trace? [yN]
> {code}
> Recommended workaround would be to prefer use of the explicit {{__}} or
> {{Column}} when there is potential for naming conflicts when using Groovy.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[DISCUSS] Handling of mismatched types as inputs to Predicates
Hi All, With regard to https://issues.apache.org/jira/browse/TINKERPOP-2375 This issue opens up the conversation of what the behavior should be in TinkerPop when there are comparisons of two objects with different types. There are probably three simple approaches that can be taken. 1. Just throw an exception when this happens. 2. Try to cast to the wider type and compare. If there are no compatible types then throw an exception. 3. Treat two objects of different types as being unequal. We currently take approach 1 in 3.5.x and a mix of approach 1/3 in 3.6.x and later. The introduction of ternary boolean starting in 3.6.x causes certain predicates to filter with a variation of approach 3 and others to throw like in approach 1. I would argue that it makes the most sense to extend approach 3 to the rest of the predicates as this seems to be the approach we've been taking recently to similar issues around filtering. Any comments on this subject would be appreciated. Thanks, Ken
Re: [DISCUSS] Relax strictness of Step classes for better extensibility
Hi All, Recently, I've been thinking about this again and decided to quickly look at some statistics of final/non-final Step classes. After a quick search of the concrete Steps, the split is roughly 118 non-final Steps and 95 final Steps. Given that more than half the Steps have already been opened, it makes sense to allow the rest of them to become non-final. I mostly agree with the three step approach from above but I wouldn't remove final from every Step in a single PR as suggested in 1. I think it still makes sense to do it on a case by case basis, however, the requirement for removing final should be very low (that is almost any reason can be given for wanting to remove final). That being said, I think this is true only for TinkerPop 3. As we move forward, I think it makes more sense for us to revisit how Steps have been implemented. Recently, a lot of the modifications have been made for the sake of performance. If we could change the way we execute the steps to allow for the optimizations in TinkerPop itself then maybe in a future version we could still keep the Steps as being final. In summary, I'm suggesting that we approve most PRs that are requesting to remove final from a Step as long as they follow some variation of the three step approach that Oleksandr suggested before. Thanks, Ken On Thu, Apr 20, 2023 at 9:52 AM Ken Hu wrote: > I agree that there are some inconsistencies that we will need to work > through. I've taken a quick look at your three PRs and they seem reasonable > to me. I will officially review them soon. > > On Thu, Apr 20, 2023 at 9:28 AM Oleksandr Porunov < > alexandr.poru...@gmail.com> wrote: > >> Hi Ken, >> >> This is a great input. Thank you for the feedback. I believe you are right >> and we should check each case separately. >> That said, I was just worried if Graph Providers start blindly replacing >> steps with optimised versions of those steps then it becomes hard to track >> how it affects TinkerPop side optimization strategies and custom Steps. >> For example, let's take a look at any random optimization strategy on >> TinkerPop's side. For example EarlyLimitStrategy >> < >> https://github.com/apache/tinkerpop/blob/3.6-dev/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/optimization/EarlyLimitStrategy.java >> > >> . >> This strategy specifically has specific logic regarding usage of >> `RangeGlobalStep`. >> I.e. `if (step instanceof RangeGlobalStep)`. >> If we check this step it's `final` (`public final class RangeGlobalStep`). >> In such a case nobody could actually easily replace this strategy without >> affecting `EarlyLimitStrategy` because we can't extend / overwrite >> RangeGlobalStep due to being final. >> Moreover `if (step instanceof RangeGlobalStep)` could actually be replaced >> by `if (RangeGlobalStep.class.equals(step.getClass()))` because we know >> there are no instances of `RangeGlobalStep` other than `RangeGlobalStep` >> itself, but the strategy tells that any instance of `RangeGlobalStep` is >> valid for optimization. >> >> The above relation of Optimizations and Steps affects many Step classes >> and >> many optimization strategies. Moreover it's kind of inconsistent that some >> of the steps are `final` and some of them are non-final even without >> having >> any other classes extending them in the TinkerPop codebase. For example, >> PropertiesStep and PropertyMapStep are non-final, but they don't have any >> classes extending them inside TinkerPop codebase. Logically Graph >> Providers >> extend and replace those steps, but then why do we keep those steps open >> if >> their codebase is bigger and more likely to be changed then the other >> `final` steps which are more likely to stay as is? (i.e. I would expect >> PropertyMap step to be changed more often then the IsStep for example, >> even >> so the latter one is non-final). >> >> I do agree with you that we should probably be curious of any methods / >> fields which we open with `protected` and don't blindly replace `private` >> fields / methods to `protected` fields / methods (as I did in the PR), but >> I doubt any Step should be `final`. >> Taking your input into account I think the way we could do, is: >> 1. Open all Step classes for extension by removing `final`. (do this in a >> single PR for all steps) >> 2. Move all `private` / `protected` utility methods out from Step classes >> into some utility classes. (do this case by case depending on the step) >> 3. Add public getter methods and / or public constructors to be able to >>
Re: [DISCUSS] Proposal to add transaction support to TinkerGraph.
Hi All, I'm going to be running some basic performance testing against this new TinkerGraph transaction implementation, the current TinkerGraph and the Neo4jGraph. The goal would be to ensure that the differences aren't too large so that users would be more willing to use the new TinkerGraph with transaction support. Below is my current proposed list of tests to run. Note that the data used is Kelvin's air-routes-latest. Read tests: 1. Test a query that traverses through the entire graph multiple times. Create tests: 1. Test creating a graph with power law distribution. Update tests: 1. Test randomly moving edges. 2. Test randomly updates properties on different elements. 3. Test updating properties on central elements. 4. Test reading and updating a property on a random element. 5. Test reading and updating a property on a central element. 6. Test upserting elements (MergeE/MergeV). Delete tests: 1. Test deleting random edges. 2. Test deleting random vertices. 3. Test deleting central vertices. There will also be testing for rollback. Please let me know if you think there are any important tests that have been missed. Thanks, Ken On Thu, Jun 15, 2023 at 10:18 PM Valentyn Kahamlyk wrote: > Hi all, > > I'm working on transactions implementation and I would like to discuss > testing of this feature. > > Currently available options for tests in TinkerPop: > 1. Unit tests. > For development I wrote bunch of unit tests for the newly introduced > TinkerTransactionGraph because they are easier to make and much more > convenient to debug. However, these tests are very complicated to be > reused by other providers. > 2. Gherkin tests. > This is the best option for providers to test some features. However, > the current implementation does not support many of the abilities > needed to test transactions, for example multi-step tests and > multi-threaded tests. The ideal solution would be to add such > features, but unfortunately it takes a lot of time and effort. > For basic checks of TinkerTransactionGraph existing Gherkin tests are > already used. > 3. Integration tests. > We have 2 kinds of integration tests in `gremlin-test` > a) process tests are mirror of Gherkin tests with some small differences. > b) structure tests are probably the best candidate to use for > TinkerTransactionGraph. There is already a set of tests for > transactions in TransactionTest and I can add more cases there. These > tests will be used in a similar way to > TinkerGraphStructureStandardTest. > 4. Gremlin Server tests. > I think these tests have a different responsibility. > 5. GLV tests. > All GLV's already have transaction tests and we can extend them to use > TinkerTransactionGraph at server-side. > > To summarize I suggest using structure tests to verify transaction > implementation. > > Regards, Valentyn > > > On Wed, May 10, 2023 at 2:59 PM Valentyn Kahamlyk < > valent...@bitquilltech.com> wrote: > > > Hi all, > > > > I opened PR with a detailed proposal for transaction implementation: > > https://github.com/apache/tinkerpop/pull/2054. > > > > Regards, Valentyn > > > > On Mon, Feb 27, 2023 at 1:22 PM Valentyn Kahamlyk < > > valent...@bitquilltech.com> wrote: > > > >> Hi all, > >> > >> Now, if you need to use transactions in tinkerpop, the only solution is > >> to use the Neo4J plugin. Unfortunately, this plugin has not been updated > >> for a long time and is only compatible with Neo4J version 3.4, which > >> reached end of life in March, 2020. > >> > >> I propose to implement native transaction support in TinkerGraph. > >> > >> The most interesting question to discuss is the level of isolation in > SQL > >> 92 terms. There are 5 options to choose from: `Read uncommitted`, `Read > >> committed`, `Repeatable read`, `Snapshot` and `Serializable`. > >> > >> In terms of balancing complexity of implementation and functionality, I > >> think the most optimal option is `Read committed` or `Repeatable read`. > >> > >> I hope to implement `Read committed`, it is enough to add versions to > the > >> Vertices and Edges, add list of updated Elements in Graph. > >> Write lock can be Vertex-level and Edge-level. > >> > >> Regards, Valentyn > >> > > >
[jira] [Commented] (TINKERPOP-2871) Borrowed count can be wildly incorrect
[ https://issues.apache.org/jira/browse/TINKERPOP-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733223#comment-17733223 ] Ken Hu commented on TINKERPOP-2871: --- I believe there were some changes to the driver in 3.6 that reduces the impact of this issue for that branch. I will have to do some testing to confirm this. > Borrowed count can be wildly incorrect > -- > > Key: TINKERPOP-2871 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2871 > Project: TinkerPop > Issue Type: Bug > Components: driver >Affects Versions: 3.5.5 >Reporter: Stephen Mallette >Priority: Critical > > It doesn't seem like it affects the algorithm for borrowing/closing > connections in the pool but the borrowed counter can sometimes reach wildly > large numbers. it shows up in the log and is confusing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (TINKERPOP-2677) Upgrade to Groovy 3.x to fix XStream security vulnerability
[ https://issues.apache.org/jira/browse/TINKERPOP-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu closed TINKERPOP-2677. - Resolution: Fixed > Upgrade to Groovy 3.x to fix XStream security vulnerability > --- > > Key: TINKERPOP-2677 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2677 > Project: TinkerPop > Issue Type: Bug > Components: groovy >Affects Versions: 3.6.0, 3.5.2 >Reporter: Divij Vaidya >Priority: Major > Fix For: 3.7.0 > > > XStream has a number of documented vulnerabilities as specified in > [https://x-stream.github.io/security.html] which are fixed in 1.4.18. Note > that 1.4.18 is not backport compatible since it uses a new whitelisting > approach for serialization. > TinkerPop has a dependency on XStream via: [1] > TinkerPop -> Groovy 2.5.x -> XStream 1.4.10 > However, Groovy 2.5.x series does not consume the version of XStream (1.4.18) > which contains the fixes for the vulnerabilities [2] but Groovy 3.x uses > XStream (1.4.18) which has the fixes for vulnerabilities. > Hence, either we convince the Groovy project to backport the vulnerability > fixes to 2.5.x series or we upgrade Groovy to 3.x for TinkerPop. > IMO, upgrading TP to use Groovy 3.x might be much easier. > [1] https://github.com/apache/tinkerpop/blob/master/pom.xml#L162 > [2]https://github.com/apache/groovy/blob/GROOVY_2_5_X/build.gradle#L165 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (TINKERPOP-2677) Upgrade to Groovy 3.x to fix XStream security vulnerability
[ https://issues.apache.org/jira/browse/TINKERPOP-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu updated TINKERPOP-2677: -- Fix Version/s: 3.7.0 > Upgrade to Groovy 3.x to fix XStream security vulnerability > --- > > Key: TINKERPOP-2677 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2677 > Project: TinkerPop > Issue Type: Bug > Components: groovy >Affects Versions: 3.6.0, 3.5.2 >Reporter: Divij Vaidya >Priority: Major > Fix For: 3.7.0 > > > XStream has a number of documented vulnerabilities as specified in > [https://x-stream.github.io/security.html] which are fixed in 1.4.18. Note > that 1.4.18 is not backport compatible since it uses a new whitelisting > approach for serialization. > TinkerPop has a dependency on XStream via: [1] > TinkerPop -> Groovy 2.5.x -> XStream 1.4.10 > However, Groovy 2.5.x series does not consume the version of XStream (1.4.18) > which contains the fixes for the vulnerabilities [2] but Groovy 3.x uses > XStream (1.4.18) which has the fixes for vulnerabilities. > Hence, either we convince the Groovy project to backport the vulnerability > fixes to 2.5.x series or we upgrade Groovy to 3.x for TinkerPop. > IMO, upgrading TP to use Groovy 3.x might be much easier. > [1] https://github.com/apache/tinkerpop/blob/master/pom.xml#L162 > [2]https://github.com/apache/groovy/blob/GROOVY_2_5_X/build.gradle#L165 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (TINKERPOP-2633) Support Gremlin Console on Java 17
[ https://issues.apache.org/jira/browse/TINKERPOP-2633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu closed TINKERPOP-2633. - Resolution: Fixed > Support Gremlin Console on Java 17 > -- > > Key: TINKERPOP-2633 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2633 > Project: TinkerPop > Issue Type: Improvement > Components: console >Affects Versions: 3.5.1 > Environment: Java 17 >Reporter: Oleksandr Porunov >Priority: Minor > Fix For: 3.7.0 > > > Currently Gremlin Console cannot be started on Java 17 following the next > error: > java.lang.IllegalArgumentException: Unsupported class file major version 61 > > It would be great if Gremlin Console supports Java 17 as it is the newest LTS > release after Java 11 and most likely people will start switching to Java 17 > sooner or later. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (TINKERPOP-2731) Bump to Spark 3.3.0
[ https://issues.apache.org/jira/browse/TINKERPOP-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Hu closed TINKERPOP-2731. - Assignee: Ken Hu Resolution: Fixed > Bump to Spark 3.3.0 > --- > > Key: TINKERPOP-2731 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2731 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop >Affects Versions: 3.6.0 >Reporter: Stephen Mallette >Assignee: Ken Hu >Priority: Minor > Fix For: 3.7.0 > > > Spark 3.3.0 allows support of JDK17 - SPARK-33772 -- This message was sent by Atlassian Jira (v8.20.10#820010)
