thanks for creating the issue in jira. i assigned myself to it to help manage it through the process.
> For #3, Do we need to take this in this work item as well ? I believe that we should handle 3a as part of this - 3b is something else and could be removed from the description for now. As for the branch............I just pushed the TINKERPOP-1919 branch: https://github.com/apache/tinkerpop/tree/TINKERPOP-1913 This was a really fast and easy first stab at this to just prove out the ideas that included: 1. a change to the server to put the "host" in the status attributes (we'll see what we ultimately want to return here as part of 3a in JIRA, but i figured that would work for now). the server changes introduced some deprecation, but it should be backward compatible 2. a change to the java driver that makes the status attributes available to the caller on the ResultSet object 3. an initial attempt at returning status attributes over a remote Traversal - i made it accessible through side-effects. not sure if this should stay that way. we've talked about having "metadata" on a traversal before. perhaps it would be best exposed that way more generally as a first class citizen, but that would entail bigger changes that might better be reserved for 3.4.0 - not sure just what we do with "remote traversals" (i.e. item 2 in JIRA) yet..... Definitely happy we're not doing this on 3.2.x. I'm not so sure I even like it on 3.3.x given what I've done so far. Depending on how things develop, we might want to push this forward to 3.4.x - i think it could be more cleanly implemented there without having to worry so hard about deprecation and other things that are binding our hands a bit. There's no target date for 3.4.x yet though I would guess some time in the summer. On Tue, Mar 6, 2018 at 5:11 PM, Ashwini Singh < ashws...@microsoft.com.invalid> wrote: > Create a incident: https://issues.apache.org/jira/browse/TINKERPOP-1913 > > For this, I am fine with that. Lets take this as part of above work item. > > For #3, Do we need to take this in this work item as well ? > > Can someone setup the branch? And we can take it from there. > > Thanks a lot guys, > Ashwini Singh > > Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows 10 > > From: Stephen Mallette<mailto:spmalle...@gmail.com> > Sent: Tuesday, March 6, 2018 3:53 AM > To: dev@tinkerpop.apache.org<mailto:dev@tinkerpop.apache.org> > Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. > > Your summary looks good to me. I also agree that we can omit 3b from this > body of work. Going back to an earlier question that you had: > > > Can we spilt the change to support for gremlin request/script first and > bytecode/traversal separately? > > I'm a bit concerned that if we don't at least have the design of the > bytecode/traversal piece thought through fully we might not get the client > side changes right to support it. Perhaps we see how things go on > implementation before making a final decision here, but as of right now, I > tend to feel like we shouldn't split this up. > > To just get started, I think I'd like to get the initial development branch > setup with Gremlin Server returning some metadata. That part is pretty > easy. Once that branch is setup, anyone contributing can just submit PRs > against that or just commit directly to it. Then we can have one PR to > evaluate through the review process for this feature > > I guess there is some question as to what version this body of work should > go to. I think I'd like to skip adding this to 3.2.8 and target 3.3.2 on > the tp33 branch. At this point, I think 3.2.x should really just contain > bug fixes and minor enhancements - I don't think this particular > enhancement qualifies as "minor". > > Ashwini, if all that sounds good, could you please create a JIRA ticket > that contains a final summary and references this thread? > > > > > > On Mon, Mar 5, 2018 at 2:47 PM, Ashwini Singh < > ashws...@microsoft.com.invalid> wrote: > > > > > To summarize what we have discussed so far: > > 1. For API using GremlinRequest/QueryScript, expose response > > attribute as part of result. Using an approach to similar to Java client > > driver (using ResultSet) . [Priority0] > > -- We rely on the last message for response attributes. > > 2. For GLV, add response attribute as part of Traversal. > [Priority > > 0] > > --Rely on the last message for attributes. > > 3. Expose other server details (like server setting). I would > > suggest to split this design discussion into two directions: > > a. Metadata for request execution: Only focuses on > details > > related to request execution. Can be achieved through #1 and #2. > > b. Metadata for Gremlin Server: Focuses on overall > > metadata for the server. Could be a separate request and fetch the data > > once for a connection. Any other suggestion ? > > > > Please add if I am missing something. > > > > Thanks a lot, > > Ashwini Singh > > Software Engineer @ Azure Cosmos DB > > > > -----Original Message----- > > From: Stephen Mallette <spmalle...@gmail.com> > > Sent: Friday, March 2, 2018 4:49 AM > > To: dev@tinkerpop.apache.org > > Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. > > > > Perhaps another useful piece of data to return from Gremlin Server: > > > > https://na01.safelinks.protection.outlook.com/?url= > > https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse% > > 2FTINKERPOP-1636&data=04%7C01%7Cashwsing%40microsoft.com% > > 7C621c0a838bd24f13eb4e08d5803bed5d%7C72f988bf86f141af91ab2d7cd011 > > db47%7C1%7C0%7C636555917242398527%7CUnknown% > 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw > > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata= > > bVd71RV9uDiWWgsjY7SYAIYEhWtLdi7Ijgv9um%2F1JoM%3D&reserved=0 > > > > On Thu, Mar 1, 2018 at 6:31 PM, Stephen Mallette <spmalle...@gmail.com> > > wrote: > > > > > I just came across this: > > > > > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissue > > > s.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1494&data=04%7C01%7Cashwsing% > > > 40microsoft.com%7C621c0a838bd24f13eb4e08d5803bed5d%7C72f988bf86f141af9 > > > 1ab2d7cd011db47%7C1%7C0%7C636555917242398527%7CUnknown%7CTWFpbGZsb3d8e > > > yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata > > > =1kY8r73w2Y3euQVnLEGIuAO0fX6ZySOzl%2BVUgbhodpE%3D&reserved=0 > > > > > > There's some ideas for what the open source Gremlin Server could > > > return there....for example, perhaps we send back the the host that > > > executed the script as the metadata for Gremlin Server? > > > > > > On Wed, Feb 28, 2018 at 5:37 PM, Ashwini Singh <ashws...@microsoft.com > . > > > invalid> wrote: > > > > > >> > > >> Completely agree on making the change for other drivers too. The plan > > >> from our side is to make changes to other drivers as well. My focus > > >> on Gremlin.Net was just for an example *. Yes, we should make change > > >> to Gremlin Server to write the metadata for testing. > > >> > > >> I agree to adding this to ResultSet and exposing this as part of > > >> every response for all the drivers and follow the same pattern as > JAVA. > > >> > > >> On the gremlin GVL part, I agree we should support this on traversal > > >> for bytecode as well. For now, we do not have support for bytecode > > >> but we are actively working on that and need support for metadata > there > > as well. > > >> Thanks for bringing this up. Just wanted to understand how we track > > >> issues at tinkerpop, Can we spilt the change to support for gremlin > > >> request/script first and bytecode/traversal separately? If we prefer > > >> to split the change in chunks, but completely fine with one change. > > >> > > >> Thanks, > > >> Ashwini Singh > > >> Software Engineer @ Azure Cosmos DB > > >> > > >> -----Original Message----- > > >> From: Stephen Mallette <spmalle...@gmail.com> > > >> Sent: Wednesday, February 28, 2018 9:58 AM > > >> To: dev@tinkerpop.apache.org > > >> Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. > > >> > > >> This newly created issue might be related to this in some way: > > >> > > >> https://na01.safelinks.protection.outlook.com/?url=https%3A%< > https://na01.safelinks.protection.outlook.com/?url=https%3A%25> > > >> 2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1906& > > >> data=04%7C01%7Cashwsing%40microsoft.com%7C614b32ff7f98 > > >> 4c66554808d57ed4d22b%7C72f988bf86f141af91ab2d7cd011db47%7C1% > > >> 7C0%7C636554374885412678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4 > > >> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1& > > >> sdata=w7pyg6JF4KMbPdpBg6t%2F8cE%2BflbnCQQdyMUgc6HDrXw%3D&reserved=0 > > >> > > >> On Wed, Feb 28, 2018 at 11:37 AM, Florian Hockmann < > > >> f...@florian-hockmann.de> > > >> wrote: > > >> > > >> > I agree that we also should make this metadata available for > > >> > traversals since we want to move users away from sending Gremlin > > >> > scripts as strings and instead use Bytecode based GLVs. > > >> > > > >> > Regarding Gremlin.Net: I think that the implementation would be > > >> > very similar to how it can be implemented in the Java driver as we > > >> > tried to stay close to the Java driver in general. The only > > >> > difference is probably that we currently don't have a ResultSet in > > >> > Gremlin.Net, but that's only because I didn't see much value in > > >> > adding that. Metadata would of course be a good argument to also > > >> > implement a ResultSet in Gremlin.Net and then the implementation > > >> > should be really basically the same as in the Java driver. > > >> > > > >> > > > >> > Am 28.02.2018 um 16:15 schrieb Stephen Mallette: > > >> > > I'm fine with using the last response message as the carrier for > > >> > > this metadata on a particular request. I can't really tell if > > >> > > there is much > > >> > work > > >> > > to do on Gremlin Server itself here. It seems like most of the > > >> > > work must occur on the various drivers (you mention the .NET api, > > >> > > but all of the drivers would need to support this feature). > > >> > > However, I would think that > > >> > we > > >> > > would want Gremlin Server itself to append in some kind metadata > > >> > > (maybe query time? something easy....) so that we could write > > >> > > tests for the drivers in TinkerPop itself. There is also the > > >> > > question of how we would expose this metadata to GLVs which don't > > >> > > see response messages at all. A traversal might need some > > >> > > metadata itself so that the user could retrieve the server > > >> > > metadata from that. The implementation between Java and the > > >> > GLVs > > >> > > might be different here as the GLV traversal class is typically > > >> > > quite lightweight and only used for generating bytecode. > > >> > > > > >> > > I'm not so sure I like the SubmitAsynWithHeaders() but I don't > > >> > > think > > >> > too > > >> > > much about how the .NET driver works. Is there a reason to not > > >> > > always return metadata? could it be expensive to do so? If we > > >> > > just added an > > >> > extra > > >> > > method how would remote traversals configure this option? I think > > >> > > we need another way. Generally speaking, for Java, I think I > > >> > > would like to see > > >> > the > > >> > > metadata available to the ResultSet somehow which would in turn > > >> > > make it pretty easy to get it on to a Traversal instance once > > >> > > that facility was made available.....but as to how to enable or > > >> > > disable the return of the metadata, i'm not sure how that should > > >> > > work just yet - i need to think on that some more. > > >> > > > > >> > > For committers who work on GLVs, please take a look at this > > >> > > thread and offer your thoughts on how this might work in the GLV > > >> > > driver you tend to have the most knowledge on. Let's see if we > > >> > > can come to one nice unified solution. At that point, we can > > >> > > setup a ticket in JIRA > > >> and go from there. > > >> > > > > >> > > Ashwini, thanks for offering a pull request for this by the way. > > >> > > Once we get consensus on how to do this, we'll see if tasks need > > >> > > to be divided > > >> > and > > >> > > how you might contribute. > > >> > > > > >> > > > > >> > > > > >> > > On Mon, Feb 26, 2018 at 6:45 PM, Ashwini Singh < > > >> > > ashws...@microsoft.com.invalid> wrote: > > >> > > > > >> > >> Hi Stephen, > > >> > >> > > >> > >> Thanks for considering the change. > > >> > >> > > >> > >> We would be more inclined towards the first approach since the > > >> > >> having a ping/pong websocket message can be a bit noisy and > > >> > >> requires > > >> > sophisticated > > >> > >> handling on the client driver side. > > >> > >> > > >> > >> For handling multiple response messages. I would suggest to rely > > >> > >> on last message as these are the metadata for request execution. > > >> > >> Partial > > >> > response > > >> > >> is very internal to the client drivers (based on limited > > >> > >> understanding > > >> > of > > >> > >> tinkerpop client drivers :) , correct me if you differ) and can > > >> > >> be > > >> > exposed > > >> > >> separately (if really needed later). > > >> > >> > > >> > >> For implementation, Let us know if we can chip in there and > > >> > >> submit > > >> PR. > > >> > The > > >> > >> high level approach to achieve this is to have corresponding > > >> > >> SubmitAsynWithHeaders() for every SubmitAsync() that returns a > > >> > >> encapsulated result with attributes and IReadOnlyCollectio<T>. > > >> > >> Let me > > >> > know > > >> > >> if you see any concerns adding a new API. > > >> > >> > > >> > >> Thanks a lot, > > >> > >> Ashwini Singh > > >> > >> Software Engineer @ Azure Cosmos DB > > >> > >> > > >> > >> > > >> > >> > > >> > >> -----Original Message----- > > >> > >> From: Stephen Mallette <spmalle...@gmail.com> > > >> > >> Sent: Friday, February 23, 2018 5:13 PM > > >> > >> To: dev@tinkerpop.apache.org > > >> > >> Subject: Re: [Discuss] Expose metadata from Gremlin Server to > > >> Clients. > > >> > >> > > >> > >> Adding those kinds of details was the reason we had the > > >> > >> ResponseStatus.attributes Map. I can really only speak for the > > >> > >> java > > >> > driver > > >> > >> as I only know that one really well (we might need other > > >> > >> TinkerPop > > >> > experts > > >> > >> to chime in for python, .net and c#). The java driver doesn't > > >> > >> really present ways to get that information easily under usage > > >> > >> that doesn't > > >> > deal > > >> > >> directly with RequestMessage directly (which people typically > > >> > >> don't > > >> do). > > >> > >> Another thing to think about is that since a single request > > >> > >> might return multiple ResponseMessage instances you might not > > >> > >> want to return that > > >> > kind > > >> > >> of data on every response - maybe just to be returned on the > > >> > >> first (or last > > >> > >> message) and then we somehow preserve that information and make > > >> > >> it accessible on the result somehow....we sorta have some kinda > > >> > >> of > > >> > precedent > > >> > >> for that with side-effect data generated by bytecode based > > >> > >> traversals - > > >> > we > > >> > >> can probably build in something similar for this sort of thing. > > >> > >> > > >> > >> I also toyed with the idea of using ping/pong websocket messages > > >> > >> to > > >> > carry > > >> > >> general information about the server to the client. Not sure if > > >> > >> any of > > >> > the > > >> > >> metadata you want to send back would fit in there, but that > > >> > >> could be another option. > > >> > >> > > >> > >> Does any of that sound helpful? > > >> > >> > > >> > >> On Fri, Feb 23, 2018 at 3:41 PM, Ashwini Singh < > > >> > >> ashws...@microsoft.com.invalid> wrote: > > >> > >> > > >> > >>> Hi All, > > >> > >>> > > >> > >>> We are working on to expose metadata as part of gremlin to > > >> > >>> response to client. The metadata is simply a property bag to > > >> > >>> provide special message/hints to the client. But currently > > >> > >>> client libraries strip off everything and only return the data > > >> > >>> to the > > >> client. > > >> > >>> > > >> > >>> Specifically, We want to expose details like Request Charge, > > >> > >>> Rate limiting/Retry policy details etc. In the other scenarios > > >> > >>> in Cosmos DB we provide these details to the client is through > > >> > >>> response headers. We did some investigation around this and one > > >> > >>> of the options is expose these is through response attributes. > > >> > >>> Gremlin Server can add metadata as part of gremlin response > > >> > >>> attributes (For example, set the property bag on > > >> > >>> ResponseStatus.Attributes for Gremlin.Net) that can be > > >> > >>> serialized > > >> by the client drivers to the clients. > > >> > >>> > > >> > >>> We would like to learn more if there are precedence around > > >> > >>> this and if there are any recommended ways to achieve this in > > >> > >>> Gremlin protocol and client drivers. > > >> > >>> > > >> > >>> Thanks a lot, > > >> > >>> Ashwini Singh > > >> > >>> Software Engineer @ Azure Cosmos DB > > >> > >>> > > >> > >>> > > >> > > > >> > > > >> > > > >> > > > > > > > > > >
