Good day. As I do not see any objections. My next step would be to create a proposal for the TP 4.0 dos.
On Thu, Dec 11, 2025 at 9:02 AM Andrii Lomakin <[email protected]> wrote: > Unfortunately, all images have been removed, so I am sending a link to the > diagrams instead > https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384 > . > > > > On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin < > [email protected]> wrote: > >> Good day. >> >> If I followed Ken's line of thought correctly, I would like to expand it >> to be part of the new proposal for the transaction API. >> First of all, about the model, I think we came to a consensus that it >> should be: >> >> [image: image.png] >> The point is that we do not have a notion of connection in TinkerPop, but >> we have a notion of GraphTraversal. >> So, from my perspective, this diagram can be changed to >> [image: image.png] >> There is a corner case for Non-TX databases that we need to discuss. >> I believe Non-TX mode is not appropriate for handling graph data >> structures. GQL, for example, requires serializable isolation to satisfy >> the standard requirements. >> Instead, we should require to have presence of a transaction, which is >> started *implicitly* when GraphTraversal is created and committed once >> it is closed. >> >> I think we DO need methods for explicit transaction management, but 90% >> of the cases will be covered by implicit transaction management >> throughout the GraphTraversal life cycle. >> 1. TX is started when GraphTraversal is created. >> 2. TX is committed when GraphTraversal is closed (which includes terminal >> operations like toList or iterate). >> 3. TX is rolled back in case of exceptional execution of >> GraphTraversal. Users can always roll back a transaction by using the >> fail() step. >> >> We also provide in the YTDB the following methods that I hope will become >> part of the framework: gSource.computeInTx(g ->{}), gSource.executeInTx(g - >> {}), gSource.autoExecuteInTx(g - {}: g) (this one executes g.iterate() on >> the returned GraphTraversal, which we find really handy). >> >> Now, regarding explicit TX management. >> I think that explicit TX management is still needed if you want to >> execute several queries and mix them with complex logic, but I think that >> methods that explicitly cause TX begin/start/commit are confusing and >> provide bad DevX. >> What I propose is to use lambda-controlled TXs (slightly different from >> the above). I think that an explicit rollback/commit of TX by the user is >> similar to the goto operator in old programming languages; it gives finer >> control over TXs, but overall only adds confusion and can be completely >> avoided in modern languages. >> >> Explicit control on TXs is proposed to be implemented using methods >> similar to above, with a different signature, namely: >> gSource.executeInTx(graphTraversalSupplier -> { >> var g = graphTraversalSupplier.get(); >> var v = g.V().next(); >> g.close() >> >> v.property("p", "val"); >> int i = v.value("int"); >> >> if(i > 0) { >> g = graphTraversalSupplier.get(); >> g.V().has("i2", i).property("i2", i + 1).iterate(); >> } >> }) >> >> In this case, the transaction is detached from GraphTraversal, and its >> lifecycle is not controlled by GraphTraversal. >> Instead, TX is committed if an exception is not thrown from the lambda >> method. >> >> To inform the user if GraphTraversal controls TX or not, a simple >> isTxAttached method is proposed to be introduced >> gSource.<V>computeInTX(graphTraversalSupplier -> {}) works in similar way >> but returns value. >> >> >> So the final diagram could look like the following: >> >> [image: image.png] >> >> Please note that in this model, there is no need for an explicit >> transaction object. >> >> On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin < >> [email protected]> wrote: >> >>> Good day, Ken. >>> >>> >This could map to multiple threads on the provider side. It's probably >>> OK if something like "g.tx()..." returns the same transaction. We could >>> require users to open multiple DriverRemoteConnections/Clients to achieve >>> this and that would be similar to other data connectivity APIs. >>> >>> Unfortunately I do not follow, could you elaborate more or give me >>> example how do you see it ? >>> >>> On Thu, Dec 4, 2025 at 7:06 PM Ken Hu <[email protected]> wrote: >>> >>>> I might be thinking about this too much for just the remote case and >>>> not considering the embedded case enough. In my mind, there just needs to >>>> be an easy way for users of the language variants (in particular JavaScript >>>> because of its single threaded nature) to start multiple transactions from >>>> a single thread. This could map to multiple threads on the provider side. >>>> It's probably OK if something like "g.tx()..." returns the same >>>> transaction. We could require users to open multiple >>>> DriverRemoteConnections/Clients to achieve this and that would be similar >>>> to other data connectivity APIs. >>>> >>>> On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev < >>>> [email protected]> wrote: >>>> >>>>> The point about having multiple concurrent transactions within the same >>>>> thread warrants a more detailed discussion. >>>>> >>>>> To compare this behavior, consider how major SQL databases handle >>>>> attempts >>>>> to start a second transaction when one is already active: >>>>> >>>>> 1. Error out: The database prevents the operation, signaling that a >>>>> transaction is already active (e.g., "Transaction already active"). >>>>> 2. Create a nested transaction (savepoint): The database creates a >>>>> sub-scope of the first transaction, not an independent transaction. If >>>>> the >>>>> parent transaction fails, the child scope also fails. >>>>> 3. Implicitly commit the first: The database automatically commits the >>>>> first transaction to open the second one (a common behavior in MySQL >>>>> DDL). >>>>> >>>>> As far as I know, only Firebird has historically allowed several truly >>>>> independent transactions concurrently on the same thread. I suspect >>>>> support >>>>> for simultaneous transactions may be even more limited among graph >>>>> database >>>>> vendors. >>>>> >>>>> On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin < >>>>> [email protected]> >>>>> wrote: >>>>> >>>>> > Good day. >>>>> > As for questions from Ken >>>>> > >What OGMs are you referring to here? Regular transactions are >>>>> complicated >>>>> > enough by itself, I would only want to introduce nested transactions >>>>> if >>>>> > there were well maintained OGMs that would have a demonstrable >>>>> benefit from >>>>> > this. >>>>> > >>>>> > We have such an OGM using TinkerPop that is well maintained, if you >>>>> mean >>>>> > by this years of support and thousands of instances running on OS >>>>> version. >>>>> > However, I have given it thought and believe that it is a relatively >>>>> rare >>>>> > case, and OGM is so complex that it can manage nested transactions >>>>> itself >>>>> > if needed. >>>>> > I think we can support nested transactions on pause for now, till we >>>>> > receive more feedback from users. >>>>> > >>>>> > There are many inconsistencies to discuss in the TX lifecycle, even >>>>> > without nested transactions. >>>>> > >>>>> > >Sessions won't exist moving forward (4.x+) as they are tied to >>>>> remote >>>>> > Groovy execution. So let's continue this conversation as simply >>>>> > transactions and not refer to sessions at all to prevent confusion, >>>>> unless >>>>> > you intend to have these changes in the 3.8.x line? >>>>> > Yeah, we already removed Groovy support in our fork, so it is even >>>>> now in >>>>> > our distribution TX, and the session is the same. >>>>> > >>>>> > >>>>> > >Could you expand a bit more on "Transaction scope is limited to the >>>>> > current thread only". >>>>> > I mean that TX visibility and manipulation are limited to a single >>>>> thread >>>>> > where it is created. >>>>> > If some of the providers support TX visibility between several >>>>> threads, >>>>> > that will only add a bonus for them if such a need arises, of course, >>>>> > without breaking the API. >>>>> > >>>>> > > I find it limiting that an attempt to open multiple transactions >>>>> from >>>>> > the same thread ends up returning the same Transaction instance >>>>> > Do you propose to support several transactions in the same thread? >>>>> > I am afraid not all current vendors can support this model. >>>>> > >>>>> > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette < >>>>> [email protected]> >>>>> > wrote: >>>>> > >>>>> >> Andrii, I'd like to just quickly clarify Ken's point stating " >>>>> Sessions >>>>> >> won't exist moving forward (4.x+) ". He's saying what you said: " >>>>> >> 'session' >>>>> >> and 'tx' are essentially the same concepts". We acknowledged that >>>>> and >>>>> >> decided to stop using "session" terminology and instead use >>>>> "transaction" >>>>> >> terminology only to avoid confusion. >>>>> >> >>>>> >> Much of what you wrote is in the spirit of what has been considered >>>>> for >>>>> >> 4.x, so that is nice alignment. I do think the goal though is to >>>>> have a >>>>> >> consistent transaction API/system that works for both embedded and >>>>> remote >>>>> >> cases. It will be interesting to see how that is achieved. >>>>> >> >>>>> >> On Tue, Dec 2, 2025 at 3:53 PM Ken Hu <[email protected]> wrote: >>>>> >> >>>>> >> > Hi Andrii, >>>>> >> > >>>>> >> > Good day to you as well. I have a couple questions about this >>>>> that I >>>>> >> would >>>>> >> > like some more details on. >>>>> >> > >>>>> >> > 1. What OGMs are you referring to here? Regular transactions are >>>>> >> > complicated enough by itself, I would only want to introduce >>>>> nested >>>>> >> > transactions if there were well maintained OGMs that would have a >>>>> >> > demonstrable benefit from this. >>>>> >> > 2. I like some of your ideas about removing Transaction as its own >>>>> >> class, >>>>> >> > but I'm not sure GraphTraversalSource is where it should be >>>>> instead. Do >>>>> >> you >>>>> >> > have some more details about what you think the Transaction API >>>>> should >>>>> >> look >>>>> >> > like? >>>>> >> > 3. Could you expand a bit more on "Transaction scope is limited >>>>> to the >>>>> >> > current thread only". I always get confused by TinkerPop's >>>>> threaded vs >>>>> >> > multithreaded transaction. I find it limiting that an attempt to >>>>> open >>>>> >> > multiple transactions from the same thread ends up returning the >>>>> same >>>>> >> > Transaction instance. Maybe I didn't quite understand what you >>>>> meant by >>>>> >> > this. >>>>> >> > 4. Sessions won't exist moving forward (4.x+) as they are tied to >>>>> remote >>>>> >> > Groovy execution. So let's continue this conversation as simply >>>>> >> > transactions and not refer to sessions at all to prevent >>>>> confusion, >>>>> >> unless >>>>> >> > you intend to have these changes in the 3.8.x line? >>>>> >> > >>>>> >> > Regards, >>>>> >> > Ken >>>>> >> > >>>>> >> > >>>>> >> > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin via dev < >>>>> >> > [email protected]> wrote: >>>>> >> > >>>>> >> > > Small clarification. >>>>> >> > > Implementation notes are related to changes in the remote >>>>> protocol. >>>>> >> > > >>>>> >> > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin < >>>>> >> > > [email protected]> >>>>> >> > > wrote: >>>>> >> > > >>>>> >> > > > To clarify: in this model, 'session' and 'tx' are essentially >>>>> the >>>>> >> same >>>>> >> > > > concepts. >>>>> >> > > > >>>>> >> > > > I also think it is essential to have a common denominator that >>>>> >> makes TP >>>>> >> > > > applications portable across implementations. >>>>> >> > > > That is why I also propose to restrict transaction scope to >>>>> the >>>>> >> thread >>>>> >> > > > scope. It will be a common denominator that will not harm user >>>>> >> > experience >>>>> >> > > > but will ensure uniform and expected API behavior. >>>>> >> > > > There should not be both `begin` and `open` methods because >>>>> the >>>>> >> > contract >>>>> >> > > > of the beginning method is vague and unpredictable. >>>>> >> > > > >>>>> >> > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii Lomakin < >>>>> >> > > > [email protected]> wrote: >>>>> >> > > > >>>>> >> > > >> Good day, >>>>> >> > > >> >>>>> >> > > >> I have a proposal to enhance the transaction handling logic >>>>> in TP, >>>>> >> > > >> specifically regarding nested transaction calls, which are >>>>> common >>>>> >> in >>>>> >> > EE >>>>> >> > > >> development, particularly when utilizing OGM. >>>>> >> > > >> >>>>> >> > > >> Currently, calling open multiple times causes an exception. >>>>> This >>>>> >> > > behavior >>>>> >> > > >> is restrictive, and the term "open" can be confusing. >>>>> >> > > >> >>>>> >> > > >> I propose a design that improves the developer experience by >>>>> using >>>>> >> an >>>>> >> > > >> internal counter for transaction calls: >>>>> >> > > >> >>>>> >> > > >> Proposed transaction counter logic >>>>> >> > > >> 1. begin/commit counter: >>>>> >> > > >> - begin would increment an internal counter. >>>>> >> > > >> - commit would decrement the internal counter, causing the >>>>> actual >>>>> >> > > >> transaction commit only when the counter reaches zero. >>>>> >> > > >> 2. rollback behavior: >>>>> >> > > >> - rollback would immediately force a transaction rollback >>>>> and >>>>> >> > > decrements >>>>> >> > > >> the internal counter >>>>> >> > > >> - the subsequent calls to rollback on the same transaction >>>>> object >>>>> >> > > should >>>>> >> > > >> be allowed until the stack trace reaches the initial begin >>>>> method >>>>> >> > call, >>>>> >> > > >> after which further rollback calls would throw an exception. >>>>> >> > > >> 3. Error handling: >>>>> >> > > >> - A commit without a matching begin (e.g., counter is >>>>> already zero) >>>>> >> > > >> should throw an exception. >>>>> >> > > >> - A rollback without a matching begin should throw an >>>>> exception >>>>> >> > > >> >>>>> >> > > >> I would also consider hiding the Transaction object >>>>> altogether and >>>>> >> > > >> delegating the logic of transaction management to >>>>> >> GraphTraversalSource >>>>> >> > > it >>>>> >> > > >> will simplify the API a lot, both for users and vendors. >>>>> >> > > >> >>>>> >> > > >> I also propose to restrict transaction scope to the thread >>>>> scope, >>>>> >> it >>>>> >> > > will >>>>> >> > > >> be a common denominator that will not harm user experience >>>>> but will >>>>> >> > > ensure >>>>> >> > > >> uniform and expected API behavior across all implementations. >>>>> >> > > >> >>>>> >> > > >> Proposed API changes >>>>> >> > > >> - I propose deprecating open and using begin with the >>>>> semantics >>>>> >> > > >> described above to avoid confusion. >>>>> >> > > >> - Transaction object is deprecated, and TX control >>>>> functionality is >>>>> >> > > >> delegated to the GraphTraversal/Graph instances. >>>>> >> > > >> - tx() method is deprecated. >>>>> >> > > >> - Transaction scope is limited to the current thread only. >>>>> >> > > >> >>>>> >> > > >> Behavior for non-direct calls >>>>> >> > > >> When begin/commit/rollback methods are not called directly, >>>>> the >>>>> >> > > >> transaction should be automatically committed by a terminal >>>>> >> operation >>>>> >> > > >> (e.g., when hasNext returns false) in both remote and >>>>> embedded >>>>> >> modes. >>>>> >> > > >> >>>>> >> > > >> Implementation notes: >>>>> >> > > >> 1. The begin command should be added. Relying on the implicit >>>>> >> start of >>>>> >> > > TX >>>>> >> > > >> by traversal can lead to non-controlled side effects. >>>>> >> > > >> 2. For the sake of optimization of the remote protocol, while >>>>> >> multiple >>>>> >> > > >> calls of begin/commit/rollback are allowed in the remote >>>>> protocol, >>>>> >> > > >> practically, it will mean that we will track the counter >>>>> locally on >>>>> >> > the >>>>> >> > > >> client and will send begin/commit/rollback only once for the >>>>> TX >>>>> >> > > lifecycle. >>>>> >> > > >> >>>>> >> > > >> This model should be uniform across all deployment types. >>>>> >> > > >> What are your thoughts on this approach? >>>>> >> > > >> >>>>> >> > > >> ----- >>>>> >> > > >> Andrii Lomakin >>>>> >> > > >> YouTrackDB development lead >>>>> >> > > >> >>>>> >> > > > >>>>> >> > > > >>>>> >> > > > -- >>>>> >> > > > Andrii Lomakin >>>>> >> > > > YouTrackDB development lead >>>>> >> > > > >>>>> >> > > >>>>> >> > > >>>>> >> > > -- >>>>> >> > > Andrii Lomakin >>>>> >> > > YouTrackDB development lead >>>>> >> > > >>>>> >> > >>>>> >> >>>>> > >>>>> > >>>>> > -- >>>>> > Andrii Lomakin >>>>> > YouTrackDB development lead >>>>> > >>>>> >>>>> >>>>> -- >>>>> Andrii Lomakin >>>>> YouTrackDB development lead >>>>> >>>> >>> >>> -- >>> Andrii Lomakin >>> YouTrackDB development lead >>> >> >> >> -- >> Andrii Lomakin >> YouTrackDB development lead >> > > > -- > Andrii Lomakin > YouTrackDB development lead > -- Andrii Lomakin YouTrackDB development lead
