Re: [Neo4j] JTA support for Neo
Ok, I have written a small post describing how this thing comes together. It is available at http://digitalstain.blogspot.com/2010/11/using-jotm-as-transactionmanager-in.html I hope it is clear enough. Questions asked here will end up as corrections to this. However, let me point out that here: https://svn.neo4j.org/laboratory/users/cgioran/recovery-robustness/ is a branched project that contains everything that is needed to run the JOTM enabled Neo, provided you have installed the jta kernel and the jotm service implementation. All configuration and dependencies are taken care of (the JOTM configuration is in the resources/ directory and the JVM parameters in the run-one script). Thank you for your time. CG ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] JTA support for Neo
On Mon, Nov 22, 2010 at 4:10 PM, Johan Svensson wrote: > This looks great, very good work! > > I would like to get this merged into trunk after we releases 1.2 (not > after this iteration but after the next one). > > The changes looks minimal to me and hopefully there are no problems > going forward with the current design. Looking forward to the guide so > me and others can try this out. > > Would it be possible to continue investigate this and see how these > changes would work in a Spring environment? The goal would be to run > on an external TM, in Spring (using annotated transactions) and have > both Neo4j and MySQL (or some other resource) participate in the same > transaction. > > Regards, > Johan Currently I am working on a branch of the recovery robustness suite that runs the jta kernel with a jotm instance. Works like a charm, after some more successful cycles I will commit it at my laboratory space, probably within the hour. This will contain a completely setup environment for running an example and the guide comes next. If this test is considered enough, I could delay integrating my JDI-based test suite and continue work on the JCA module. Personally, I feel confident with the robustness suite, so if there are no objections until I am done with the guide, I will begin with the connector. cheers, CG ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] JTA support for Neo
This looks great, very good work! I would like to get this merged into trunk after we releases 1.2 (not after this iteration but after the next one). The changes looks minimal to me and hopefully there are no problems going forward with the current design. Looking forward to the guide so me and others can try this out. Would it be possible to continue investigate this and see how these changes would work in a Spring environment? The goal would be to run on an external TM, in Spring (using annotated transactions) and have both Neo4j and MySQL (or some other resource) participate in the same transaction. Regards, Johan On Sat, Nov 20, 2010 at 6:40 PM, Chris Gioran wrote: >> IMHO you should start a branch >> in the SVN so others can look at the code. > > So, at > > https://svn.neo4j.org/laboratory/users/cgioran/neo4j-kernel-jta/ > > you can find the kernel component with my changes incorporated. The > classes added are > > org.neo4j.kernel.impl.transaction.TransactionManagerImpl > org.neo4j.kernel.impl.transaction.TransactionManagerService > > that are the hooks for providing custom transaction managers. > The first is an extension of javax.transaction.TransactionManager > adding support for > startup and shutdown, an operation present in all tx managers but not > part of their API. This > provides the ability to plugin custom implementations in the TxModule. > The second is a convenience class that is extended by tx managers that > are to be provided as a service. > > Also, changes are present in > > org.neo4j.kernel.impl.transaction.TxModule > > for using this new way of doing things, > > org.neo4j.kernel.impl.transaction.TxManager > org.neo4j.kernel.impl.transaction.ReadOnlyTxManager > > for them to fit in this and > > org.neo4j.kernel.EmbeddedGraphDbImpl > org.neo4j.kernel.Config > > to bind them. > > This fork is (or should be) completely compatible with the official > kernel, so it can be used as a drop in replacement. Any deviation is a > bug and if reported it will be fixed. > > The second project is at > > https://svn.neo4j.org/laboratory/users/cgioran/JOTMService/ > > and is a sample implementation of a tx manager service for JOTM. To > use this, build it, add the resulting jar to your classpath and, if > you are using the new jta fork of the kernel, you can pass a > configuration parameter of "tx_manager_impl"="jotm" to your > EmbeddedGraphDatabase and presto!, if all is well you will be using a > JOTM TxManager to do your thing. > Of course, the jotm libraries must be also in your classpath, version 2.1.9 > > If this way of doing things is met with approval, I will write a > complete guide to using the above, implementation and design details > and as a result a how to for adding more external tx managers. > > There is more to come. > > CG ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] JTA support for Neo
> IMHO you should start a branch > in the SVN so others can look at the code. So, at https://svn.neo4j.org/laboratory/users/cgioran/neo4j-kernel-jta/ you can find the kernel component with my changes incorporated. The classes added are org.neo4j.kernel.impl.transaction.TransactionManagerImpl org.neo4j.kernel.impl.transaction.TransactionManagerService that are the hooks for providing custom transaction managers. The first is an extension of javax.transaction.TransactionManager adding support for startup and shutdown, an operation present in all tx managers but not part of their API. This provides the ability to plugin custom implementations in the TxModule. The second is a convenience class that is extended by tx managers that are to be provided as a service. Also, changes are present in org.neo4j.kernel.impl.transaction.TxModule for using this new way of doing things, org.neo4j.kernel.impl.transaction.TxManager org.neo4j.kernel.impl.transaction.ReadOnlyTxManager for them to fit in this and org.neo4j.kernel.EmbeddedGraphDbImpl org.neo4j.kernel.Config to bind them. This fork is (or should be) completely compatible with the official kernel, so it can be used as a drop in replacement. Any deviation is a bug and if reported it will be fixed. The second project is at https://svn.neo4j.org/laboratory/users/cgioran/JOTMService/ and is a sample implementation of a tx manager service for JOTM. To use this, build it, add the resulting jar to your classpath and, if you are using the new jta fork of the kernel, you can pass a configuration parameter of "tx_manager_impl"="jotm" to your EmbeddedGraphDatabase and presto!, if all is well you will be using a JOTM TxManager to do your thing. Of course, the jotm libraries must be also in your classpath, version 2.1.9 If this way of doing things is met with approval, I will write a complete guide to using the above, implementation and design details and as a result a how to for adding more external tx managers. There is more to come. CG ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] JTA support for Neo
> Chris, > This sounds like some major progress! Meh. It was some work for me, mainly making the vm crash at will and getting JOTM to enlist/recover the resource managers. Things worked out of the box from the side of neo. > IMHO you should start a branch > in the SVN so others can look at the code. That would be the ideal way, yes. Would you set up a laboratory space for me? > Having the compatibility > tests is great - there are other breaking test that Mattias and Johan > are using to purposely break recovery on both the normal server and HA > setups. They are run on a number of amazon instances and it would be > great if your tests could be integrated and run there, too, so we can > find failing recovery scenarios. I assume you are referring to qa/recovery-robustness. I haven't studied its mechanics yet but I could put it there, I suppose. From a cursory look I understood that it runs from a script so my tests could fit there. > any suggestions on making the use of external TX managers easier or > document the setup process somewhere? For my part I could post an article on my blag(TM) on how I have setup JOTM as a Neo service and how to enable it. I would prefer to do that however after the code has been seen from eyeballs other than mine and actually certified not to be a big, fat, slobbering mess. This way the API will be standardized and people will have something concrete to read about. cheers, CG ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] JTA support for Neo
Chris, This sounds like some major progress! IMHO you should start a branch in the SVN so others can look at the code. Having the compatibility tests is great - there are other breaking test that Mattias and Johan are using to purposely break recovery on both the normal server and HA setups. They are run on a number of amazon instances and it would be great if your tests could be integrated and run there, too, so we can find failing recovery scenarios. any suggestions on making the use of external TX managers easier or document the setup process somewhere? /Peter On Thursday, November 18, 2010, Chris Gioran wrote: >> If you can get the above test case working using other TMs together >> with Neo4j would be great! > > OK, it took some time, since I messed with some things for the first time. > I have created the following: > - I have added two classes in the kernel module that provide support > for registration > of TransactionManager implementations as services. Like the lucene > indexer, if the jar > is in the classpath it will be picked up and made available. The configuration > passed to the GraphDatabase must define the name of the TxManager to > use, defaulting > of course to the native one. These changes required some modifications > in TxManager, ReadOnlyTxManager, > TxModule, EmbeddedGraphDbImpl and Config. > - A sample implementation of the above for providing a JOTM instance as a > TransactionManager. > - A test "platform" that, through JDI crashes the system at specific points, > up to a resolution of line of code. This way at any point in the transaction > the system can be brought down in flames. This aspect is still a > little awkward to > integrate since a new JVM process must be started and terminated. At the time > being I use it as a standalone test (launched at the console). > > Using the above, after crashing a neo instance that uses a lucene > XaResource and (of course) > a Neo XaResource at any point during commit or rollback, using JOTM > 2.1.9, restating the system brings it in > a consistent state, committing all pending resources or rolling back, > whatever is needed. > There are many test cases still to be implemented to > make sure that the setup is production ready but what I have tested so > far passes. This includes > of course the stock tests for the kernel component, ensuring (to > whatever possible extent) that my > changes do not mess up with normal operations. > >>> - Find the least intrusive way of making neo fit in the picture, in >>> terms of configuration/code changes etc, approve and commit those. > > See above. > > What I would like as feedback is ways the community would integrate > these capabilities in their > environment. This way I can provide integration methods that will make > the procedure less risky > and more transparent, as well as develop test cases that do not at the > moment occur to me. > > As this feature requires changes in the neo kernel, I am still > searching for the best way to make > the code available. I will inform you when that changes, hopefully soon. > Also, if there is interest, I could write an article on how this is > implemented. > > What I will test next is how this thing plays with JDBC, seeing as > this would be a common case. Also, > I have read up on JCA and there is some preliminary code to produce a > .rar archive that will provide neo > functionality within a container, although this still very premature. > > Thank you for your time, > > CG > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] JTA support for Neo
> If you can get the above test case working using other TMs together > with Neo4j would be great! OK, it took some time, since I messed with some things for the first time. I have created the following: - I have added two classes in the kernel module that provide support for registration of TransactionManager implementations as services. Like the lucene indexer, if the jar is in the classpath it will be picked up and made available. The configuration passed to the GraphDatabase must define the name of the TxManager to use, defaulting of course to the native one. These changes required some modifications in TxManager, ReadOnlyTxManager, TxModule, EmbeddedGraphDbImpl and Config. - A sample implementation of the above for providing a JOTM instance as a TransactionManager. - A test "platform" that, through JDI crashes the system at specific points, up to a resolution of line of code. This way at any point in the transaction the system can be brought down in flames. This aspect is still a little awkward to integrate since a new JVM process must be started and terminated. At the time being I use it as a standalone test (launched at the console). Using the above, after crashing a neo instance that uses a lucene XaResource and (of course) a Neo XaResource at any point during commit or rollback, using JOTM 2.1.9, restating the system brings it in a consistent state, committing all pending resources or rolling back, whatever is needed. There are many test cases still to be implemented to make sure that the setup is production ready but what I have tested so far passes. This includes of course the stock tests for the kernel component, ensuring (to whatever possible extent) that my changes do not mess up with normal operations. >> - Find the least intrusive way of making neo fit in the picture, in >> terms of configuration/code changes etc, approve and commit those. See above. What I would like as feedback is ways the community would integrate these capabilities in their environment. This way I can provide integration methods that will make the procedure less risky and more transparent, as well as develop test cases that do not at the moment occur to me. As this feature requires changes in the neo kernel, I am still searching for the best way to make the code available. I will inform you when that changes, hopefully soon. Also, if there is interest, I could write an article on how this is implemented. What I will test next is how this thing plays with JDBC, seeing as this would be a common case. Also, I have read up on JCA and there is some preliminary code to produce a .rar archive that will provide neo functionality within a container, although this still very premature. Thank you for your time, CG ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] JTA support for Neo
Hi Chris, On Tue, Nov 9, 2010 at 7:34 PM, Chris Gioran wrote: >> Chris, >> Awesome! I think the next step would be to start testing things when >> neo4j needs to recover, rollback etc, I think this is where the >> problems arise :) >> >> Also, any chance of making a maven project out of it and having the >> project as a test component somewhere in the Svn repo, so it can be >> run as part of the QA process for releases etc? > > OK, so the plan I have in my mind is this: > > - Do runs to see what problems exist when rolling back/starting in > recovery mode with the only resource being neo. > - See how the whole thing works when another XA compatible resource is > thrown in the mix, probably a RDBMS, checking that 2PC works. Yes, 2PC is what needs to be tested since on a 1PC the xa resource can during recovery figure out if a transaction should be committed or not. For a prepared 2PC transaction the global TM has to tell the resource to commit or rollback during recovery. Create a test that does the following: o create a transaction and enlist two resources (for example nioneodb and lucene-index, create a node and index it) o let the TM prepare both resources o let the TM send commit to one resource but crash the system before commit is sent to the other resource The system is now in a inconsistent state. One resource will have committed the changes and the other is just prepared. The global TM should detect this while doing recovery and invoke recover() on the appropriate resources and tell them what to do with the transactions that are in prepared state. How the global TM should get hold of a XaResource (creating it if needed) to invoke recover while investigating its transaction log is not really specified. Everyone seems to hack their own solution (JNDI, serialization to disk etc). If you can get the above test case working using other TMs together with Neo4j would be great! Regards, Johan > - Find the least intrusive way of making neo fit in the picture, in > terms of configuration/code changes etc, approve and commit those. > - Write test cases and a maven project so that it can be integrated in > the release cycle to be checked for correct functionality. > - After that probably I would like to fill in the gaps so that from an > app server I can do a container managed tx over a jdbc connection and > a neo connection. After all, this is the ultimate purpose of this > exercise. > > I will fill you in as I go through each of the above. Thanks for your time. > > cheers, > CG > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] JTA support for Neo
OK, I have a question. The XA/Open spec in paragraph 2.3 states that the TM can discard its knowledge of the current global tx after the RMs have committed or rolled back. Neo takes advantage of this fact in several places, where during the completion of the current tx either way (commit or rollback) it asks the TM for the current tx to retrieve related resources. In fact, the TxManager class that is provided with Neo as a top level TM "forgets" (i.e. unmaps the tx from the thread) *after* calling afterCompletion() hooks, so everything works fine out of the box. JOTM on the other hand is not consistent (and, eventually, spec compliant), at least this is what is seems to me. Let me explain. JOTM (its Current class, that is the TM implementation) has a commit() method that first asks RMs to commit() and then dissociates the global tx from the thread. But on rollback, first it dissociates the tx from the thread and then, from a method local copy it asks the RMs to rollback(). This leaves any RM to get a null reference when asking the JOTM TM for the current tx, which leads to failures in the process. If the rollback() code in Current is changed to first ask the RMs to rollback() and then forget the global tx, everything works as it should. My point is that, from a first view, this is a bug in JOTM, not in Neo. Does anyone have any experience with this? Am I interpreting this wrong or should it be taken upstream? cheers, CG ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] JTA support for Neo
This is seriously cool, Chris, Let the list know if you need any support, I can stay up some nights for hid :) Peter On Tuesday, November 9, 2010, Chris Gioran wrote: >> Chris, >> Awesome! I think the next step would be to start testing things when >> neo4j needs to recover, rollback etc, I think this is where the >> problems arise :) >> >> Also, any chance of making a maven project out of it and having the >> project as a test component somewhere in the Svn repo, so it can be >> run as part of the QA process for releases etc? > > OK, so the plan I have in my mind is this: > > - Do runs to see what problems exist when rolling back/starting in > recovery mode with the only resource being neo. > - See how the whole thing works when another XA compatible resource is > thrown in the mix, probably a RDBMS, checking that 2PC works. > - Find the least intrusive way of making neo fit in the picture, in > terms of configuration/code changes etc, approve and commit those. > - Write test cases and a maven project so that it can be integrated in > the release cycle to be checked for correct functionality. > - After that probably I would like to fill in the gaps so that from an > app server I can do a container managed tx over a jdbc connection and > a neo connection. After all, this is the ultimate purpose of this > exercise. > > I will fill you in as I go through each of the above. Thanks for your time. > > cheers, > CG > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] JTA support for Neo
> Chris, > Awesome! I think the next step would be to start testing things when > neo4j needs to recover, rollback etc, I think this is where the > problems arise :) > > Also, any chance of making a maven project out of it and having the > project as a test component somewhere in the Svn repo, so it can be > run as part of the QA process for releases etc? OK, so the plan I have in my mind is this: - Do runs to see what problems exist when rolling back/starting in recovery mode with the only resource being neo. - See how the whole thing works when another XA compatible resource is thrown in the mix, probably a RDBMS, checking that 2PC works. - Find the least intrusive way of making neo fit in the picture, in terms of configuration/code changes etc, approve and commit those. - Write test cases and a maven project so that it can be integrated in the release cycle to be checked for correct functionality. - After that probably I would like to fill in the gaps so that from an app server I can do a container managed tx over a jdbc connection and a neo connection. After all, this is the ultimate purpose of this exercise. I will fill you in as I go through each of the above. Thanks for your time. cheers, CG ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] JTA support for Neo
Chris, Awesome! I think the next step would be to start testing things when neo4j needs to recover, rollback etc, I think this is where the problems arise :) Also, any chance of making a maven project out of it and having the project as a test component somewhere in the Svn repo, so it can be run as part of the QA process for releases etc? /Peter On Tuesday, November 9, 2010, Chris Gioran wrote: > Hi people, > > I have started work on providing support for integrating Neo in a JTA > environment. This is the first time I have done something like this so > some feedback would be welcome. > > My setup is this: I have an ApacheDS instance running to provide an > LDAP service to store objects. I have tweaked (very slightly) a > standalone JOTM instance to provide its TransactionManager via JNDI to > the directory (I prefer that over RMI). I then start an > EmbeddedGraphDatabase and replace the native neo TxManager with the > JOTM provided implementation as I retrieve it from the directory, both > in the PersistenceModule and the TxModule. From there, instead of > asking the graphDb to beginTx() I ask the JOTM TransactionManager to > begin(). I perform some Node creations and they are persisted on > commit(). Also, via a debugger, I can see that the XAResources of Neo > are indeed enlisted on the remove Transaction. > > I know that this is very premature and for the time being the changes > are very rough but I think I have a proof of concept. Comments? Does > anyone have any experience in a similar setting? Next steps? > > This has a very high coolness factor, btw :) > > cheers, > CG > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] JTA support for Neo
Hi people, I have started work on providing support for integrating Neo in a JTA environment. This is the first time I have done something like this so some feedback would be welcome. My setup is this: I have an ApacheDS instance running to provide an LDAP service to store objects. I have tweaked (very slightly) a standalone JOTM instance to provide its TransactionManager via JNDI to the directory (I prefer that over RMI). I then start an EmbeddedGraphDatabase and replace the native neo TxManager with the JOTM provided implementation as I retrieve it from the directory, both in the PersistenceModule and the TxModule. From there, instead of asking the graphDb to beginTx() I ask the JOTM TransactionManager to begin(). I perform some Node creations and they are persisted on commit(). Also, via a debugger, I can see that the XAResources of Neo are indeed enlisted on the remove Transaction. I know that this is very premature and for the time being the changes are very rough but I think I have a proof of concept. Comments? Does anyone have any experience in a similar setting? Next steps? This has a very high coolness factor, btw :) cheers, CG ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
