Thanks so much for your thoughtful response. I recognize now that my expectations of maturity are a bit out of sync with reality at this moment in time.
I'd like to offer just an opinion that trying to keep up with the constant changes/additions in all of the different SQL databases in the code generator seems like a long term uneconomical use of your time. This problem will only get worse as you onboard support for more databases that have non-standard SQL/DDL which may be something you would be considering for the commercial version of the product. just for an example use case, we have been using flyway for many years, to the point where we had over 100 files doing incremental changes to the database. As a performance improvement, we did a pgDump of the database state ( not the table data... ) and replaced the 100 incremental files. A lot of this dump is obscure and postgres-specific and not something we actually want to "think" about. it just works when creating an image of the database for testing and as a basis for further incremental changes. I could imagine this happening for all of your supported databases. This puts you in the position of having to interpret (even if "ignore") lots and lots of obscure database-specific code. This would terrify me from a product maintenance perspective. Not sure how I would deal with it, but it seems to be a real problem. In the case of JPA, I'm just wondering if there might be an opportunity for a different approach. I notice that JPA/Hibernate generates "companion" classes for each entity ( they take the class name and append an underscore character ) I am assuming that the JPA/Hibernate folks already have developed everything needed to directly interpret source code to produce these companion classes. These companion classes may have all of the needed information for generating the JOOQ classes. Perhaps there is an opportunity to collaborate with the Hibernate folks in order to create a bridge, or perhaps even extend the companion classes. Here, you might be able to take advantage of a very mature and highly maintained project to supercharge the JOOQ code generator and provide broad compatibility with the JPA ecosystem. On Thursday, April 4, 2019 at 4:08:18 AM UTC-4, Lukas Eder wrote: > > Hello, and thank you very much for your feedback > > We'll certainly not get "out of the business of trying to emulate database > languages" :) In fact, we've only just started and we'll invest much more > in that area in the future! > > Surely, you've hit some limitations as most new users do, and your > feedback is very valuable to us to improve quite a few things. But your > having hit limitations doesn't mean we're on a completely wrong track. It > does mean, however, that what we've built so far is insufficient for you. > > The JPADatabase and DDLDatabase are tools that work to some extent. They > are far from complete or perfect, and definitely need much more thought and > work. The JPADatabase should definitely be able to run during the > annotation processing phase as that's what a lot of people do with entities > anyway, e.g. when generating code for use with the Criteria Query API. > There's a pending feature request for this: > https://github.com/jOOQ/jOOQ/issues/6945 > > At this point, it is unclear how much of the jOOQ code generator logic > will work wrapped as an annotation processor, because it was built without > taking into account any restrictions of *when* it is run. In particular, > using Hibernate to reverse engineer the entities might not work at this > stage, so we might have to interpret the JPA annotations ourselves, which > would be a prohibitive amount of work. This will need a lot of > investigation. We'll hopefully support it eventually, but we're not there > yet. > > DDLDatabase evolves as the parser evolves. This feature will never be 100% > complete, as vendors have tons of functionality that we currently don't > have in the jOOQ API yet. Extending the parser only would be simple, but > the way we work here is that we almost always add a feature both to the > parser and the API and, if possible, emulate it. Your specific DDLDatabase > finding about sequences will be implemented in jOOQ 3.12, which is indeed > not available to you yet. > > Both of the JPADatabase and the DDLDatabase work quite well on an 80/20 > philosophy for a lot of simpler projects, and we're improving things over > time. > > Right now, given the limitations you've encountered, I would assume you'd > have to revert to the default mode of operation in jOOQ and connect to the > PostgreSQL database after migrating your schema with Flyway. Since you're > using DDL as your source of truth for the schema (which is the recommended > approach from a jOOQ perspective), I think that it would be viable to > expect updated generated jOOQ code to be available only after a migration > execution. You've probably encountered this blog post already, but for the > record and other readers: > > https://blog.jooq.org/2014/06/25/flyway-and-jooq-for-unbeatable-sql-development-productivity/ > > Regarding your specific findings: > > 1. setting up multiple modules is not possible due to the entity >> interdependence issues, among other things >> > > I understand. Surely, this would be beneficial in the long run, but is not > viable in the short run. > > >> 2. the "naive" approach of simply adding a generated-sources execution >> phase clearly doesn't work since the compiled entity classes aren't there >> yet and JOOQ doesn't generate anything >> > > Indeed, this doesn't work with the JPADatabase, but it would work with the > DDLDatabase and the PostgresDatabase, which I would recommend here. > > >> 3. trying to compile all of the sources first, and then running the JOOQ >> class generator afterward doesn't work, since some of the application >> classes refer to the JOOQ generated classes, which don't exist. The >> compile phase fails and the needed entity classes are never generated. >> > > A lot of people extract the jOOQ schema into a new module (e.g. the > database module, which could also host all your DDL scripts and the Flyway > migration) that all other modules depend on. I don't think this is a bad > idea, even if it might mean some restructuring of your projects. > > I do understand that you have a status quo that makes this difficult right > now, but perhaps this is a good occasion to review the status quo... E.g. > having dependencies from entities back to services is probably not what you > really want to have. > > >> 4. trying to add a compile step during the generate-sources phase just >> for the entities. This doesn't work since there is a bug with the >> maven-compiler-plugin that causes dependent classes to be compiled without >> the annotation processors (lombok) active. > > > Yes, I don't think that Maven is built for such customisation. The Maven > way to do this is to cleanly factor out modules, which leads to more > predictable behaviour than tweaking the phases. > > Myself, it took me a long time to accept Maven's rigor. In the old days, > with ant, I could just create a build in any way I wanted. But after years > of adhereing to its rigor and giving in to its model, I do appreciate that > this has led to much cleaner project layouts, and more importantly, to > forceful avoidance of dependency cycles. > > Again, I understand that this doesn't help you immediately, but my advice > here is: Instead of trying to shoehorn jOOQ into an already difficult to > maintain setup, it might really be a good opportunity to think about the > long term strategy to clean things up. I'm sure that also your JPA using > logic will profit greatly from this. > > I hope this helps, > Lukas > > On Thu, Apr 4, 2019 at 5:57 AM sdwarwick <[email protected] > <javascript:>> wrote: > >> Thinking that the long term solution for JOOQ is to get out of the >> business of trying to emulate database languages. perhaps something like >> this would have been a solution: >> >> https://github.com/opentable/otj-pg-embedded/blob/master/README.md >> >> >> On Wednesday, April 3, 2019 at 3:19:27 PM UTC-4, sdwarwick wrote: >>> >>> This has proven to be far more difficult than I realized, particularly >>> since the application is very large and there is no appetite for >>> restructuring, creating multiple modules or modifying existing classes. >>> >>> Notes: >>> >>> 1. The strategy to use @entity scanning was driven by the need to 1) >>> keep database userIds and passwords out of maven and Intellij >>> configuration >>> files - they are only set at the environment level in production and 2) >>> ensure that JOOQ tracks the class entity structures, not the status of >>> the >>> database. >>> 2. Entity classes are not always pure data - they sometimes have >>> references to service classes. In fact, some are even Autowired using >>> Aspectj and @configurable. >>> 3. Generate-sources requires the compiled version of the Entity >>> classes. >>> 4. Compiling an Entity class triggers the compiling of any dependent >>> classes (maven-compiler-plugin) >>> 5. "maven clean package" should work. >>> 6. The ultimate desire was to have the same level of fluidity we get >>> from JPA / hibernate / lombok , where generated sources and annotations >>> are processed in real time during editing and are essentially invisible. >>> >>> Right now, modifying an entity class doesn't require any special >>> processing >>> in the IDE and even can be done during debugging with JRebel integrated >>> into Intellij >>> >>> Findings >>> >>> 1. setting up multiple modules is not possible due to the entity >>> interdependence issues, among other things. >>> 2. the "naive" approach of simply adding a generated-sources >>> execution phase clearly doesn't work since the compiled entity classes >>> aren't there yet and JOOQ doesn't generate anything >>> 3. trying to compile all of the sources first, and then running the >>> JOOQ class generator afterward doesn't work, since some of the >>> application >>> classes refer to the JOOQ generated classes, which don't exist. The >>> compile phase fails and the needed entity classes are never generated. >>> 4. trying to add a compile step during the generate-sources phase >>> just for the entities. This doesn't work since there is a bug with the >>> maven-compiler-plugin that causes dependent classes to be compiled >>> without >>> the annotation processors (lombok) active. >>> >>> Conclusion - the current structure of JOOQ makes integration with >>> existing JPA / hibernate applications quite difficult. Not sure how to >>> proceed. >>> >>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "jOOQ User Group" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "jOOQ User Group" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
