Hey Chris- Thanks for taking the first pass in to the generator code.
This got me thinking— Can we make a JDK and tools break to decouple the complexity that comes with dropping annogen AND trying to maintain JDK 8 support? We leveled ActiveMQ v6.x to JDK 17— keeping JDK 8 (language) support is two JDK LTS jumps back, and going to be increasingly difficult to maintain language AND tooling+dependency support going forward. Not to mention, being able to leverage JDK 17 text blocks might be super handy =) Riffing here.. 1. Create an activemq-openwire-jdk8 module (generate and ‘leave’ the classes and test code there — like you mentioned) 2. Modernize open wire-generator away from gram/groovy/annogen as needed, leverage text blocks where we can (license, headers, etc), and perhaps something like JavaPoet for java class generation. Thoughts? -Matt Pavlovich > On Dec 5, 2023, at 6:44 AM, Christopher Shannon > <christopher.l.shan...@gmail.com> wrote: > > I started looking at the Openwire generator issues and the biggest problem > is the use of a super ancient library for things like javadoc parsing which > no longer works on JDK 11+ https://github.com/codehaus/annogen > > The entire doclet API was changed so we would need to update the openwire > generator to replace that library which would require a substantial rewrite > or we would need to fix that library. Based on what I am seeing I think > fixing the Openwire generator code might be a longer term task depending on > how bad the changes are. > > The generator should still work with JDK 8 so if it's going to take a long > time to fix we may need to generate a new Openwire version in an older 5.x > branch using JDK 8 and then just copy the generated code to the new > versions. > > On Mon, Dec 4, 2023 at 1:51 PM Matt Pavlovich <mattr...@gmail.com> wrote: > >> Hi Chris- >> >> My comments in-line below. For the most part, I agree in full. Any of the >> work that we can break up into minor or patch releases would be good from >> the standpoint of getting some runtime of all the changes under our belt >> before rolling out the major breaking change parts. >> >>> On Dec 4, 2023, at 6:57 AM, Christopher Shannon < >> christopher.l.shan...@gmail.com> wrote: >>> >>> We can have a roadmap and targets for things but the main thing I'm >> pushing >>> back on is having firm dates as there is a lot of unknown. There is a lot >>> of work left to do and it will take time to get things correct. We don't >>> want to have a rushed/bad implementation that is buggy. >> >> Agreed. I think getting the three other JMS 2.0 impl features are fairly >> straight-forward, and getting those out would allow them to bake-in a bit >> while we sort out Shared Durable Topic Subscription (SDTS) support. >> >>> For one thing, if we are going to add support for shared subscriptions >> and >>> shared durables we need an actual design to implement it as there's more >>> than one way it could be done. >>> >>> One approach I have been thinking about is the following and some of the >>> things off the top of my head that need to be done. There's probably a >> lot >>> more I'm not thinking of that would come up as working on it. >>> >>> 1. New openwire version that supports new commands for a shared topic >>> consumer and shared durable consumer. We need to fix the openwire >> generator >>> first and also fix the CVE in it for generated code. >> >> Agree. We need to clean-up openwire generator (and probably re-home >> kahadb-proto) for JDK, modernization and just ease of maintenance. >> >>> 2. Client changes to support the new commands and properly creating >>> consumers >> >> Yep. >> >>> 3. Some way to map the shared subscriptions. As previously discussed, >>> since shared subs can be implemented with queues, we can leverage the >> fact >>> that the broker already supports Virtual topics and composite >> destinations. >>> I think a good approach would be to have a new VT namespace that is >>> dedicated just to shared subscriptions. When a user creates a shared >> sub >>> with the new command we can then behind the scenes in the broker >> generate a >>> virtual topic that maps to the new shared sub name and virtual topic >> queue >>> using the namespace. Since it is a special namespace then on recovery >> we >>> could detect the virtual topic is a shared subscription and recover >> things >>> correctly. >> >> The subName+clientId may be stored in the ‘id’ field of the Virtual Topic— >> similar to how it’s done with the optional MQTT mapping. But yeah, a full >> deep-dive and analysis is warranted. >> >>> 4. We probably need to make some enhancements to virtual topic support. >>> It may work out of the box bu I'm not sure. The good news is we >> shouldn't >>> need any other special stuff in KahaDB or elsewhere in the code if we >> can >>> just re-use Virtual topics. >> >> Yep, also— standardizing some of the Virtual Destination behaviors would >> be good. For example, Composite Queue/Topic doesn’t support the transaction >> fan-out when the ack mode is non-transacted. >> >>> 5. All of the rules in the spec need to be validated >>> which will take work to get right. There are a lot of rules to enforce >> with >>> the API that we will need to check broker side which is non trivial. >> For >>> example, "A shared durable subscription and an unshared durable >>> subscription may not have the same name and client identifier (if >> set). If >>> an unshared durable subscription already exists with the same name and >>> client identifier (if set) then a JMSRuntimeException is thrown." >> >> Also— network consumers always throw topics for a curve. >> >>> 6. We need to decide if other protocols will map as well...are we >>> supporting MQTT/AMQP/STOMP ? If so, how are they going to map shared >> subs? >> >> >>> 7. We probably need new metrics/monitoring of things. >> >> Yes. +1 >> >>> 8. We of course will need a lot of testing to verify all the new >>> behavior works which will be a good amount of work. >> >> Yep. >> >> Thanks, >> Matt >> >> >>> On Mon, Dec 4, 2023 at 7:23 AM Jean-Baptiste Onofré <j...@nanthrax.net> >> wrote: >>> >>>> Ok. But in that case, can we target SMX 7 with full JMS 2 support ? >>>> >>>> I think it’s important to have a kind of roadmap. >>>> >>>> So let’s use 6.x for incremental work and 7 when complete (without >> strong >>>> commitment on date). >>>> >>>> Regards >>>> JB >>>> >>>> Le lun. 4 déc. 2023 à 13:18, Christopher Shannon < >>>> christopher.l.shan...@gmail.com> a écrit : >>>> >>>>> I don't see how we can release shared subscription support for 6.1.0 at >>>>> this point. We haven't even come up with a plan of how we are going to >>>>> implement it. There's multiple ways it could be done and probably >>>> requires >>>>> protocol changes. We have to decide how much work is done by the broker >>>> and >>>>> where. >>>>> >>>>> On Mon, Dec 4, 2023 at 2:18 AM Jean-Baptiste Onofré <j...@nanthrax.net> >>>>> wrote: >>>>> >>>>>> I think it would better to complete JMS 2 in 6.1.0 including shared >>>> topic >>>>>> subscriptions. >>>>>> We already did 6.0.x with partial JMS 2 support, which is so so from >>>> user >>>>>> perspective. >>>>>> >>>>>> I would prefer to wait few weeks for 6.1.0 to give us time to complete >>>>> JMS >>>>>> 2. >>>>>> >>>>>> Regards >>>>>> JB >>>>>> >>>>>> Le lun. 4 déc. 2023 à 07:52, Matt Pavlovich <mattr...@gmail.com> a >>>>> écrit : >>>>>> >>>>>>> Hey JB -- >>>>>>> >>>>>>> These JMS 2.0 features are planned for v6.1.0: >>>>>>> >>>>>>> AMQ-8464 PR #1046 6.1.0, 5.18.x JMSConsumer .receiveBody(Class) >>>>> methods >>>>>>> AMQ-8320 PR #982 6.1.0, 5.18.x Delivery Delay Support for Message >>>>>>> DeliveryDelay feature >>>>>>> AMQ-8324 PR #1045 6.1.0, 5.18.x JMSProducer features Completion >>>>> Listener >>>>>>> async send support >>>>>>> >>>>>>> This would just leave Shared Topic Subscriptions, which is currently >>>>>>> planned for v6.2.0. >>>>>>> >>>>>>> AMQ-8323 6.2.0, 5.18.x Shared Topic Consumer Multi-consumer >>>>>> (queue-like) >>>>>>> consuming from topic subscriptions >>>>>>> >>>>>>> Reference: >>>>>>> https://activemq.apache.org/jms2 >>>>>>> >>>>>>> I think this would work well, since we have Virtual Topic support >>>>> (which >>>>>>> is better anyway). >>>>>>> >>>>>>> Thanks, >>>>>>> Matt >>>>>>> >>>>>>> >>>>>>>> On Dec 2, 2023, at 11:00 PM, Jean-Baptiste Onofré <j...@nanthrax.net >>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> I think it's really important to focus on JMS 2 complete impl for >>>>>> 6.1.0. >>>>>>>> That's the most important. >>>>>>>> >>>>>>>> I started to work on some impl, a couple are a little longer to >>>> impl, >>>>>>>> require tests etc. >>>>>>>> I don't think early January is reasonable. I would rather try at >>>> the >>>>>>>> end of January. >>>>>>>> >>>>>>>> I would rather: >>>>>>>> 1. Focus on 6.0.2 for fixes (I'm preparing 5.18.x/5.17.x too as >>>> they >>>>>>>> include fixes as well) >>>>>>>> 2. Focus on 6.1.0 to complete JMS 2.x support. That's probably the >>>>>>>> most important (honestly, I'm not a big fan of JMS 2.x support in >>>>>>>> ActiveMQ 6.0.x, it could be confusing for users). >>>>>>>> >>>>>>>> Regards >>>>>>>> JB >>>>>>>> >>>>>>>> On Sat, Dec 2, 2023 at 4:10 PM Matt Pavlovich <mattr...@gmail.com> >>>>>>> wrote: >>>>>>>>> >>>>>>>>> All- >>>>>>>>> >>>>>>>>> I’ve started organizing some JIRAs for v6.1.0. I’m thinking >>>>>>> early-January for release target timeframe. >>>>>>>>> >>>>>>>>> - Additional JMS 2.0 impls >>>>>>>>> - New features for observability >>>>>>>>> - Code base modernization >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> Matt Pavlovich >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >> >>
