The issue is the generator tool relying on the doclet API that wtas removed in JDK 11+ so to run the current generator we have to run the tool with JDK 8 at build time. We would need to re-write all that parsing with the new API. https://openjdk.org/jeps/221 Unfortunately trying to update everything to use the new api is going to be a lot of work I think
As an alternative, Tim reminded me he started work on an openwire generator that is annotation based and lives here: https://github.com/apache/activemq-openwire/tree/main It would likely be a better idea to take over that project and update it as needed as it looks like it still works but is a bit out of date. On Tue, Dec 5, 2023 at 3:30 PM Matt Pavlovich <mattr...@gmail.com> wrote: > Hey Chris- > > Thanks for taking the first pass in to the generator code. > > This got me thinking— Can we make a JDK and tools break to decouple the > complexity that comes with dropping annogen AND trying to maintain JDK 8 > support? > > We leveled ActiveMQ v6.x to JDK 17— keeping JDK 8 (language) support is > two JDK LTS jumps back, and going to be increasingly difficult to maintain > language AND tooling+dependency support going forward. Not to mention, > being able to leverage JDK 17 text blocks might be super handy =) > > > Riffing here.. > > 1. Create an activemq-openwire-jdk8 module (generate and ‘leave’ the > classes and test code there — like you mentioned) > > 2. Modernize open wire-generator away from gram/groovy/annogen as needed, > leverage text blocks where we can (license, headers, etc), and perhaps > something like JavaPoet for java class generation. > > Thoughts? > > -Matt Pavlovich > > > On Dec 5, 2023, at 6:44 AM, Christopher Shannon < > christopher.l.shan...@gmail.com> wrote: > > > > I started looking at the Openwire generator issues and the biggest > problem > > is the use of a super ancient library for things like javadoc parsing > which > > no longer works on JDK 11+ https://github.com/codehaus/annogen > > > > The entire doclet API was changed so we would need to update the openwire > > generator to replace that library which would require a substantial > rewrite > > or we would need to fix that library. Based on what I am seeing I think > > fixing the Openwire generator code might be a longer term task depending > on > > how bad the changes are. > > > > The generator should still work with JDK 8 so if it's going to take a > long > > time to fix we may need to generate a new Openwire version in an older > 5.x > > branch using JDK 8 and then just copy the generated code to the new > > versions. > > > > On Mon, Dec 4, 2023 at 1:51 PM Matt Pavlovich <mattr...@gmail.com> > wrote: > > > >> Hi Chris- > >> > >> My comments in-line below. For the most part, I agree in full. Any of > the > >> work that we can break up into minor or patch releases would be good > from > >> the standpoint of getting some runtime of all the changes under our belt > >> before rolling out the major breaking change parts. > >> > >>> On Dec 4, 2023, at 6:57 AM, Christopher Shannon < > >> christopher.l.shan...@gmail.com> wrote: > >>> > >>> We can have a roadmap and targets for things but the main thing I'm > >> pushing > >>> back on is having firm dates as there is a lot of unknown. There is a > lot > >>> of work left to do and it will take time to get things correct. We > don't > >>> want to have a rushed/bad implementation that is buggy. > >> > >> Agreed. I think getting the three other JMS 2.0 impl features are fairly > >> straight-forward, and getting those out would allow them to bake-in a > bit > >> while we sort out Shared Durable Topic Subscription (SDTS) support. > >> > >>> For one thing, if we are going to add support for shared subscriptions > >> and > >>> shared durables we need an actual design to implement it as there's > more > >>> than one way it could be done. > >>> > >>> One approach I have been thinking about is the following and some of > the > >>> things off the top of my head that need to be done. There's probably a > >> lot > >>> more I'm not thinking of that would come up as working on it. > >>> > >>> 1. New openwire version that supports new commands for a shared topic > >>> consumer and shared durable consumer. We need to fix the openwire > >> generator > >>> first and also fix the CVE in it for generated code. > >> > >> Agree. We need to clean-up openwire generator (and probably re-home > >> kahadb-proto) for JDK, modernization and just ease of maintenance. > >> > >>> 2. Client changes to support the new commands and properly creating > >>> consumers > >> > >> Yep. > >> > >>> 3. Some way to map the shared subscriptions. As previously discussed, > >>> since shared subs can be implemented with queues, we can leverage the > >> fact > >>> that the broker already supports Virtual topics and composite > >> destinations. > >>> I think a good approach would be to have a new VT namespace that is > >>> dedicated just to shared subscriptions. When a user creates a shared > >> sub > >>> with the new command we can then behind the scenes in the broker > >> generate a > >>> virtual topic that maps to the new shared sub name and virtual topic > >> queue > >>> using the namespace. Since it is a special namespace then on recovery > >> we > >>> could detect the virtual topic is a shared subscription and recover > >> things > >>> correctly. > >> > >> The subName+clientId may be stored in the ‘id’ field of the Virtual > Topic— > >> similar to how it’s done with the optional MQTT mapping. But yeah, a > full > >> deep-dive and analysis is warranted. > >> > >>> 4. We probably need to make some enhancements to virtual topic > support. > >>> It may work out of the box bu I'm not sure. The good news is we > >> shouldn't > >>> need any other special stuff in KahaDB or elsewhere in the code if we > >> can > >>> just re-use Virtual topics. > >> > >> Yep, also— standardizing some of the Virtual Destination behaviors would > >> be good. For example, Composite Queue/Topic doesn’t support the > transaction > >> fan-out when the ack mode is non-transacted. > >> > >>> 5. All of the rules in the spec need to be validated > >>> which will take work to get right. There are a lot of rules to enforce > >> with > >>> the API that we will need to check broker side which is non trivial. > >> For > >>> example, "A shared durable subscription and an unshared durable > >>> subscription may not have the same name and client identifier (if > >> set). If > >>> an unshared durable subscription already exists with the same name and > >>> client identifier (if set) then a JMSRuntimeException is thrown." > >> > >> Also— network consumers always throw topics for a curve. > >> > >>> 6. We need to decide if other protocols will map as well...are we > >>> supporting MQTT/AMQP/STOMP ? If so, how are they going to map shared > >> subs? > >> > >> > >>> 7. We probably need new metrics/monitoring of things. > >> > >> Yes. +1 > >> > >>> 8. We of course will need a lot of testing to verify all the new > >>> behavior works which will be a good amount of work. > >> > >> Yep. > >> > >> Thanks, > >> Matt > >> > >> > >>> On Mon, Dec 4, 2023 at 7:23 AM Jean-Baptiste Onofré <j...@nanthrax.net> > >> wrote: > >>> > >>>> Ok. But in that case, can we target SMX 7 with full JMS 2 support ? > >>>> > >>>> I think it’s important to have a kind of roadmap. > >>>> > >>>> So let’s use 6.x for incremental work and 7 when complete (without > >> strong > >>>> commitment on date). > >>>> > >>>> Regards > >>>> JB > >>>> > >>>> Le lun. 4 déc. 2023 à 13:18, Christopher Shannon < > >>>> christopher.l.shan...@gmail.com> a écrit : > >>>> > >>>>> I don't see how we can release shared subscription support for 6.1.0 > at > >>>>> this point. We haven't even come up with a plan of how we are going > to > >>>>> implement it. There's multiple ways it could be done and probably > >>>> requires > >>>>> protocol changes. We have to decide how much work is done by the > broker > >>>> and > >>>>> where. > >>>>> > >>>>> On Mon, Dec 4, 2023 at 2:18 AM Jean-Baptiste Onofré <j...@nanthrax.net > > > >>>>> wrote: > >>>>> > >>>>>> I think it would better to complete JMS 2 in 6.1.0 including shared > >>>> topic > >>>>>> subscriptions. > >>>>>> We already did 6.0.x with partial JMS 2 support, which is so so from > >>>> user > >>>>>> perspective. > >>>>>> > >>>>>> I would prefer to wait few weeks for 6.1.0 to give us time to > complete > >>>>> JMS > >>>>>> 2. > >>>>>> > >>>>>> Regards > >>>>>> JB > >>>>>> > >>>>>> Le lun. 4 déc. 2023 à 07:52, Matt Pavlovich <mattr...@gmail.com> a > >>>>> écrit : > >>>>>> > >>>>>>> Hey JB -- > >>>>>>> > >>>>>>> These JMS 2.0 features are planned for v6.1.0: > >>>>>>> > >>>>>>> AMQ-8464 PR #1046 6.1.0, 5.18.x JMSConsumer .receiveBody(Class) > >>>>> methods > >>>>>>> AMQ-8320 PR #982 6.1.0, 5.18.x Delivery Delay Support for > Message > >>>>>>> DeliveryDelay feature > >>>>>>> AMQ-8324 PR #1045 6.1.0, 5.18.x JMSProducer features Completion > >>>>> Listener > >>>>>>> async send support > >>>>>>> > >>>>>>> This would just leave Shared Topic Subscriptions, which is > currently > >>>>>>> planned for v6.2.0. > >>>>>>> > >>>>>>> AMQ-8323 6.2.0, 5.18.x Shared Topic Consumer Multi-consumer > >>>>>> (queue-like) > >>>>>>> consuming from topic subscriptions > >>>>>>> > >>>>>>> Reference: > >>>>>>> https://activemq.apache.org/jms2 > >>>>>>> > >>>>>>> I think this would work well, since we have Virtual Topic support > >>>>> (which > >>>>>>> is better anyway). > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Matt > >>>>>>> > >>>>>>> > >>>>>>>> On Dec 2, 2023, at 11:00 PM, Jean-Baptiste Onofré < > j...@nanthrax.net > >>>>> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>> Hi > >>>>>>>> > >>>>>>>> I think it's really important to focus on JMS 2 complete impl for > >>>>>> 6.1.0. > >>>>>>>> That's the most important. > >>>>>>>> > >>>>>>>> I started to work on some impl, a couple are a little longer to > >>>> impl, > >>>>>>>> require tests etc. > >>>>>>>> I don't think early January is reasonable. I would rather try at > >>>> the > >>>>>>>> end of January. > >>>>>>>> > >>>>>>>> I would rather: > >>>>>>>> 1. Focus on 6.0.2 for fixes (I'm preparing 5.18.x/5.17.x too as > >>>> they > >>>>>>>> include fixes as well) > >>>>>>>> 2. Focus on 6.1.0 to complete JMS 2.x support. That's probably the > >>>>>>>> most important (honestly, I'm not a big fan of JMS 2.x support in > >>>>>>>> ActiveMQ 6.0.x, it could be confusing for users). > >>>>>>>> > >>>>>>>> Regards > >>>>>>>> JB > >>>>>>>> > >>>>>>>> On Sat, Dec 2, 2023 at 4:10 PM Matt Pavlovich <mattr...@gmail.com > > > >>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> All- > >>>>>>>>> > >>>>>>>>> I’ve started organizing some JIRAs for v6.1.0. I’m thinking > >>>>>>> early-January for release target timeframe. > >>>>>>>>> > >>>>>>>>> - Additional JMS 2.0 impls > >>>>>>>>> - New features for observability > >>>>>>>>> - Code base modernization > >>>>>>>>> > >>>>>>>>> Thanks! > >>>>>>>>> Matt Pavlovich > >>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >> > >> > >
