Hi Yeah these starters are not in use and should be removed
camel-debezium-common-starter http-common-starter jetty-common-starter On Mon, Oct 18, 2021 at 8:34 PM David Jencks <david.a.jen...@gmail.com> wrote: > > I can easily remove the empty starter json files, but if e.g. > camel-debezium-common-starter isn’t a starter, what is it? Removing the 3 > starter projects from camel-spring-boot results in integration test failures. > Does some other process install these as dependencies as appropriate? Are > they actually unnecessary but something needs more tweaking? > > David Jencks > > > On Oct 18, 2021, at 12:17 AM, Claus Ibsen <claus.ib...@gmail.com> wrote: > > > > Hi > > > > The -common JARs are not starters, and they should not be listed. > > > > On Mon, Oct 18, 2021 at 7:45 AM David Jencks <david.a.jen...@gmail.com > > <mailto:david.a.jen...@gmail.com>> wrote: > >> > >> Locally I got the ability to show unused starter json files working, and > >> fixed the obvious problems: > >> > >> avro-rpc was using avro > >> paho-mqtt5 was using paho > >> in 3.7.x, grape and joor were set up wrong. > >> > >> There are still some “starters” i’m not sure about: > >> > >> debezium-common > >> http-common > >> jetty-common > >> > >> In a previous commit I added empty json files for these on the theory that > >> if they were starters they should have instructions on how to use them, > >> but they don’t have corresponding main camel doc pages. Are these > >> starters automatically installed by the corresponding non-common starter > >> artifacts such as camel-debezium-mongodb-starter? If so, I can remove the > >> empty json files. If not, how should their usage be documented? > >> > >> I’m hoping to get my updated Antora extensions published in the next > >> couple of days, at which point we can upgrade Camel to use them. > >> > >> David Jencks > >> > >>> On Oct 13, 2021, at 8:14 PM, David Jencks <david.a.jen...@gmail.com> > >>> wrote: > >>> > >>> They are part of the camel-core-starter, although it’s hard to find the 2 > >>> options per language among the 147 options for the starter. This suggests > >>> that, for starters that encompass more than one component, it would be > >>> useful to only show the options relevant to that component on the > >>> components’ page. > >>> > >>> In any case I believe everything is working properly and I’ve merged all > >>> the 9 PRs. > >>> > >>> The core and non-core languages are currently in different tables on the > >>> spring-boot list page: I should be able to put them back into one table > >>> after upgrading antora-indexer. > >>> > >>> David Jencks > >>> > >>>> On Oct 12, 2021, at 11:03 PM, David Jencks <david.a.jen...@gmail.com > >>>> <mailto:david.a.jen...@gmail.com> <mailto:david.a.jen...@gmail.com > >>>> <mailto:david.a.jen...@gmail.com>>> wrote: > >>>> > >>>> There’s at least one problem…. (also asked on zulip) > >>>> > >>>> Are camel-core-languages actually part of camel-spring-boot? > >>>> > >>>> They are listed in the table in "list of starters" but with an > >>>> artifactid of "camel-base" which isn't a starter. The current site > >>>> doesn't have autoconfiguration info for them either. Since they are in > >>>> the table, I added info to them for my PRs, but it's at least partly > >>>> wrong, since it comes out as needing camel-core-languages-starter, which > >>>> doesn't exist. cf. > >>>> https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages > >>>> > >>>> <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages> > >>>> > >>>> <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages > >>>> > >>>> <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages>>, > >>>> > >>>> https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration > >>>> > >>>> <https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration> > >>>> > >>>> <https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration > >>>> > >>>> <https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration>>. > >>>> Since they say they are part of camel-core, I've used core.json to > >>>> construct the autoconfig options table, but that doesn't look very > >>>> relevant to me. > >>>> > >>>> Is there a starter needed for them? If so, which one? camel-core-starter > >>>> seems like a possibility, is it correct? > >>>> > >>>> If no starter is needed, why are they in the spring-boot starters table? > >>>> > >>>> I hope to be able to merge everything tomorrow, it got too late tonight. > >>>> > >>>> David Jencks > >>>> > >>>>> On Oct 12, 2021, at 7:41 PM, David Jencks <david.a.jen...@gmail.com > >>>>> <mailto:david.a.jen...@gmail.com> <mailto:david.a.jen...@gmail.com > >>>>> <mailto:david.a.jen...@gmail.com>>> wrote: > >>>>> > >>>>> The part of this work I’m ready for now is done, except for some > >>>>> commit-squashing and dealing with check style errors :-) > >>>>> > >>>>> Preview at https://pr-644--camel.netlify.app > >>>>> <https://pr-644--camel.netlify.app/> > >>>>> <https://pr-644--camel.netlify.app/ > >>>>> <https://pr-644--camel.netlify.app/>> > >>>>> > >>>>> I seem to only be able to write AsciiDoc by now…. :-) Here’s a list of > >>>>> the differences between table rows new/old: > >>>>> > >>>>> —— > >>>>> = Differences between spring-boot list-old and list > >>>>> > >>>>> == latest (main) > >>>>> > >>>>> === In list, not list-old: > >>>>> > >>>>> * disruptor-vm-component > >>>>> * splunk-hec-component > >>>>> > >>>>> * (3 bindy dataformats collapsed to one link, corresponding to the > >>>>> single main bindy doc page) > >>>>> > >>>>> * csimple-language > >>>>> > >>>>> == 3.12.x > >>>>> === In list, not list-old: > >>>>> > >>>>> * disruptor-vm-component > >>>>> * splunk-hec-component > >>>>> > >>>>> * (3 bindy dataformats collapsed to one link, corresponding to the > >>>>> single main bindy doc page) > >>>>> > >>>>> * csimple-language > >>>>> > >>>>> == 3.11.x > >>>>> === In list, not list-old: > >>>>> > >>>>> * splunk-hec-component > >>>>> * (3 bindy dataformats collapsed to one link, corresponding to the > >>>>> single main bindy doc page) > >>>>> * csimple-language > >>>>> > >>>>> Moved to separate table: > >>>>> > >>>>> * camel-spring-cloud > >>>>> * camel-spring-cloud-consul > >>>>> * camel-spring-cloud-netflix > >>>>> * camel-spring-cloud-zookeeper > >>>>> > >>>>> == 3.7.x > >>>>> === In list, not list-old: > >>>>> * splunk-hec-component > >>>>> * (3 bindy dataformats collapsed to one link, corresponding to the > >>>>> single main bindy doc page) > >>>>> * csimple-language > >>>>> > >>>>> > >>>>> Moved to separate table: > >>>>> > >>>>> * camel-spring-cloud > >>>>> * camel-spring-cloud-consul > >>>>> * camel-spring-cloud-netflix > >>>>> * camel-spring-cloud-zookeeper > >>>>> > >>>>> —— > >>>>> > >>>>> There are also some presentation differences in the tables, the new > >>>>> ones follow the tables in main camel components more closely, e.g. > >>>>> > >>>>> <td class="tableblock halign-left valign-top"><p > >>>>> class="tableblock">Stable</p></td> > >>>>>>> > >>>>> <td class="tableblock halign-left valign-top"><p > >>>>> class="tableblock">Stable-deprecated</p></td> > >>>>> > >>>>> as the deprecation marker. > >>>>> > >>>>> Also quite a few core languages now list the starter as > >>>>> > >>>>> <td class="tableblock halign-left valign-top"><p > >>>>> class="tableblock">camel-core-languages-starter</p></td> > >>>>> rather than > >>>>> <td class="tableblock halign-left valign-top"><p > >>>>> class="tableblock">camel-base</p></td> > >>>>> > >>>>> Is this correct? > >>>>> > >>>>> There are 8 PRs: > >>>>> (Camel) > >>>>> Main spring boot jsonpath (allow me to squash and merge) > >>>>> <https://github.com/apache/camel/pull/6261 > >>>>> <https://github.com/apache/camel/pull/6261>> > >>>>> Camel 3.12.x spring boot jsonpath (please let me squash and merge) > >>>>> <https://github.com/apache/camel/pull/6263 > >>>>> <https://github.com/apache/camel/pull/6263>> > >>>>> Camel 3.11.x spring boot jsonpath (please let me squash and merge) > >>>>> <https://github.com/apache/camel/pull/6264 > >>>>> <https://github.com/apache/camel/pull/6264>> > >>>>> Camel 3.7.x spring boot jsonpath (please let me squash and merge) > >>>>> <https://github.com/apache/camel/pull/6262 > >>>>> <https://github.com/apache/camel/pull/6262>> > >>>>> (Camel-spring-boot) > >>>>> Main jsonpath (please let me squash and merge) > >>>>> <https://github.com/apache/camel-spring-boot/pull/388 > >>>>> <https://github.com/apache/camel-spring-boot/pull/388>> > >>>>> Camel spring boot 3.12.x jsonpath (please let me squash and merge) > >>>>> <https://github.com/apache/camel-spring-boot/pull/387 > >>>>> <https://github.com/apache/camel-spring-boot/pull/387>> > >>>>> Camel spring boot 3.11.x jsonpath (please let me squash and merge) > >>>>> <https://github.com/apache/camel-spring-boot/pull/386 > >>>>> <https://github.com/apache/camel-spring-boot/pull/386>> > >>>>> Camel spring boot 3.7.x jsonpath (please let me squash and merge) > >>>>> <https://github.com/apache/camel-spring-boot/pull/385 > >>>>> <https://github.com/apache/camel-spring-boot/pull/385>> > >>>>> > >>>>> If everything looks good I plan to squash each PR to 2 or 3 commits > >>>>> (code changes and generated changes) and rebase/merge. I don’t think > >>>>> any playbook changes are needed. > >>>>> > >>>>> I’m completing a big upgrade of the antora-indexer and jsonpath > >>>>> extensions, and plan to finish that next. It will involve changing the > >>>>> syntax of most uses of these extensions. After that I expect to be > >>>>> able to implement automated checking for starters that aren’t linked to > >>>>> by main camel components. I think there are about 5, but have no way > >>>>> to find them at this point. > >>>>> > >>>>> > >>>>> David Jencks > >>>>> > >>>>>> On Oct 12, 2021, at 12:01 AM, Claus Ibsen <claus.ib...@gmail.com > >>>>>> <mailto:claus.ib...@gmail.com> <mailto:claus.ib...@gmail.com > >>>>>> <mailto:claus.ib...@gmail.com>>> wrote: > >>>>>> > >>>>>> On Mon, Oct 11, 2021 at 8:07 AM David Jencks <david.a.jen...@gmail.com > >>>>>> <mailto:david.a.jen...@gmail.com> <mailto:david.a.jen...@gmail.com > >>>>>> <mailto:david.a.jen...@gmail.com>>> wrote: > >>>>>>> > >>>>>>> I’ve been working on generating the spring-boot autoconfig info from > >>>>>>> the json files generated during the spring-boot build, and the spring > >>>>>>> boot “list.adoc” tables using the indexer extension (like the > >>>>>>> components tables are generated). > >>>>>>> > >>>>>>> My understanding of the spring-boot stuff is that: > >>>>>>> > >>>>>>> - any Camel component can be used in Spring > >>>>>>> > >>>>>>> - Some camel components/dataformats… can participate in spring-boot > >>>>>>> auto configure. These components are identified as having a > >>>>>>> corresponding camel-*-starter project in the spring-boot repo. > >>>>>>> > >>>>>>> - There are 3 kinds of these: > >>>>>>> > >>>>>>> 1. There’s (generated) java code and a generated json file > >>>>>>> > >>>>>>> 2. There’s generated java code but no generated json file (e.g. > >>>>>>> jasypt) > >>>>>>> > >>>>>>> 3. There’s no java code and no json file, just some properties files > >>>>>>> etc. (e.g. aws-xray) > >>>>>>> > >>>>>>> I’m really not familiar with spring-boot or the philosophy behind it, > >>>>>>> but my guess is that if there’s a spring-boot starter jar then spring > >>>>>>> will create a singleton instance of something in the jar, and if > >>>>>>> there’s a json file that will guide or describe configuring this > >>>>>>> instance with data from somewhere. If this is roughly correct, then > >>>>>>> the 2nd and 3rd kinds correspond to non-configurable instances. > >>>>>>> > >>>>>>> I’ve discovered that: > >>>>>>> > >>>>>>> - several components have starters that are not listed in the > >>>>>>> existing list > >>>>>>> > >>>>>> > >>>>>> Oh which ones for example is that? > >>>>>> > >>>>>> For Camel on Spring Boot then we only support the -starter JARs. So if > >>>>>> there is not a starter then its not supported. > >>>>>> And yes some of the starters are just shallow and empty/almost empty. > >>>>>> > >>>>>> But it's our way to curate what we support and make sense to use on > >>>>>> Spring Boot. > >>>>>> > >>>>>> For Camel on Karaf its the features.xml file that defines what we > >>>>>> support there. > >>>>>> And ditto for Quarkus with the extensions. > >>>>>> > >>>>>> > >>>>>>> - quite a few components main adoc documentation do not include the > >>>>>>> relevant autoconfig information (even if it’s “no options”) > >>>>>>> Most of these are linked to from the “list” page, so I’d imagine it > >>>>>>> would be pretty confusing to follow the link and find no information > >>>>>>> about spring-boot. > >>>>>>> > >>>>>>> Evidently, the current process is not maintainable. > >>>>>>> > >>>>>>> The (new, jsonpath based) autoconfig doc generation is based on > >>>>>>> putting the name of the appropriate json file in the main adoc file > >>>>>>> as a header attribute. > >>>>>>> > >>>>>>> I think we can produce something more resilient by: > >>>>>>> > >>>>>>> - for kinds (2) and (3), put a no-options appropriately named json > >>>>>>> file in the starter project under src/main/docs. This will not get > >>>>>>> added to the starter jar, unlike the properly generated json files. > >>>>>>> > >>>>>>> - put the corresponding header attribute in the appropriate main adoc > >>>>>>> file > >>>>>>> > >>>>>>> Now we should be able to do a basic check, that the number of unique > >>>>>>> json file names from header attributes is equal to the number of json > >>>>>>> files, which should be equal to the number of starters. This should > >>>>>>> be possible to do as part of the website build. > >>>>>>> > >>>>>>> This won’t check that for instance with a starter that serves many > >>>>>>> components, all the components will refer to the starter information, > >>>>>>> but it will detect most or all of the problems I’v found so far. > >>>>>>> > >>>>>>> The preview for https://github.com/apache/camel-website/pull/644 > >>>>>>> <https://github.com/apache/camel-website/pull/644> > >>>>>>> <https://github.com/apache/camel-website/pull/644 > >>>>>>> <https://github.com/apache/camel-website/pull/644>> shows how far > >>>>>>> I’ve gotten with this: the appropriate page to look at is > >>>>>>> https://pr-644--camel.netlify.app/camel-spring-boot/latest/list-2.html > >>>>>>> > >>>>>>> <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list-2.html> > >>>>>>> > >>>>>>> <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list-2.html > >>>>>>> > >>>>>>> <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list-2.html>> > >>>>>>> This has, in addition to the tables of spring-boot > >>>>>>> autoconfigure-enabled components, counts of json files used and > >>>>>>> existing (there are 5 unused), and tables of non-spring-boot-enabled > >>>>>>> components/dataformats… > >>>>>>> All the dataformats and languages are set up properly. There are 2 > >>>>>>> components that aren’t spring-boot-enabled and a whole lot of “others" > >>>>>>> > >>>>>>> I think that actually failing the build when there are unused .json > >>>>>>> files and also finding out which json files are unused will have to > >>>>>>> wait for an upgrade to the antora-indexer… unfortunately this will > >>>>>>> require some syntax changes. > >>>>>>> > >>>>>>> I’d like to polish these changes, replace the list page with the new > >>>>>>> one (with the extra info removed), port this work to the other > >>>>>>> relevant branches, and then pursue releasing the upgraded > >>>>>>> antora-indexer extension and upgrading the syntax, and then work on > >>>>>>> making the checks work. > >>>>>>> > >>>>>>> Does this seem reasonable? > >>>>>>> > >>>>>> > >>>>>> Yeah sure continue your great work and effort on improving the > >>>>>> documentation for all the Camel projects on the website. > >>>>>> > >>>>>> > >>>>>> > >>>>>>> —— > >>>>>>> > >>>>>>> Another feature of the preview is that I found out how to make the > >>>>>>> option names in all the jsonpath generated tables link to themselves > >>>>>>> and display a marker on hover, just like section titles, so you can > >>>>>>> now easily copy the link to an option and paste it somewhere. (the > >>>>>>> names have been links for a while, but there was no practical way to > >>>>>>> find out what the link was!) > >>>>>>> > >>>>>>> E.g. … > >>>>>>> https://pr-644--camel.netlify.app/components/latest/amqp-component.html#_endpoint_query_option_disableReplyTo > >>>>>>> > >>>>>>> <https://pr-644--camel.netlify.app/components/latest/amqp-component.html#_endpoint_query_option_disableReplyTo><https://pr-644--camel.netlify.app/components/latest/amqp-component.html#_endpoint_query_option_disableReplyTo > >>>>>>> > >>>>>>> <https://pr-644--camel.netlify.app/components/latest/amqp-component.html#_endpoint_query_option_disableReplyTo>> > >>>>>>> > >>>>>>> David Jencks > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Claus Ibsen > >>>>>> ----------------- > >>>>>> http://davsclaus.com <http://davsclaus.com/> <http://davsclaus.com/ > >>>>>> <http://davsclaus.com/>> @davsclaus > >>>>>> Camel in Action 2: https://www.manning.com/ibsen2 > >>>>>> <https://www.manning.com/ibsen2> <https://www.manning.com/ibsen2 > >>>>>> <https://www.manning.com/ibsen2>> > >>>> > >>> > >> > > > > > > -- > > Claus Ibsen > > ----------------- > > http://davsclaus.com <http://davsclaus.com/> @davsclaus > > Camel in Action 2: https://www.manning.com/ibsen2 > > <https://www.manning.com/ibsen2> -- Claus Ibsen ----------------- http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
