Hi The -common JARs are not starters, and they should not be listed.
On Mon, Oct 18, 2021 at 7:45 AM David Jencks <david.a.jen...@gmail.com> wrote: > > Locally I got the ability to show unused starter json files working, and > fixed the obvious problems: > > avro-rpc was using avro > paho-mqtt5 was using paho > in 3.7.x, grape and joor were set up wrong. > > There are still some “starters” i’m not sure about: > > debezium-common > http-common > jetty-common > > In a previous commit I added empty json files for these on the theory that if > they were starters they should have instructions on how to use them, but they > don’t have corresponding main camel doc pages. Are these starters > automatically installed by the corresponding non-common starter artifacts > such as camel-debezium-mongodb-starter? If so, I can remove the empty json > files. If not, how should their usage be documented? > > I’m hoping to get my updated Antora extensions published in the next couple > of days, at which point we can upgrade Camel to use them. > > David Jencks > > > On Oct 13, 2021, at 8:14 PM, David Jencks <david.a.jen...@gmail.com> wrote: > > > > They are part of the camel-core-starter, although it’s hard to find the 2 > > options per language among the 147 options for the starter. This suggests > > that, for starters that encompass more than one component, it would be > > useful to only show the options relevant to that component on the > > components’ page. > > > > In any case I believe everything is working properly and I’ve merged all > > the 9 PRs. > > > > The core and non-core languages are currently in different tables on the > > spring-boot list page: I should be able to put them back into one table > > after upgrading antora-indexer. > > > > David Jencks > > > >> On Oct 12, 2021, at 11:03 PM, David Jencks <david.a.jen...@gmail.com > >> <mailto:david.a.jen...@gmail.com>> wrote: > >> > >> There’s at least one problem…. (also asked on zulip) > >> > >> Are camel-core-languages actually part of camel-spring-boot? > >> > >> They are listed in the table in "list of starters" but with an artifactid > >> of "camel-base" which isn't a starter. The current site doesn't have > >> autoconfiguration info for them either. Since they are in the table, I > >> added info to them for my PRs, but it's at least partly wrong, since it > >> comes out as needing camel-core-languages-starter, which doesn't exist. > >> cf. > >> https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages > >> > >> <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list.html#_camel_languages>, > >> > >> https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration > >> > >> <https://pr-644--camel.netlify.app/components/latest/languages/header-language.html#_spring_boot_auto_configuration>. > >> Since they say they are part of camel-core, I've used core.json to > >> construct the autoconfig options table, but that doesn't look very > >> relevant to me. > >> > >> Is there a starter needed for them? If so, which one? camel-core-starter > >> seems like a possibility, is it correct? > >> > >> If no starter is needed, why are they in the spring-boot starters table? > >> > >> I hope to be able to merge everything tomorrow, it got too late tonight. > >> > >> David Jencks > >> > >>> On Oct 12, 2021, at 7:41 PM, David Jencks <david.a.jen...@gmail.com > >>> <mailto:david.a.jen...@gmail.com>> wrote: > >>> > >>> The part of this work I’m ready for now is done, except for some > >>> commit-squashing and dealing with check style errors :-) > >>> > >>> Preview at https://pr-644--camel.netlify.app > >>> <https://pr-644--camel.netlify.app/> > >>> > >>> I seem to only be able to write AsciiDoc by now…. :-) Here’s a list of > >>> the differences between table rows new/old: > >>> > >>> —— > >>> = Differences between spring-boot list-old and list > >>> > >>> == latest (main) > >>> > >>> === In list, not list-old: > >>> > >>> * disruptor-vm-component > >>> * splunk-hec-component > >>> > >>> * (3 bindy dataformats collapsed to one link, corresponding to the single > >>> main bindy doc page) > >>> > >>> * csimple-language > >>> > >>> == 3.12.x > >>> === In list, not list-old: > >>> > >>> * disruptor-vm-component > >>> * splunk-hec-component > >>> > >>> * (3 bindy dataformats collapsed to one link, corresponding to the single > >>> main bindy doc page) > >>> > >>> * csimple-language > >>> > >>> == 3.11.x > >>> === In list, not list-old: > >>> > >>> * splunk-hec-component > >>> * (3 bindy dataformats collapsed to one link, corresponding to the single > >>> main bindy doc page) > >>> * csimple-language > >>> > >>> Moved to separate table: > >>> > >>> * camel-spring-cloud > >>> * camel-spring-cloud-consul > >>> * camel-spring-cloud-netflix > >>> * camel-spring-cloud-zookeeper > >>> > >>> == 3.7.x > >>> === In list, not list-old: > >>> * splunk-hec-component > >>> * (3 bindy dataformats collapsed to one link, corresponding to the single > >>> main bindy doc page) > >>> * csimple-language > >>> > >>> > >>> Moved to separate table: > >>> > >>> * camel-spring-cloud > >>> * camel-spring-cloud-consul > >>> * camel-spring-cloud-netflix > >>> * camel-spring-cloud-zookeeper > >>> > >>> —— > >>> > >>> There are also some presentation differences in the tables, the new ones > >>> follow the tables in main camel components more closely, e.g. > >>> > >>> <td class="tableblock halign-left valign-top"><p > >>> class="tableblock">Stable</p></td> > >>> >> > >>> <td class="tableblock halign-left valign-top"><p > >>> class="tableblock">Stable-deprecated</p></td> > >>> > >>> as the deprecation marker. > >>> > >>> Also quite a few core languages now list the starter as > >>> > >>> <td class="tableblock halign-left valign-top"><p > >>> class="tableblock">camel-core-languages-starter</p></td> > >>> rather than > >>> <td class="tableblock halign-left valign-top"><p > >>> class="tableblock">camel-base</p></td> > >>> > >>> Is this correct? > >>> > >>> There are 8 PRs: > >>> (Camel) > >>> Main spring boot jsonpath (allow me to squash and merge) > >>> <https://github.com/apache/camel/pull/6261> > >>> Camel 3.12.x spring boot jsonpath (please let me squash and merge) > >>> <https://github.com/apache/camel/pull/6263> > >>> Camel 3.11.x spring boot jsonpath (please let me squash and merge) > >>> <https://github.com/apache/camel/pull/6264> > >>> Camel 3.7.x spring boot jsonpath (please let me squash and merge) > >>> <https://github.com/apache/camel/pull/6262> > >>> (Camel-spring-boot) > >>> Main jsonpath (please let me squash and merge) > >>> <https://github.com/apache/camel-spring-boot/pull/388> > >>> Camel spring boot 3.12.x jsonpath (please let me squash and merge) > >>> <https://github.com/apache/camel-spring-boot/pull/387> > >>> Camel spring boot 3.11.x jsonpath (please let me squash and merge) > >>> <https://github.com/apache/camel-spring-boot/pull/386> > >>> Camel spring boot 3.7.x jsonpath (please let me squash and merge) > >>> <https://github.com/apache/camel-spring-boot/pull/385> > >>> > >>> If everything looks good I plan to squash each PR to 2 or 3 commits (code > >>> changes and generated changes) and rebase/merge. I don’t think any > >>> playbook changes are needed. > >>> > >>> I’m completing a big upgrade of the antora-indexer and jsonpath > >>> extensions, and plan to finish that next. It will involve changing the > >>> syntax of most uses of these extensions. After that I expect to be able > >>> to implement automated checking for starters that aren’t linked to by > >>> main camel components. I think there are about 5, but have no way to > >>> find them at this point. > >>> > >>> > >>> David Jencks > >>> > >>>> On Oct 12, 2021, at 12:01 AM, Claus Ibsen <claus.ib...@gmail.com > >>>> <mailto:claus.ib...@gmail.com>> wrote: > >>>> > >>>> On Mon, Oct 11, 2021 at 8:07 AM David Jencks <david.a.jen...@gmail.com > >>>> <mailto:david.a.jen...@gmail.com>> wrote: > >>>>> > >>>>> I’ve been working on generating the spring-boot autoconfig info from > >>>>> the json files generated during the spring-boot build, and the spring > >>>>> boot “list.adoc” tables using the indexer extension (like the > >>>>> components tables are generated). > >>>>> > >>>>> My understanding of the spring-boot stuff is that: > >>>>> > >>>>> - any Camel component can be used in Spring > >>>>> > >>>>> - Some camel components/dataformats… can participate in spring-boot > >>>>> auto configure. These components are identified as having a > >>>>> corresponding camel-*-starter project in the spring-boot repo. > >>>>> > >>>>> - There are 3 kinds of these: > >>>>> > >>>>> 1. There’s (generated) java code and a generated json file > >>>>> > >>>>> 2. There’s generated java code but no generated json file (e.g. jasypt) > >>>>> > >>>>> 3. There’s no java code and no json file, just some properties files > >>>>> etc. (e.g. aws-xray) > >>>>> > >>>>> I’m really not familiar with spring-boot or the philosophy behind it, > >>>>> but my guess is that if there’s a spring-boot starter jar then spring > >>>>> will create a singleton instance of something in the jar, and if > >>>>> there’s a json file that will guide or describe configuring this > >>>>> instance with data from somewhere. If this is roughly correct, then > >>>>> the 2nd and 3rd kinds correspond to non-configurable instances. > >>>>> > >>>>> I’ve discovered that: > >>>>> > >>>>> - several components have starters that are not listed in the existing > >>>>> list > >>>>> > >>>> > >>>> Oh which ones for example is that? > >>>> > >>>> For Camel on Spring Boot then we only support the -starter JARs. So if > >>>> there is not a starter then its not supported. > >>>> And yes some of the starters are just shallow and empty/almost empty. > >>>> > >>>> But it's our way to curate what we support and make sense to use on > >>>> Spring Boot. > >>>> > >>>> For Camel on Karaf its the features.xml file that defines what we > >>>> support there. > >>>> And ditto for Quarkus with the extensions. > >>>> > >>>> > >>>>> - quite a few components main adoc documentation do not include the > >>>>> relevant autoconfig information (even if it’s “no options”) > >>>>> Most of these are linked to from the “list” page, so I’d imagine it > >>>>> would be pretty confusing to follow the link and find no information > >>>>> about spring-boot. > >>>>> > >>>>> Evidently, the current process is not maintainable. > >>>>> > >>>>> The (new, jsonpath based) autoconfig doc generation is based on putting > >>>>> the name of the appropriate json file in the main adoc file as a header > >>>>> attribute. > >>>>> > >>>>> I think we can produce something more resilient by: > >>>>> > >>>>> - for kinds (2) and (3), put a no-options appropriately named json file > >>>>> in the starter project under src/main/docs. This will not get added to > >>>>> the starter jar, unlike the properly generated json files. > >>>>> > >>>>> - put the corresponding header attribute in the appropriate main adoc > >>>>> file > >>>>> > >>>>> Now we should be able to do a basic check, that the number of unique > >>>>> json file names from header attributes is equal to the number of json > >>>>> files, which should be equal to the number of starters. This should be > >>>>> possible to do as part of the website build. > >>>>> > >>>>> This won’t check that for instance with a starter that serves many > >>>>> components, all the components will refer to the starter information, > >>>>> but it will detect most or all of the problems I’v found so far. > >>>>> > >>>>> The preview for https://github.com/apache/camel-website/pull/644 > >>>>> <https://github.com/apache/camel-website/pull/644> shows how far I’ve > >>>>> gotten with this: the appropriate page to look at is > >>>>> https://pr-644--camel.netlify.app/camel-spring-boot/latest/list-2.html > >>>>> <https://pr-644--camel.netlify.app/camel-spring-boot/latest/list-2.html> > >>>>> This has, in addition to the tables of spring-boot > >>>>> autoconfigure-enabled components, counts of json files used and > >>>>> existing (there are 5 unused), and tables of non-spring-boot-enabled > >>>>> components/dataformats… > >>>>> All the dataformats and languages are set up properly. There are 2 > >>>>> components that aren’t spring-boot-enabled and a whole lot of “others" > >>>>> > >>>>> I think that actually failing the build when there are unused .json > >>>>> files and also finding out which json files are unused will have to > >>>>> wait for an upgrade to the antora-indexer… unfortunately this will > >>>>> require some syntax changes. > >>>>> > >>>>> I’d like to polish these changes, replace the list page with the new > >>>>> one (with the extra info removed), port this work to the other relevant > >>>>> branches, and then pursue releasing the upgraded antora-indexer > >>>>> extension and upgrading the syntax, and then work on making the checks > >>>>> work. > >>>>> > >>>>> Does this seem reasonable? > >>>>> > >>>> > >>>> Yeah sure continue your great work and effort on improving the > >>>> documentation for all the Camel projects on the website. > >>>> > >>>> > >>>> > >>>>> —— > >>>>> > >>>>> Another feature of the preview is that I found out how to make the > >>>>> option names in all the jsonpath generated tables link to themselves > >>>>> and display a marker on hover, just like section titles, so you can now > >>>>> easily copy the link to an option and paste it somewhere. (the names > >>>>> have been links for a while, but there was no practical way to find out > >>>>> what the link was!) > >>>>> > >>>>> E.g. … > >>>>> https://pr-644--camel.netlify.app/components/latest/amqp-component.html#_endpoint_query_option_disableReplyTo > >>>>> > >>>>> <https://pr-644--camel.netlify.app/components/latest/amqp-component.html#_endpoint_query_option_disableReplyTo> > >>>>> > >>>>> David Jencks > >>>> > >>>> > >>>> > >>>> -- > >>>> Claus Ibsen > >>>> ----------------- > >>>> http://davsclaus.com <http://davsclaus.com/> @davsclaus > >>>> Camel in Action 2: https://www.manning.com/ibsen2 > >>>> <https://www.manning.com/ibsen2> > >> > > > -- Claus Ibsen ----------------- http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
