Re: Adding something new into Camel DSL -- but how?
Hi Steven Good to know. Sorry I am too focused on 3.14 LTS release, a few last things to clear up. About ascii doc links, then they can be tricky. There has been some improvements/updates to the website build and whatnot, so its a bit in "flux" at the moment. So if you need to continue then do not add links with xref yet, but add a TODO etc in the doc then we can look at that later. On Sat, Dec 4, 2021 at 4:51 PM Steve973 wrote: > > Hi, Claus, and thanks for the reply. I created the ticket on 31 Oct: > https://issues.apache.org/jira/projects/CAMEL/issues/CAMEL-17154 and I am > working on it in the evenings as time allows. I have finished the initial > implementation, and I am working on tests and the asciidoc stuff. So far, > I have gotten through at least half of the happy-path unit tests, and I > have some questions about aspects of the asciidoc format and content, > mostly related to the links that I see in some of the other adoc files that > I am using as examples. I just added my notes about camel-core, and I > added a comment with some of my design and implementation ideas. Any of my > questions can wait until the 3.14 LTS release is complete. > > Thanks again, > Steve > > On Tue, Nov 30, 2021 at 3:28 PM Claus Ibsen wrote: > > > Hi Steven > > > > > > On Sun, Nov 28, 2021 at 5:01 PM Steve973 wrote: > > > > > > Hi, Claus. I have done what I hope was a fairly full tracing through the > > > code of camel-core, and I can now say that I appreciate your concern > > about > > > adding things to core! I also took a closer look at the Saga and Kamelet > > > components, and that helped to correct a misconception that I had -- > > > components are not solely for implementing things like transport > > > implementations into Camel, and that it is (arguably) a more suitable > > > mechanism for adding anything that does not have a strict and solid > > reason > > > for existing in core. So I will implement this EIP as a component, and > > not > > > inside of core. I created a markdown file with my notes when I traced > > > through core. If you, or anyone else, might be interested, I can attach > > it > > > to a reply here. > > > > > > > Thanks for your patience and impressive to hear that you dived into > > the deep source code and > > found the good, bad, "and ugly" of our huge code base. > > > > One "bad" about EIPs is that the model is defined tightly into the > > same JAR which means > > that adding new EIPs is harder - or that users can't develop custom > > EIPs and plugin as they need. > > > > But on the other side then everybody of Camel has the same DSL, and > > therefore its "standard" and familiar. > > > > About your new dynamic eip, then did you create a JIRA for it? If not > > you are welcome to do so. > > And then attach the notes in the JIRA so its not lost, and we have it > > tracked for everybody to see, > > and also easier find in years to come. > > > > I am a bit busy this week with Red Hat virtual company event the > > entire week. And on the same time I must finish some work on health > > checks. > > However I should get some "clear timer" later in December. > > > > And we need Camel 3.14 LTS released first also. So have some patience, > > and then we have better clear road for 3.15 onwards to add new EIPs. > > > > > > > Thanks, > > > Steve > > > > > > On Wed, Nov 24, 2021 at 10:20 AM Claus Ibsen > > wrote: > > > > > > > Hi > > > > > > > > Adding EIPs or changes to the DSL is not a Camel end user task, and is > > > > not documented thoroughly. > > > > Adding new EIPs happens more rarely and there is maybe a little > > > > hesitation to add because when its in the DSL it stays for a long > > > > time. > > > > Also the DSL can become bloated - there are some mistakes from Camel > > > > v1 and v2 that we have today. > > > > > > > > That said to add a new EIP there are 2 ways > > > > > > > > - The EIP is implemented fully in the core, so all its code are in > > core. > > > > - The EIP is implemeted in a component and are located in camel-xxx in > > > > the components folder. Examples are circuit breakers, saga, kamelet > > > > and maybe others I forgot. > > > > > > > > So first you need to find out if its the former or the latter. However > > > > for both cases, the the DSL model are in the core and there is a fair > > > > bit of code to write to add a new model. > > > > Therefore its a good idea to find a simple EIP and see where its used > > > > in the code - can be ConvertBodyDefinition (are in core). Or if its > > > > outside then maybe KameletDefinition which is the latest EIP added. > > > > > > > > The reifer is the "guy" that sits between the model (DSL) and the > > > > processor (implementation), so all 3 are separated in their own JARs. > > > > > > > > And on top of that then there is also the new YAML DSL, so adding a > > > > new EIP can require some coding there too, but that can be done > > > > afterwards. > > > > At first is to get the regular Java and XML DSLs to work. T
Re: Adding something new into Camel DSL -- but how?
Hi, Claus, and thanks for the reply. I created the ticket on 31 Oct: https://issues.apache.org/jira/projects/CAMEL/issues/CAMEL-17154 and I am working on it in the evenings as time allows. I have finished the initial implementation, and I am working on tests and the asciidoc stuff. So far, I have gotten through at least half of the happy-path unit tests, and I have some questions about aspects of the asciidoc format and content, mostly related to the links that I see in some of the other adoc files that I am using as examples. I just added my notes about camel-core, and I added a comment with some of my design and implementation ideas. Any of my questions can wait until the 3.14 LTS release is complete. Thanks again, Steve On Tue, Nov 30, 2021 at 3:28 PM Claus Ibsen wrote: > Hi Steven > > > On Sun, Nov 28, 2021 at 5:01 PM Steve973 wrote: > > > > Hi, Claus. I have done what I hope was a fairly full tracing through the > > code of camel-core, and I can now say that I appreciate your concern > about > > adding things to core! I also took a closer look at the Saga and Kamelet > > components, and that helped to correct a misconception that I had -- > > components are not solely for implementing things like transport > > implementations into Camel, and that it is (arguably) a more suitable > > mechanism for adding anything that does not have a strict and solid > reason > > for existing in core. So I will implement this EIP as a component, and > not > > inside of core. I created a markdown file with my notes when I traced > > through core. If you, or anyone else, might be interested, I can attach > it > > to a reply here. > > > > Thanks for your patience and impressive to hear that you dived into > the deep source code and > found the good, bad, "and ugly" of our huge code base. > > One "bad" about EIPs is that the model is defined tightly into the > same JAR which means > that adding new EIPs is harder - or that users can't develop custom > EIPs and plugin as they need. > > But on the other side then everybody of Camel has the same DSL, and > therefore its "standard" and familiar. > > About your new dynamic eip, then did you create a JIRA for it? If not > you are welcome to do so. > And then attach the notes in the JIRA so its not lost, and we have it > tracked for everybody to see, > and also easier find in years to come. > > I am a bit busy this week with Red Hat virtual company event the > entire week. And on the same time I must finish some work on health > checks. > However I should get some "clear timer" later in December. > > And we need Camel 3.14 LTS released first also. So have some patience, > and then we have better clear road for 3.15 onwards to add new EIPs. > > > > Thanks, > > Steve > > > > On Wed, Nov 24, 2021 at 10:20 AM Claus Ibsen > wrote: > > > > > Hi > > > > > > Adding EIPs or changes to the DSL is not a Camel end user task, and is > > > not documented thoroughly. > > > Adding new EIPs happens more rarely and there is maybe a little > > > hesitation to add because when its in the DSL it stays for a long > > > time. > > > Also the DSL can become bloated - there are some mistakes from Camel > > > v1 and v2 that we have today. > > > > > > That said to add a new EIP there are 2 ways > > > > > > - The EIP is implemented fully in the core, so all its code are in > core. > > > - The EIP is implemeted in a component and are located in camel-xxx in > > > the components folder. Examples are circuit breakers, saga, kamelet > > > and maybe others I forgot. > > > > > > So first you need to find out if its the former or the latter. However > > > for both cases, the the DSL model are in the core and there is a fair > > > bit of code to write to add a new model. > > > Therefore its a good idea to find a simple EIP and see where its used > > > in the code - can be ConvertBodyDefinition (are in core). Or if its > > > outside then maybe KameletDefinition which is the latest EIP added. > > > > > > The reifer is the "guy" that sits between the model (DSL) and the > > > processor (implementation), so all 3 are separated in their own JARs. > > > > > > And on top of that then there is also the new YAML DSL, so adding a > > > new EIP can require some coding there too, but that can be done > > > afterwards. > > > At first is to get the regular Java and XML DSLs to work. The XML DSL > > > is source generated with a maven plugin, and it can "have weird > > > problems" if the DSL model is wrong. > > > So if you have a very complex EIP then it can be tricky sometimes. > > > > > > > > > > > > On Mon, Nov 22, 2021 at 10:16 PM Steve973 wrote: > > > > > > > > Hi, all. I am working on a new version of the Dynamic Router > processor > > > > that implements the pattern spec much more closely, and I have the > > > initial > > > > implementation complete, but I want to test it (among other ways) by > > > using > > > > it in the DSL. I haven't seen any documentation on how to do this, > or > > > any > > > > tuto
Re: Adding something new into Camel DSL -- but how?
Hi Steven On Sun, Nov 28, 2021 at 5:01 PM Steve973 wrote: > > Hi, Claus. I have done what I hope was a fairly full tracing through the > code of camel-core, and I can now say that I appreciate your concern about > adding things to core! I also took a closer look at the Saga and Kamelet > components, and that helped to correct a misconception that I had -- > components are not solely for implementing things like transport > implementations into Camel, and that it is (arguably) a more suitable > mechanism for adding anything that does not have a strict and solid reason > for existing in core. So I will implement this EIP as a component, and not > inside of core. I created a markdown file with my notes when I traced > through core. If you, or anyone else, might be interested, I can attach it > to a reply here. > Thanks for your patience and impressive to hear that you dived into the deep source code and found the good, bad, "and ugly" of our huge code base. One "bad" about EIPs is that the model is defined tightly into the same JAR which means that adding new EIPs is harder - or that users can't develop custom EIPs and plugin as they need. But on the other side then everybody of Camel has the same DSL, and therefore its "standard" and familiar. About your new dynamic eip, then did you create a JIRA for it? If not you are welcome to do so. And then attach the notes in the JIRA so its not lost, and we have it tracked for everybody to see, and also easier find in years to come. I am a bit busy this week with Red Hat virtual company event the entire week. And on the same time I must finish some work on health checks. However I should get some "clear timer" later in December. And we need Camel 3.14 LTS released first also. So have some patience, and then we have better clear road for 3.15 onwards to add new EIPs. > Thanks, > Steve > > On Wed, Nov 24, 2021 at 10:20 AM Claus Ibsen wrote: > > > Hi > > > > Adding EIPs or changes to the DSL is not a Camel end user task, and is > > not documented thoroughly. > > Adding new EIPs happens more rarely and there is maybe a little > > hesitation to add because when its in the DSL it stays for a long > > time. > > Also the DSL can become bloated - there are some mistakes from Camel > > v1 and v2 that we have today. > > > > That said to add a new EIP there are 2 ways > > > > - The EIP is implemented fully in the core, so all its code are in core. > > - The EIP is implemeted in a component and are located in camel-xxx in > > the components folder. Examples are circuit breakers, saga, kamelet > > and maybe others I forgot. > > > > So first you need to find out if its the former or the latter. However > > for both cases, the the DSL model are in the core and there is a fair > > bit of code to write to add a new model. > > Therefore its a good idea to find a simple EIP and see where its used > > in the code - can be ConvertBodyDefinition (are in core). Or if its > > outside then maybe KameletDefinition which is the latest EIP added. > > > > The reifer is the "guy" that sits between the model (DSL) and the > > processor (implementation), so all 3 are separated in their own JARs. > > > > And on top of that then there is also the new YAML DSL, so adding a > > new EIP can require some coding there too, but that can be done > > afterwards. > > At first is to get the regular Java and XML DSLs to work. The XML DSL > > is source generated with a maven plugin, and it can "have weird > > problems" if the DSL model is wrong. > > So if you have a very complex EIP then it can be tricky sometimes. > > > > > > > > On Mon, Nov 22, 2021 at 10:16 PM Steve973 wrote: > > > > > > Hi, all. I am working on a new version of the Dynamic Router processor > > > that implements the pattern spec much more closely, and I have the > > initial > > > implementation complete, but I want to test it (among other ways) by > > using > > > it in the DSL. I haven't seen any documentation on how to do this, or > > any > > > tutorials. Have I missed something? Could anyone give some kind of > > > overview? I could probably use a different processor and trace up > > through > > > the classes to see what all is involved, but I don't want to miss > > anything, > > > and this would be great documentation to have. If it would be helpful > > for > > > other contributors, I would not mind putting it together for inclusion in > > > the overall documentation. > > > > > > Thanks, > > > Steve > > > > > > > > -- > > Claus Ibsen > > - > > http://davsclaus.com @davsclaus > > Camel in Action 2: https://www.manning.com/ibsen2 > > -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
Re: Adding something new into Camel DSL -- but how?
Hi, Claus. I have done what I hope was a fairly full tracing through the code of camel-core, and I can now say that I appreciate your concern about adding things to core! I also took a closer look at the Saga and Kamelet components, and that helped to correct a misconception that I had -- components are not solely for implementing things like transport implementations into Camel, and that it is (arguably) a more suitable mechanism for adding anything that does not have a strict and solid reason for existing in core. So I will implement this EIP as a component, and not inside of core. I created a markdown file with my notes when I traced through core. If you, or anyone else, might be interested, I can attach it to a reply here. Thanks, Steve On Wed, Nov 24, 2021 at 10:20 AM Claus Ibsen wrote: > Hi > > Adding EIPs or changes to the DSL is not a Camel end user task, and is > not documented thoroughly. > Adding new EIPs happens more rarely and there is maybe a little > hesitation to add because when its in the DSL it stays for a long > time. > Also the DSL can become bloated - there are some mistakes from Camel > v1 and v2 that we have today. > > That said to add a new EIP there are 2 ways > > - The EIP is implemented fully in the core, so all its code are in core. > - The EIP is implemeted in a component and are located in camel-xxx in > the components folder. Examples are circuit breakers, saga, kamelet > and maybe others I forgot. > > So first you need to find out if its the former or the latter. However > for both cases, the the DSL model are in the core and there is a fair > bit of code to write to add a new model. > Therefore its a good idea to find a simple EIP and see where its used > in the code - can be ConvertBodyDefinition (are in core). Or if its > outside then maybe KameletDefinition which is the latest EIP added. > > The reifer is the "guy" that sits between the model (DSL) and the > processor (implementation), so all 3 are separated in their own JARs. > > And on top of that then there is also the new YAML DSL, so adding a > new EIP can require some coding there too, but that can be done > afterwards. > At first is to get the regular Java and XML DSLs to work. The XML DSL > is source generated with a maven plugin, and it can "have weird > problems" if the DSL model is wrong. > So if you have a very complex EIP then it can be tricky sometimes. > > > > On Mon, Nov 22, 2021 at 10:16 PM Steve973 wrote: > > > > Hi, all. I am working on a new version of the Dynamic Router processor > > that implements the pattern spec much more closely, and I have the > initial > > implementation complete, but I want to test it (among other ways) by > using > > it in the DSL. I haven't seen any documentation on how to do this, or > any > > tutorials. Have I missed something? Could anyone give some kind of > > overview? I could probably use a different processor and trace up > through > > the classes to see what all is involved, but I don't want to miss > anything, > > and this would be great documentation to have. If it would be helpful > for > > other contributors, I would not mind putting it together for inclusion in > > the overall documentation. > > > > Thanks, > > Steve > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2 >
Re: Adding something new into Camel DSL -- but how?
Hi, Claus, and thank you for the heads up. Since this is an alternate Dynamic Router implementation, it seems like it might be best suited to be implemented in the core, unless it would make better sense to implement it as a component for possible inclusion, later, in core. Either way, does it make sense to call it DynamicRouter2? The main difference that distinguishes this implementation from the current dynamic router is that this one has a control channel for potential recipients to register their rules for accepting messages, or to de-register later. It is a bit similar to the Choice processor. Since there is "mina" and "mina2", it looks like that establishes the convention for naming alternate/subsequent implementations. I will trace another processor through the code (perhaps Choice), and see how it is all tied together. While I am doing this, I can take good notes. Would it be useful to formally document this process? I would be glad to do it, and I would need to know about your documentation standards and format, and where I might include it. Lastly, for now, is there any special process for getting something new approved for inclusion in core, or is it the usual merge request process? If you would like to hear about my design and implementation considerations, I can write something up and send it to you for review. I have the initial/basic processor implementation complete, and it is just waiting for integration. I am eager to get it integrated well enough to write some tests to verify its functionality. Thanks again! Steve On Wed, Nov 24, 2021 at 10:20 AM Claus Ibsen wrote: > Hi > > Adding EIPs or changes to the DSL is not a Camel end user task, and is > not documented thoroughly. > Adding new EIPs happens more rarely and there is maybe a little > hesitation to add because when its in the DSL it stays for a long > time. > Also the DSL can become bloated - there are some mistakes from Camel > v1 and v2 that we have today. > > That said to add a new EIP there are 2 ways > > - The EIP is implemented fully in the core, so all its code are in core. > - The EIP is implemeted in a component and are located in camel-xxx in > the components folder. Examples are circuit breakers, saga, kamelet > and maybe others I forgot. > > So first you need to find out if its the former or the latter. However > for both cases, the the DSL model are in the core and there is a fair > bit of code to write to add a new model. > Therefore its a good idea to find a simple EIP and see where its used > in the code - can be ConvertBodyDefinition (are in core). Or if its > outside then maybe KameletDefinition which is the latest EIP added. > > The reifer is the "guy" that sits between the model (DSL) and the > processor (implementation), so all 3 are separated in their own JARs. > > And on top of that then there is also the new YAML DSL, so adding a > new EIP can require some coding there too, but that can be done > afterwards. > At first is to get the regular Java and XML DSLs to work. The XML DSL > is source generated with a maven plugin, and it can "have weird > problems" if the DSL model is wrong. > So if you have a very complex EIP then it can be tricky sometimes. > > > > On Mon, Nov 22, 2021 at 10:16 PM Steve973 wrote: > > > > Hi, all. I am working on a new version of the Dynamic Router processor > > that implements the pattern spec much more closely, and I have the > initial > > implementation complete, but I want to test it (among other ways) by > using > > it in the DSL. I haven't seen any documentation on how to do this, or > any > > tutorials. Have I missed something? Could anyone give some kind of > > overview? I could probably use a different processor and trace up > through > > the classes to see what all is involved, but I don't want to miss > anything, > > and this would be great documentation to have. If it would be helpful > for > > other contributors, I would not mind putting it together for inclusion in > > the overall documentation. > > > > Thanks, > > Steve > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2 >
Re: Adding something new into Camel DSL -- but how?
Hi Adding EIPs or changes to the DSL is not a Camel end user task, and is not documented thoroughly. Adding new EIPs happens more rarely and there is maybe a little hesitation to add because when its in the DSL it stays for a long time. Also the DSL can become bloated - there are some mistakes from Camel v1 and v2 that we have today. That said to add a new EIP there are 2 ways - The EIP is implemented fully in the core, so all its code are in core. - The EIP is implemeted in a component and are located in camel-xxx in the components folder. Examples are circuit breakers, saga, kamelet and maybe others I forgot. So first you need to find out if its the former or the latter. However for both cases, the the DSL model are in the core and there is a fair bit of code to write to add a new model. Therefore its a good idea to find a simple EIP and see where its used in the code - can be ConvertBodyDefinition (are in core). Or if its outside then maybe KameletDefinition which is the latest EIP added. The reifer is the "guy" that sits between the model (DSL) and the processor (implementation), so all 3 are separated in their own JARs. And on top of that then there is also the new YAML DSL, so adding a new EIP can require some coding there too, but that can be done afterwards. At first is to get the regular Java and XML DSLs to work. The XML DSL is source generated with a maven plugin, and it can "have weird problems" if the DSL model is wrong. So if you have a very complex EIP then it can be tricky sometimes. On Mon, Nov 22, 2021 at 10:16 PM Steve973 wrote: > > Hi, all. I am working on a new version of the Dynamic Router processor > that implements the pattern spec much more closely, and I have the initial > implementation complete, but I want to test it (among other ways) by using > it in the DSL. I haven't seen any documentation on how to do this, or any > tutorials. Have I missed something? Could anyone give some kind of > overview? I could probably use a different processor and trace up through > the classes to see what all is involved, but I don't want to miss anything, > and this would be great documentation to have. If it would be helpful for > other contributors, I would not mind putting it together for inclusion in > the overall documentation. > > Thanks, > Steve -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
Adding something new into Camel DSL -- but how?
Hi, all. I am working on a new version of the Dynamic Router processor that implements the pattern spec much more closely, and I have the initial implementation complete, but I want to test it (among other ways) by using it in the DSL. I haven't seen any documentation on how to do this, or any tutorials. Have I missed something? Could anyone give some kind of overview? I could probably use a different processor and trace up through the classes to see what all is involved, but I don't want to miss anything, and this would be great documentation to have. If it would be helpful for other contributors, I would not mind putting it together for inclusion in the overall documentation. Thanks, Steve