Francisco, You're spot on on the purpose of the current Process Definition on DI: it provides complementary information to process instances.
Example? Process definition (ie. diagram) for the current instance. On Thu, Jul 11, 2024 at 12:48 PM Francisco Javier Tirado Sarti <[email protected]> wrote: > > Hi, > Since DI already has a ProcessDefinition entity with a service URL, we feel > that adding deployment status was not breaking anything that was not > already broken. > I guess it all depends on what we think should be the purpose of DI. > If it is just to offer information about process instances, Enrique is > right, this is not the place to monitor which process definitions are > available or not. > However, if the purpose of DI is just that, why is the process definition > entity there? as supporting information for the process instance only? > I feel we need to clearly define this before discarding Walters' proposal. > If we take a strict approach, then it means that any software deploying > runtimes and data index will have to develop its own deployment component > with its own db. That's probably fair, but should be written somewhere. > > > > On Tue, Jul 9, 2024 at 3:39 PM Alex Porcelli <[email protected]> wrote: > > > Walter, > > > > I have to agree with Enrique… although your ask is minimal in the code > > changes and impact, the problem is more on the message that it sends. > > > > As pointed out by last email from Enrique, we - as Apache KIE - don’t > > control or have visibility over the whole environment, so it’s virtually > > impossible to guarantee that something is set to not available is really > > not available, and vice versa. > > > > I understand that the serverless operator aims to provide an experience, > > but it also has limitation over the installation… so probably this kind of > > information would be better stored outside the Apache KIE systems / in your > > case, in the downstream product database. > > > > Regards, > > _____________ > > Alex Porcelli > > http://porcelli.me > > > > > > On Tue, Jul 9, 2024 at 1:59 AM Enrique Gonzalez Martinez < > > [email protected]> wrote: > > > > > Hi Walter, > > > > > > I see some problems aside from going on in mixing the data index with > > data > > > does not belong to it like deployment mgmt. E.g. The service url is there > > > not because the DI but the gateway component (mutations) that makes > > changes > > > in runtimes. Something that needs to change in the future. But i digress. > > > > > > Back to this is deployment management. Basically the requirement is to > > > compute the current catalog of process definitions deployed in the > > > environment. > > > > > > I am not against but I am not comfortable mixing with the data index. > > > Because it is not part of the data index goal; provide the last state > > > information of the process instances. > > > > > > Regarding how this approach works and given you are capable of counting > > > properly instances. > > > > > > 1. Let's say you bring down one workflow. The system will be marked as > > > undeployed/deleted. You don't specify how alive process instance will be > > > processed or if you will perform some other action. > > > The other problem i see is trying to already solve platform problems. I > > > mean, if the platform already can compute which containers are deployed > > and > > > therefore which is the proper catalog why should we polute the system > > with > > > this ? Would not make more sense in telling the user how to query that > > info > > > in the platform ? Making platform independent does not give a proper > > > benefits of chasing multiple platform problems regarding deployment / > > > undeployment operations, url changes, counting instances... Will never > > get > > > it right. It is an unsustainable approach. > > > > > > El lun, 8 jul 2024, 11:18, Walter Medvedeo <[email protected]> > > > escribió: > > > > > > > Hi Enrique, the requirement we have from SonataFlow users is very > > simple. > > > > > > > > Imagine the ProcesssDefinitions in the DI as a catalog of all the > > > existing > > > > definitions. > > > > > > > > Similar to the sonataflow-console, the jbpm-console, etc, user > > > > applications query the ProcessDefintions to get the list with all the > > > > existing definitions, and present that list for example in a UI. > > > > > > > > So, imagine that with the ProcessDefinitions query you get the > > following > > > > list: > > > > > > > > purchase-order, serviceUrl_1 > > > > new-customer, serviceUrl_2 > > > > etc. > > > > > > > > Similar to what we do in our consoles, in their UIs, users can pick any > > > > process from that list to create new instances, etc. > > > > > > > > All good. > > > > > > > > In a later point in time, an administrator, decides that the > > > > "new-customer" process is out from the catalog. We won't work with that > > > > ProcessDefinition anymore. > > > > > > > > Very simplified, In SonataFlow Operator driven setups, you can do it by > > > > telling the operator to "un-deploy" the worfklow. As part of this > > > > procedure, the operator can send a basic event to the data index > > telling > > > > that situation. > > > > > > > > So, at the data index, we need to mark the "new-customer" process > > > > definition as "deleted", or better said as "logically deleted". > > (Because, > > > > as I explained in previous comments, we don't want, at least by now, do > > > > physical deletions in the DI) > > > > > > > > To do that, we need to add a "new field". (Simplest approach, we don't > > > > want more elaborated modelings) > > > > > > > > I suggested to name that field as "status", to keep it completely > > > > unrelated from the type of setup/deployment, because, as I mentioned, > > > > kogito supports a plenty of setups, docker, docker compose, Sonataflow > > > > operator driven, standalone JVM, etc. > > > > > > > > So, introduced that we could for example mark "status" = "unavailable", > > > > and maybe this introduced confusion. If that is the case, forget about > > > the > > > > world unavailable, and to follow the reasoning, imagine we mark that > > > field > > > > with "status" = "deleted" (logical deletion) > > > > > > > > Finally, by adding this simple field/concept, totally decoupled from > > the > > > > various setups that people can use, we can facilitate SonataFlow users > > > > applications to query the DI, and only get the ProcessDefinitons that > > are > > > > not "deleted" (logically deleted) > > > > > > > > Select * from ProcessDefinitions where status != deleted > > > > > > > > > > > > Regards, > > > > Walter. > > > > > > > > > > > > > > > > > > > > On 2024/07/04 11:35:08 Enrique Gonzalez Martinez wrote: > > > > > Hi walter, > > > > > I am not certain about the req. You have few things in motion in here > > > > > > > > > > 1. The req says the deployment is still required : Then you have to > > do > > > > > nothing as this information is just available in the index. Query the > > > > > active process instance of a certain procces id and version. You > > dont > > > > need > > > > > to access the process definition. > > > > > > > > > > The req says you need to know if the deployment is required but you > > did > > > > not > > > > > deploy data index. Then you need to check if there pricess instance > > > > > belonging to that deployment. > > > > > > > > > > The req says you need to know if a deployment is > > unavailable/available > > > i > > > > > would say you need to send events related to that deployment and this > > > is > > > > > more tricky. Because you need to somehow define first what is > > > available. > > > > It > > > > > seems your definition is related to what is deployed. In that sense i > > > > would > > > > > rely into the platform. As if you dont want to operate with certain > > > > process > > > > > you just need to drop the container running it. It will be shown in > > the > > > > > platform tool like dooker etc.... > > > > > But if we want to keep track of history i will leave things opening. > > > For > > > > > instance i am interested in we start issuing these events to those > > > > > deployments that are still managing some process instances but dont > > > > accept > > > > > new ones or deployments that have migrated the new process > > definitions > > > to > > > > > new deployments...so i would try to not mess too much with the > > process > > > > > definition but afd a new event in this case open to new transition > > and > > > > > states.... And you can store that in any way you fancy in the data > > > index. > > > > > > > > > > El jue, 4 jul 2024, 13:03, Walter Medvedeo <[email protected]> > > > > escribió: > > > > > > > > > > > Hi Enrique, > > > > > > maybe I couldn't express the concept well. > > > > > > > > > > > > But let's say that the DI has the catalog of all the existing > > > > > > ProcessDefinitions. > > > > > > > > > > > > To mark a definition as "unavailable" is kind of logically remove > > it > > > > from > > > > > > the Catalog. > > > > > > > > > > > > > > > > > > On 2024/07/04 09:59:00 Enrique Gonzalez Martinez wrote: > > > > > > > Hi walter, this won't give you the data you are aiming for. > > > > > > > > > > > > > > If you have two instances and one goes down the state will be set > > > as > > > > not > > > > > > > available unless you compute something in the data index so at > > > least > > > > you > > > > > > > will need more information or duplicate lines. This is unlikely > > to > > > > be the > > > > > > > responsibility of the data index. > > > > > > > > > > > > > > El jue, 4 jul 2024, 11:28, Walter Medvedeo <[email protected] > > > > > > > > > escribió: > > > > > > > > > > > > > > > Hi, the idea is to introduce the requirement/concept first, and > > > > > > summarize > > > > > > > > the impact on the information registered in the DI as much as > > > > possible. > > > > > > > > > > > > > > > > And thus, very summarized, the proposal is to include one more > > > > filed in > > > > > > > > the ProcessDefinitions information registered by the DI, that > > > will > > > > let > > > > > > us > > > > > > > > know if a given definition is still "available" to create > > > > instances. > > > > > > > > > > > > > > > > That is a requirement we have from SonataFlow users. > > > > > > > > > > > > > > > > > > > > > > > > On 2024/07/03 20:50:20 Jason Porter wrote: > > > > > > > > > I am not exactly sure what the proposal is here. I see a > > bunch > > > of > > > > > > > > background info for this new feature, but I don't see an actual > > > > > > proposal. > > > > > > > > > > > > > > > > > > On 2024/07/03 15:06:21 Walter Medvedeo wrote: > > > > > > > > > > Hello Guys, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Let me share some requirement we have. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Right now, as part of the ProcessDefinition information we > > > > store > > > > > > in the > > > > > > > > > > Data Index, we record for example the processId and the > > > > > > serviceURL. > > > > > > > > > > > > > > > > > > > > That information is good for users to know, for instance, > > > which > > > > > > are the > > > > > > > > > > existing ProcessDefitnions, and, based on that information, > > > > they > > > > > > can > > > > > > > > for > > > > > > > > > > example start a new process instance by concatenating the > > > > > > serviceURL + > > > > > > > > “/” > > > > > > > > > > + processId. > > > > > > > > > > > > > > > > > > > > In the context of a SonataFlow Operator driven setups, > > users > > > > need > > > > > > to > > > > > > > > know > > > > > > > > > > if a ProcessDefinition is still “available”. > > > > > > > > > > > > > > > > > > > > For example, users can deploy a WF, use it for some time, > > and > > > > then, > > > > > > > > decide > > > > > > > > > > to un-deploy it. From this point, no more instances of that > > > > > > workflow > > > > > > > > can be > > > > > > > > > > created. The user has just decided to remove that > > deployment > > > > from > > > > > > the > > > > > > > > > > ecosystem. > > > > > > > > > > > > > > > > > > > > However, in the data-index, that definition is still > > > recorded. > > > > And > > > > > > > > users > > > > > > > > > > want to keep it there, since they want to keep consistent > > > > > > information > > > > > > > > about > > > > > > > > > > the already executed instances etc. (The intention is to > > not > > > > > > introduce > > > > > > > > any > > > > > > > > > > information deleting procedure at the DI, etc.) > > > > > > > > > > > > > > > > > > > > On the other hand, users need a way to filter which > > > > definitions are > > > > > > > > still > > > > > > > > > > “available” by using the standard graphql api. > > > > > > > > > > > > > > > > > > > > What we are evaluating is to include a “status” field in > > the > > > > > > > > > > ProcessDefinitions, with the values available/unavailable, > > > and, > > > > > > that > > > > > > > > will > > > > > > > > > > be updated as usual via ClouldEvents. In the case of > > > > SonataFlow, > > > > > > those > > > > > > > > > > events will be produced by the operator, which governs the > > > > > > > > > > deployment/undeployment, etc. Other setups might just > > ignore > > > > that > > > > > > > > field. > > > > > > > > > > > > > > > > > > > > Note: we are looking for a very minimalist approach that > > > lets > > > > us > > > > > > > > record > > > > > > > > > > the situation, and facilitate SonataFlow users to query > > that > > > > > > situation > > > > > > > > by > > > > > > > > > > using the standard Graphql api. > > > > > > > > > > > > > > > > > > > > This is not any sort of more elaborated clustering > > > > > > > > > > management/monitoring/etc. solution. > > > > > > > > > > > > > > > > > > > > Independently of fine grained details about the event to > > > > introduce, > > > > > > > > which > > > > > > > > > > carries more or less the information we are already sending > > > in > > > > the > > > > > > > > other DI > > > > > > > > > > events. > > > > > > > > > > > > > > > > > > > > From the point of view of let’s say non SonataFlow setups, > > do > > > > you > > > > > > guys > > > > > > > > see > > > > > > > > > > any objections? > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > > > > > > > Walter. > > > > > > > > > > ResponderReenviar > > > > > > > > > > Añadir reacción > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
