Hi, Since DI already has a ProcessDefinition entity with a service URL, we feel that adding deployment status was not breaking anything that was not already broken. I guess it all depends on what we think should be the purpose of DI. If it is just to offer information about process instances, Enrique is right, this is not the place to monitor which process definitions are available or not. However, if the purpose of DI is just that, why is the process definition entity there? as supporting information for the process instance only? I feel we need to clearly define this before discarding Walters' proposal. If we take a strict approach, then it means that any software deploying runtimes and data index will have to develop its own deployment component with its own db. That's probably fair, but should be written somewhere.
On Tue, Jul 9, 2024 at 3:39 PM Alex Porcelli <a...@porcelli.me> wrote: > Walter, > > I have to agree with Enrique… although your ask is minimal in the code > changes and impact, the problem is more on the message that it sends. > > As pointed out by last email from Enrique, we - as Apache KIE - don’t > control or have visibility over the whole environment, so it’s virtually > impossible to guarantee that something is set to not available is really > not available, and vice versa. > > I understand that the serverless operator aims to provide an experience, > but it also has limitation over the installation… so probably this kind of > information would be better stored outside the Apache KIE systems / in your > case, in the downstream product database. > > Regards, > _____________ > Alex Porcelli > http://porcelli.me > > > On Tue, Jul 9, 2024 at 1:59 AM Enrique Gonzalez Martinez < > elguard...@gmail.com> wrote: > > > Hi Walter, > > > > I see some problems aside from going on in mixing the data index with > data > > does not belong to it like deployment mgmt. E.g. The service url is there > > not because the DI but the gateway component (mutations) that makes > changes > > in runtimes. Something that needs to change in the future. But i digress. > > > > Back to this is deployment management. Basically the requirement is to > > compute the current catalog of process definitions deployed in the > > environment. > > > > I am not against but I am not comfortable mixing with the data index. > > Because it is not part of the data index goal; provide the last state > > information of the process instances. > > > > Regarding how this approach works and given you are capable of counting > > properly instances. > > > > 1. Let's say you bring down one workflow. The system will be marked as > > undeployed/deleted. You don't specify how alive process instance will be > > processed or if you will perform some other action. > > The other problem i see is trying to already solve platform problems. I > > mean, if the platform already can compute which containers are deployed > and > > therefore which is the proper catalog why should we polute the system > with > > this ? Would not make more sense in telling the user how to query that > info > > in the platform ? Making platform independent does not give a proper > > benefits of chasing multiple platform problems regarding deployment / > > undeployment operations, url changes, counting instances... Will never > get > > it right. It is an unsustainable approach. > > > > El lun, 8 jul 2024, 11:18, Walter Medvedeo <wmedve...@apache.org> > > escribió: > > > > > Hi Enrique, the requirement we have from SonataFlow users is very > simple. > > > > > > Imagine the ProcesssDefinitions in the DI as a catalog of all the > > existing > > > definitions. > > > > > > Similar to the sonataflow-console, the jbpm-console, etc, user > > > applications query the ProcessDefintions to get the list with all the > > > existing definitions, and present that list for example in a UI. > > > > > > So, imagine that with the ProcessDefinitions query you get the > following > > > list: > > > > > > purchase-order, serviceUrl_1 > > > new-customer, serviceUrl_2 > > > etc. > > > > > > Similar to what we do in our consoles, in their UIs, users can pick any > > > process from that list to create new instances, etc. > > > > > > All good. > > > > > > In a later point in time, an administrator, decides that the > > > "new-customer" process is out from the catalog. We won't work with that > > > ProcessDefinition anymore. > > > > > > Very simplified, In SonataFlow Operator driven setups, you can do it by > > > telling the operator to "un-deploy" the worfklow. As part of this > > > procedure, the operator can send a basic event to the data index > telling > > > that situation. > > > > > > So, at the data index, we need to mark the "new-customer" process > > > definition as "deleted", or better said as "logically deleted". > (Because, > > > as I explained in previous comments, we don't want, at least by now, do > > > physical deletions in the DI) > > > > > > To do that, we need to add a "new field". (Simplest approach, we don't > > > want more elaborated modelings) > > > > > > I suggested to name that field as "status", to keep it completely > > > unrelated from the type of setup/deployment, because, as I mentioned, > > > kogito supports a plenty of setups, docker, docker compose, Sonataflow > > > operator driven, standalone JVM, etc. > > > > > > So, introduced that we could for example mark "status" = "unavailable", > > > and maybe this introduced confusion. If that is the case, forget about > > the > > > world unavailable, and to follow the reasoning, imagine we mark that > > field > > > with "status" = "deleted" (logical deletion) > > > > > > Finally, by adding this simple field/concept, totally decoupled from > the > > > various setups that people can use, we can facilitate SonataFlow users > > > applications to query the DI, and only get the ProcessDefinitons that > are > > > not "deleted" (logically deleted) > > > > > > Select * from ProcessDefinitions where status != deleted > > > > > > > > > Regards, > > > Walter. > > > > > > > > > > > > > > > On 2024/07/04 11:35:08 Enrique Gonzalez Martinez wrote: > > > > Hi walter, > > > > I am not certain about the req. You have few things in motion in here > > > > > > > > 1. The req says the deployment is still required : Then you have to > do > > > > nothing as this information is just available in the index. Query the > > > > active process instance of a certain procces id and version. You > dont > > > need > > > > to access the process definition. > > > > > > > > The req says you need to know if the deployment is required but you > did > > > not > > > > deploy data index. Then you need to check if there pricess instance > > > > belonging to that deployment. > > > > > > > > The req says you need to know if a deployment is > unavailable/available > > i > > > > would say you need to send events related to that deployment and this > > is > > > > more tricky. Because you need to somehow define first what is > > available. > > > It > > > > seems your definition is related to what is deployed. In that sense i > > > would > > > > rely into the platform. As if you dont want to operate with certain > > > process > > > > you just need to drop the container running it. It will be shown in > the > > > > platform tool like dooker etc.... > > > > But if we want to keep track of history i will leave things opening. > > For > > > > instance i am interested in we start issuing these events to those > > > > deployments that are still managing some process instances but dont > > > accept > > > > new ones or deployments that have migrated the new process > definitions > > to > > > > new deployments...so i would try to not mess too much with the > process > > > > definition but afd a new event in this case open to new transition > and > > > > states.... And you can store that in any way you fancy in the data > > index. > > > > > > > > El jue, 4 jul 2024, 13:03, Walter Medvedeo <wmedve...@apache.org> > > > escribió: > > > > > > > > > Hi Enrique, > > > > > maybe I couldn't express the concept well. > > > > > > > > > > But let's say that the DI has the catalog of all the existing > > > > > ProcessDefinitions. > > > > > > > > > > To mark a definition as "unavailable" is kind of logically remove > it > > > from > > > > > the Catalog. > > > > > > > > > > > > > > > On 2024/07/04 09:59:00 Enrique Gonzalez Martinez wrote: > > > > > > Hi walter, this won't give you the data you are aiming for. > > > > > > > > > > > > If you have two instances and one goes down the state will be set > > as > > > not > > > > > > available unless you compute something in the data index so at > > least > > > you > > > > > > will need more information or duplicate lines. This is unlikely > to > > > be the > > > > > > responsibility of the data index. > > > > > > > > > > > > El jue, 4 jul 2024, 11:28, Walter Medvedeo <wmedve...@apache.org > > > > > > > escribió: > > > > > > > > > > > > > Hi, the idea is to introduce the requirement/concept first, and > > > > > summarize > > > > > > > the impact on the information registered in the DI as much as > > > possible. > > > > > > > > > > > > > > And thus, very summarized, the proposal is to include one more > > > filed in > > > > > > > the ProcessDefinitions information registered by the DI, that > > will > > > let > > > > > us > > > > > > > know if a given definition is still "available" to create > > > instances. > > > > > > > > > > > > > > That is a requirement we have from SonataFlow users. > > > > > > > > > > > > > > > > > > > > > On 2024/07/03 20:50:20 Jason Porter wrote: > > > > > > > > I am not exactly sure what the proposal is here. I see a > bunch > > of > > > > > > > background info for this new feature, but I don't see an actual > > > > > proposal. > > > > > > > > > > > > > > > > On 2024/07/03 15:06:21 Walter Medvedeo wrote: > > > > > > > > > Hello Guys, > > > > > > > > > > > > > > > > > > > > > > > > > > > Let me share some requirement we have. > > > > > > > > > > > > > > > > > > > > > > > > > > > Right now, as part of the ProcessDefinition information we > > > store > > > > > in the > > > > > > > > > Data Index, we record for example the processId and the > > > > > serviceURL. > > > > > > > > > > > > > > > > > > That information is good for users to know, for instance, > > which > > > > > are the > > > > > > > > > existing ProcessDefitnions, and, based on that information, > > > they > > > > > can > > > > > > > for > > > > > > > > > example start a new process instance by concatenating the > > > > > serviceURL + > > > > > > > “/” > > > > > > > > > + processId. > > > > > > > > > > > > > > > > > > In the context of a SonataFlow Operator driven setups, > users > > > need > > > > > to > > > > > > > know > > > > > > > > > if a ProcessDefinition is still “available”. > > > > > > > > > > > > > > > > > > For example, users can deploy a WF, use it for some time, > and > > > then, > > > > > > > decide > > > > > > > > > to un-deploy it. From this point, no more instances of that > > > > > workflow > > > > > > > can be > > > > > > > > > created. The user has just decided to remove that > deployment > > > from > > > > > the > > > > > > > > > ecosystem. > > > > > > > > > > > > > > > > > > However, in the data-index, that definition is still > > recorded. > > > And > > > > > > > users > > > > > > > > > want to keep it there, since they want to keep consistent > > > > > information > > > > > > > about > > > > > > > > > the already executed instances etc. (The intention is to > not > > > > > introduce > > > > > > > any > > > > > > > > > information deleting procedure at the DI, etc.) > > > > > > > > > > > > > > > > > > On the other hand, users need a way to filter which > > > definitions are > > > > > > > still > > > > > > > > > “available” by using the standard graphql api. > > > > > > > > > > > > > > > > > > What we are evaluating is to include a “status” field in > the > > > > > > > > > ProcessDefinitions, with the values available/unavailable, > > and, > > > > > that > > > > > > > will > > > > > > > > > be updated as usual via ClouldEvents. In the case of > > > SonataFlow, > > > > > those > > > > > > > > > events will be produced by the operator, which governs the > > > > > > > > > deployment/undeployment, etc. Other setups might just > ignore > > > that > > > > > > > field. > > > > > > > > > > > > > > > > > > Note: we are looking for a very minimalist approach that > > lets > > > us > > > > > > > record > > > > > > > > > the situation, and facilitate SonataFlow users to query > that > > > > > situation > > > > > > > by > > > > > > > > > using the standard Graphql api. > > > > > > > > > > > > > > > > > > This is not any sort of more elaborated clustering > > > > > > > > > management/monitoring/etc. solution. > > > > > > > > > > > > > > > > > > Independently of fine grained details about the event to > > > introduce, > > > > > > > which > > > > > > > > > carries more or less the information we are already sending > > in > > > the > > > > > > > other DI > > > > > > > > > events. > > > > > > > > > > > > > > > > > > From the point of view of let’s say non SonataFlow setups, > do > > > you > > > > > guys > > > > > > > see > > > > > > > > > any objections? > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > > > > > Walter. > > > > > > > > > ResponderReenviar > > > > > > > > > Añadir reacción > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > > > > > > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > > > > > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > > > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > > For additional commands, e-mail: dev-h...@kie.apache.org > > > > > > > > >
