Francisco, Seems that a few of us believe this would be better addressed by an external system.
On Wed, Jul 17, 2024 at 7:14 AM Francisco Javier Tirado Sarti < [email protected]> wrote: > Hi, > Yes, process definition is used because runtime tooling is relying on > process definition to be able to render a graphical representation of the > runtime progress. > It can be argued that a runtime tool might want to know in which servers of > the cluster that process definition is deployed and the status of that > deployment. So, the same serviceURL is there, a deployment status flag is > not a crazy idea. > > On Thu, Jul 11, 2024 at 6:45 PM Alex Porcelli <[email protected]> wrote: > > > Francisco, > > > > You're spot on on the purpose of the current Process Definition on DI: > > it provides complementary information to process instances. > > > > Example? Process definition (ie. diagram) for the current instance. > > > > On Thu, Jul 11, 2024 at 12:48 PM Francisco Javier Tirado Sarti > > <[email protected]> wrote: > > > > > > Hi, > > > Since DI already has a ProcessDefinition entity with a service URL, we > > feel > > > that adding deployment status was not breaking anything that was not > > > already broken. > > > I guess it all depends on what we think should be the purpose of DI. > > > If it is just to offer information about process instances, Enrique is > > > right, this is not the place to monitor which process definitions are > > > available or not. > > > However, if the purpose of DI is just that, why is the process > definition > > > entity there? as supporting information for the process instance only? > > > I feel we need to clearly define this before discarding Walters' > > proposal. > > > If we take a strict approach, then it means that any software deploying > > > runtimes and data index will have to develop its own deployment > component > > > with its own db. That's probably fair, but should be written somewhere. > > > > > > > > > > > > On Tue, Jul 9, 2024 at 3:39 PM Alex Porcelli <[email protected]> wrote: > > > > > > > Walter, > > > > > > > > I have to agree with Enrique… although your ask is minimal in the > code > > > > changes and impact, the problem is more on the message that it sends. > > > > > > > > As pointed out by last email from Enrique, we - as Apache KIE - don’t > > > > control or have visibility over the whole environment, so it’s > > virtually > > > > impossible to guarantee that something is set to not available is > > really > > > > not available, and vice versa. > > > > > > > > I understand that the serverless operator aims to provide an > > experience, > > > > but it also has limitation over the installation… so probably this > > kind of > > > > information would be better stored outside the Apache KIE systems / > in > > your > > > > case, in the downstream product database. > > > > > > > > Regards, > > > > _____________ > > > > Alex Porcelli > > > > http://porcelli.me > > > > > > > > > > > > On Tue, Jul 9, 2024 at 1:59 AM Enrique Gonzalez Martinez < > > > > [email protected]> wrote: > > > > > > > > > Hi Walter, > > > > > > > > > > I see some problems aside from going on in mixing the data index > with > > > > data > > > > > does not belong to it like deployment mgmt. E.g. The service url is > > there > > > > > not because the DI but the gateway component (mutations) that makes > > > > changes > > > > > in runtimes. Something that needs to change in the future. But i > > digress. > > > > > > > > > > Back to this is deployment management. Basically the requirement is > > to > > > > > compute the current catalog of process definitions deployed in the > > > > > environment. > > > > > > > > > > I am not against but I am not comfortable mixing with the data > index. > > > > > Because it is not part of the data index goal; provide the last > state > > > > > information of the process instances. > > > > > > > > > > Regarding how this approach works and given you are capable of > > counting > > > > > properly instances. > > > > > > > > > > 1. Let's say you bring down one workflow. The system will be marked > > as > > > > > undeployed/deleted. You don't specify how alive process instance > > will be > > > > > processed or if you will perform some other action. > > > > > The other problem i see is trying to already solve platform > > problems. I > > > > > mean, if the platform already can compute which containers are > > deployed > > > > and > > > > > therefore which is the proper catalog why should we polute the > system > > > > with > > > > > this ? Would not make more sense in telling the user how to query > > that > > > > info > > > > > in the platform ? Making platform independent does not give a > proper > > > > > benefits of chasing multiple platform problems regarding > deployment / > > > > > undeployment operations, url changes, counting instances... Will > > never > > > > get > > > > > it right. It is an unsustainable approach. > > > > > > > > > > El lun, 8 jul 2024, 11:18, Walter Medvedeo <[email protected]> > > > > > escribió: > > > > > > > > > > > Hi Enrique, the requirement we have from SonataFlow users is very > > > > simple. > > > > > > > > > > > > Imagine the ProcesssDefinitions in the DI as a catalog of all the > > > > > existing > > > > > > definitions. > > > > > > > > > > > > Similar to the sonataflow-console, the jbpm-console, etc, user > > > > > > applications query the ProcessDefintions to get the list with > all > > the > > > > > > existing definitions, and present that list for example in a UI. > > > > > > > > > > > > So, imagine that with the ProcessDefinitions query you get the > > > > following > > > > > > list: > > > > > > > > > > > > purchase-order, serviceUrl_1 > > > > > > new-customer, serviceUrl_2 > > > > > > etc. > > > > > > > > > > > > Similar to what we do in our consoles, in their UIs, users can > > pick any > > > > > > process from that list to create new instances, etc. > > > > > > > > > > > > All good. > > > > > > > > > > > > In a later point in time, an administrator, decides that the > > > > > > "new-customer" process is out from the catalog. We won't work > with > > that > > > > > > ProcessDefinition anymore. > > > > > > > > > > > > Very simplified, In SonataFlow Operator driven setups, you can do > > it by > > > > > > telling the operator to "un-deploy" the worfklow. As part of > this > > > > > > procedure, the operator can send a basic event to the data index > > > > telling > > > > > > that situation. > > > > > > > > > > > > So, at the data index, we need to mark the "new-customer" process > > > > > > definition as "deleted", or better said as "logically deleted". > > > > (Because, > > > > > > as I explained in previous comments, we don't want, at least by > > now, do > > > > > > physical deletions in the DI) > > > > > > > > > > > > To do that, we need to add a "new field". (Simplest approach, we > > don't > > > > > > want more elaborated modelings) > > > > > > > > > > > > I suggested to name that field as "status", to keep it completely > > > > > > unrelated from the type of setup/deployment, because, as I > > mentioned, > > > > > > kogito supports a plenty of setups, docker, docker compose, > > Sonataflow > > > > > > operator driven, standalone JVM, etc. > > > > > > > > > > > > So, introduced that we could for example mark "status" = > > "unavailable", > > > > > > and maybe this introduced confusion. If that is the case, forget > > about > > > > > the > > > > > > world unavailable, and to follow the reasoning, imagine we mark > > that > > > > > field > > > > > > with "status" = "deleted" (logical deletion) > > > > > > > > > > > > Finally, by adding this simple field/concept, totally decoupled > > from > > > > the > > > > > > various setups that people can use, we can facilitate SonataFlow > > users > > > > > > applications to query the DI, and only get the ProcessDefinitons > > that > > > > are > > > > > > not "deleted" (logically deleted) > > > > > > > > > > > > Select * from ProcessDefinitions where status != deleted > > > > > > > > > > > > > > > > > > Regards, > > > > > > Walter. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 2024/07/04 11:35:08 Enrique Gonzalez Martinez wrote: > > > > > > > Hi walter, > > > > > > > I am not certain about the req. You have few things in motion > in > > here > > > > > > > > > > > > > > 1. The req says the deployment is still required : Then you > have > > to > > > > do > > > > > > > nothing as this information is just available in the index. > > Query the > > > > > > > active process instance of a certain procces id and version. > You > > > > dont > > > > > > need > > > > > > > to access the process definition. > > > > > > > > > > > > > > The req says you need to know if the deployment is required but > > you > > > > did > > > > > > not > > > > > > > deploy data index. Then you need to check if there pricess > > instance > > > > > > > belonging to that deployment. > > > > > > > > > > > > > > The req says you need to know if a deployment is > > > > unavailable/available > > > > > i > > > > > > > would say you need to send events related to that deployment > and > > this > > > > > is > > > > > > > more tricky. Because you need to somehow define first what is > > > > > available. > > > > > > It > > > > > > > seems your definition is related to what is deployed. In that > > sense i > > > > > > would > > > > > > > rely into the platform. As if you dont want to operate with > > certain > > > > > > process > > > > > > > you just need to drop the container running it. It will be > shown > > in > > > > the > > > > > > > platform tool like dooker etc.... > > > > > > > But if we want to keep track of history i will leave things > > opening. > > > > > For > > > > > > > instance i am interested in we start issuing these events to > > those > > > > > > > deployments that are still managing some process instances but > > dont > > > > > > accept > > > > > > > new ones or deployments that have migrated the new process > > > > definitions > > > > > to > > > > > > > new deployments...so i would try to not mess too much with the > > > > process > > > > > > > definition but afd a new event in this case open to new > > transition > > > > and > > > > > > > states.... And you can store that in any way you fancy in the > > data > > > > > index. > > > > > > > > > > > > > > El jue, 4 jul 2024, 13:03, Walter Medvedeo < > [email protected] > > > > > > > > > escribió: > > > > > > > > > > > > > > > Hi Enrique, > > > > > > > > maybe I couldn't express the concept well. > > > > > > > > > > > > > > > > But let's say that the DI has the catalog of all the existing > > > > > > > > ProcessDefinitions. > > > > > > > > > > > > > > > > To mark a definition as "unavailable" is kind of logically > > remove > > > > it > > > > > > from > > > > > > > > the Catalog. > > > > > > > > > > > > > > > > > > > > > > > > On 2024/07/04 09:59:00 Enrique Gonzalez Martinez wrote: > > > > > > > > > Hi walter, this won't give you the data you are aiming for. > > > > > > > > > > > > > > > > > > If you have two instances and one goes down the state will > > be set > > > > > as > > > > > > not > > > > > > > > > available unless you compute something in the data index so > > at > > > > > least > > > > > > you > > > > > > > > > will need more information or duplicate lines. This is > > unlikely > > > > to > > > > > > be the > > > > > > > > > responsibility of the data index. > > > > > > > > > > > > > > > > > > El jue, 4 jul 2024, 11:28, Walter Medvedeo < > > [email protected] > > > > > > > > > > > > > escribió: > > > > > > > > > > > > > > > > > > > Hi, the idea is to introduce the requirement/concept > > first, and > > > > > > > > summarize > > > > > > > > > > the impact on the information registered in the DI as > much > > as > > > > > > possible. > > > > > > > > > > > > > > > > > > > > And thus, very summarized, the proposal is to include one > > more > > > > > > filed in > > > > > > > > > > the ProcessDefinitions information registered by the DI, > > that > > > > > will > > > > > > let > > > > > > > > us > > > > > > > > > > know if a given definition is still "available" to create > > > > > > instances. > > > > > > > > > > > > > > > > > > > > That is a requirement we have from SonataFlow users. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 2024/07/03 20:50:20 Jason Porter wrote: > > > > > > > > > > > I am not exactly sure what the proposal is here. I see > a > > > > bunch > > > > > of > > > > > > > > > > background info for this new feature, but I don't see an > > actual > > > > > > > > proposal. > > > > > > > > > > > > > > > > > > > > > > On 2024/07/03 15:06:21 Walter Medvedeo wrote: > > > > > > > > > > > > Hello Guys, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Let me share some requirement we have. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Right now, as part of the ProcessDefinition > > information we > > > > > > store > > > > > > > > in the > > > > > > > > > > > > Data Index, we record for example the processId and > > the > > > > > > > > serviceURL. > > > > > > > > > > > > > > > > > > > > > > > > That information is good for users to know, for > > instance, > > > > > which > > > > > > > > are the > > > > > > > > > > > > existing ProcessDefitnions, and, based on that > > information, > > > > > > they > > > > > > > > can > > > > > > > > > > for > > > > > > > > > > > > example start a new process instance by concatenating > > the > > > > > > > > serviceURL + > > > > > > > > > > “/” > > > > > > > > > > > > + processId. > > > > > > > > > > > > > > > > > > > > > > > > In the context of a SonataFlow Operator driven > setups, > > > > users > > > > > > need > > > > > > > > to > > > > > > > > > > know > > > > > > > > > > > > if a ProcessDefinition is still “available”. > > > > > > > > > > > > > > > > > > > > > > > > For example, users can deploy a WF, use it for some > > time, > > > > and > > > > > > then, > > > > > > > > > > decide > > > > > > > > > > > > to un-deploy it. From this point, no more instances > of > > that > > > > > > > > workflow > > > > > > > > > > can be > > > > > > > > > > > > created. The user has just decided to remove that > > > > deployment > > > > > > from > > > > > > > > the > > > > > > > > > > > > ecosystem. > > > > > > > > > > > > > > > > > > > > > > > > However, in the data-index, that definition is still > > > > > recorded. > > > > > > And > > > > > > > > > > users > > > > > > > > > > > > want to keep it there, since they want to keep > > consistent > > > > > > > > information > > > > > > > > > > about > > > > > > > > > > > > the already executed instances etc. (The intention is > > to > > > > not > > > > > > > > introduce > > > > > > > > > > any > > > > > > > > > > > > information deleting procedure at the DI, etc.) > > > > > > > > > > > > > > > > > > > > > > > > On the other hand, users need a way to filter which > > > > > > definitions are > > > > > > > > > > still > > > > > > > > > > > > “available” by using the standard graphql api. > > > > > > > > > > > > > > > > > > > > > > > > What we are evaluating is to include a “status” > field > > in > > > > the > > > > > > > > > > > > ProcessDefinitions, with the values > > available/unavailable, > > > > > and, > > > > > > > > that > > > > > > > > > > will > > > > > > > > > > > > be updated as usual via ClouldEvents. In the case of > > > > > > SonataFlow, > > > > > > > > those > > > > > > > > > > > > events will be produced by the operator, which > governs > > the > > > > > > > > > > > > deployment/undeployment, etc. Other setups might just > > > > ignore > > > > > > that > > > > > > > > > > field. > > > > > > > > > > > > > > > > > > > > > > > > Note: we are looking for a very minimalist approach > > that > > > > > lets > > > > > > us > > > > > > > > > > record > > > > > > > > > > > > the situation, and facilitate SonataFlow users to > query > > > > that > > > > > > > > situation > > > > > > > > > > by > > > > > > > > > > > > using the standard Graphql api. > > > > > > > > > > > > > > > > > > > > > > > > This is not any sort of more elaborated clustering > > > > > > > > > > > > management/monitoring/etc. solution. > > > > > > > > > > > > > > > > > > > > > > > > Independently of fine grained details about the event > > to > > > > > > introduce, > > > > > > > > > > which > > > > > > > > > > > > carries more or less the information we are already > > sending > > > > > in > > > > > > the > > > > > > > > > > other DI > > > > > > > > > > > > events. > > > > > > > > > > > > > > > > > > > > > > > > From the point of view of let’s say non SonataFlow > > setups, > > > > do > > > > > > you > > > > > > > > guys > > > > > > > > > > see > > > > > > > > > > > > any objections? > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > > > > > > > > > > > Walter. > > > > > > > > > > > > ResponderReenviar > > > > > > > > > > > > Añadir reacción > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > > > For additional commands, e-mail: > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > >
