Ok, nice, It will be good to see all this added into the jobs service. If you have guys any design document, or prototype you can share at some point, it'll be great.
Thanks, Walter. On 2023/11/17 10:19:35 Enrique Gonzalez Martinez wrote: > 1. There are not distributed tx... There are several tx. > 2. V7 has this problem regarding the singleton service. As we relied on > eap/wildfly. This one has an interesting concept called timer partition so > u can split in fact a singleton service in a cluster environment without > colliding or multiplying the trigger if that is u worry about. Also there > are a ways to create locks through the state so no other instances will > clash when they are manipulating the jobs. > > El vie, 17 nov 2023, 10:04, Walter Medvedeo <[email protected]> escribió: > > > Hi Enrique, > > so according with 2) you will provide two transport types? > > 2.1) local access via Pojos > > 2.2) rest client invoking the job service in a distributed tx > > > > Finally, on last thing that is important to consider is that the jobs > > service is currently a singleton service, so, depending on how much of the > > logic of the job service you'll execute in the kogito.-runtime side, this > > might introduce an additional challenge. > > > > Regards, > > Walter. > > > > On 2023/11/16 16:14:06 Enrique Gonzalez Martinez wrote: > > > HI Walter: > > > > > > Not sure if I understood properly what you are assessing here. > > > > > > The idea is to have a > > > 1) core component (let's say an interface job service) that is > > > responsible of the logic of the job service (including storage) > > > 2) two different types of transport (in vm) deployed within the > > > runtime and just passing pojo from the runtime to the job service and > > > a different as a rest client invoking the job service in a distributed > > tx. > > > > > > It does not really matter where the transaction is executed because > > > the only difference is the transport. Forget for an instance the > > > reactive stuff for now. > > > > > > if you have an in-vm deployed / colocated / embedded (same jvm) and > > > there is an overdue trigger... the lifecycle will still be the same > > > (scheduled - completed) but the entire thing will happen in the same > > > tx and jvm. > > > > > > if you have in a different subsystem (2 jvm) what will happen is that > > > the way you send the message will should be involved in the tx so what > > > will you have is > > > > > > # runtime > > > executed some stuff > > > create job service > > > -send message to job service > > > > > > # job service > > > > > > the other jvm > > > consume message > > > create job > > > schedule for execution > > > > > > that will be the other tx in the second jvm > > > > > > > > > that is what we want to achieve. There might be cases where reactive > > > might work out or not, we should design this properly. > > > > > > > > > It is not possible to make all combinations available to the end user > > > (and it is not even needed - it won't add any value) > > > > > > > > > > > > El jue, 16 nov 2023 a las 15:32, Walter Medvedeo > > > (<[email protected]>) escribió: > > > > > > > > Maybe I'm overthinking, or having a technical gap :(, but what I don't > > see right now guys is how to solve this challenge. (in case of option2) > > > > > > > > Let me explain in a very simplified way. At some point in time we need > > to persist the runtime info and the jobs service info in single write. > > > > > > > > Scenario1: (single write) > > > > > > > > 1) OpenTransaction > > > > > > > > 2) Write process the process information with a non-reactive > > datasource. > > > > 3) Write the job-service related information to create a job, this > > time we use not only a different datasource, but also a reactive-datasource. > > > > > > > > 4) Commit Transaction > > > > > > > > What I don't see is how this will work. > > > > > > > > > > > > Scenario2: (overdue trigger) > > > > > > > > 1) OpenTransaction > > > > > > > > 2) Write process the process information with a non-reactive > > datasource. > > > > 3) Write the job-service related information to create a job, this > > time we use not only a different datasource, but also a reactive-datasource. > > > > 3.5) It's detected that the job is overdue and must be executed, > > let's execute it. > > > > > > > > 4) Commit Transaction > > > > 5) (another posibility instead of 3.5) > > > > After the commit, it's detected that the job is overdue and must be > > executed, let's execute it. > > > > > > > > Now, where do the code to achieve the execution of 3.5, or 5 execute? > > in the kogito-runtimes JVM or in the jobs-service JVM? > > > > > > > > Regards, > > > > Walter. > > > > > > > > > > > > On 2023/11/16 10:38:13 Enrique Gonzalez Martinez wrote: > > > > > Hi Walter: > > > > > > > > > > 1) you can still be reactive if you like if you build a transport > > > > > layer being reactive. so moving that to the transport layer you will > > > > > have the same benefits not blocking the IO at the endpoint level IMO. > > > > > In any case using this sort of approach. > > > > > 2) that should not invalidate the logic. It is to say. The job > > service > > > > > will the the create - schedule - fire execution in place. > > > > > > > > > > Ideally this should be like > > > > > > > > > > as colocated service > > > > > runtime -> job service API -> embedded job service proxy (in vm > > > > > transport) -> job service > > > > > > > > > > as distributed > > > > > runtime -> job service API -> rest job service proxy (rest client) || > > > > > wire || -> endpoint (reactive or not) -> job service > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > El jue, 16 nov 2023 a las 10:15, Walter Medvedeo > > > > > (<[email protected]>) escribió: > > > > > > > > > > > > Hi Alex, and guys, thanks for starting discussion. > > > > > > > > > > > > Let me add some comments that I think might be useful to consider > > for the evaluation/implementation. > > > > > > > > > > > > Regarding option 2) we must keep in mind that right now, the jobs > > service DB writes are reactive, as well as the scheduling process, etc. If > > we keep this approach, I think we have a potential challenge here on how to > > integrate the non reactive kogito-runtime DB writes (and it's synchronous > > execution model in general) with the subsequent reactive schedule of the > > job. Since in the end, we want all this to happen in the same transaction. > > > > > > What are the plan in that regard? > > > > > > > > > > > > > > > > > > Another thing to keep in mind is the treatment of overdue jobs. > > There are situations where the job being created has already overpassed the > > execution time, in these cases, it's automatically fired, and the > > corresponding associated action is executed immediatelly. If I don't recall > > wrong, this is decided as part of the scheduling process. I think we must > > be sure that we can keep the possibility to fire ouverdue jobs (something > > that is optional configurable) at the same time we don't want the execution > > of the job is produced in the kogito-runtimes side but in the jobs-service. > > > > > > > > > > > > Regards, > > > > > > Walter. > > > > > > > > > > > > > > > > > > On 2023/11/15 20:58:11 Alex Porcelli wrote: > > > > > > > Similar to Enrique, I'd +1 the second option.. as it aligns > > better > > > > > > > with the current data-index approach. It would allow, for sake of > > > > > > > simplicity. > > > > > > > > > > > > > > Are we ok if we move forward with the second option as a > > proposal to > > > > > > > be implemented? > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 14, 2023 at 4:40 AM Enrique Gonzalez Martinez > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > That case is fixed as it uses the EventPublisher interface. > > > > > > > > > > > > > > > > El mar, 14 nov 2023, 10:25, Pere Fernandez (apache) < > > [email protected]> > > > > > > > > escribió: > > > > > > > > > > > > > > > > > Something we may also need to think about is the > > communication between the > > > > > > > > > Job-Service and the Data-Index. Currently it Job-Service > > needs to message > > > > > > > > > the Data-Index when a Job status changes. I think that for > > the purpose of > > > > > > > > > the Simplified Architecture it should be able to write > > directly into the > > > > > > > > > Data-Index DB, in a similar manner as the runtime does when > > the > > > > > > > > > `kogito-addons-quarkus-data-index-persistence` addons are > > present. > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: [email protected] > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
