Let’s do that let me know your availability we can stream it also ! Let me know your preferred platform !
On Tuesday, October 26, 2021, VICTOR MANUEL ROMERO RODRIGUEZ < victor.rom...@fintecheando.mx> wrote: > Saransh > > I would like to have a discussion, if we can record it, publish it in > youtube for example and share the link here again, in order to allow the > community to be in the loop. > > Does it work? > > Regards > > El lun, 25 oct 2021 a las 22:37, Saransh Sharma (<sara...@muellners.com>) > escribió: > >> Timestamping could also be added as part of the key. >> >> I agree with Victor here, we need to keep it simple, Hey @Victor Romero >> <victor.rom...@fintecheando.mx> would you like to discuss this and >> flesh it out together ? >> >> On Tue, Oct 26, 2021 at 9:01 AM VICTOR MANUEL ROMERO RODRIGUEZ < >> victor.rom...@fintecheando.mx> wrote: >> >>> I will extend the previous email, >>> >>> At asynchronous transactions: >>> >>> The trigger (or POST) for doing asynchronous transactions while sending >>> notifications (email, sms, push) over MQ/JMS queues, the user/customer >>> experience will be affected by receiving twice or more payment >>> reminders, payment notification (they can think that the payment was >>> applied twice), account creations, loan applications, etc and the cost >>> associated for each message sent over the notification channel. >>> >>> At journey entry: >>> >>> The trigger (or POST) for doing any transactions (monetary or non >>> monetary) only 1 entry must be recorded in all the systems so then the >>> logging and monitoring systems (fraud, alerting, etc) can correlate the >>> transaction (using surrogate keys) and later the data scientist or data >>> engineers won't be blaming the CBS engineering team. >>> >>> At transaction caching: >>> >>> We got a use case where the loan origination process was taking some >>> time to execute the process and because of the demand of the financial >>> product people was trying to execute several times to have a loan, so then >>> the business requirement was to keep on cache the same "loan request" if >>> the people ID was the same (in our case the National Population ID) and not >>> executing the query to the external systems (Credit Bureau, Black List had >>> a high cost per each query). So then we saved execution time and money. >>> >>> ****TL;TR**** >>> What I am trying to express is that the level, use cases, and user >>> history (or whatever it can be the concept about is it use) have a wide >>> range of "idempotent" implementations for the potential solutions should be >>> evaluated carefully depending on them. >>> >>> I think that the ID of the "idempotent" key should have enough >>> entropy to avoid any collision in the middleware, cache, db or messaging >>> level and the recording of it must be at UTC time either if it handles >>> synchronous or asynchronous API calls. >>> >>> ******** >>> >>> Best regards >>> >>> Victor >>> >>> >>> El lun, 25 oct 2021 a las 19:39, VICTOR MANUEL ROMERO RODRIGUEZ (< >>> victor.rom...@fintecheando.mx>) escribió: >>> >>>> Hello, >>>> >>>> I agree that a key must be used on the server side (Consumer) to avoid >>>> any duplicate record, most of the time I used to look at the Software >>>> Patterns like the Martin Fowler's articles: >>>> >>>> https://martinfowler.com/articles/patterns-of-distributed-systems/ >>>> idempotent-receiver.html >>>> >>>> The pattern described there is more complex than the Saransh's proposal >>>> (which is a good starting point), there are many approaches to add this >>>> feature (Idempotency) on Apache Fineract, which I think should be flexible >>>> enough to handle business rules. >>>> >>>> Idempotency can be at different levels (even using the POST/PUT), >>>> example: >>>> >>>> At DB level: >>>> >>>> The POST can be executed to create a new customer and the DB table has >>>> as a key or unique the email used in the account creating, then a >>>> duplicated entry at DB level should be thrown and the API Rest should send >>>> this error. Then this API Rest is Idempotent because at DB level there is a >>>> restriction. >>>> >>>> At API REST per business rule level: >>>> >>>> The POST for a login should be idempotent because only one session is >>>> created at a time per user/customer and in case of X number of failed >>>> retries the account must be locked for a period of time or unlocked by a >>>> manual/automatic process. Once the session is created, if a POST for login >>>> sends a 200 status, then for a period of time the possibility to have >>>> another login must be blocked until the current active session expires >>>> (expiry time is set by the digital delivery channel owner like Internet or >>>> Mobile banking). >>>> >>>> At API REST per consistency rule level: >>>> >>>> The POST for doing a payment (third party same bank or interbank >>>> transfer) the idempotency key must be used for keeping the track of the >>>> transaction status, holding the amount, settlement or reversal of the >>>> amount, digital receipt, digital invoice generation, accounting record, >>>> etc) in order to avoid duplication of charges in the origin account. Even >>>> with confirmation/warning pop ups/windows errors occur. So then the handle >>>> of the idempotency key must be handled and implemented in the Front end >>>> application, the Server (and the internal consumer). >>>> >>>> And by the way the transactions must be done at UTC time, not at TZ >>>> (another change in the Apache Fineract's tables should be done for the >>>> offset time zone recording) . >>>> >>>> Best regards >>>> >>>> Victor >>>> >>>> El sáb, 23 oct 2021 a las 0:05, Saransh Sharma (<sara...@muellners.com>) >>>> escribió: >>>> >>>>> This is a curious problem in computer science and especially in >>>>> distributed networks. >>>>> >>>>> API Idempotency is a major issue for software like Fineract, unless >>>>> the API handles drop off and recognize the second call or any n call as >>>>> either duplicate or important in any given case, like how stripe does >>>>> this >>>>> by specifying a key from the client end ensures that even in case of >>>>> failure the charge does not happen twice! >>>>> >>>>> This can lead to financial loss in case of real payment taking place! >>>>> >>>>> I think, as of right now there is no such mechanism in Fineract 1.x >>>>> that eliminates such risk. Except, on the validation side, but that’s not >>>>> enough. >>>>> >>>>> To have this implemented we have done some work on this last year , >>>>> maybe we can present some code also ! >>>>> >>>>> I have used some cryptography libs to have the client-side send a >>>>> unique key that’s generated by the private key infrastructure and the >>>>> client-side has to send these as headers and on the server-side, the >>>>> signatures are validated thus making the idempotency in POST and PUT >>>>> request. >>>>> >>>>> The code on the client side >>>>> >>>>> var keys = { >>>>> alice: >>>>> '38f93bdda21a5c4a7bae4eb75bb7811cbc3eb627176805c1009ff2099263c6ad', >>>>> bob: >>>>> '09880c962437080d72f72c8c63a69efd65d086c9e7851a87b76373eb6ce9aab5'}; >>>>> // GET >>>>> for(k in keys) { >>>>> var url = 'APIURL'; >>>>> var dataToSign = url; >>>>> var options = { >>>>> url: url, >>>>> headers: { >>>>> 'x-identity': auth.getPublicKeyFromPrivateKey(keys[k]), >>>>> 'x-signature': auth.sign(dataToSign, keys[k]) >>>>> } >>>>> }; >>>>> >>>>> >>>>> The above code generates these Keys are now passed on each request to >>>>> the server, and the server using the middleware in case of Fineract have >>>>> to do two things >>>>> >>>>> 1. VerifySignature >>>>> 2. GetSinFromThePublicKey >>>>> >>>>> I started writing some code for this and quickly realized that it's >>>>> actually doable, it requires around 20 hours of work, There are some >>>>> dependency that needs to be added on the server side and some quick >>>>> changes >>>>> on the middleware and its done. >>>>> >>>>> We presented a paper on this last year on Apache Con ! Remember >>>>> something about using zero knowledge of proof and password less auth . >>>>> >>>>> Let me know , I would like to make sure that this gets added in this >>>>> upcoming release. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Friday, October 22, 2021, James Dailey <jamespdai...@gmail.com> >>>>> wrote: >>>>> >>>>>> Devs - >>>>>> >>>>>> Through a conversation with a computer science person, I've learned >>>>>> that Fineract should have a clearly articulated approach with regard to >>>>>> idempotency. This is particularly true when thinking about how fineract >>>>>> interacts with payment systems. >>>>>> >>>>>> "When building a system to move money, it is paramount that >>>>>> operations that move money are idempotent. Failure to do this might >>>>>> result >>>>>> in accidentally double charging your customer, or paying a vendor >>>>>> multiple >>>>>> times. The risk of this happening is elevated based on the way software >>>>>> is >>>>>> typically built today, since developers take advantage of scalable >>>>>> systems >>>>>> that process multiple items in parallel. " >>>>>> (https://www.moderntreasury.com/journal/why-idempotency- >>>>>> matters-in-payments) >>>>>> >>>>>> Or this article on Stripe engineering calling for liberal use of >>>>>> Idempotency: >>>>>> >>>>>> "The easiest way to address inconsistencies in distributed state >>>>>> caused by failures is to implement server endpoints so that they’re >>>>>> idempotent, which means that they can be called any number of times while >>>>>> guaranteeing that side effects only occur once." ( >>>>>> https://stripe.com/blog/idempotency) >>>>>> >>>>>> and this article in Apache Camel >>>>>> https://camel.apache.org/camel-kafka-connector/latest/ >>>>>> user-guide/idempotency.html >>>>>> >>>>>> So, my understanding may be wrong and we've got this covered, or >>>>>> maybe we just consider this overkill. Perhaps there is something in the >>>>>> "magic" that's developed in the CQRS plus in-memory database queuing and >>>>>> our clever transaction state management, but... I don't know. >>>>>> >>>>>> Or perhaps we haven't really had to think about this because we >>>>>> haven't had that many large scale deployments and the scalability issues >>>>>> associated with multiple parallel calls haven't come to the fore, or >>>>>> haven't come back to the list. So, that's the intent of this email. >>>>>> >>>>>> Do we have a need now for layering on Idempotency checks? What is the >>>>>> right approach? Has anyone already implemented this, or decided not to >>>>>> for >>>>>> very good reasons? What would be the right approach to this? Implement >>>>>> where? Time limited to 3 hrs? Etc... >>>>>> >>>>>> So, this is a call to discuss. I'm far from an expert on this. >>>>>> >>>>>> >>>>>> >> >> -- >> >> Saransh Sharma >> *Research Partner* >> *Muellner Internet Pvt Ltd * >> >> ------------------------------------------------------------ >> ---------------------------------- >> >> The idea of separation of me and you is amazing. >> ------------------------------------------------------------ >> ---------------------------------- >> *Company Website <https://www.muellners.com/> **Company Linkedin >> <https://www.linkedin.com/company/muellners> *Company Facebook >> <https://www.facebook.com/muellners> >> >> This mail is governed by Muellner® Internet Private Limited's IT policy. >> The information contained in this e-mail and any accompanying documents >> may contain information that is confidential or otherwise protected from >> disclosure. If you are not the intended recipient of this message, or if >> this message has been addressed to you in error, please immediately alert >> the sender by reply e-mail and then delete this message, including any >> attachments. Any dissemination, distribution or other use of the contents >> of this message by anyone other than the intended recipient is strictly >> prohibited. All messages sent to and from this e-mail address may be >> monitored as permitted by applicable law and regulations to ensure >> compliance with our internal policies and to protect our business. E-mails >> are not secure and cannot be guaranteed to be error free as they can be >> intercepted, amended, lost or destroyed, or contain viruses. You are deemed >> to have accepted these risks if you communicate with us by e-mail. >> > -- Sent from Gmail Mobile
