Hey Myrle please refer to this update document This report is not confidential and is open for suggestions
Any microservice architecture that is defined and is in practice with the active forces and that comes with solution as reference from link <https://microservices.io/patterns/microservices.html> Forces - There is a team of developers working on the application - New team members must quickly become productive - The application must be easy to understand and modify - You want to practice continuous deployment of the application - You must run multiple instances of the application on multiple machines in order to satisfy scalability and availability requirements - You want to take advantage of emerging technologies (frameworks, programming languages, etc) Solution Define an architecture that structures the application as a set of loosely coupled, collaborating services. This approach corresponds to the Y-axis of the Scale Cube <https://microservices.io/articles/scalecube.html>. Each service is: - Highly maintainable and testable - enables rapid and frequent development and deployment - Loosely coupled with other services - enables a team to work independently the majority of time on their service(s) without being impacted by changes to other services and without affecting other services - Independently deployable - enables a team to deploy their service without having to coordinate with other teams - Capable of being developed by a small team - essential for high productivity by avoiding the high communication head of large teams The above defined terms clearly outlines the idea of a microservices, the original premise is that any team or an individual could develop their own set of service without bothering about the other set of services and connect with other services with proper set of auth and middleware kind of approach like using a service broker or message delivery service. This is what microservice means , now this term has been defined by experts and developers who have tested this approach in different scenarios. Microservice is more like a design pattern. There is no fixed idea or a tech stack like for example in cases like two different services communicating with each other over the internet or intranet can also be encompassed as microservice. Microservice is a quasi distributed system that reduces the monolithic approach of dealing with a single application or service. Now developers face the complexity of understanding new ideas like connecting with other services using Eureka, ideas like circuit breaker. It becomes more difficult to test the services since services are isolated. Fineract CN Primary architecture of fineract is based on the idea of Microservices the project are structured primarily with three different sub projects 1. API 2. Service 3. Component test Lets understand what does these terms are API deals with functions that define the domain , defines the client with mapping of GET and other HTTP verbs and validation; it acts like an interface for communication. Using Active MQ that emits the events from the API Services deals with the registration with Eureka, migration of DB and we can assume that it does the configuration for spring services also. Command handler in the code is defined in the services project. Mapping and repository is also defined here, so to speak, services is the communication between other services and is deployed also at the runtime. Component test as the name suggests does the testing for the single service by mocking using test libs and suite. Most of the project is structured in the above approach , the first approach is time consuming defining three different sub projects. Any simple developer would want to have a single project and use libs/dependency to achieve the above task. Proposing a simple structure here , we can take out the API / Service / Component test as root project and come up with below file structure src/main/java/org/apache/fineract/cn/ 1. logging/ 2. client / 3. config / 4. domain / 5. repository / 6. security / 7. service / 8. web /rest The above structure is simplified and does the same as above and reduces these sub-projects to a single project ,and is more known to the developers and adopts the naming convention that is familiar. It removes the burden for maintaining three separate projects. One might say that the internal libs and code patterns are different and, I assume that we can simply adopt those things here also in the same project structure. Lets focus on the other side of the project which are other services and lets see what they are and if any improvements can be made. Provisioner This does the resource binding and more specifically with respect to the tenant and allocates the DB , that gets binded by the Anubis and Identity service, at the same time this service tracks which tenant has access to the which service or application Currently this service assumes that all services are going to use the same kind of DB which is Postgres at the moment and hence the migration might be the same. As of right now , as the document describes here the link that it does the “magic” , The magic it does is that it binds the identity service and other services and work upon the magic like indexing the service related information in namespaces of cassandra but when you dig deep the services are attached to a single DB and creates different tenant 1. I believe developers should have the choice to define the service that they want to attach or it should be done from the developer's side. 2. Having each service spits out its complete state on bus or through Active MQ could do the work or log level DB like blockchain and Apache Kafka 3. The provisioner does the work for attaching all the services like Rhythm , Office and accounting etc. Suggestions This service makes sense, but definitely we can use something that is more mature and or focus on making this service like a gateway that does other things also. The idea is that any developer will bind the service to this approach is self-referential in nature. Currently services can manage their own sub-services and it should be left to the developers. We could provide them an approach that if you use this design pattern then you have some advantages. Users are forced to maintain the services, rather than developers defining the services. In my opinion each service should provision its own resource and maintain its own configuration. Then it should be left with developers to index their services . Security point of view : What if the service users are compromised then service configuration and service information are revealed. Identity As the name suggests this service provides a JWT based token where user rights are delegated and services can interact with each other. In my opinion this service can be really phased out with something like this below Suggestions Passwords are no longer adequate. Users worldwide are victims of multiple malfeasances like brute force attacks, injection attacks, phishing, unsafe credentials, data theft, among others. The benefits of using passwords - which have increasingly become predictable, leave users vulnerable to data and identity theft. Even the strongest passwords are easy to crack and prone to phishing nonetheless. Hence, given all these nuisances, there’s a need to eliminate character-based authentication protocols, which would ultimately benefit all developers as well as end-users. To replace this conventional but antiquated protocol with a secure alternative, would be Passwordless Authentication. BitRupee is a secure passwordless authentication protocol that uses cryptography where one’s identity is presented with anonymity. This type of multi-factor authentication layer approves a signature only after two or more verification factors (that are secured with a cryptographic key pair) are authorized. The BitRupee system is decentralized and only stores a public key in the database. The system creates a public and private key, of which the user is only able to access the private key. Further after signing the key, the user sends the information over the API to the server, where it checks its validity with the public key and grants access accordingly. Rhythm This service describes itself as some kind of self timer that's ticking and observing activity and sending notifications. Sounds like a great service. (Although it does not observe anything :) and is not self timed) 1. The service/developers should independently decide which logic/service they want to schedule and if they have something to trigger using some kind of central bus should be triggered to a different service. 2. This service can be defined as design patterns rather than complete service that is outsourced to do something that any independent service designer would perform on its own. Suggestions This service can be clubbed with a single service or perhaps the wilder idea is to club this service in the command or let developers choose to subscribe to the messages from a central bus aka log service. One use case is that any service spits out knowing that there is a service that does interest-recalculation, the service loan just need to spit out “re-calculate” through a pub/sub or peer method where “interest” service reacts to the message by simply processing the command , this is quite simplified and leaves a lot to developers to act on which message and command. Lets focus on the libraries that are not deployable but binded as dependency to each service. Anubis This service does the validation of the identity that any service is trying to bind that to, this service represent the permissions defined by the developers for their service Suggestions This lib does not make any sense , what if developers can define their own permissions and could declare that to other services. Why validate ? Permitted Feign Client This service allows the clients to communicate with delegated rights and access between services. This service more or less does the permission for the services in between. Suggestions We could use the Netflix Spring Feign Client . again the permissions can be defined on the runtime by developers, I think that’s what they would like to do. Services can come up with its own defined permission. Test A generic service that provides utility for testing the components. API Already defined in the above section, this is a feign client. Command This is the CQRS service library that supports asynchronous calls for read and write operations over a NOSQL DB, Hence this library does binding using a bus. Although this library claims to be non central that I could not verify. Suggestions This service could be further extended in a completely distributed manner on log basis rather than relying on cassandra, one could use Blockchain like DB that does store everything on log basis. The central bus I keep referring to could be replaced with mentioned and is fault tolerant also. Hence service would not need to just need to include this service to access information about other services also. This service could be clubbed with rhythm here. PostgresQL and Data Persistence Library for managing tenant aware DB for Postgres. Suggestions How can we make a standard lib? Casandra Lib for managing the service calls for Cassandra DB, Interestingly the cost of managing the cassandra service is around 250 US per month at any given cloud service provider. How can one afford that? Suggestions Myrle, how can the unbanked afford to run this service? Please provide me with the answer. Async This library handles mutli-threading for commands lib above Suggestions Have you ever heard of RXJava ? Crypto This lib does the hashing of the password Suggestions Maybe we can use the tested libs instead of this one. Lang Conclusion The above project structure could be simplified by reducing the set of libs and allowing a log level DB in place. This would mean moving towards distributed architecture and harnessing the true propositions of the microservices. Provisioner does a great job but the services should get to know by communicating with each other what they have and what methods they can share that other services can trigger using client or message logging using the central bus with appropriate permission stub type defined. One may say that these libraries are binded in such a way that it does the most of the work for the developers, this comes at the cost of maintaining the project. My argument is that developers are unable to contribute back because they find it difficult to understand the project from the structure perspective. The library the way they have been defined are ambiguous and not properly documented. Looks like the original author has tried to most effectively reduce the work for the developers and at the cost of increasing dependency on the immature libraries that could be outsourced to other open source libraries that are well behaved and are not vulnerable to bugs and upgrades etc. The command lib could be extended further so that services could use the command in terms of reading the information / messages and each service can make sense or interpret in their own approach. This reduces the size of the project from 40 * 3 = 120 projects including sub-projects from 120 to only 40 root projects with fewer than 5 libraries to maintain. In nutshell, once we reduce the project to such a level then we have a chance to keep the pace quickly or else one more year the project will be left behind amongst the growing number of digital financial services. One more interesting feature that will be achieved is if we end up using blockchain, the replacement of the accounting and transaction, the transaction for any MFI/FI can be recorded by the native token. Smart contract functionality can be achieved also if using something on those lines. PS the document is here for the WIP https://docs.google.com/document/d/1v3VDwAXxlV9xP38CP8OeYYYQBixu_3KrRXBhTsUjPMs/edit#
