Hi Isura, On Fri, Jun 7, 2019 at 9:16 AM Isura Karunaratne <[email protected]> wrote:
> > > On Wed, Jun 5, 2019 at 9:34 AM Ruwan Abeykoon <[email protected]> wrote: > >> Hi All, >> The "User Store" configuration can be considered as a deployment artifact >> if we look at the following aspects. >> >> a) "Secondary User Stores" are added, removed and updated per tenant >> basis. (Same as SP, and IdP configs) >> b) "User Store", "IdP" and "SP", XACML policies, etc behaves as a >> collection of business rules, defining the authentication and authorization >> flows per the tenant. >> c) A change in a particular "User Store" usually affect the >> "Authentication" decisions done via SP config. Hence they have tight >> coupling. >> d) All "User Store", "SP", "IdP", etc, need to be taken as one unit, when >> we consider environment to environment promotion of these configs. >> (Dev->QA->staging->Prod) >> >> Hence IMHO, treating "User Store" as the file-based artifact was a right >> decision, when our products have been designed for deployment on bear-metal >> or VM. However moving forward to container, and cloud nativeness, posses >> the challenge on sharing these artifacts. >> >> Also, considering the CI/CD pipelines, the governance aspect of changing >> the configurations, etc, these type of configs need to be considered as >> artifacts. We might need to version control these artifacts in future and >> may need to push and pull them from VCS systems. >> >> What we need to do is to have a delegation pattern implemented for all >> current file based (and registry based artifacts), where we can switch the >> repository from file based one to different system. DB based repository >> would be the first such(simple) implementation. We may need to implement >> Git based repository when we properly support cloud use cases for example. >> We can abstract the storage system, and retain all the parsing and >> generation logic unchanged for artifacts. it would be a minimal change and >> most versatile way to extend IMO. >> >> We would need to implement property or "environment variable" binding >> logic, to get proper support for environment to environment promotion of >> artifacts. yet, it can be done with a separate effort than this IMO. >> >> Hence +1 to treat >> >> - Persist data as a blob (marshalled to text form) >> - In a separate table structure. >> >> Cheers, >> Ruwan A >> >> >> On Tue, Jun 4, 2019 at 3:50 PM Johann Nallathamby <[email protected]> >> wrote: >> >>> +1 to get rid of the artifacts for user stores. I think this was a wrong >>> decision we made early on. >>> >>> On Tue, Jun 4, 2019 at 1:19 PM Hasanthi Purnima Dissanayake < >>> [email protected]> wrote: >>> >>>> Hi All, >>>> >>>> *Problem * >>>> Currently, some artifacts like userstores , tenants' data, etc are >>>> stored in the file system (not in the database). So when using a clustered >>>> setup those artifacts should be shared among all the nodes by using one of >>>> the following file sharing mechanisms. >>>> >>>> - Dep Sync >>>> - rSync >>>> - Shared File System >>>> >>>> >>>> *Solution * >>>> In order to avoid a shared file system and to reduce the deployment and >>>> maintenance overhead, those artifacts ca be persisted in the database >>>> itself. >>>> >>>> *Approach* >>>> After discussing with @Ruwan Abeykoon <[email protected]> and @Isura >>>> Karunaratne <[email protected]> we have two options to persist above >>>> discussed artifact details. >>>> >>>> - In the configuration store which is already implemented as >>>> discussed in [1][2]. >>>> - In a separate table structure. >>>> >>>> If we are to go with option 01, then we need to consider the artifacts >>>> as configurations and persist in the existing schema. >>>> >>> The advantage of using this is we can re-use the existing implementation >>>> including the database schema and existing rest APIs and functionalities >>>> (pagination, searching, etc) . >>>> >>> >>> >> > Hi all, > > >> The drawback is the conceptual difference between an artifact and >>>> configuration. >>>> >>> >>> I think this is not a problem. In fact I believe we made a wrong >>> decision of considering user stores as artifacts. I can't remember exactly >>> as to why we decided like that. User stores are not development artifacts; >>> they are one time configurations. They don't have metadata, versioning, >>> lifecycles or any other properties associated with other artifacts in WSO2. >>> >>> >>>> Further if we are to use the configuration store there is no way to >>>> include specific input validations for the user store configuration >>>> feature. >>>> >>> >>> I am not sure I completely understand this. But please check the SAML2 >>> metadata for identity provider feature where I believe we've done some >>> validations on the properties. I think they were done through interceptors >>> in identity provider service. >>> >> Yes. If we have an interception layer, we can do the validation in that > level. Currently, we don't have an intercepting layer for configuration > management, we can treat this as an improvement to the config management > feature. > > >>>> If we are to go with the option 02, then the flow will be as follows. >>>> >>>> *Existing Flow* >>>> >>>> [image: Untitled Diagram (9).png] >>>> >>>> >>>> >>>> >>>> >>>> >>>> *Suggested Flow* >>>> >>>> [image: Untitled Diagram (10).png] >>>> >>>> >>>> >>>> - With the suggested approach, as the storage mechanisms, file >>>> system and database can be used and any other storage mechanism is >>>> pluggable. >>>> >>>> I don't agree with classifying user stores as artifacts. Therefore I >>> guess for me this option doesn't qualify at all :). >>> >>> However, just to understand this option better, have we come across any >>> customer who has asked us to support multiple types of storage mechanisms >>> for artifacts and/or configurations? Where is this requirement coming from? >>> I've only seen such requirements for user stores. For configurations I feel >>> this is just over engineering. May be it is a valid requirement for >>> artifacts? Even if we agree that there are valid reasons to support this >>> then it has to be supported for all configurations and/or artifacts. >>> >> *I think you've missed the important question here :). Where is this requirement coming from? To have multiple storage locations for configurations?* Regards, Johann. > >>>> - There should be a way to identify the repository where the data >>>> is loaded from. The repository can be the file system, database or any >>>> other storage mechanism. >>>> >>>> It sounds like this can get too complicated. >>> >> I think this is just a flag which we can set while loading the user > stores. This will help to show where the user store is configured. Ex. file > system, database or GIT location. > >> >>>> - In both the read write operations the enduser should have the >>>> control to decide the storage mechanism. >>>> >>>> Hmm.. this sounds more like a requirement to optimize database read >>> write performance. Doesn't sound right for artifacts. >>> >> This way we can support the backward compatibility. It is true that we > can support backward compatibility via a configuration, but this is more > flexible. > >> >>>> - If the user needs to migrate a userstore from one storage >>>> mechanism (file system) to another then they can do it via UI. >>>> >>>> Again too many options for the user can make the product fragile. >>> >> Users may require to migrate the existing file-based configuration to > JDBC tables. In such cases, this will help to migrate configuration rather > than doing them manually. > >> >>> >>>> When persisting the data in the database there are two options we can >>>> use : >>>> >>>> - Persist data as a blob >>>> >>>> If we persist as blob then we lose the granular control over each >>> property for validation, transformation, etc. >>> >>>> >>>> - Persists data as key value pair >>>> >>>> +1 for this. >>> >>> >>>> If we are to go with the option one then we can persist the file as a >>>> blob and reuse most of the existing parsing logics. >>>> >>> >>> Given the understanding I think I prefer option 1 with properties. >>> >>> Thanks & Regards, >>> Johann. >>> >> > Cheers, > Isura. > >> >>> >>>> >>>> Highly appreciate your suggestions and feedbacks on the above approach. >>>> >>>> [1] [Architecture][IAM][JDBC based Configuration Store] Database Schema >>>> [2] [Architecture] [IS] JDBC based Configuration Store for WSO2 IS >>>> >>>> Thanks, >>>> Hasanthi >>>> >>>> -- >>>> >>>> Hasanthi Dissanayake | Senior Software Engineer | WSO2 Inc. >>>> (m) +94718407133 | (w) +94112145345 | Email: [email protected] >>>> >>>> >>> >>> -- >>> *Johann Dilantha Nallathamby* | Associate Director/Solutions Architect >>> | WSO2 Inc. >>> (m) +94 (77) 7776950 | (w) +94 (11) 2145345 | (e) [email protected] >>> [image: Signature.jpg] >>> >> >> >> -- >> Ruwan Abeykoon | Director/Architect | WSO2 Inc. >> (w) +947435800 | Email: [email protected] >> >> > > -- > > *Isura Dilhara Karunaratne* > Technical Lead | WSO2 <http://wso2.com/> > *lean.enterprise.middleware* > Email: [email protected] > Mob : +94 772 254 810 > Blog : https://medium.com/@isurakarunaratne > > > > -- *Johann Dilantha Nallathamby* | Associate Director/Solutions Architect | WSO2 Inc. (m) +94 (77) 7776950 | (w) +94 (11) 2145345 | (e) [email protected] [image: Signature.jpg]
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
