+1 to get rid of the artifacts for user stores. I think this was a wrong decision we made early on.
On Tue, Jun 4, 2019 at 1:19 PM Hasanthi Purnima Dissanayake < hasan...@wso2.com> wrote: > Hi All, > > *Problem * > Currently, some artifacts like userstores , tenants' data, etc are stored > in the file system (not in the database). So when using a clustered setup > those artifacts should be shared among all the nodes by using one of the > following file sharing mechanisms. > > - Dep Sync > - rSync > - Shared File System > > > *Solution * > In order to avoid a shared file system and to reduce the deployment and > maintenance overhead, those artifacts ca be persisted in the database > itself. > > *Approach* > After discussing with @Ruwan Abeykoon <ruw...@wso2.com> and @Isura > Karunaratne <is...@wso2.com> we have two options to persist above > discussed artifact details. > > - In the configuration store which is already implemented as discussed > in [1][2]. > - In a separate table structure. > > If we are to go with option 01, then we need to consider the artifacts as > configurations and persist in the existing schema. > The advantage of using this is we can re-use the existing implementation > including the database schema and existing rest APIs and functionalities > (pagination, searching, etc) . > > The drawback is the conceptual difference between an artifact and > configuration. > I think this is not a problem. In fact I believe we made a wrong decision of considering user stores as artifacts. I can't remember exactly as to why we decided like that. User stores are not development artifacts; they are one time configurations. They don't have metadata, versioning, lifecycles or any other properties associated with other artifacts in WSO2. > Further if we are to use the configuration store there is no way to > include specific input validations for the user store configuration feature. > I am not sure I completely understand this. But please check the SAML2 metadata for identity provider feature where I believe we've done some validations on the properties. I think they were done through interceptors in identity provider service. > > If we are to go with the option 02, then the flow will be as follows. > > *Existing Flow* > > [image: Untitled Diagram (9).png] > > > > > > > *Suggested Flow* > > [image: Untitled Diagram (10).png] > > > > - With the suggested approach, as the storage mechanisms, file system > and database can be used and any other storage mechanism is pluggable. > > I don't agree with classifying user stores as artifacts. Therefore I guess for me this option doesn't qualify at all :). However, just to understand this option better, have we come across any customer who has asked us to support multiple types of storage mechanisms for artifacts and/or configurations? Where is this requirement coming from? I've only seen such requirements for user stores. For configurations I feel this is just over engineering. May be it is a valid requirement for artifacts? Even if we agree that there are valid reasons to support this then it has to be supported for all configurations and/or artifacts. > > - There should be a way to identify the repository where the data is > loaded from. The repository can be the file system, database or any other > storage mechanism. > > It sounds like this can get too complicated. > > - In both the read write operations the enduser should have the > control to decide the storage mechanism. > > Hmm.. this sounds more like a requirement to optimize database read write performance. Doesn't sound right for artifacts. > > - If the user needs to migrate a userstore from one storage mechanism > (file system) to another then they can do it via UI. > > Again too many options for the user can make the product fragile. > When persisting the data in the database there are two options we can use : > > - Persist data as a blob > > If we persist as blob then we lose the granular control over each property for validation, transformation, etc. > > - Persists data as key value pair > > +1 for this. > If we are to go with the option one then we can persist the file as a blob > and reuse most of the existing parsing logics. > Given the understanding I think I prefer option 1 with properties. Thanks & Regards, Johann. > > Highly appreciate your suggestions and feedbacks on the above approach. > > [1] [Architecture][IAM][JDBC based Configuration Store] Database Schema > [2] [Architecture] [IS] JDBC based Configuration Store for WSO2 IS > > Thanks, > Hasanthi > > -- > > Hasanthi Dissanayake | Senior Software Engineer | WSO2 Inc. > (m) +94718407133 | (w) +94112145345 | Email: hasan...@wso2.com > > -- *Johann Dilantha Nallathamby* | Associate Director/Solutions Architect | WSO2 Inc. (m) +94 (77) 7776950 | (w) +94 (11) 2145345 | (e) joh...@wso2.com [image: Signature.jpg]
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
