I think the latter is better. Thanks, Bhathiya
On Mon, Oct 29, 2018 at 2:20 PM Dushani Wellappili <dusha...@wso2.com> wrote: > Hi all, > > The current implementation allows sending the fileName in 'Add a new > Document to API' REST call and add it to the DB. We ignore the file name > which sent in the FileDetail object in 'Upload the content of an API' REST > call and expects the user to always send the file name in DocumentDTO. Is > this behavior acceptable or do we need to allow the file name to be taken > when uploading the file content? WDYT? > > Thanks > > *Dushani Wellappili* > Software Engineer - WSO2 > > Email : dusha...@wso2.com > Mobile : +94779367571 > Web : https://wso2.com/ > > > > > On Wed, Oct 24, 2018 at 4:46 PM Dushani Wellappili <dusha...@wso2.com> > wrote: > >> Thank you both for the clarification. Will remove the meta data word >> usage from REST APIs and do the needful to get both content for url and >> inline docs using 'Get a document of API' REST call, and get content for >> FILE type documents using ' Get content of document' REST call. >> >> Regards, >> >> *Dushani Wellappili* >> Software Engineer - WSO2 >> >> Email : dusha...@wso2.com >> Mobile : +94779367571 >> Web : https://wso2.com/ >> >> >> >> >> On Wed, Oct 24, 2018 at 3:39 PM Mushthaq Rumy <musht...@wso2.com> wrote: >> >>> +1 for removing the word metadata from the REST API as the word metadata >>> is no longer used. And we would need to get the content for url and inline >>> types as they are displayed in the details page where as it make sense to >>> have a 2nd call to get the file content as it will be retrieved when the >>> download button is clicked. >>> >>> Thanks & Regards, >>> Mushthaq >>> >>> On Wed, 24 Oct 2018, 15:30 Uvindra Dias Jayasinha, <uvin...@wso2.com> >>> wrote: >>> >>>> >>>> >>>> On Wed, 24 Oct 2018 at 15:05, Dushani Wellappili <dusha...@wso2.com> >>>> wrote: >>>> >>>>> Hi all, >>>>> >>>>> As discussed, now all API documentation data will be stored in only >>>>> one table, which is the AM_API_DOCS with the following table schema. >>>>> >>>>> `AM_API_DOCS` ( >>>>> `UUID` VARCHAR(255), >>>>> `API_ID` VARCHAR(255), >>>>> `NAME` VARCHAR(255), >>>>> `SUMMARY` VARCHAR(1024), >>>>> `TYPE` VARCHAR(255), >>>>> `OTHER_TYPE_NAME` VARCHAR(255), >>>>> `CONTENT` VARCHAR(1024), >>>>> `CONTENT_BINARY_VALUE` LONGBLOB, >>>>> `FILE_NAME` VARCHAR(255), >>>>> `SOURCE_TYPE` VARCHAR(255), >>>>> `VISIBILITY` VARCHAR(30), >>>>> `AM_DOC_PERMISSION` int(11) DEFAULT '7', >>>>> CREATED_BY VARCHAR(100), >>>>> CREATED_TIME TIMESTAMP(6) DEFAULT CURRENT_TIMESTAMP(6), >>>>> UPDATED_BY VARCHAR(100), >>>>> LAST_UPDATED_TIME TIMESTAMP(6) DEFAULT CURRENT_TIMESTAMP(6), >>>>> PRIMARY KEY (`UUID`), >>>>> FOREIGN KEY (`API_ID`) REFERENCES `AM_API`(`UUID`) ON UPDATE CASCADE ON >>>>> DELETE CASCADE >>>>> ) >>>>> >>>>> The REST API call for adding documentation will add the values except >>>>> for CONTENT_BINARY_VALUE column when SOURCE_TYPE is 'FILE'. The CONTENT >>>>> column will have either the documentation URL or the inline content. >>>>> >>>>> When adding a documentation of source type either URL or INLINE, there >>>>> will be only REST call needed which is to add a new Document to API. When >>>>> adding a documentation of source type 'FILE', we would first do the 'Add a >>>>> new Document to API' REST call and then do the 'Upload content of an API >>>>> document' which will update the same record's CONTENT_BINARY_VALUE column. >>>>> >>>>> Similarly, for the REST call of getting a document of an API, we will >>>>> get values except the value in CONTENT_BINARY_VALUE column. For the REST >>>>> call of getting content of the document of an API, the >>>>> CONTENT_BINARY_VALUE >>>>> column value will be retrieved after checking whether the SourceType is >>>>> 'FILE'. >>>>> >>>>> One of the few concerns I have on the above design is, whether it is >>>>> okay to retrieve both URL value and inline content in 'Get a document of >>>>> an >>>>> API' call which we expect only to retrieve API doc metadata. If so, I >>>>> suppose we can't use the term API doc metadata for that REST API operation >>>>> description anymore? If not, we can change the REST API to only >>>>> retrieve doc metadata without the CONTENT column value and retrieve doc >>>>> content for all 3 types of docs using 'Get the content of an API document' >>>>> REST call. >>>>> >>>> >>>> This table is no longer just a meta data table so Im +1 for stopping >>>> the use of the word meta data in the REST API. We need to be able to >>>> retrieve the Inline and URL content in asingle call when we are listing the >>>> documents in the UI. >>>> >>>> The other concern is, in the above schema, we have only one of UPDATED_BY, >>>> LAST_UPDATED_TIME values for each document. So for both REST calls of >>>> getting the document (metadata) and getting document content data, is it >>>> okay to check with the same fingerprint (LAST_UPDATED_TIME), or do we need >>>> to add another separate audit columns for content updates as >>>> CONTENT_LAST_UPDATED_TIME, CONTENT_UPDATED_BY? >>>> >>>> We can simply update UPDATED_BY, LAST_UPDATED_TIME again at the time >>>> when we actually do a document upload. So that way the fingerprint will >>>> always be the latest. No need to add another column since the meta and >>>> content data are actually related. >>>> >>>> Appreciate your response. >>>> >>>> Thank you >>>> >>>> *Dushani Wellappili* >>>> Software Engineer - WSO2 >>>> >>>> Email : dusha...@wso2.com >>>> Mobile : +94779367571 >>>> Web : https://wso2.com/ >>>> >>>> >>>> >>>> >>>> On Tue, Oct 23, 2018 at 11:09 AM Harsha Kumara <hars...@wso2.com> >>>> wrote: >>>> >>>>> >>>>> >>>>> On Tue, Oct 23, 2018 at 10:07 AM Uvindra Dias Jayasinha < >>>>> uvin...@wso2.com> wrote: >>>>> >>>>>> @Bhathiya Jayasekara <bhath...@wso2.com> >>>>>> yes, this is also an option. In reality though very few customers >>>>>> upload lots of additional docs. We could just rename DOC_META_DATA table >>>>>> to >>>>>> DOCS and store the docs in that table itself. So at the very minimum we >>>>>> have a swagger and ballerina definition per an API stored in the >>>>>> RESOURCES >>>>>> table. >>>>>> >>>>> Definitely we should have a separate table for docs. >>>>> >>>>>> >>>>>> Can we make a call on this soon? Implementation is getting delayed >>>>>> because of this. >>>>>> >>>>>> On Mon, 22 Oct 2018 at 21:08, Bhathiya Jayasekara <bhath...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> This looks ok. Btw, I was wondering if it's a good idea to have all >>>>>>> files (swaggers, wsdls, docs etc.) in a single table. Won't that make >>>>>>> the >>>>>>> table grow faster and make querying slower? How about keeping a separate >>>>>>> table for this? Then we can have a foreign key as well. >>>>>>> >>>>>>> Thanks, >>>>>>> Bhathiya >>>>>>> >>>>>>> On Mon, Oct 22, 2018 at 2:57 PM Uvindra Dias Jayasinha < >>>>>>> uvin...@wso2.com> wrote: >>>>>>> >>>>>>>> +1, CONTENT is fine >>>>>>>> >>>>>>>> On Mon, 22 Oct 2018 at 14:45, Mushthaq Rumy <musht...@wso2.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Uvindra, >>>>>>>>> >>>>>>>>> Please find the data types below >>>>>>>>> >>>>>>>>> *AM_API_DOC_META_DATA* >>>>>>>>> API_ID (This will be a foreign key referred to AM_API table) >>>>>>>>> VARCHAR(255) >>>>>>>>> RESOURCE_ID VARCHAR(255) >>>>>>>>> DOC_CONTENT VARCHAR(1024) >>>>>>>>> >>>>>>>>> *AM_API_RESOURCES* >>>>>>>>> RESOURCE_NAME VARCHAR(255) >>>>>>>>> >>>>>>>>> And I think we can change it DOC_CONTENT to CONTENT . >>>>>>>>> WDYT? >>>>>>>>> >>>>>>>>> Thanks & Regards, >>>>>>>>> Mushthaq >>>>>>>>> >>>>>>>>> On Mon, Oct 22, 2018 at 2:35 PM Uvindra Dias Jayasinha < >>>>>>>>> uvin...@wso2.com> wrote: >>>>>>>>> >>>>>>>>>> HI Mushtaq, >>>>>>>>>> >>>>>>>>>> Can you provide the data types of the columns so that this is >>>>>>>>>> more clear? I believe that DOC_CONTENT should be a VARCHAR, in that >>>>>>>>>> case >>>>>>>>>> better to call it something like SHORT_CONTENT, since this >>>>>>>>>> communicates >>>>>>>>>> that it is meant to share relatively short text values such as URL >>>>>>>>>> and >>>>>>>>>> inline text. The name DOC_CONTENT is misleading and might suggest we >>>>>>>>>> can >>>>>>>>>> store larger documents in this column. >>>>>>>>>> >>>>>>>>>> On Mon, 22 Oct 2018 at 14:02, Mushthaq Rumy <musht...@wso2.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi All, >>>>>>>>>>> >>>>>>>>>>> We had an internal discussion and decided to do some >>>>>>>>>>> modifications in the DB Schema related to API Docs >>>>>>>>>>> (AM_API_RESOURCES and >>>>>>>>>>> AM_API_DOC_META_DATA). We will be removing the foreign key >>>>>>>>>>> constraint added >>>>>>>>>>> to the AM_API_DOC_META_DATA table as it it has a primary key >>>>>>>>>>> referred to >>>>>>>>>>> the primary key of AM_API_RESOURCES which is not a good practice. >>>>>>>>>>> >>>>>>>>>>> And the following columns will be added to the respective tables. >>>>>>>>>>> >>>>>>>>>>> *AM_API_DOC_META_DATA* >>>>>>>>>>> API_ID (This will be a foreign key referred to AM_API table) >>>>>>>>>>> RESOURCE_ID >>>>>>>>>>> DOC_CONTENT >>>>>>>>>>> >>>>>>>>>>> *AM_API_RESOURCES* >>>>>>>>>>> RESOURCE_NAME >>>>>>>>>>> >>>>>>>>>>> So the content of INLINE and URL of API documents will be saved >>>>>>>>>>> in *AM_API_DOC_META_DATA (*DOC_CONTENT column*) *and content of >>>>>>>>>>> FILE of API documents will be saved in *AM_API_RESOURCES *which >>>>>>>>>>> will have a reference in *AM_API_DOC_META_DATA (*RESOURCE_ID >>>>>>>>>>> column*)*. >>>>>>>>>>> >>>>>>>>>>> Thanks & Regards, >>>>>>>>>>> Mushthaq >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, Oct 19, 2018 at 9:23 PM Harsha Kumara <hars...@wso2.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Oct 19, 2018 at 10:30 AM Uvindra Dias Jayasinha < >>>>>>>>>>>> uvin...@wso2.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Here is my take on this. When I originally designed the schema >>>>>>>>>>>>> I wasn't taking into consideration any of the practical >>>>>>>>>>>>> implications >>>>>>>>>>>>> associated with API resources being saved and retrieved at DB >>>>>>>>>>>>> level. But >>>>>>>>>>>>> now that we are at implementation stage some of these >>>>>>>>>>>>> implications are much >>>>>>>>>>>>> more clearer now. >>>>>>>>>>>>> >>>>>>>>>>>>> The AM_API_RESOURCES is a generic API resource table(For >>>>>>>>>>>>> storing all file based resources associated with APIs). It will >>>>>>>>>>>>> be storing >>>>>>>>>>>>> the Swagger file, Ballerina file and documentation associated >>>>>>>>>>>>> with the API. >>>>>>>>>>>>> >>>>>>>>>>>>> The AM_API_DOC_META_DATA table is specialized to store >>>>>>>>>>>>> additional meta data only associated with documentation. >>>>>>>>>>>>> >>>>>>>>>>>>> Practically we need to do two calls for document uploads and >>>>>>>>>>>>> adding meta data because we are dealing with two different content >>>>>>>>>>>>> types(application/json for meta data and multipart/form-data for >>>>>>>>>>>>> the file). >>>>>>>>>>>>> >>>>>>>>>>>>> All files have a name associated with them so it makes sense >>>>>>>>>>>>> to have the file name in the AM_API_RESOURCES table. I don't >>>>>>>>>>>>> think its a >>>>>>>>>>>>> good idea to have a NULL value in a column that we are going to >>>>>>>>>>>>> update >>>>>>>>>>>>> later, this could lead to all kinds of complications that we will >>>>>>>>>>>>> need to >>>>>>>>>>>>> handle at code level. So its better to have the file name in >>>>>>>>>>>>> AM_API_RESOURCES where we can ensure that we always have a valid >>>>>>>>>>>>> name at >>>>>>>>>>>>> the time of upload. It is also very easy for us to enforce that a >>>>>>>>>>>>> file name >>>>>>>>>>>>> for a given type does not get duplicated with a table level >>>>>>>>>>>>> constraint if >>>>>>>>>>>>> we go with this option. >>>>>>>>>>>>> >>>>>>>>>>>>> Joining between two tables like this in case we need to get >>>>>>>>>>>>> the file name is trivial so I don't think we should let that >>>>>>>>>>>>> affect us >>>>>>>>>>>>> coming up with the best possible solution. >>>>>>>>>>>>> >>>>>>>>>>>> +1 it's a not good practice to add record which will update >>>>>>>>>>>> from null to some value cause of update going for another table. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> So Im +1 for option 2. WDYT? >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, 18 Oct 2018 at 17:31, Mushthaq Rumy <musht...@wso2.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Adding @dev-wso2 <d...@wso2.org> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Oct 18, 2018 at 5:25 PM Nuwan Dias <nuw...@wso2.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please discuss technical problems externally. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Thu, Oct 18, 2018 at 3:44 PM Mushthaq Rumy < >>>>>>>>>>>>>>> musht...@wso2.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> While I was implementing the view page for API document >>>>>>>>>>>>>>>> (File) I came across an issue where we get the file name as >>>>>>>>>>>>>>>> null when using >>>>>>>>>>>>>>>> the micro-service to get the content of the the API document. >>>>>>>>>>>>>>>> While analyzing, when adding a file as an API document, I >>>>>>>>>>>>>>>> found out that first we save only the doc metadata and then >>>>>>>>>>>>>>>> we save the >>>>>>>>>>>>>>>> file content using a second call. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> After analyzing the DB scripts I figured out that the >>>>>>>>>>>>>>>> fileName is stored in AM_API_DOC_META_DATA table and the >>>>>>>>>>>>>>>> content is stored >>>>>>>>>>>>>>>> in AM_API_RESOURCES. So during the first call we do not have >>>>>>>>>>>>>>>> the file name >>>>>>>>>>>>>>>> and it is saved as null. During the second call the fileName >>>>>>>>>>>>>>>> is passed to >>>>>>>>>>>>>>>> the micro-service but it is not stored anywhere. Hence, the >>>>>>>>>>>>>>>> fileName is >>>>>>>>>>>>>>>> null when we get the content of the file. So to solve this >>>>>>>>>>>>>>>> issue, I thought >>>>>>>>>>>>>>>> of two solutions. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 1. During the second call while adding a file document for >>>>>>>>>>>>>>>> API as we get the FileName to the micro-service we can >>>>>>>>>>>>>>>> retrieve the >>>>>>>>>>>>>>>> document metadata using the documentId and update the fileName >>>>>>>>>>>>>>>> apart from >>>>>>>>>>>>>>>> saving the content. Hence, it will be available when >>>>>>>>>>>>>>>> retrieving the content. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 2. We can change the fileName field from >>>>>>>>>>>>>>>> AM_API_DOC_META_DATA to AM_API_RESOURCES as the content of the >>>>>>>>>>>>>>>> document is >>>>>>>>>>>>>>>> stored in this table. And while saving the content we can save >>>>>>>>>>>>>>>> it with the >>>>>>>>>>>>>>>> fileName. Hence, it will be available when retrieving the >>>>>>>>>>>>>>>> content. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> IMO as option 1 will have more DB calls, option 2 would be >>>>>>>>>>>>>>>> the preferred solution. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Appreciate your valuable inputs. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks & Regards, >>>>>>>>>>>>>>>> Mushthaq >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Mushthaq Rumy >>>>>>>>>>>>>>>> *Senior Software Engineer* >>>>>>>>>>>>>>>> Mobile : +94 (0) 779 492140 >>>>>>>>>>>>>>>> Email : musht...@wso2.com >>>>>>>>>>>>>>>> WSO2, Inc.; http://wso2.com/ >>>>>>>>>>>>>>>> lean . enterprise . middleware. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> <http://wso2.com/signature> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> *Nuwan Dias* | Director | WSO2 Inc. >>>>>>>>>>>>>>> (m) +94 777 775 729 | (e) nuw...@wso2.com >>>>>>>>>>>>>>> [image: Signature.jpg] >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Mushthaq Rumy >>>>>>>>>>>>>> *Senior Software Engineer* >>>>>>>>>>>>>> Mobile : +94 (0) 779 492140 >>>>>>>>>>>>>> Email : musht...@wso2.com >>>>>>>>>>>>>> WSO2, Inc.; http://wso2.com/ >>>>>>>>>>>>>> lean . enterprise . middleware. >>>>>>>>>>>>>> >>>>>>>>>>>>>> <http://wso2.com/signature> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Uvindra >>>>>>>>>>>>> >>>>>>>>>>>>> Mobile: 777733962 >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>>> *Harsha Kumara* >>>>>>>>>>>> >>>>>>>>>>>> Associate Technical Lead, WSO2 Inc. >>>>>>>>>>>> Mobile: +94775505618 >>>>>>>>>>>> Email: hars...@wso2.coim >>>>>>>>>>>> Blog: harshcreationz.blogspot.com >>>>>>>>>>>> >>>>>>>>>>>> GET INTEGRATION AGILE >>>>>>>>>>>> Integration Agility for Digitally Driven Business >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Mushthaq Rumy >>>>>>>>>>> *Senior Software Engineer* >>>>>>>>>>> Mobile : +94 (0) 779 492140 >>>>>>>>>>> Email : musht...@wso2.com >>>>>>>>>>> WSO2, Inc.; http://wso2.com/ >>>>>>>>>>> lean . enterprise . middleware. >>>>>>>>>>> >>>>>>>>>>> <http://wso2.com/signature> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Regards, >>>>>>>>>> Uvindra >>>>>>>>>> >>>>>>>>>> Mobile: 777733962 >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Mushthaq Rumy >>>>>>>>> *Senior Software Engineer* >>>>>>>>> Mobile : +94 (0) 779 492140 >>>>>>>>> Email : musht...@wso2.com >>>>>>>>> WSO2, Inc.; http://wso2.com/ >>>>>>>>> lean . enterprise . middleware. >>>>>>>>> >>>>>>>>> <http://wso2.com/signature> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> Uvindra >>>>>>>> >>>>>>>> Mobile: 777733962 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Bhathiya Jayasekara* >>>>>>> *Technical Lead,* >>>>>>> *WSO2 inc., http://wso2.com <http://wso2.com>* >>>>>>> >>>>>>> *Phone: +94715478185* >>>>>>> *LinkedIn: http://www.linkedin.com/in/bhathiyaj >>>>>>> <http://www.linkedin.com/in/bhathiyaj>* >>>>>>> *Twitter: https://twitter.com/bhathiyax >>>>>>> <https://twitter.com/bhathiyax>* >>>>>>> *Blog: http://movingaheadblog.blogspot.com >>>>>>> <http://movingaheadblog.blogspot.com/>* >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> Uvindra >>>>>> >>>>>> Mobile: 777733962 >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> *Harsha Kumara* >>>>> >>>>> Associate Technical Lead, WSO2 Inc. >>>>> Mobile: +94775505618 >>>>> Email: hars...@wso2.coim >>>>> Blog: harshcreationz.blogspot.com >>>>> >>>>> GET INTEGRATION AGILE >>>>> Integration Agility for Digitally Driven Business >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Uvindra >>>> >>>> Mobile: 777733962 >>>> >>> -- *Bhathiya Jayasekara* *Technical Lead,* *WSO2 inc., http://wso2.com <http://wso2.com>* *Phone: +94715478185* *LinkedIn: http://www.linkedin.com/in/bhathiyaj <http://www.linkedin.com/in/bhathiyaj>* *Twitter: https://twitter.com/bhathiyax <https://twitter.com/bhathiyax>* *Blog: http://movingaheadblog.blogspot.com <http://movingaheadblog.blogspot.com/>*
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
