Hi Megala, AFAIU There are two problems as where we store the file contents(supporting clustering cases) and file structure of application content.
Storage Location - This should be written as extension where you can store in local directory, FTP, NFS or someone can customize based on requirement. Application store structure - I prefer option 1 as it will be more manageable per application which makes separate storage space for application contents(apk,icon etc). As an example we can store like <application_id>/<version>/<application_content_name>/content_data I option 2 you will have less folders but you may need to concat file names for support for multiple application versions and their contents. In option 1 delete application operation also easy. Do we need real requirement to search by image name,apk name or other content name. If we have that kind of requiremnt as well then option 1 will have some cons. Thanks, Waruna On Wed, Aug 16, 2017 at 6:02 PM, Charitha Goonetilleke <charit...@wso2.com> wrote: > Hi Nuwan, > > As our previous experience with similar approach we have used to store > binaries in DB, it cause for scallability issues in later when growing the > size of the DB in GBs. So storing files as binaries in DB is highly > depending on the DB engine used and the growth rate of the data. IMO, these > concerns and issues might be critical and complex than the complexity we > are going to introduce with FS based solution. So I think it is best to > store files in FS rather than adding it to DB. > > Anyway as you have raised, there might be situations where, NFS > integration is not possible. So I'm +1 for having few alternative ways to > handle this as well. In addition to the RSync, we can also use shared > volume or even S3 bucket if some one is going to deploy this in AWS. > > @Megala, I'm not clear with the reason behind this statement in under > option 2 > > "but this may not be possible in real production scenarios depending on > the size of the images." > > > On Wed, Aug 16, 2017 at 5:46 PM, Nuwan Dias <nuw...@wso2.com> wrote: > >> If you require NFS anyway (for other usecases), then I think its fine. >> But if you're introducing the complexity of NFS just for this use case, >> then I think it would be an overkill. >> >> BTW, not every one is ok with setting up a NFS. So if you need to share >> files between nodes you may want to have some other recommendations handy >> as well (RSync). >> >> Is there a problem is storing this file on the DB instead of the FS? >> >> On Wed, Aug 16, 2017 at 5:41 PM, Megala Uthayakumar <meg...@wso2.com> >> wrote: >> >>> Hi Nuwan, >>> >>> As mentioned by Harshan user needs to configure NFS when configuring >>> this in HA setup. We do accept local file path or NFS url in the >>> configuration file for the path. >>> >>> Thanks. >>> >>> On Wed, Aug 16, 2017 at 5:01 PM, Nuwan Dias <nuw...@wso2.com> wrote: >>> >>>> When you have two nodes of the IoT server running for HA, and if a >>>> given image/file is stored on the file system of one server, how do you >>>> make that image/file available for a request that's served via the other >>>> server? >>>> >>>> On Wed, Aug 16, 2017 at 4:54 PM, Megala Uthayakumar <meg...@wso2.com> >>>> wrote: >>>> >>>>> Hi All, >>>>> >>>>> Currently we are working on writing a mobile application store for IOT >>>>> server. For that we have decided to save the images and binary files >>>>> related with applications in the file system(This decision was taken >>>>> considering the file size). File location will be derived from a >>>>> configuration. While implementing this, we have come-up with the following >>>>> suggestions for saving files as per the meeting we had internally. >>>>> >>>>> *Option 1* >>>>> For each application we will have a folder with the name as the >>>>> generated ID for the application from the database. Each will hold all the >>>>> relevant artifacts such as icon, screen-shots and binary files. >>>>> In this case, there were 2 suggestions >>>>> >>>>> 1. For each app, the icon and screen-shots will be saved with the >>>>> same name. Ex - For icons the image file name will be 'icon' and >>>>> like-wise >>>>> for others as well. So in this case, we do not need to save image >>>>> names in >>>>> the database. >>>>> 2. Need to dynamically generate some random names. >>>>> >>>>> If we consider first approach, if we use the same name for all the >>>>> application, there is possible chance an attacker may get all the image >>>>> files if the name is known to them. AFAIU this can happen even if we do >>>>> not use the 1st approach and use the second approach in which we use >>>>> random >>>>> names, as this can happen only if the root path for saving the artifacts >>>>> are compromised and without the name also attacker can do a "listFiles" >>>>> request and get all the data. >>>>> >>>>> *Option 2* >>>>> For binary files and image files we will have separate location and >>>>> each respective files will be saved in these location separately by >>>>> generating a unique name. This may be helpful if we can cache the images >>>>> and improve the performance, but this may not be possible in real >>>>> production scenarios depending on the size of the images. >>>>> >>>>> What would be the most preferable option? Comments and suggestions on >>>>> this regard is highly appreciated. >>>>> >>>>> Thanks. >>>>> >>>>> Regards, >>>>> Megala >>>>> >>>>> -- >>>>> Megala Uthayakumar >>>>> >>>>> Software Engineer >>>>> Mobile : 0779967122 >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "WSO2 IoT Team Group" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to iot-group+unsubscr...@wso2.com. >>>>> For more options, visit https://groups.google.com/a/wso2.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> Nuwan Dias >>>> >>>> Software Architect - WSO2, Inc. http://wso2.com >>>> email : nuw...@wso2.com >>>> Phone : +94 777 775 729 <+94%2077%20777%205729> >>>> >>> >>> >>> >>> -- >>> Megala Uthayakumar >>> >>> Software Engineer >>> Mobile : 0779967122 >>> >> >> >> >> -- >> Nuwan Dias >> >> Software Architect - WSO2, Inc. http://wso2.com >> email : nuw...@wso2.com >> Phone : +94 777 775 729 <+94%2077%20777%205729> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "WSO2 IoT Team Group" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to iot-group+unsubscr...@wso2.com. >> For more options, visit https://groups.google.com/a/wso2.com/d/optout. >> > > > > -- > *Charitha Goonetilleke* > Senior Software Engineer > WSO2 Inc.; http://wso2.com > lean.enterprise.middleware > > mobile: +94 77 751 3669 <%2B94777513669> > Twitter:@CharithaWs <https://twitter.com/CharithaWs>, fb: charithag > <https://www.facebook.com/charithag>, linkedin: charithag > <http://www.linkedin.com/in/charithag> > > <http://wso2.com/signature> > > -- > You received this message because you are subscribed to the Google Groups > "WSO2 IoT Team Group" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to iot-group+unsubscr...@wso2.com. > For more options, visit https://groups.google.com/a/wso2.com/d/optout. > -- Regards, Waruna Lakshitha Jayaweera Senior Software Engineer WSO2 Inc; http://wso2.com phone: +94713255198 http://waruapz.blogspot.com/
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
