Hi Chan, In SS we have the HDFS browser facility. This is a feature where you can upload/download files to the HDFS. Please refer https://docs.wso2.org/display/SS110/HDFS+Browser. Please check if this fits your requirement.
Also, as far as UES is concerned, implementing the HDFS UI as a jaggery app is a roadmap item https://redmine.wso2.com/issues/1744. Mutitenancy is yet to be completed and in the current release is not production recommended. With the introduction of Hadoop 2.x in SS (in a future release) we are planning to look into possibilities of federating the file system and thereby address multitenancy. It is yet to be decided. On Fri, Feb 28, 2014 at 6:33 AM, Chan <[email protected]> wrote: > *Simple spec for the Unified Storage proposal * > The goal of the Unified Storage proposal is to provide a REST API and > jaggery module wrapper which will provide below features - > > - Persist files > - Get files > > > *REST API* > * Persist file* > > URL - storage/resource/ > VERB - PUT > > Headers > > consumer token - Get the token of the provider > > consumer user - User that's authenticated to the service provider > > Body > > File data > > Response > > Returns id > > > *Get file* > > URL - storage/ > resource > /:id > > VERB - GET > Attributes > > id - The ID obtained from the PUT call > > Headers > > consumer token - Get the token of the provider > > consumer user - User that's authenticated to the service provider > > Response > > Returns the file > > *Jaggery Module* > A jaggery module will be implemented which wrap the REST API and provide a > developer friendly javascript api. > > var storage = require("storage").storage; > /* > To be configured with > { > provider-name: sdf > provider-secret: sdf > } > */ > storage.configure({}); > var file = new File(); > > /* > persist a file > meta will contain > { > id = id to access the file > } > */ > var meta = storage.put(file); > > /* > request a file by sending the file id > */ > var file = storage.get(id); > > > We can improve on the spec by providing security, multi-tenancy > *Buckets* > A bucket is a namespace for files or objects. This will be useful for a > use-case where namespaces for files are required. > > *Multi-tenancy* > There are some limitations in multi-tenancy in HDFS (@Deep or @Shani can > better explain this). We can implement the tenant isolation in the storage > app layer. > > *Security* > A token can be requested from storage for a resource before issuing the > url to a client. A use-case is where store takes a token and append it to > the url before sending it to the android device. There will be a governance > model on the time period and invalidation to secure the executable > resource. Also another layer of security will be implemented so that > Storage app can trust the Service-Provider (which is Publisher or Store). > > Any ideas are welcome. > > Cheers~ > > > On Mon, Feb 24, 2014 at 1:23 PM, Chan <[email protected]> wrote: > >> @Dilshan as I have mentioned - SS only handles provisioning of storage. >> The proposed solution is for File Storage (not for provisioning). >> >> >> On Mon, Feb 24, 2014 at 12:51 PM, Dilshan Edirisuriya >> <[email protected]>wrote: >> >>> Hi Chan, >>> >>> I think we discussed this few months ago with SS team (Prabath, Deep). >>> What was the final conclusion? >>> >>> Regards, >>> >>> Dilshan >>> >>> >>> On Thu, Feb 20, 2014 at 7:49 PM, Chan <[email protected]> wrote: >>> >>>> Hi folks, >>>> In our platform - we don't yet have a solution for storing static files >>>> in the cloud. A solution that provides a simple service of persisting a >>>> file and getting back a URI. The Enterprise Store team built a storage >>>> module in the Publisher app that provides persisting of files to a database >>>> and getting back URIs. My proposal is to build a jaggery-app that >>>> specifically handles file storage with a REST interface extending what the >>>> ES team has done. >>>> >>>> *Use case* >>>> I will be using the Enterprise Store as an example for the Storage App. >>>> [image: Inline image 1] >>>> 1:- Developer uploads a file to the publisher app using an entry form >>>> 2:- Publisher app calls the storage module to contact the storage >>>> provider >>>> 3:- Storage module contacts the Storage app with a HTTP PUT call and >>>> returns the URI of the file persisted. Publisher app will persist this URI >>>> to the registry along with other information. >>>> 4:- A client contacts the server requesting an asset page. >>>> 5:- The server will contact the storage module to request a token. It >>>> will in turn call the Storage app requesting a token. After retrieving the >>>> token It will append the token to the URI when rendering the pages server >>>> side. >>>> 6:- Browser will contact the Storage app directly when requesting the >>>> file using HTTP GET. Since the browser is passing a valid token - Storage >>>> app will return the file in the response. >>>> >>>> *Architecture overview* >>>> Storage app will have service providers. These service providers will >>>> be apps that require the Storage service from the Storage app. First let's >>>> start off with the API. Storage app will provide an HTTP interface as below >>>> GET >>>> >>>> GET interface is used to request a file. The request can be >>>> for permanent links or token based links. >>>> >>>> PUT >>>> >>>> PUT can only called by a registered service provider. A token (Provider >>>> token) has to be passed to validate the service provider. It will also >>>> allow options of - security, permanent links etc. >>>> >>>> DELETE >>>> >>>> URI is passed to the DELETE API as well as the provider token. Used to >>>> delete files. >>>> >>>> >>>> Next is what the real storage options are. For the first cut we can >>>> have the storage app connected to a database. Afterwards we can move to >>>> HDFS type storage or Cassandra perhaps. >>>> >>>> As with Multi-tenancy - we can handle multi-tenancy the same way we >>>> handle it for jaggery apps (SaaS multi-tenancy). >>>> >>>> Ultimate objective of this proposal is to build a cloud storage service >>>> like Amazon S3, Google Cloud storage [1]. WDYT guys? >>>> >>>> [1] - >>>> http://www.quora.com/Amazon-S3/What-are-the-best-alternatives-to-S3-and-what-are-the-pros-and-cons >>>> >>>> Cheers~ >>>> -- >>>> Chan (Dulitha Wijewantha) >>>> Software Engineer - Mobile Development >>>> WSO2Mobile >>>> Lean.Enterprise.Mobileware >>>> * ~Email [email protected] <[email protected]>* >>>> * ~Mobile +94712112165 <%2B94712112165>* >>>> >>>> * ~Website dulithawijewantha.com <http://dulithawijewantha.com/> * >>>> >>>> * ~Blog blog.dulithawijewantha.com >>>> <http://dulichan.github.io/chan/>* >>>> * ~Twitter @dulitharw <https://twitter.com/dulitharw>* >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Dilshan Edirisuriya >>> Senior Software Engineer - WSO2Mobile >>> Mob: + 94 772245502 >>> http://wso2mobile.com/ >>> >> >> >> >> -- >> Chan (Dulitha Wijewantha) >> Software Engineer - Mobile Development >> WSO2Mobile >> Lean.Enterprise.Mobileware >> * ~Email [email protected] <[email protected]>* >> * ~Mobile +94712112165 <%2B94712112165>* >> >> * ~Website dulithawijewantha.com <http://dulithawijewantha.com/> * >> >> * ~Blog blog.dulithawijewantha.com >> <http://dulichan.github.io/chan/>* >> * ~Twitter @dulitharw <https://twitter.com/dulitharw>* >> > > > > -- > Chan (Dulitha Wijewantha) > Software Engineer - Mobile Development > WSO2Mobile > Lean.Enterprise.Mobileware > * ~Email [email protected] <[email protected]>* > * ~Mobile +94712112165 <%2B94712112165>* > > * ~Website dulithawijewantha.com <http://dulithawijewantha.com/> * > > * ~Blog blog.dulithawijewantha.com > <http://dulichan.github.io/chan/>* > * ~Twitter @dulitharw <https://twitter.com/dulitharw>* > -- Thanks and Regards *,Shani Ranasinghe* Software Engineer WSO2 Inc.; http://wso2.com lean.enterprise.middleware mobile: +94 77 2273555 linked in: lk.linkedin.com/pub/shani-ranasinghe/34/111/ab
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
