Re: [whatwg] Shared storage
(Apologies...resending due to my inadvertent use of generic whatwg subject...) On 10/28/2014 12:06 PM,whatwg-requ...@lists.whatwg.org wrote: Date: Tue, 28 Oct 2014 17:33:02 + (UTC) From: Ian Hicksoni...@hixie.ch Subject: Re: [whatwg] Shared storage On Sat, 15 Feb 2014, Brett Zamir wrote: The desktop PC thankfully evolved into allowing third-party software which could create and edit files shareable by other third-party software which would have the same rights to do the same. The importance of this can hardly be overestimated. Yet today, on the web, there appears to be no standard way to create content in such an agnostic manner whereby users have full, built-in, locally-controlled portability of their data. Why can't you just do the same as used to be done? Download the resource locally (save, using a href download), then upload it to the new site (open, using input type=file)? Yes, as mentioned by others, this can become a terrible user experience, and this is not to speak of user agent reasons. Besides the added challenges mentioned by Katelyn Gadd when live updates occur on multiple instances of the file across different applications, there is also the even more common use case of a particular app being able to remember the files used previously and let the user access them without needing to remember their exact location in a hierarchy (though allowing a hierarchy is desirable to the user for flexibility in organization). Such recall of, and access to, previously used files without repeated need for manual selection is commonly found in apps which use the likes of a recent files drop-down or, perhaps even more commonly, by a set of tabs which open with the last set of used files. There is also the specific desirability for functionality to iterate through file names (or other shared data) so that apps can provide their own UI, perhaps filtered down by file type according to the types of files consumable by the app, such as an IDE project viewer which still allows the user to group files app-agnostically and where they wish along with other file types. The ability to store files of different types within the same user-viewable and user-creatable folder is compelling because the user has freedom to group like content together even if the file types differ. A user might wish to store an email draft, a word processing file, and a set of images all in the same folder to keep track of them, even if a given web app might not utilize all of these types. While there are apps which aggregate data from different sources and then let the user tag them in such a manner as to mimic this functionality, this is again application-specific and not necessarily portable or as flexibly under user control as would be a shared and hierarchical file storage area. Best, Brett
Re: [whatwg] Shared storage
On Sat, 15 Feb 2014, Brett Zamir wrote: The desktop PC thankfully evolved into allowing third-party software which could create and edit files shareable by other third-party software which would have the same rights to do the same. The importance of this can hardly be overestimated. Yet today, on the web, there appears to be no standard way to create content in such an agnostic manner whereby users have full, built-in, locally-controlled portability of their data. Why can't you just do the same as used to be done? Download the resource locally (save, using a href download), then upload it to the new site (open, using input type=file)? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Shared storage
On Tue, Oct 28, 2014 at 6:33 PM, Ian Hickson i...@hixie.ch wrote: On Sat, 15 Feb 2014, Brett Zamir wrote: The desktop PC thankfully evolved into allowing third-party software which could create and edit files shareable by other third-party software which would have the same rights to do the same. The importance of this can hardly be overestimated. Yet today, on the web, there appears to be no standard way to create content in such an agnostic manner whereby users have full, built-in, locally-controlled portability of their data. Why can't you just do the same as used to be done? Download the resource locally (save, using a href download), then upload it to the new site (open, using input type=file)? Because that's a terrible user experience? --tobie
Re: [whatwg] Shared storage
Tobie Langel tobie.lan...@gmail.com writes: On Tue, Oct 28, 2014 at 6:33 PM, Ian Hickson i...@hixie.ch wrote: On Sat, 15 Feb 2014, Brett Zamir wrote: The desktop PC thankfully evolved into allowing third-party software which could create and edit files shareable by other third-party software which would have the same rights to do the same. The importance of this can hardly be overestimated. Yet today, on the web, there appears to be no standard way to create content in such an agnostic manner whereby users have full, built-in, locally-controlled portability of their data. Why can't you just do the same as used to be done? Download the resource locally (save, using a href download), then upload it to the new site (open, using input type=file)? Because that's a terrible user experience? If that is indeed the case, the terrible user experience is likely a feature of your user agent. Many mobile UAs currently offer several alternatives to the standard file-picker, no change in HTML needed. -- Nils Dagsson Moskopp // erlehmann http://dieweltistgarnichtso.net
Re: [whatwg] Shared storage
A smarter file-picker is not going to address the significant UX and performance issues with save-open for moving files between applications. Many common workflows involve having files open in multiple editors at once, saving in one app, reloading in others. Some applications have built in affordances for this, like text editors and IDEs that automatically reload when the content has changed on disk. One real-world example: Unity's project management lets you pull various resources into your game, then live-edit the resources in Maya, Photoshop, etc with the IDE automatically pulling in the changes and running builds to update the content. If this kind of interchange and interaction requires literally doing save-confirm save-open-pick saved file from wherever it went *every time* nobody will use these applications. The applications that get used will be ones that figure out a clever set of workarounds like the postmessage/blessed domain one mentioned in the original post. That sort of approach is something that will easily lead to a walled garden library that 'solves the problem' and becomes widely adopted, when HTML5 itself could probably tackle the issue without a bunch of new scaffolding or significant new APIs. Mind you, I'm not saying this is an easy problem to solve - just that it's worth solving and that save-open is not a solution. One idea: 'transferable' support for indexedDB such that a blessed domain can postMessage an indexedDB database over to another domain, to allow shared access to storage for a specific database. This doesn't address all the problems in the original post, but simplifies the task of sharing content between applications. This would probably also pay dividends for web workers. (Actually, is this already specced? I only know that worker access to indexedDB wasn't available in Gecko or Blink the last time I checked and didn't seem to be a planned addition.) Another option: Some simple protocol for creating 'shared' databases using a special token in the name you pass to indexedDB.open? I can see how this might be a compat problem, but it would be a pretty low-overhead way to expose basic support for this. Perhaps the unique token you pass would be created by a feature-detectable API, sort of like how createObjectURL works. That would give you a single point where you can insert approval/feedback UI and do any necessary I/O to set up sharing, while the rest of the database flow remains unchanged. Synchronization between two processes mutually touching the DB is hard but you could just punt on that like is currently done with typed arrays - neutering or equivalent. Alternately, a single database-wide lock would probably cover things - that's the approach SQLite used for a long time. -kg On 28 October 2014 12:17, Nils Dagsson Moskopp n...@dieweltistgarnichtso.net wrote: Tobie Langel tobie.lan...@gmail.com writes: On Tue, Oct 28, 2014 at 6:33 PM, Ian Hickson i...@hixie.ch wrote: On Sat, 15 Feb 2014, Brett Zamir wrote: The desktop PC thankfully evolved into allowing third-party software which could create and edit files shareable by other third-party software which would have the same rights to do the same. The importance of this can hardly be overestimated. Yet today, on the web, there appears to be no standard way to create content in such an agnostic manner whereby users have full, built-in, locally-controlled portability of their data. Why can't you just do the same as used to be done? Download the resource locally (save, using a href download), then upload it to the new site (open, using input type=file)? Because that's a terrible user experience? If that is indeed the case, the terrible user experience is likely a feature of your user agent. Many mobile UAs currently offer several alternatives to the standard file-picker, no change in HTML needed. -- Nils Dagsson Moskopp // erlehmann http://dieweltistgarnichtso.net
Re: [whatwg] Shared storage
On 15 February 2014 03:04, Brett Zamir bret...@yahoo.com wrote: *The opportunity and current obstacles* The desktop PC thankfully evolved into allowing third-party software which could create and edit files shareable by other third-party software which would have the same rights to do the same. The importance of this can hardly be overestimated. Yet today, on the web, there appears to be no standard way to create content in such an agnostic manner whereby users have full, built-in, locally-controlled portability of their data. *Workarounds* Sure, there is postMessage or CORS requests which can be used to allow one site to be the arbiter of this data. And one could conceivably create a shared data store built upon even postMessage alone, even one which can work fully offline through cache manifests and localStorage or IndexedDB (I have begun some work on this concept at https://gist.github.com/brettz9/8876920 ), but this can only work if: 1. A site or set of domains is trusted to host the shared content. 2. Instead of being built into the browser, it requires that the shared storage site be visited at least one time. *Proposal* 1. Add support for sharedStorage (similar to globalStorage but requiring approval), SharedIndexedDB, and SharedFileWriter/SharedFileSystem which, when used, would cause the browser to prompt the user to require user approval whenever storing or retrieving from such data stores (with an option to remember the choice for a particular site/domain), informing users of potential risks depending on how the data might be used, and potentially allowing them to view, on the spot, the specific data that was being stored. Optional API methods could deter XSS by doing selective escaping, but the potential for abuse should not be used as an excuse for preventing arbitrary shared storage, since again, it is worked well on the desktop, despite risks there, and as works with postMessage despite it also having risks. 2. Add support for corresponding ReadonlyShared storage mechanisms, namespaced by the origin site of the data. A site, http://example.com might add such shared storage under example.com which http://some-other-site.example could retrieve but not alter or delete (unless perhaps a grave warning were given to users about the fact that this was not the same domain). This would have the benefit above postMessage in that if the origin site goes down, third party sites would still be able to have access to the data. 3. Encourage browsers to allow direct editing of this stored data in a human-readable manner (with files at least being ideally directly viewable from the OS desktop). I proposed something similar earlier, and received a reply about doing this through shared workers, but as I understood it, I did not like that possibility because: a. it would limit the neutrality of the storage, creating one site as an unchallengeable arbiter of the data b. it would increase complexity for developers c. it would presumably depend on the setting of CORS directives to distinguish it from same-domain shared workers. While https://wiki.mozilla.org/WebAPI/DeviceStorageAPI appears to meet a subset of these needs, it does not meet all. +1 Thank you, Brett
Re: [whatwg] Shared storage
On 28 October 2014 21:32, Nils Dagsson Moskopp n...@dieweltistgarnichtso.net wrote: Melvin Carvalho melvincarva...@gmail.com writes: On 15 February 2014 03:04, Brett Zamir bret...@yahoo.com wrote: *The opportunity and current obstacles* The desktop PC thankfully evolved into allowing third-party software which could create and edit files shareable by other third-party software which would have the same rights to do the same. The importance of this can hardly be overestimated. Yet today, on the web, there appears to be no standard way to create content in such an agnostic manner whereby users have full, built-in, locally-controlled portability of their data. *Workarounds* Sure, there is postMessage or CORS requests which can be used to allow one site to be the arbiter of this data. And one could conceivably create a shared data store built upon even postMessage alone, even one which can work fully offline through cache manifests and localStorage or IndexedDB (I have begun some work on this concept at https://gist.github.com/brettz9/8876920 ), but this can only work if: 1. A site or set of domains is trusted to host the shared content. 2. Instead of being built into the browser, it requires that the shared storage site be visited at least one time. *Proposal* 1. Add support for sharedStorage (similar to globalStorage but requiring approval), SharedIndexedDB, and SharedFileWriter/SharedFileSystem which, when used, would cause the browser to prompt the user to require user approval whenever storing or retrieving from such data stores (with an option to remember the choice for a particular site/domain), informing users of potential risks depending on how the data might be used, and potentially allowing them to view, on the spot, the specific data that was being stored. Optional API methods could deter XSS by doing selective escaping, but the potential for abuse should not be used as an excuse for preventing arbitrary shared storage, since again, it is worked well on the desktop, despite risks there, and as works with postMessage despite it also having risks. 2. Add support for corresponding ReadonlyShared storage mechanisms, namespaced by the origin site of the data. A site, http://example.com might add such shared storage under example.com which http://some-other-site.example could retrieve but not alter or delete (unless perhaps a grave warning were given to users about the fact that this was not the same domain). This would have the benefit above postMessage in that if the origin site goes down, third party sites would still be able to have access to the data. 3. Encourage browsers to allow direct editing of this stored data in a human-readable manner (with files at least being ideally directly viewable from the OS desktop). I proposed something similar earlier, and received a reply about doing this through shared workers, but as I understood it, I did not like that possibility because: a. it would limit the neutrality of the storage, creating one site as an unchallengeable arbiter of the data b. it would increase complexity for developers c. it would presumably depend on the setting of CORS directives to distinguish it from same-domain shared workers. While https://wiki.mozilla.org/WebAPI/DeviceStorageAPI appears to meet a subset of these needs, it does not meet all. +1 Stop doing this. Excuse me? -- Nils Dagsson Moskopp // erlehmann http://dieweltistgarnichtso.net
Re: [whatwg] Shared storage
Melvin Carvalho melvincarva...@gmail.com writes: On 28 October 2014 21:32, Nils Dagsson Moskopp n...@dieweltistgarnichtso.net wrote: Melvin Carvalho melvincarva...@gmail.com writes: On 15 February 2014 03:04, Brett Zamir bret...@yahoo.com wrote: *The opportunity and current obstacles* The desktop PC thankfully evolved into allowing third-party software which could create and edit files shareable by other third-party software which would have the same rights to do the same. The importance of this can hardly be overestimated. Yet today, on the web, there appears to be no standard way to create content in such an agnostic manner whereby users have full, built-in, locally-controlled portability of their data. *Workarounds* Sure, there is postMessage or CORS requests which can be used to allow one site to be the arbiter of this data. And one could conceivably create a shared data store built upon even postMessage alone, even one which can work fully offline through cache manifests and localStorage or IndexedDB (I have begun some work on this concept at https://gist.github.com/brettz9/8876920 ), but this can only work if: 1. A site or set of domains is trusted to host the shared content. 2. Instead of being built into the browser, it requires that the shared storage site be visited at least one time. *Proposal* 1. Add support for sharedStorage (similar to globalStorage but requiring approval), SharedIndexedDB, and SharedFileWriter/SharedFileSystem which, when used, would cause the browser to prompt the user to require user approval whenever storing or retrieving from such data stores (with an option to remember the choice for a particular site/domain), informing users of potential risks depending on how the data might be used, and potentially allowing them to view, on the spot, the specific data that was being stored. Optional API methods could deter XSS by doing selective escaping, but the potential for abuse should not be used as an excuse for preventing arbitrary shared storage, since again, it is worked well on the desktop, despite risks there, and as works with postMessage despite it also having risks. 2. Add support for corresponding ReadonlyShared storage mechanisms, namespaced by the origin site of the data. A site, http://example.com might add such shared storage under example.com which http://some-other-site.example could retrieve but not alter or delete (unless perhaps a grave warning were given to users about the fact that this was not the same domain). This would have the benefit above postMessage in that if the origin site goes down, third party sites would still be able to have access to the data. 3. Encourage browsers to allow direct editing of this stored data in a human-readable manner (with files at least being ideally directly viewable from the OS desktop). I proposed something similar earlier, and received a reply about doing this through shared workers, but as I understood it, I did not like that possibility because: a. it would limit the neutrality of the storage, creating one site as an unchallengeable arbiter of the data b. it would increase complexity for developers c. it would presumably depend on the setting of CORS directives to distinguish it from same-domain shared workers. While https://wiki.mozilla.org/WebAPI/DeviceStorageAPI appears to meet a subset of these needs, it does not meet all. +1 Stop doing this. Excuse me? If I am not mistaken, you full-quoted an entire Email just to add „+1“. If you would have done similar in the Google+ Ghetto or on Facebook, I would have no problem with your behaviour. However, on a mailing list, you are annoying people, probably the few hundred or thousand who are subscribed in this case. You are wasting their time – and yours, if I might add … since the WHATWG was not a democracy last time I checked. -- Nils Dagsson Moskopp // erlehmann http://dieweltistgarnichtso.net
[whatwg] Shared storage
*The opportunity and current obstacles* The desktop PC thankfully evolved into allowing third-party software which could create and edit files shareable by other third-party software which would have the same rights to do the same. The importance of this can hardly be overestimated. Yet today, on the web, there appears to be no standard way to create content in such an agnostic manner whereby users have full, built-in, locally-controlled portability of their data. *Workarounds* Sure, there is postMessage or CORS requests which can be used to allow one site to be the arbiter of this data. And one could conceivably create a shared data store built upon even postMessage alone, even one which can work fully offline through cache manifests and localStorage or IndexedDB (I have begun some work on this concept at https://gist.github.com/brettz9/8876920 ), but this can only work if: 1. A site or set of domains is trusted to host the shared content. 2. Instead of being built into the browser, it requires that the shared storage site be visited at least one time. *Proposal* 1. Add support for sharedStorage (similar to globalStorage but requiring approval), SharedIndexedDB, and SharedFileWriter/SharedFileSystem which, when used, would cause the browser to prompt the user to require user approval whenever storing or retrieving from such data stores (with an option to remember the choice for a particular site/domain), informing users of potential risks depending on how the data might be used, and potentially allowing them to view, on the spot, the specific data that was being stored. Optional API methods could deter XSS by doing selective escaping, but the potential for abuse should not be used as an excuse for preventing arbitrary shared storage, since again, it is worked well on the desktop, despite risks there, and as works with postMessage despite it also having risks. 2. Add support for corresponding ReadonlyShared storage mechanisms, namespaced by the origin site of the data. A site, http://example.com might add such shared storage under example.com which http://some-other-site.example could retrieve but not alter or delete (unless perhaps a grave warning were given to users about the fact that this was not the same domain). This would have the benefit above postMessage in that if the origin site goes down, third party sites would still be able to have access to the data. 3. Encourage browsers to allow direct editing of this stored data in a human-readable manner (with files at least being ideally directly viewable from the OS desktop). I proposed something similar earlier, and received a reply about doing this through shared workers, but as I understood it, I did not like that possibility because: a. it would limit the neutrality of the storage, creating one site as an unchallengeable arbiter of the data b. it would increase complexity for developers c. it would presumably depend on the setting of CORS directives to distinguish it from same-domain shared workers. While https://wiki.mozilla.org/WebAPI/DeviceStorageAPI appears to meet a subset of these needs, it does not meet all. Thank you, Brett
